Guide: Voice Agent with Video (Customer Service Rep)

Use case: Screening customer service candidates with a structured video interview. The AI interviewer asks 5 targeted questions, scores each response, and evaluates communication skills — all without any recruiter time. Ideal for mid-volume roles where you want to assess soft skills and verbal ability before a live interview.

What You’ll Build

A single-stage hiring pipeline:

Video Interview — An AI interviewer (Sarah) conducts a 5-question video interview. Candidates see a webcam-enabled interface, the AI asks questions conversationally with follow-ups, and each answer is scored on a 1–5 scale. You get a scorecard, transcript, and video recording.

Step 1: Create the Agent

Create a posting with a single web_interview workflow step. Setting "video": true enables the candidate’s webcam during the interview.

curl -X POST "https://api.heymilo.network/api/v4/postings" \
  -H "X-API-KEY: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Customer Service Representative",
    "description": "We are hiring Customer Service Representatives to join our support team. The ideal candidate has 1+ years of experience in customer-facing roles, strong verbal and written communication skills, and the ability to de-escalate tense situations. This role handles inbound calls and live chat for our SaaS platform. Must be comfortable with CRM tools and have a stable internet connection for remote work.",
    "interviewer_name": "Sarah",
    "language": "en",
    "company_overview": "BrightDesk is a fast-growing SaaS company providing helpdesk software to over 5,000 businesses worldwide.",
    "job_overview": "Handle inbound support tickets and calls, troubleshoot product issues, and deliver an exceptional customer experience.",
    "workflow": [
      {
        "type": "web_interview",
        "config": {
          "video": true,
          "num_questions": 5
        },
        "questions": [
          {
            "modality": "voice",
            "text": "Tell me about a time you turned an unhappy customer into a satisfied one. What was the situation and what did you do?",
            "evaluation_criteria": "Assess empathy, problem-solving, and de-escalation ability. Look for a specific example with a clear resolution.",
            "score_of_1": "Cannot provide an example or describes a negative interaction with no resolution.",
            "score_of_5": "Describes a detailed scenario showing active listening, empathy, creative problem-solving, and a positive outcome.",
            "question_type": "open-ended",
            "score_weight": 3.0,
            "min_follow_ups": 1,
            "max_follow_ups": 2
          },
          {
            "modality": "voice",
            "text": "How do you prioritize when you have multiple customers waiting for help at the same time?",
            "evaluation_criteria": "Evaluate time management and ability to handle pressure. Look for a systematic approach to triage.",
            "score_of_1": "No clear strategy; seems overwhelmed by the scenario.",
            "score_of_5": "Describes a clear triage system — urgency assessment, setting expectations, and efficient resolution.",
            "question_type": "open-ended",
            "score_weight": 2.0,
            "min_follow_ups": 1,
            "max_follow_ups": 2
          },
          {
            "modality": "voice",
            "text": "Walk me through how you would troubleshoot a customer who says the software is not working. What questions would you ask?",
            "evaluation_criteria": "Assess technical troubleshooting methodology and communication clarity. Look for structured diagnostic approach.",
            "score_of_1": "Jumps to conclusions without asking clarifying questions.",
            "score_of_5": "Asks smart clarifying questions (browser, error messages, steps to reproduce) and explains thinking clearly.",
            "question_type": "open-ended",
            "score_weight": 2.0,
            "min_follow_ups": 1,
            "max_follow_ups": 2
          },
          {
            "modality": "voice",
            "text": "Describe your experience with CRM or helpdesk tools. Which ones have you used and how did you use them?",
            "evaluation_criteria": "Gauge familiarity with support tooling (Zendesk, Freshdesk, Intercom, Salesforce, etc.). Look for practical usage, not just name-dropping.",
            "score_of_1": "No experience with any support tools.",
            "score_of_5": "Experienced with multiple tools, can describe workflows like ticket routing, macros, reporting, and escalation.",
            "question_type": "open-ended",
            "score_weight": 1.5,
            "min_follow_ups": 0,
            "max_follow_ups": 1
          },
          {
            "modality": "voice",
            "text": "Why are you interested in a customer service role at a SaaS company? What excites you about this type of work?",
            "evaluation_criteria": "Assess genuine motivation and culture fit. Look for enthusiasm about helping people and interest in technology.",
            "score_of_1": "Generic answer with no connection to customer service or SaaS.",
            "score_of_5": "Shows genuine passion for helping customers, interest in tech products, and alignment with the role.",
            "question_type": "open-ended",
            "score_weight": 1.5,
            "min_follow_ups": 0,
            "max_follow_ups": 1
          }
        ]
      }
    ]
  }'

Response:

{
  "object": "posting",
  "data": {
    "object": "posting",
    "id": "B5E7F8A2",
    "status": "active",
    "urls": {
      "candidate_url": "https://candidates.heymilo.io/brightdesk/customer-service-representative",
      "ingestion_url_key": "xK9mNpQ2rS4tU6vW8yZ0aB3cD5eF7gH9iJ1kL",
      "review_url": "https://admin.heymilo.ai/w/WORKSPACE_ID/lab2/B5E7F8A2/1-details"
    },
    "created_at": 1739612400.0,
    "posting": { ... }
  }
}

Tip: The score_weight field controls how much each question contributes to the overall score. In this example, the de-escalation question (weight 3.0) counts twice as much as the motivation question (weight 1.5). The AI calculates an overall match score (0–100) using these weights.

Step 2: Manage Questions After Creation

List all interview questions

curl -X GET "https://api.heymilo.network/api/v4/postings/B5E7F8A2/questions?modality=voice" \
  -H "X-API-KEY: your_api_key_here"

Response:

{
  "object": "list",
  "data": [
    {
      "id": "Q001",
      "modality": "voice",
      "text": "Tell me about a time you turned an unhappy customer into a satisfied one...",
      "rank": 1,
      "score_weight": 3.0,
      "score_of_1": "Cannot provide an example...",
      "score_of_5": "Describes a detailed scenario..."
    },
    ...
  ],
  "total_count": 5,
  "has_more": false,
  "url": "/v4/postings/B5E7F8A2/questions"
}

Update a question’s scoring criteria

Perhaps after reviewing early results, you want to adjust the evaluation criteria for question 3:

curl -X PATCH "https://api.heymilo.network/api/v4/postings/B5E7F8A2/questions/Q003" \
  -H "X-API-KEY: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "evaluation_criteria": "Assess technical troubleshooting methodology. Must ask at least 2 clarifying questions before suggesting a solution. Look for browser/device checks and reproduction steps.",
    "score_weight": 3.0
  }'

Reorder questions

Change the interview flow so the motivation question comes first:

curl -X POST "https://api.heymilo.network/api/v4/postings/B5E7F8A2/questions/reorder" \
  -H "X-API-KEY: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "modality": "voice",
    "question_ids": ["Q005", "Q001", "Q002", "Q003", "Q004"]
  }'

Step 3: Ingest Candidates

Single candidate

curl -X POST "https://api.heymilo.network/api/v4/postings/B5E7F8A2/candidates" \
  -H "X-API-KEY: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Priya Patel",
    "email": "priya.patel@email.com",
    "metadata": {
      "source": "linkedin",
      "recruiter": "jsmith"
    }
  }'

Response:

{
  "object": "candidate",
  "data": {
    "object": "candidate",
    "interview_id": "E8F9A0B1C2D3E4F5",
    "interview_url": "https://candidates.heymilo.io/brightdesk/i/E8F9A0B1C2D3E4F5",
    "posting_id": "B5E7F8A2"
  }
}

Share the interview_url with the candidate via email, SMS, or your ATS. When they click it, they’ll see a video interview interface with Sarah as the interviewer.

Bulk ingest from your ATS export

curl -X POST "https://api.heymilo.network/api/v4/postings/B5E7F8A2/candidates/bulk" \
  -H "X-API-KEY: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "data": [
      {"name": "Alex Chen", "email": "alex.c@email.com", "metadata": {"source": "greenhouse"}},
      {"name": "Jordan Lee", "email": "j.lee@email.com", "metadata": {"source": "greenhouse"}},
      {"name": "Taylor Kim", "email": "t.kim@email.com", "metadata": {"source": "greenhouse"}}
    ]
  }'

Phone number is optional for video-only interviews. Candidates access the interview through the interview_url link. However, if you want HeyMilo to send SMS reminders, include phone_number in the request.

Step 4: See Results

List all candidates with scores

curl -X GET "https://api.heymilo.network/api/v4/postings/B5E7F8A2/candidates?limit=50" \
  -H "X-API-KEY: your_api_key_here"

This returns each candidate’s name, email, overall score, and workflow status — useful for building a leaderboard or filtering by score threshold.

Get the full scorecard for a candidate

curl -X GET "https://api.heymilo.network/api/v4/interviews/E8F9A0B1C2D3E4F5/data" \
  -H "X-API-KEY: your_api_key_here"

Response (after completion):

{
  "object": "interview_data",
  "data": {
    "object": "interview_data",
    "interview_id": "E8F9A0B1C2D3E4F5",
    "posting_id": "B5E7F8A2",
    "name": "Priya Patel",
    "email": "priya.patel@email.com",
    "score": 82.0,
    "status": "completed",
    "web_interview": {
      "object": "web_interview_result",
      "interview_id": "E8F9A0B1C2D3E4F5",
      "match_score": 82,
      "highlights": [
        "Excellent de-escalation skills with a specific, compelling example",
        "Strong communication clarity throughout the interview"
      ],
      "lowlights": [
        "Limited experience with enterprise helpdesk tools"
      ],
      "speech_score": 88.0,
      "scorecard": [
        {
          "question_id": "Q001",
          "question": "Tell me about a time you turned an unhappy customer into a satisfied one...",
          "evaluation_score": 5,
          "evaluation_summary": "Provided an excellent example of handling an angry customer whose order was delayed. Showed active listening, apologized sincerely, offered a discount, and followed up the next day.",
          "evaluation_criteria": "Assess empathy, problem-solving, and de-escalation ability.",
          "score_weight": 3.0,
          "rank": 1,
          "not_scored": false,
          "chat_transcript": [
            {"speaker": "interviewer", "text": "Tell me about a time you turned an unhappy customer into a satisfied one...", "timestamp_str": "00:01:15"},
            {"speaker": "candidate", "text": "Absolutely. At my last job at a retail company, I had a customer who was furious because...", "timestamp_str": "00:01:22"}
          ]
        },
        {
          "question_id": "Q002",
          "question": "How do you prioritize when you have multiple customers waiting?",
          "evaluation_score": 4,
          "evaluation_summary": "Described a clear triage system based on urgency and SLA timers. Could improve on setting customer expectations during wait times.",
          "score_weight": 2.0,
          "rank": 2,
          "not_scored": false
        }
      ],
      "transcript": [
        {"speaker": "interviewer", "text": "Hi Priya! I'm Sarah from BrightDesk. Thanks for joining us today...", "timestamp_str": "00:00:00"},
        {"speaker": "candidate", "text": "Hi Sarah, thank you for having me! I'm really excited about this opportunity.", "timestamp_str": "00:00:08"}
      ],
      "audio_recording_url": "https://cdn.heymilo.io/recordings/E8F9A0B1.mp3",
      "video_recording_url": "https://cdn.heymilo.io/recordings/E8F9A0B1.mp4",
      "tags": ["empathetic", "structured-thinker", "strong-communicator"]
    }
  }
}

Key things to look at in the results

Field	What It Tells You
`score`	Overall weighted match score (0–100)
`speech_score`	Communication quality rating
`scorecard[].evaluation_score`	Per-question score (1–5)
`scorecard[].evaluation_summary`	AI’s reasoning for the score
`highlights` / `lowlights`	Quick strengths and areas for improvement
`video_recording_url`	Watch the full interview yourself
`transcript`	Read the complete conversation

Set up webhooks

Get notified as soon as interviews are complete:

curl -X POST "https://api.heymilo.network/api/v4/webhooks" \
  -H "X-API-KEY: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "posting_id": "B5E7F8A2",
    "url": "https://your-system.com/heymilo-callback",
    "event_type": "report_available"
  }'

Summary

Step	Endpoint	What Happens
Create agent	`POST /v4/postings`	Video interview pipeline with 5 weighted questions
Manage questions	`GET/POST/PATCH/DELETE /v4/postings/{id}/questions`	Adjust questions, weights, and order at any time
Reorder questions	`POST /v4/postings/{id}/questions/reorder`	Change the interview question sequence
Ingest candidate	`POST /v4/postings/{id}/candidates`	Candidate receives an interview link
Candidate interviews	(automatic)	Sarah conducts a video interview with follow-ups
Get results	`GET /v4/interviews/{id}/data`	Scorecard, transcript, speech score, and recordings
Webhook	`POST /v4/webhooks`	Real-time notification when report is ready

Getting Started

Your Dashboard

Interviewers

Interviews

Candidate Experience

Sourcing

Tools

Outreach

Inviting Candidates

Integrations

Data

Settings & Admin

API (Next Gen)

Public API

Webhook Management

Voice agent video

Guide: Voice Agent with Video (Customer Service Rep)

What You’ll Build

Step 1: Create the Agent

Step 2: Manage Questions After Creation

List all interview questions

Update a question’s scoring criteria

Reorder questions

Step 3: Ingest Candidates

Single candidate

Bulk ingest from your ATS export

Step 4: See Results

List all candidates with scores

Get the full scorecard for a candidate

Key things to look at in the results

Set up webhooks

Summary

Getting Started

Your Dashboard

Interviewers

Interviews

Candidate Experience

Sourcing

Tools

Outreach

Inviting Candidates

Integrations

Data

Settings & Admin

API (Next Gen)

Public API

Webhook Management

Documentation Index

​Guide: Voice Agent with Video (Customer Service Rep)

​What You’ll Build

​Step 1: Create the Agent

​Step 2: Manage Questions After Creation

​List all interview questions

​Update a question’s scoring criteria

​Reorder questions

​Step 3: Ingest Candidates

​Single candidate

​Bulk ingest from your ATS export

​Step 4: See Results

​List all candidates with scores

​Get the full scorecard for a candidate

​Key things to look at in the results

​Set up webhooks

​Summary

Guide: Voice Agent with Video (Customer Service Rep)

What You’ll Build

Step 1: Create the Agent

Step 2: Manage Questions After Creation

List all interview questions

Update a question’s scoring criteria

Reorder questions

Step 3: Ingest Candidates

Single candidate

Bulk ingest from your ATS export

Step 4: See Results

List all candidates with scores

Get the full scorecard for a candidate

Key things to look at in the results

Set up webhooks

Summary