> ## Documentation Index
> Fetch the complete documentation index at: https://docs.heymilo.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Voice agent video
# Guide: Voice Agent with Video (Customer Service Rep)
> **Use case**: Screening customer service candidates with a structured video interview. The AI interviewer asks 5 targeted questions, scores each response, and evaluates communication skills — all without any recruiter time. Ideal for mid-volume roles where you want to assess soft skills and verbal ability before a live interview.
## What You'll Build
A single-stage hiring pipeline:
1. **Video Interview** — An AI interviewer (Sarah) conducts a 5-question video interview. Candidates see a webcam-enabled interface, the AI asks questions conversationally with follow-ups, and each answer is scored on a 1–5 scale. You get a scorecard, transcript, and video recording.
***
## Step 1: Create the Agent
Create a posting with a single `web_interview` workflow step. Setting `"video": true` enables the candidate's webcam during the interview.
```bash theme={null}
curl -X POST "https://api.heymilo.network/api/v4/postings" \
-H "X-API-KEY: your_api_key_here" \
-H "Content-Type: application/json" \
-d '{
"title": "Customer Service Representative",
"description": "We are hiring Customer Service Representatives to join our support team. The ideal candidate has 1+ years of experience in customer-facing roles, strong verbal and written communication skills, and the ability to de-escalate tense situations. This role handles inbound calls and live chat for our SaaS platform. Must be comfortable with CRM tools and have a stable internet connection for remote work.",
"interviewer_name": "Sarah",
"language": "en",
"company_overview": "BrightDesk is a fast-growing SaaS company providing helpdesk software to over 5,000 businesses worldwide.",
"job_overview": "Handle inbound support tickets and calls, troubleshoot product issues, and deliver an exceptional customer experience.",
"workflow": [
{
"type": "web_interview",
"config": {
"video": true,
"num_questions": 5
},
"questions": [
{
"modality": "voice",
"text": "Tell me about a time you turned an unhappy customer into a satisfied one. What was the situation and what did you do?",
"evaluation_criteria": "Assess empathy, problem-solving, and de-escalation ability. Look for a specific example with a clear resolution.",
"score_of_1": "Cannot provide an example or describes a negative interaction with no resolution.",
"score_of_5": "Describes a detailed scenario showing active listening, empathy, creative problem-solving, and a positive outcome.",
"question_type": "open-ended",
"score_weight": 3.0,
"min_follow_ups": 1,
"max_follow_ups": 2
},
{
"modality": "voice",
"text": "How do you prioritize when you have multiple customers waiting for help at the same time?",
"evaluation_criteria": "Evaluate time management and ability to handle pressure. Look for a systematic approach to triage.",
"score_of_1": "No clear strategy; seems overwhelmed by the scenario.",
"score_of_5": "Describes a clear triage system — urgency assessment, setting expectations, and efficient resolution.",
"question_type": "open-ended",
"score_weight": 2.0,
"min_follow_ups": 1,
"max_follow_ups": 2
},
{
"modality": "voice",
"text": "Walk me through how you would troubleshoot a customer who says the software is not working. What questions would you ask?",
"evaluation_criteria": "Assess technical troubleshooting methodology and communication clarity. Look for structured diagnostic approach.",
"score_of_1": "Jumps to conclusions without asking clarifying questions.",
"score_of_5": "Asks smart clarifying questions (browser, error messages, steps to reproduce) and explains thinking clearly.",
"question_type": "open-ended",
"score_weight": 2.0,
"min_follow_ups": 1,
"max_follow_ups": 2
},
{
"modality": "voice",
"text": "Describe your experience with CRM or helpdesk tools. Which ones have you used and how did you use them?",
"evaluation_criteria": "Gauge familiarity with support tooling (Zendesk, Freshdesk, Intercom, Salesforce, etc.). Look for practical usage, not just name-dropping.",
"score_of_1": "No experience with any support tools.",
"score_of_5": "Experienced with multiple tools, can describe workflows like ticket routing, macros, reporting, and escalation.",
"question_type": "open-ended",
"score_weight": 1.5,
"min_follow_ups": 0,
"max_follow_ups": 1
},
{
"modality": "voice",
"text": "Why are you interested in a customer service role at a SaaS company? What excites you about this type of work?",
"evaluation_criteria": "Assess genuine motivation and culture fit. Look for enthusiasm about helping people and interest in technology.",
"score_of_1": "Generic answer with no connection to customer service or SaaS.",
"score_of_5": "Shows genuine passion for helping customers, interest in tech products, and alignment with the role.",
"question_type": "open-ended",
"score_weight": 1.5,
"min_follow_ups": 0,
"max_follow_ups": 1
}
]
}
]
}'
```
**Response:**
```json theme={null}
{
"object": "posting",
"data": {
"object": "posting",
"id": "B5E7F8A2",
"status": "active",
"urls": {
"candidate_url": "https://candidates.heymilo.io/brightdesk/customer-service-representative",
"ingestion_url_key": "xK9mNpQ2rS4tU6vW8yZ0aB3cD5eF7gH9iJ1kL",
"review_url": "https://admin.heymilo.ai/w/WORKSPACE_ID/lab2/B5E7F8A2/1-details"
},
"created_at": 1739612400.0,
"posting": { ... }
}
}
```
**Tip**: The `score_weight` field controls how much each question contributes to the overall score. In this example, the de-escalation question (weight 3.0) counts twice as much as the motivation question (weight 1.5). The AI calculates an overall match score (0–100) using these weights.
***
## Step 2: Manage Questions After Creation
### List all interview questions
```bash theme={null}
curl -X GET "https://api.heymilo.network/api/v4/postings/B5E7F8A2/questions?modality=voice" \
-H "X-API-KEY: your_api_key_here"
```
**Response:**
```json theme={null}
{
"object": "list",
"data": [
{
"id": "Q001",
"modality": "voice",
"text": "Tell me about a time you turned an unhappy customer into a satisfied one...",
"rank": 1,
"score_weight": 3.0,
"score_of_1": "Cannot provide an example...",
"score_of_5": "Describes a detailed scenario..."
},
...
],
"total_count": 5,
"has_more": false,
"url": "/v4/postings/B5E7F8A2/questions"
}
```
### Update a question's scoring criteria
Perhaps after reviewing early results, you want to adjust the evaluation criteria for question 3:
```bash theme={null}
curl -X PATCH "https://api.heymilo.network/api/v4/postings/B5E7F8A2/questions/Q003" \
-H "X-API-KEY: your_api_key_here" \
-H "Content-Type: application/json" \
-d '{
"evaluation_criteria": "Assess technical troubleshooting methodology. Must ask at least 2 clarifying questions before suggesting a solution. Look for browser/device checks and reproduction steps.",
"score_weight": 3.0
}'
```
### Reorder questions
Change the interview flow so the motivation question comes first:
```bash theme={null}
curl -X POST "https://api.heymilo.network/api/v4/postings/B5E7F8A2/questions/reorder" \
-H "X-API-KEY: your_api_key_here" \
-H "Content-Type: application/json" \
-d '{
"modality": "voice",
"question_ids": ["Q005", "Q001", "Q002", "Q003", "Q004"]
}'
```
***
## Step 3: Ingest Candidates
### Single candidate
```bash theme={null}
curl -X POST "https://api.heymilo.network/api/v4/postings/B5E7F8A2/candidates" \
-H "X-API-KEY: your_api_key_here" \
-H "Content-Type: application/json" \
-d '{
"name": "Priya Patel",
"email": "priya.patel@email.com",
"metadata": {
"source": "linkedin",
"recruiter": "jsmith"
}
}'
```
**Response:**
```json theme={null}
{
"object": "candidate",
"data": {
"object": "candidate",
"interview_id": "E8F9A0B1C2D3E4F5",
"interview_url": "https://candidates.heymilo.io/brightdesk/i/E8F9A0B1C2D3E4F5",
"posting_id": "B5E7F8A2"
}
}
```
Share the `interview_url` with the candidate via email, SMS, or your ATS. When they click it, they'll see a video interview interface with Sarah as the interviewer.
### Bulk ingest from your ATS export
```bash theme={null}
curl -X POST "https://api.heymilo.network/api/v4/postings/B5E7F8A2/candidates/bulk" \
-H "X-API-KEY: your_api_key_here" \
-H "Content-Type: application/json" \
-d '{
"data": [
{"name": "Alex Chen", "email": "alex.c@email.com", "metadata": {"source": "greenhouse"}},
{"name": "Jordan Lee", "email": "j.lee@email.com", "metadata": {"source": "greenhouse"}},
{"name": "Taylor Kim", "email": "t.kim@email.com", "metadata": {"source": "greenhouse"}}
]
}'
```
**Phone number is optional** for video-only interviews. Candidates access the interview through the `interview_url` link. However, if you want HeyMilo to send SMS reminders, include `phone_number` in the request.
***
## Step 4: See Results
### List all candidates with scores
```bash theme={null}
curl -X GET "https://api.heymilo.network/api/v4/postings/B5E7F8A2/candidates?limit=50" \
-H "X-API-KEY: your_api_key_here"
```
This returns each candidate's name, email, overall score, and workflow status — useful for building a leaderboard or filtering by score threshold.
### Get the full scorecard for a candidate
```bash theme={null}
curl -X GET "https://api.heymilo.network/api/v4/interviews/E8F9A0B1C2D3E4F5/data" \
-H "X-API-KEY: your_api_key_here"
```
**Response (after completion):**
```json theme={null}
{
"object": "interview_data",
"data": {
"object": "interview_data",
"interview_id": "E8F9A0B1C2D3E4F5",
"posting_id": "B5E7F8A2",
"name": "Priya Patel",
"email": "priya.patel@email.com",
"score": 82.0,
"status": "completed",
"web_interview": {
"object": "web_interview_result",
"interview_id": "E8F9A0B1C2D3E4F5",
"match_score": 82,
"highlights": [
"Excellent de-escalation skills with a specific, compelling example",
"Strong communication clarity throughout the interview"
],
"lowlights": [
"Limited experience with enterprise helpdesk tools"
],
"speech_score": 88.0,
"scorecard": [
{
"question_id": "Q001",
"question": "Tell me about a time you turned an unhappy customer into a satisfied one...",
"evaluation_score": 5,
"evaluation_summary": "Provided an excellent example of handling an angry customer whose order was delayed. Showed active listening, apologized sincerely, offered a discount, and followed up the next day.",
"evaluation_criteria": "Assess empathy, problem-solving, and de-escalation ability.",
"score_weight": 3.0,
"rank": 1,
"not_scored": false,
"chat_transcript": [
{"speaker": "interviewer", "text": "Tell me about a time you turned an unhappy customer into a satisfied one...", "timestamp_str": "00:01:15"},
{"speaker": "candidate", "text": "Absolutely. At my last job at a retail company, I had a customer who was furious because...", "timestamp_str": "00:01:22"}
]
},
{
"question_id": "Q002",
"question": "How do you prioritize when you have multiple customers waiting?",
"evaluation_score": 4,
"evaluation_summary": "Described a clear triage system based on urgency and SLA timers. Could improve on setting customer expectations during wait times.",
"score_weight": 2.0,
"rank": 2,
"not_scored": false
}
],
"transcript": [
{"speaker": "interviewer", "text": "Hi Priya! I'm Sarah from BrightDesk. Thanks for joining us today...", "timestamp_str": "00:00:00"},
{"speaker": "candidate", "text": "Hi Sarah, thank you for having me! I'm really excited about this opportunity.", "timestamp_str": "00:00:08"}
],
"audio_recording_url": "https://cdn.heymilo.io/recordings/E8F9A0B1.mp3",
"video_recording_url": "https://cdn.heymilo.io/recordings/E8F9A0B1.mp4",
"tags": ["empathetic", "structured-thinker", "strong-communicator"]
}
}
}
```
### Key things to look at in the results
| Field | What It Tells You |
| -------------------------------- | ----------------------------------------- |
| `score` | Overall weighted match score (0–100) |
| `speech_score` | Communication quality rating |
| `scorecard[].evaluation_score` | Per-question score (1–5) |
| `scorecard[].evaluation_summary` | AI's reasoning for the score |
| `highlights` / `lowlights` | Quick strengths and areas for improvement |
| `video_recording_url` | Watch the full interview yourself |
| `transcript` | Read the complete conversation |
### Set up webhooks
Get notified as soon as interviews are complete:
```bash theme={null}
curl -X POST "https://api.heymilo.network/api/v4/webhooks" \
-H "X-API-KEY: your_api_key_here" \
-H "Content-Type: application/json" \
-d '{
"posting_id": "B5E7F8A2",
"url": "https://your-system.com/heymilo-callback",
"event_type": "report_available"
}'
```
***
## Summary
| Step | Endpoint | What Happens |
| -------------------- | --------------------------------------------------- | --------------------------------------------------- |
| Create agent | `POST /v4/postings` | Video interview pipeline with 5 weighted questions |
| Manage questions | `GET/POST/PATCH/DELETE /v4/postings/{id}/questions` | Adjust questions, weights, and order at any time |
| Reorder questions | `POST /v4/postings/{id}/questions/reorder` | Change the interview question sequence |
| Ingest candidate | `POST /v4/postings/{id}/candidates` | Candidate receives an interview link |
| Candidate interviews | *(automatic)* | Sarah conducts a video interview with follow-ups |
| Get results | `GET /v4/interviews/{id}/data` | Scorecard, transcript, speech score, and recordings |
| Webhook | `POST /v4/webhooks` | Real-time notification when report is ready |