Setup Mistakes
1. Vague or Missing Scoring Rubrics
The mistake: Setting up questions without defining what a good answer looks like. Why it matters: The AI scores based on your rubrics. If you write “evaluate based on experience,” the AI has no clear standard. You’ll get inconsistent scores that are hard to trust. The fix: For each question, write out what a strong answer includes and what a weak answer looks like. Be specific. “3+ years of customer support experience with examples of handling escalations” is much better than “relevant experience.”2. Too Many Questions
The mistake: Adding 15+ questions to a single interview because “we want to be thorough.” Why it matters: Long interviews have dramatically higher drop-off rates. Candidates abandon interviews that feel like they’ll never end, especially for hourly roles. The fix: Stick to 5-8 questions for voice/video interviews and 3-5 for SMS. Focus on the questions that actually differentiate candidates. If you can’t tell the difference between a great and mediocre answer, the question probably isn’t worth asking.3. Wrong Interview Type for the Audience
The mistake: Using voice/video (phone) for hourly warehouse workers who can’t take a phone call during their shift, or using SMS for executive roles. Why it matters: The interview format needs to match your candidates’ reality. A warehouse worker is more likely to respond to a text on their break. An executive expects a more polished experience. The fix: Use SMS for high-volume hourly hiring, voice for professional and customer-facing roles, and web for remote/tech positions. See Interview Types for a full comparison.4. Not Testing the Interview Yourself
The mistake: Going live without ever clicking the interview link yourself. Why it matters: You might have confusing questions, a weird welcome message, or a broken flow that you’d catch immediately if you experienced it as a candidate. The fix: Before going live with any interviewer, click the link and complete the interview yourself. It takes 10 minutes and you’ll catch issues every time.Usage Mistakes
5. Not Sending Reminders
The mistake: Sending one invitation email and waiting. Why it matters: Candidates are busy. Many intend to complete the interview but forget. A single reminder can boost completion rates by 20-30%. The fix: Always send at least one follow-up 2-3 days after the initial invitation. Use the templates in Inviting Candidates & Driving Completion.6. Ignoring Low Completion Rates
The mistake: Only looking at candidates who completed and ignoring the drop-off. Why it matters: If only 30% of invited candidates complete the interview, you’re missing 70% of your pipeline. Some of those incomplete candidates could be your best hires. The fix: Monitor completion rates in Your Dashboard. If rates drop below 60-70%, investigate: Are interviews too long? Are invitations unclear? Is the interview type wrong for the audience?7. Not Calibrating Scores
The mistake: Taking scores at face value without ever reading the actual transcripts. Why it matters: Early on, your rubrics might not perfectly capture what you’re looking for. A candidate scoring 3.5 might actually be great, or a 4.5 might have red flags that the score doesn’t reflect. The fix: For your first 10-20 interviews, read the full transcripts alongside the scores. This helps you trust the system and adjust rubrics where needed. After calibrating, you can rely more on scores for quick filtering. For workspace-wide tuning, see Adjusting Scoring (Interview Evaluation).8. Not Updating the Knowledge Base
The mistake: Setting up the Knowledge Base once and never updating it. Why it matters: If your benefits change, your office moves, or your team structure shifts, the AI will give candidates outdated information during interviews. That’s a bad look. The fix: Review your Knowledge Base quarterly, or whenever there’s a significant company update. Deactivate items that are no longer accurate.Next Steps
- Creating an Interviewer: Follow the full setup guide to avoid common pitfalls
- Quick Reference Cheat Sheet: Common tasks at a glance
- How Scoring Works: Understand scoring so you can write better rubrics