43,000+ AI screening calls were made: Check out the full analysis

  • Jaymin Bhuptani
  • November 18, 2026
  • 7 min read
43,000+ AI screening calls were made: Check out the full analysis

At Uplers, we have an interesting advantage when it comes to building AI.

We have a 50-person recruitment team, including experienced principal recruiters who have closed complex roles for early-stage startups. Alongside them, we have a 10-person engineering team building AI solutions for our customers and our own teams.

This means we can test AI alongside experienced recruiters doing the same work every day, rather than evaluating it in isolation.

Most of our customers are early-stage startups. Many don’t have an ATS or an internal recruiter, and the profiles we send often go directly to the founder. So when we started building Reya, our voice AI agent, the question wasn’t whether AI could make screening calls.

It was whether AI could help us send fewer, better-validated candidates to startup founders.

Since May, Reya has made 43,492 screening calls, spoken to 35,789 candidates, and spent 7,145 hours on the phone. A lot has worked. Quite a few things have broken along the way, too.

Here are 9 lessons from building and testing voice AI in a real recruitment workflow.

1. Scaling conversations was easy. Scaling good decisions wasn’t.

Voice AI can run multiple conversations in parallel, something human recruiters simply can’t do. That gave us significant leverage. With Reya working alongside a small group of recruiters, we moved from sharing roughly 18 profiles a day to around 50.

But we soon realised that more conversations didn’t necessarily mean better screening. If AI is speaking to the wrong candidates, asking irrelevant questions, or continuing conversations that should have ended, increasing capacity only creates more wasted effort.

We had to start looking beyond how many calls Reya could make and focus on how much useful work those calls actually produced.

2. Longer conversations weren’t always better.

One of Reya’s strengths was her ability to go deeper into a candidate’s experience. She could prepare using the candidate’s CV, the job description, the client’s must-have requirements, and relevant technical context. She could then ask follow-up questions based on the answers.

In one month of data, 32.1% of Reya’s answered calls lasted more than 10 minutes, while 23.4% of recruiters’ answered calls lasted more than 5 minutes.

These aren’t directly comparable measures, but they did show that candidates were willing to engage in fairly detailed conversations with AI.

Initially, we saw that as a positive sign. Our experienced recruiters helped us see the other side. A good recruiter can sometimes tell within two minutes that someone isn’t right for a role. Reya might continue asking questions for another 15 minutes.

Longer conversations were useful when a candidate warranted deeper assessment, but unnecessary when the mismatch was already clear. We’re still improving Reya’s ability to recognise when to probe further and when to stop.

It’s been a useful reminder that the metric you’re improving isn’t always the outcome you should be optimising for.

3. Our biggest early problem wasn’t intelligence. It was latency.

An early version of Reya had a frustrating delay. A candidate would answer the phone and sometimes hear silence for eight to ten seconds before Reya responded. By then, the experience already felt broken, regardless of how intelligent the next question was.

We eventually reduced that delay to around 1.5–2 seconds.

It sounds like a small engineering improvement, but it made a substantial difference to how natural the conversations felt. If I were building a voice product again, I’d prioritise latency much earlier instead of focusing only on making the model more sophisticated. People experience the delay before they experience the intelligence.

4. Improving the voice made a bigger difference than we expected.

Our early version sounded robotic, and candidates were noticeably more guarded during conversations. We worked on tone, pacing, conversational language, interruptions, and how naturally Reya responded. Those changes improved call completion by roughly 5–7 percentage points. That surprised me because we tend to think AI product quality is primarily about model capability.

But in a voice conversation, trust is also influenced by timing, tone, rhythm, and whether the interaction feels comfortable. A technically capable agent isn’t necessarily a good conversational experience.

5. We started with a voice agent and ended up building a workflow.

Candidates don’t always behave the way you expect them to. They miss calls, ask to see the JD first, want to reschedule, or have outdated CVs. Some stop responding on one channel but answer immediately on another. And that’s okay. So we wanted to work around these dynamic behaviour.

Voice alone couldn’t handle all of this. Over time, Reya became part of a broader workflow involving voice calls, WhatsApp, email, follow-ups, and rescheduling. We also ran into operational problems we hadn’t anticipated.

For example, when too many calls went out from the same number, it started getting flagged as spam. We had to distribute calls across multiple numbers, which materially improved pickup rates. These weren’t problems with the AI model itself, but they affected whether the system could actually do its job.

One of our biggest learnings was that building a good AI feature isn’t enough. The entire workflow around it needs to work.

6. One of our best AI use cases came from a candidate complaint.

This was a problem that had bothered me for a while. Whenever I spoke to candidates about what we could improve, one complaint kept coming up. They didn’t mind recruiters from Uplers calling them. It was a job opportunity, after all.

What frustrated them was having to repeat the same information to different recruiters. Their notice period, compensation, career history, and reasons for looking for a change. We had tried addressing this through process improvements, but it was difficult to make the experience consistent across the team.

AI gave us a better way to solve it. We started storing candidate context centrally so future conversations could build on information already shared. Now candidates have to repeat themselves less, and the same information helps us identify more relevant opportunities for them later.

The benefit wasn’t limited to recruiter efficiency. It also improved the candidate experience. This changed how I think about identifying AI use cases. Some of the best opportunities don’t begin with asking what AI can automate. They begin with understanding what’s frustrating the user and finding a better way to solve it.

And as that shared context grows, it becomes increasingly useful.

7. Then candidates started bringing AI into the interviews, too.

We began noticing candidates who appeared to be reading answers generated by other AI tools during screening conversations. The obvious response might be to detect AI usage and reject the candidate.

I don’t think that’s the right approach. People are going to use AI, both at work and during the hiring process. The more important question is whether the candidate actually understands the answer.

Can they explain their reasoning? Can they go deeper when questioned? Can they apply that knowledge to a real problem?

We’re training Reya to recognise certain signals and probe further when necessary. Those observations can then be flagged for a human recruiter. The objective isn’t to catch candidates using AI. It’s to understand whether their capabilities are genuine.

Hiring someone into a role they cannot perform is bad for the startup and bad for the candidate.

I think this will become an increasingly important challenge in hiring: evaluating whether someone genuinely understands and owns an answer, rather than simply whether they can produce the right one.

8. Our best recruiters became more important, not less.

One of the most valuable parts of our setup is having experienced recruiters working alongside the engineering team. They bring judgment that AI still struggles to replicate.

They know when a compensation gap might be negotiable, when a technically correct answer lacks depth, or when a candidate’s interest in joining a startup doesn’t seem convincing. They can also recognise when the founder’s requirement itself needs to be challenged.

That experience continuously helps us improve Reya. Human recruiters are still involved before many candidates reach our customers, and their feedback helps us understand where the system needs improvement.

Our operating model is gradually taking shape around three things: AI handling more of the repetitive work, recruiters contributing their judgment, and founder feedback helping us improve both. If you’re building AI for any expert workflow, I’d strongly recommend involving your best people in the development process.

Their experience and judgment are some of the most valuable inputs you have.

9. Not every candidate wants the same AI experience.

Another thing we underestimated was how differently candidates respond to AI. Less-experienced candidates, particularly those actively or urgently looking for opportunities, tend to be more comfortable starting with an AI conversation. Senior candidates behave differently.

At around 8–10+ years of experience, we’re seeing more resistance to AI-first screening. Some don’t want their first serious conversation about a career move to be with an AI agent.

We haven’t completely solved this yet. We’re still experimenting with different approaches. Should AI handle the first conversation and bring in a recruiter later? Should a human recruiter lead and use AI for structured assessment? Should certain roles remain human-led throughout?

The answer may vary depending on the candidate and the role. Just because AI can handle the first interaction doesn’t mean it should be the default for everyone.

Our scoreboard so far

After more than 43,000 conversations, here’s where things stand.

What’s working

  • Scale: Reya can run multiple screening conversations simultaneously.
  • Speed: We’ve materially increased the number of profiles our team can process and share.
  • Depth: AI can prepare using role-specific context and ask adaptive technical follow-up questions.
  • Candidate experience: Improving latency and voice quality has increased call completion.
  • Workflow: Combining voice, WhatsApp, email, and follow-ups works better than voice alone.
  • Shared memory: Retaining candidate context reduces repetition and helps with future matching.

What still needs work

  • Knowing when to stop: Reya doesn’t always recognise when a conversation has gone far enough.
  • Measuring quality: Longer conversations and higher engagement don’t necessarily mean better screening.
  • Senior candidate adoption: AI-first screening doesn’t work equally well for every candidate.
  • AI-assisted answers: Assessing genuine understanding is becoming more complicated.
  • Human judgment: Experienced recruiters still handle nuance, exceptions, and context better.

Six things I’d do differently if we were starting again

1. Define the business outcome before the AI metrics: Calls made, conversation duration, and completion rates matter only if they improve the quality and speed of hiring.

2. Involve your best human operators from the beginning: Their experience helps identify mistakes and improve the product much faster.

3. Fix latency and the user experience early: A sophisticated model won’t compensate for an uncomfortable conversation.

4. Build shared memory from day one: Retaining context makes future interactions more useful and reduces repetition for users.

5. Design the complete workflow: The AI conversation is only one part of getting the job done.

6. Be deliberate about where humans remain involved: Not every decision or interaction needs to be automated.

Where we go from here

After 43,000+ conversations, I’m more optimistic about voice AI than when we started.

But I’m also more convinced that the goal shouldn’t be to replace recruiters. It should be to help them make better hiring decisions.

Because founders don’t need more profiles. They need fewer profiles that are actually worth interviewing.

And if you’re an early-stage founder struggling with hiring, happy to have a no-pressure conversation to understand your challenges and see how we can help.

Master of turning ideas into impact, is a leader who blends vision with action. Passionate about innovation and building businesses that thrive, he’s a problem-solver at heart. Equal parts strategist and motivator, Jaymin leads with humor, heart, and an eye on what’s next.
Jaymin Bhuptani

Jaymin BhuptaniLinkedin

CEO