These three tools are frequently compared as if they were alternatives to each other. They are not, quite. ElevenLabs is primarily a voice synthesis layer that has expanded into full agents. Vapi and Retell are orchestration platforms that assemble telephony, transcription, reasoning and synthesis into a single pipeline, and both can use ElevenLabs voices inside them.
Understanding that distinction is most of the decision. The rest is a question of how much control you want and how much assembly you are willing to do.
A note on what this article is and is not. Pricing and feature sets in this category change every few months. We have deliberately avoided quoting specific rates that would be wrong by the time you read this. What we have written instead is the set of criteria that stay stable, and what each platform is architecturally good at.
The layer they each occupy
Recall the four layer structure of a voice agent. Telephony connects the call. Speech to text transcribes the caller. A reasoning layer decides what to say and when to act. Text to speech produces the reply.
ElevenLabs started at the fourth layer and is generally regarded as the leader there. Its voice quality is the reason it is in most production stacks, whether or not it is the platform running the agent. It has since built a conversational agent product that covers the other layers too.
Vapi and Retell both started as orchestration. They handle the whole pipeline, let you plug in your choice of transcription and synthesis providers, and expose the reasoning layer for you to configure. Neither is trying to be the best voice model. They are trying to be the best assembly of models.
The practical consequence. If your primary concern is how the agent sounds, that decision is largely independent of which orchestration platform you pick, because most orchestration platforms can call ElevenLabs. If your primary concern is how the agent behaves, the orchestration platform is what matters.
An analogy. Think of it as building a recording studio. ElevenLabs is the microphone, and it is a very good microphone. Vapi and Retell are the mixing desk. Arguing about which mixing desk has the better microphone misses that you will probably use the same microphone on either desk.
What to actually evaluate
Feature checklists in this space are close to useless, because everyone ships everyone else's features within a quarter. These are the criteria that actually differ in practice and that you will still care about a year from now.
Latency and how it is achieved
Every platform will quote a latency figure. Ask how it is measured, because the number that matters is end to end, from the caller finishing a sentence to audio starting to play, under real network conditions with your integrations in the loop. A benchmark taken without tool calls is not representative of your agent.
Ask specifically about how the platform handles endpoint detection, meaning deciding that the caller has finished speaking. This is where a surprising amount of perceived latency comes from, and it is where platforms genuinely differ.
How it handles interruption
Callers talk over agents. Constantly. The question is what happens when they do. Does the agent stop speaking immediately, does it finish its sentence, and does it correctly discard what it was about to say?
This is worth testing rather than reading about. Get a trial account, build a trivial agent, and interrupt it repeatedly. The differences are obvious in thirty seconds of trying and invisible in documentation.
Tool calling and integration model
This is the criterion that most often determines the outcome of a project, and it is the one most often skipped during evaluation.
How does the platform let the agent call your systems? Is it a webhook you host, a native integration, or a function definition you supply? What happens when your endpoint is slow, and what does the caller hear during the wait? What happens when your endpoint fails, and can the agent recover gracefully or does the call collapse?
That last question matters more than anything on the feature list. An agent that says the appointment is booked when the write actually failed is worse than no agent at all.
Observability
After launch, you need to see what happened on every call. Full transcripts, the tool calls the agent made and what they returned, latency broken down by stage, and the point at which calls were escalated or abandoned.
Platforms differ significantly here and it is easy to overlook during evaluation, because during evaluation you are watching every call yourself. Six months in, when you have five thousand calls a month, observability is the entire product.
Portability
Ask how much of your work moves if you leave. Conversation logic expressed as prompts and function definitions is fairly portable. Logic expressed in a proprietary visual builder is not.
This is not an argument against visual builders, which are genuinely faster to work in. It is an argument for knowing what you are trading.
Compliance and data handling
Where are recordings stored, for how long, which subprocessors touch the audio, and can you turn recording off entirely. In healthcare, finance and legal contexts this often narrows the field before any other criterion is considered, so check it first rather than last.
How the three compare in practice
ElevenLabs is the default choice for the voice itself. If the way your agent sounds is a competitive consideration, and in consumer facing work it usually is, this is where the quality difference is audible to a caller. Its agent product is a reasonable option when you want a single vendor and your integration needs are modest.
Vapi is generally the choice of teams who want control. It exposes the pipeline, lets you swap components, and assumes you are comfortable writing code. That is a strength if you have engineers and a burden if you do not.
Retell tends to appeal to teams who want a shorter path to something working, with more of the assembly handled for you. The trade is the usual one, less configuration surface in exchange for less to configure.
Trillet, which we work with directly, sits in the orchestration category and is worth evaluating alongside the other two, particularly where the priority is getting a working agent connected to real business systems rather than maximum flexibility.
Any of these can produce a good agent. We have seen good and bad implementations on all of them, and the variance between implementations on the same platform is larger than the variance between platforms.
The mistake almost everyone makes
Teams spend three weeks comparing platforms and three days on conversation design and integration. This is backwards, and it is the single most reliable predictor of a disappointing outcome.
The platform is roughly twenty percent of the result. The other eighty percent is what your agent asks, in what order, how it recovers from confusion, when it escalates, and whether it can actually read and write to the systems your business runs on. None of that is determined by which platform you chose.
The analogy. Choosing a voice platform is like choosing a statistical package for a research project. SPSS, R and Stata will all produce the same regression coefficients. If your study design is flawed, none of them will save you, and no reviewer will care which one you used.
A practical selection process
If you want a defensible decision without spending a month on it, this sequence works.
Write the requirements first. One page. What call types, what systems it must reach, what languages, what regulatory constraints, what your escalation rules are. Do this before looking at any platform, or you will unconsciously write the requirements to match whichever platform you saw first.
Eliminate on compliance. If you are in a regulated sector, this typically removes options immediately and saves you evaluating them.
Build the same trivial agent on two platforms. Not a demo from their documentation. Your actual highest volume call type, connected to a real test instance of your real system. Two days each is enough.
Interrupt it, confuse it, and break the integration on purpose. Unplug the test endpoint mid call and see what the caller hears. This single test tells you more than any comparison table.
Choose, and then stop. The switching cost later is real but not catastrophic if you keep your conversation logic portable. Do not spend another month.
What we would tell you if you asked directly
For most businesses building their first voice agent, the platform is not the decision that determines success. Pick a mainstream orchestration platform, use a high quality voice, and put your effort into the conversation design, the integrations and the escalation rules.
If you want help specifying the requirements or a working agent built on a platform that suits your constraints, get in touch and we will scope it properly.
Frequently asked questions
Can I use ElevenLabs voices inside Vapi or Retell? Generally yes. Most orchestration platforms let you select a synthesis provider. This is why the voice decision and the platform decision are largely separable.
Which is cheapest? The per minute cost is driven mostly by which models and voices you select rather than which orchestration platform wraps them. Platform fees exist but are usually a minority of the total. Compare a like for like configuration, not headline rates.
How hard is it to switch platforms later? Prompts and function definitions move fairly easily. Anything built in a proprietary visual flow builder generally does not. Keep your conversation logic in a form you could hand to another platform.
Do I need to pick one at all, or can I use several? Some teams run different platforms for different call types. This is workable but doubles your maintenance and observability burden. For a first deployment, pick one.
Is a self assembled stack ever the right answer? It can be, if you have specific latency or compliance requirements no platform meets, and you have the engineering capacity to maintain it. For most businesses the orchestration layer is not where you want to spend your engineering time.
