Pydantic AI vs LiveKit Agents
LiveKit Agents is a voice-agent framework built on LiveKit’s WebRTC transport: rooms, SIP telephony, turn detection, noise cancellation, mid-call handoffs, and plugins for STT, LLM and TTS vendors, with a cascaded pipeline as the default. Pydantic AI’s realtime support is a speech-to-speech agent loop on four providers behind one API, and it is the same typed Agent that runs as text, in a web chat or behind your API: the call uses the same tools, dependencies and capabilities, becomes ordinary message history you can hand to a text agent for structured output, and is traced end to end in Logfire. You bring the transport; with LiveKit, the transport is the product.
Pydantic AI is one part of a stack: the Harness SDK for capabilities and complete agents, Pydantic Evals for evaluation, Pydantic Graph for graph-based workflows, Pydantic Logfire for observability, and Pydantic itself for validation. The tables below cover the whole of it.
| LiveKit Agents | Pydantic AI and Harness SDK | |
|---|---|---|
| Language | Python (also Node) | Python |
| License | Apache-2.0 | MIT |
| Model providers | Many (plugins) | Many |
| Extensibility | Pipeline nodes (stt_node, llm_node, …) | Capabilities and toolsets; 50+ with the Harness SDK |
| Harnesses | Build your own | Built-in Coder and Researcher, or compose your own |
| Observability | OpenTelemetry | OpenTelemetry, including Pydantic Logfire |
| Durable execution | No | Seven integrations |
| Interfaces | WebRTC rooms, telephony, text sessions | CLI, web chat, AG-UI, Vercel AI, ACP (experimental), GitHub Agentic Workflows |
| Realtime voice | Speech-to-speech and cascaded STT + LLM + TTS | Speech-to-speech, four providers |
| Evals | Yes | Pydantic Evals |
| Image generation | No | Image Generation |
Our realtime support means speech-to-speech models: one persistent connection, audio in and audio out, on the four providers below. LiveKit also runs the cascaded pipeline, which we do not. If you are choosing a voice stack, these are the rows that decide it:
| LiveKit Agents | Pydantic AI | |
|---|---|---|
| Speech-to-speech providers | Plugins for several providers | Four behind one API: OpenAI, Azure OpenAI, Gemini Live, xAI; ElevenLabs in #7964 |
| Cascaded STT + LLM + TTS | Yes, the default; dozens of STT and TTS plugins | Not built in; compose it yourself around a text agent |
| Audio transport | WebRTC rooms via LiveKit server or Cloud | Yours: browser WebRTC sideband or WebSocket relay |
| Telephony | SIP in and out, DTMF, transfers; numbers on Cloud | Bridge a provider such as Twilio |
| Turn detection | Silero VAD, own turn-detector model, adaptive interruption | Provider turn detection, barge-in, push-to-talk |
| Noise cancellation | Krisp and ai-coustics plugins; enhanced models on Cloud | Provider-side only |
| Hand off to another agent mid-call | Yes, context carried over | No; delegate from a tool instead |
| Tools mid-call | @function_tool, MCP | The same tools, toolsets and dependencies as a text agent |
| Capabilities mid-call | No equivalent | Capabilities and hooks, with documented limits |
| After the call | session.history, SessionReport JSON | Agent.run() on the call’s history for structured output or follow-up |
| Observability | OpenTelemetry; Insights on Cloud | OpenTelemetry: session, turn and tool spans, usage attributed per response |
| Evals | pytest framework with an LLM judge; simulations on Cloud | Pydantic Evals on the text hand-off; nothing realtime-specific yet |
| Deployment | Agent server, dispatch, jobs; Cloud or self-host | Your process, your backend |
| The same agent without voice | Text-only sessions, still a room and a server | run(), CLI, web chat, AG-UI, Vercel AI |