Pydantic AI vs LiveKit Agents
LiveKit Agents is a voice-agent framework built on LiveKit’s WebRTC transport: rooms, SIP telephony, turn detection, noise cancellation, mid-call handoffs, and plugins for STT, LLM and TTS vendors, with a cascaded pipeline as the default. Pydantic AI’s realtime support is a speech-to-speech agent loop on five providers behind one API, and it is the same typed Agent that runs as text, in a web chat or behind your API: the call uses the same tools, dependencies and capabilities, becomes ordinary message history you can hand to a text agent for structured output, and is traced end to end in Logfire. You bring the transport; with LiveKit, the transport is the product.
Pydantic AI is one part of a stack: the Harness SDK for capabilities and complete agents, Pydantic Evals for evaluation, Pydantic Graph for graph-based workflows, Pydantic Logfire for observability, and Pydantic itself for validation. The tables below cover the whole of it.
| LiveKit Agents | Pydantic AI and Harness SDK | |
|---|---|---|
| Language | Python (also Node) | Python |
| License | Apache-2.0 | MIT |
| Model providers | Many (plugins) | Many |
| Extensibility | Pipeline nodes (stt_node, llm_node, …) | Capabilities and toolsets; 90+ with the Harness SDK |
| Harnesses | Build your own | Built-in Coder and Researcher, or compose your own |
| Observability | OpenTelemetry | OpenTelemetry, including Pydantic Logfire |
| Enterprise support | With a LiveKit Cloud plan | For the framework itself, no platform required |
| Durable execution | No | Nine integrations |
| Interfaces | WebRTC rooms, telephony, text sessions | CLI, web chat, AG-UI, Vercel AI, ACP (experimental), GitHub Agentic Workflows |
| Realtime voice | Speech-to-speech and cascaded STT + LLM + TTS | Speech-to-speech, five providers |
| Evals | Yes | Pydantic Evals |
| Image generation | No | Image Generation |
Our realtime support means speech-to-speech models: one persistent connection, audio in and audio out, on the five providers below. LiveKit also runs a cascaded pipeline you compose from plugins; we don’t build one in, though ElevenLabs Agents and Azure AI Voice Live’s cascade models run hosted ones. If you are choosing a voice stack, these are the rows that decide it:
| LiveKit Agents | Pydantic AI | |
|---|---|---|
| Speech-to-speech providers | Plugins for several providers | Five behind one API: OpenAI, Azure OpenAI, Gemini Live, xAI, ElevenLabs |
| Cascaded STT + LLM + TTS | Yes, the default; dozens of STT and TTS plugins | Hosted only, via ElevenLabs Agents or Azure AI Voice Live; otherwise compose it yourself around a text agent |
| Audio transport | WebRTC rooms via LiveKit server or Cloud | Yours: browser WebRTC sideband or WebSocket relay |
| Telephony | SIP in and out, DTMF, transfers; numbers on Cloud | Bridge a provider such as Twilio |
| Turn detection | Silero VAD, own turn-detector model, adaptive interruption | Provider turn detection, barge-in, push-to-talk |
| Noise cancellation | Krisp and ai-coustics plugins; enhanced models on Cloud | Provider-side only |
| Hand off to another agent mid-call | Yes, context carried over | No; delegate from a tool instead |
| Tools mid-call | @function_tool, MCP | The same tools, toolsets and dependencies as a text agent |
| Capabilities mid-call | No equivalent | Capabilities and hooks, with documented limits |
| After the call | session.history, SessionReport JSON | Agent.run() on the call’s history for structured output or follow-up; a LiveKit call’s history converts too |
| Observability | OpenTelemetry; Insights on Cloud | OpenTelemetry: session, turn and tool spans, usage attributed per response |
| Evals | pytest framework with an LLM judge; simulations on Cloud | Pydantic Evals on the text hand-off; nothing realtime-specific yet |
| Deployment | Agent server, dispatch, jobs; Cloud or self-host | Your process, your backend; Hosting the backend covers long-lived sessions, routing, and draining |
| The same agent without voice | Text-only sessions, still a room and a server | run(), CLI, web chat, AG-UI, Vercel AI |
If you keep LiveKit’s cascaded pipeline for the call, you can still use a Pydantic AI agent afterwards for structured output or follow-up. LiveKit records the call as a ChatContext in AgentSession.history, not as Pydantic AI message history, so convert its items to ModelRequests and ModelResponses first:
from livekit.agents import AgentSession, llm
from pydantic import BaseModel
from pydantic_ai import (
Agent,
ModelMessage,
ModelRequest,
ModelResponse,
TextPart,
ToolCallPart,
ToolReturnPart,
UserPromptPart,
)
def to_model_messages(chat_ctx: llm.ChatContext) -> list[ModelMessage]:
messages: list[ModelMessage] = []
for item in chat_ctx.items:
if isinstance(item, llm.ChatMessage):
text = item.text_content # the transcript; images and audio are dropped
if not text or item.role in ('system', 'developer'):
continue # the LiveKit agent's instructions; the text agent has its own
if item.role == 'user':
messages.append(ModelRequest(parts=[UserPromptPart(text)]))
else:
if item.interrupted:
text += ' [Interrupted]'
messages.append(ModelResponse(parts=[TextPart(text)]))
elif isinstance(item, llm.FunctionCall):
part = ToolCallPart(item.name, item.arguments, item.call_id)
messages.append(ModelResponse(parts=[part]))
elif isinstance(item, llm.FunctionCallOutput):
outcome = 'failed' if item.is_error else 'success'
part = ToolReturnPart(item.name, item.output, item.call_id, outcome=outcome)
messages.append(ModelRequest(parts=[part]))
return messages
class CallSummary(BaseModel):
summary: str
follow_up_needed: bool
notetaker = Agent('openai:gpt-6.1-sol', output_type=CallSummary)
async def summarize_call(session: AgentSession) -> CallSummary:
result = await notetaker.run(
'Summarize the call.', message_history=to_model_messages(session.history)
)
return result.output
Pydantic AI merges consecutive requests or responses, such as a spoken reply followed by a tool call, before sending the history to the model. Agent handoffs and configuration updates in the LiveKit history have no Pydantic AI equivalent and are skipped.
Last checked: 2026-10-26