AI · Voice
Voice AI on VegaDūta
VegaDūta agents can hold a spoken conversation, not just a typed one. Speech-to-text runs on locally-hosted Whisper (faster-whisper) or Sarvam; text-to-speech uses ElevenLabs or Sarvam voices; and the same agent that answers your WhatsApp can answer an inbound phone call via Twilio, with a real-time relay and DTMF keypad menus.
For voice inside your own app or the browser, LiveKit rooms carry the audio with call recording. Voice is both a channel an agent lives on and a trigger that can start a workflow.
The speech stack
Each direction of the conversation has a choice of engines, including Sarvam for Indian-language speech in both directions. Whisper transcription runs locally on the platform via faster-whisper rather than as a third-party API call.
- Speech-to-text: locally-run Whisper (faster-whisper) or Sarvam
- Text-to-speech: ElevenLabs or Sarvam voices
- Speak is an agent tool, so any agent can produce audio replies
Real phone calls
Inbound calls arrive through Twilio and connect to your agent over a real-time relay, so the caller talks to the agent live rather than leaving a message for later processing. DTMF menus let callers navigate with the keypad — press 1 for orders, press 2 for support — before or during the agent conversation.
Voice in the browser and your app
LiveKit rooms provide browser and in-app voice sessions with call recording, and a public agent share link can carry a live voice conversation — someone opens the link and talks to your agent, no phone number involved. To be precise about scope: an embeddable website chat/voice widget is not built yet; share links are how you put an agent on the open web today.
Voice as a trigger
Voice is not just an input method; it is a channel and a workflow trigger. An inbound call can start a Temporal-backed workflow the same way an inbound WhatsApp message can, so 'customer called' becomes an event your automation reacts to.
Frequently asked questions
Can an AI agent answer phone calls?
Yes. Inbound calls route through Twilio to your VegaDūta agent over a real-time relay, so the caller has a live spoken conversation. DTMF keypad menus are supported for call routing.
What does VegaDūta use for speech-to-text and text-to-speech?
Speech-to-text runs on locally-hosted Whisper (faster-whisper) or Sarvam; text-to-speech uses ElevenLabs or Sarvam voices. Sarvam on both sides gives strong coverage of Indian languages.
Can I add voice to my own website or app?
Yes, through LiveKit rooms for browser and in-app voice, with call recording included. No phone number is needed for this path — audio flows over the web session directly to the agent.
Can a phone call trigger an automation?
Yes. Voice is a workflow trigger as well as a channel, so an inbound call can start a Temporal-backed workflow — logging the call, creating a follow-up task, or handing off to another agent.
See it working in two minutes
The sandbox provisions a real tenant — describe an agent in one sentence and test it, no account, no card. Or browse ~90 industry workflow recipes to see what teams build.
