Voice AI agent stack · L3 · 25 min
A hosted voice agent, from the xAI docs
Learn in five minutes
- The Speech to Speech page says you connect with a WebSocket to wss://api.x.ai/v1/realtime and pass model as a query parameter.
- That page says grok-voice-latest is an alias for grok-voice-3.0, and that grok-voice-2.0 is deprecated.
- Session fields on that page include instructions, voice, tools, and turn_detection. The tools it lists are file_search, web_search, x_search, mcp, and function.
- Authentication on that page is an ephemeral token for a client-side app, or an API key in the Authorization header for the server side only.
- The Voice overview says all audio data is processed in real time and never stored or used for training. That is the page's statement. This lab sends no audio.
Quote the page you opened. The quick-start sample uses the voice name eve and server_vad. Those are sample fields, not a contact-center standard. This page states no price.
Visual
A pin line, and nothing sent
- Write the model query value you would pin: grok-voice-3.0, because the page says the alias points there.
- Write the sample voice and the sample turn detection beside the date you read the page.
- Write 'ephemeral token on the client, API key only on the server.'
- Do not paste a key. Do not open the socket.
Quiz
Ecosystem
- Ephemeral token
- The Speech to Speech page's client-side credential, so the API key stays off the browser.
Sisters: Speech-to-text, Language model, Text-to-speech, WebRTC, SIP.
A cascaded pipeline and a speech-to-speech model are two shapes the LiveKit docs name. The xAI Speech to Speech API is a hosted speech-to-speech session.
Pin the model id, the voice, and the date you read the docs.
Config
- Write the goal in one sentence.
- Name the input and the allowed output.
- Save the trace next to the result.
Troubleshooting
- The answer looks fluent and is wrong. Check whether a source passage actually contains the claim.
- The loop never stops. Set a step cap and stop when the tool result repeats.
Sources and go deeper
- Speech to Speech, docs.x.ai vendor The page title is Speech to Speech. The body names the Grok Speech to Speech API and also calls the site SpaceXSI Docs. last checked 2026-10
- Voice Overview, docs.x.ai vendor The page calls them SpaceXSI Voice APIs, powered by Grok, and separates Speech to Speech, Text to Speech, and Speech to Text. last checked 2026-10