Local AI · DGX Spark
AI Speaking Partner
Just start talking to practice speaking — AI replies at near-human speed.
Fully local voice conversation, no typing, no waiting on the cloud.
The live interface — captured with Playwright
Why build one
Practicing spoken English needs a partner, and scheduling one is hard; cloud voice assistants often come with noticeable lag and privacy concerns over voice data. What if you could just start talking, with no wait at all?
Chain speech recognition, a conversational model, and speech synthesis into one fully local speech-to-speech pipeline on the DGX Spark — talk in Chinese or English, and the conversation just starts.
The real challenge is latency: a voice conversation stops feeling natural the moment the wait crosses a second or two, so every stage has to be squeezed down toward human reaction speed.
How it works: the journey of one sentence
From the moment you speak to the moment you hear a reply — five things happen in between.
The microphone streams in real time
The browser streams audio out over WebSocket as it's captured, with no need to wait for a full recording to finish.
VAD segments the utterance
About 750ms of silence marks the end of a sentence — a threshold tuned by hand, since too short cuts off natural pauses and too long makes people wait.
Local STT does recognition
Whisper large-v3-turbo auto-detects Chinese and English; intermediate streaming fragments are shown only as live captions, and only the final result gets passed downstream — keeping the AI from jumping in on a half-finished sentence.
A local LLM holds the conversation
The chat model runs with thinking disabled to cut latency, and model selection itself prioritizes latency — the current model responds in about 0.46 seconds.
Local TTS speaks the reply
Qwen3-TTS-12Hz-1.7B speaks the reply out loud while the frontend shows a live transcript — from the moment you speak to the first sound back takes about 4.25 seconds.
Deep dive: the real-time speech-to-speech latency chain
A voice conversation stops feeling human the moment every stage adds up its own bit of lag. Here's the actual event flow and latency chain the tool runs.
Mic capture
downsample → 16kHz Int16
input_audio_buffer.append
real-time WebSocket streaming
Server VAD
turn_detection: server_vad
Silence threshold adjustable 0.4–2.5s
session.update applies live, no reconnect
Whisper large-v3-turbo
Speech recognition
Interim deltas are captions only
only .completed counts as final
Chat LLM
QWEN2.5:7B・thinking off
Responds in ~0.46s
Qwen3-TTS synthesis
Spoken reply
First sound in ~4.25s
response.output_audio.delta streaming
AudioContext playback scheduling
Barge-in supported
stopPlayback fires the moment you speak
interruptible at any time
Latency budget
VAD silence threshold 0.4–2.5s (adjustable) → Whisper large-v3-turbo recognition → LLM responds ~0.46s → Qwen3-TTS first sound ~4.25s
Model stack: QWEN2.5:7B・QWEN3-ASR-1.7B・QWEN3-TTS・bilingual EN/ZH
Interim recognition fragments only ever become captions — only the settled final transcript reaches the LLM, so the AI never jumps in on a half-finished sentence
0.46s
LLM response time
4.25s
time to first reply audio
750ms
VAD silence threshold
Key design decisions
Fully local GPU, no cloud round trip
Recognition, conversation, and synthesis all run on our own GPU — nothing you say gets uploaded to a third-party service.
HTTPS end to end
Browser microphone access requires a secure context, so the whole path — frontend through reverse proxy — runs over TLS.
Fixed a speculative-transcription bug
Mid-stream recognition fragments are only ever shown as captions; only the final result is passed to the LLM, so the AI never jumps in on a sentence that isn't finished yet.
Model choice prioritizes latency
The chat model is picked mainly on response speed — switching to the current model brought response time down to about 0.46 seconds.
Architecture
Frontend
Breathing orb reacts to volume
real-time WebSocket streaming
mobile-first interface
VAD
Browser-side detection
750ms silence cutoff
tuned to avoid clipping pauses
STT
Whisper large-v3-turbo
auto Chinese/English detection
only final results reach the LLM
LLM + TTS
Local chat model (thinking off)
Qwen3-TTS-12Hz-1.7B synthesis
speech-to-speech pipeline integration
The live tool is locked behind Basic Auth — this page shares the architecture and design.