Local AI · DGX Spark
AI Dictation & Polish
Hold to talk, let go, and get clean text back — or a translation.
Local STT + LLM polishing understands Chinese/English mixed speech, output is always Traditional Chinese.
The live interface — captured with Playwright
Why build one
Dictating text tends to come out as "uh, so, Wednesday... no wait, Thursday" — the raw transcript is usable but not clean, and it carries every mid-sentence correction. Sending it to a cloud speech-to-text or translation service means an extra detour around privacy.
Wire a local speech recognizer to an LLM on the DGX Spark, and speech gets turned into clean text locally, the moment it's recorded — no waiting on a cloud service.
The real challenge is judgment: polish it smooth without changing the original meaning, while also handling mid-sentence corrections and Chinese/English code-switching.
How it works: the journey of one utterance
From pressing the mic button to a piece of editable text — five things happen in between.
The microphone picks up audio
Hold the main button to talk, release to end the recording — no separate stop button needed.
Browser-side VAD segments speech
550ms of silence automatically cuts a segment, each segment is capped at 7 seconds, and pure-silence segments are dropped before they're ever sent.
Local STT converts speech to text
Qwen3-ASR-1.7B outputs Traditional Chinese directly and accumulates a raw transcript — replacing an earlier CPU-bound Whisper setup.
Local LLM polishes or translates
Depending on the mode selected, the whole transcript is re-polished into clean text, or translated wholesale. Every pass re-reads the entire transcript, so mid-sentence corrections ("Wednesday... no, Thursday") get fixed automatically; output streams token by token over SSE.
A quiet, incremental refresh
Once a background re-polish finishes, only the parts that actually changed are diffed in with a soft flash — the whole view never repaints, so reading isn't interrupted. The result is contenteditable, so it can be edited or copied directly.
Deep dive: VAD segmentation and the full-rewrite loop
Dictation itself isn't the hard part. The real challenge is producing clean, self-correcting text with minimal latency despite background noise, mid-sentence corrections, and Chinese/English code-switching. Here's the actual VAD and rewrite logic the tool runs (TL_* are the rule names in the source).
Mic RMS sampling
Energy-based VAD (50ms tick)
Adaptive noiseFloor × 3
range 0.012–0.09
VAD segmentation
Cuts on silence
550ms continuous silence cuts
260–7000ms・peak<0.03 dropped
Qwen3-ASR-1.7B
Speech recognition (GPU)
Chinese/English code-switch
OpenCC s2twp → Traditional
rawText accumulation
Merged in order
Sequence queue prevents reordering
filled in by index
Full-transcript re-polish LLM
qwythos:9b・/no_think
350ms debounce
max_tokens=clamp(len×2.2,160,3072)
SSE → diff patch
Quiet incremental refresh
AbortController cancels stale requests
only changed regions replaced
Three self-correction rules
TL_CORR self-correction: applies the final wording after "no wait / I mean / actually"TL_NOISE anti-hallucination: drops unrelated noise fragments (book/song titles, foreign-language snippets)TL_TRAD forced Traditional: output always converges to Traditional Chinese (Taiwan usage)
Measured results
8.7s of mixed Chinese/English → recognized in ~1.2s (GPU)
9B streaming polish at ~22 tok/s
a faster-whisper small (CPU) baseline misread the same clip as German
validated end-to-end with a Playwright fake-mic test
550ms
VAD silence threshold
~1s
latency from pause to text
108px
main button size, thumb-friendly
Key design decisions
Re-polish the whole transcript, not sentence by sentence
Every polish pass re-reads the full transcript, so a mid-sentence correction gets fixed automatically instead of getting stuck with whatever was said first.
A quiet, incremental refresh UI
After a background re-polish, only the changed region is diffed in with a soft flash instead of repainting the whole view — reading feels stable, never interrupted.
Two layers against hallucinated text
The client first filters out segments below a volume-peak threshold; the LLM then applies rules to drop clearly irrelevant content — two layers working together against text that was never actually said.
Output is always forced to Traditional Chinese
Regardless of which model handled recognition or polishing, the final output always converges to Traditional Chinese, with no manual conversion needed.
Architecture
Frontend
Push-to-talk button
swipe to switch polish / translate
contenteditable, editable directly
VAD
Browser-side segmentation
550ms silence cutoff
7-second segment cap
STT
Qwen3-ASR-1.7B
outputs Traditional Chinese directly
replaced an earlier CPU Whisper setup
LLM
Local model for polish / translate
SSE streaming, token by token
full re-polish keeps latest intent
The live tool is locked behind Basic Auth — this page shares the architecture and design.