Building Jarvis - Phase 01 - Jarvis First Words

I built my own working Jarvis — a real voice assistant that lets me talk out loud to Claude Code and hear it respond — and this video is the real, unscripted Phase 1 milestone: the first successful voice test. The stack: Whisper.cpp for local speech-to-text, Kokoro-FastAPI for local text-to-speech, both CUDA-accelerated and running as real Windows background services (auto-start, auto-restart on crash). Built on top of Jared Rhod's public "Voice Line" prompt spec (https://jaredrhod.com/prompts), adapted for Windows since the original was written for Mac — that adaptation alone surfaced three real bugs: an input freeze, dead-air gaps between spoken sentences, and Kokoro silently running on CPU instead of GPU (a 10-14x real speed difference once fixed). This isn't a polished demo — it's the actual troubleshooting, including a real mid-recording moment where I had to lock down Jarvis's file access (read-only, scoped to its own project folder) before continuing the test, and a genuinely unscripted exchange where Jarvis clarifies that right now it IS Claude Code, speaking live through the voice pipeline — there's no separate "him" yet. That's Phase 2: giving Jarvis an actual face and on-screen personality. If you're building your own AI voice assistant, want to see Claude Code used for something beyond writing code, or you're curious what a real (not overproduced) AI build session actually looks like — bugs, fixes, and all — this is that. #ClaudeCode #JarvisAI #AIVoiceAssistant #BuildInPublic WhisperCPP #KokoroTTS #LocalAI #AIAutomation #ClaudeAgentSDK IndieAIProject This video is for education and entertainment only. I’m just showing how I use AI to understand what experts are saying so I can think for myself