NVIDIA Nemotron Voice Agent powers open-source Halloween installation with speech, LLM, and 3D animation
A NVIDIA engineer used the Nemotron Voice Agent, Nemotron 3.5 Bolt E4B, Magpie TTS Zeroshot, and Audio2Face-3D to build an interactive, grumpy pirate-themed haunted house projection on a single workstation.
1 source · cross-referenced
- An NVIDIA engineer built an interactive Halloween projection named MonsterHouse using the open Nemotron Voice Agent stack.
- The installation runs on a single workstation with an NVIDIA RTX PRO 6000 Blackwell Max-Q and an NVIDIA RTX PRO 4000 Blackwell.
- Components include Nemotron 3 ASR for speech-to-text, Nemotron 3.5 Bolt E4B 4GB for dialogue, Magpie TTS Zeroshot for voice synthesis, and Audio2Face-3D for 3D facial animation.
An NVIDIA engineer built an interactive Halloween installation called MonsterHouse using the NVIDIA Nemotron Voice Agent and related open models. The project runs on a single workstation equipped with an NVIDIA RTX PRO 6000 Blackwell Max-Q and an NVIDIA RTX PRO 4000 Blackwell.
The installation listens to visitors via NVIDIA Nemotron 3 ASR, which converts speech to text. It then generates responses using NVIDIA Nemotron 3.5 Bolt E4B 4GB, a small open language model purpose-built for conversational agents. A LoRA adapter supplies the grumpy pirate personality. Responses are voiced in real time with NVIDIA Magpie TTS Zeroshot, and a 3D wooden mouth is animated using NVIDIA Audio2Face-3D within Unreal Engine 5.
The engineer also used NVIDIA Nemotron 3 Nano Omni to describe costumes and YOLO with ByteTrack to count and track visitors. All components are coordinated by the Nemotron Voice Agent running locally on the workstation.
The project’s character design features a garage door with jagged wooden lips and fiery animations, controlled by a custom rig exported from Autodesk Maya into Unreal Engine. The backstory frames the house as a reincarnated pirate ship haunted by its past, which informed the conversational flow and grumpy responses.
The engineer tested more than a dozen models before selecting Bolt E4B due to its smaller footprint and suitability for local, offline conversational agents. The 4-bit quantized release was used as the default response model, with a BF16 checkpoint available for comparison.
To shape the agent’s behavior, the engineer created a dataset of over 5,000 multi-turn conversations and trained LoRA adapters for each candidate model. Training times varied from about an hour to a couple of hours per model, with a local agent and a large LLM used to generate and judge the dataset.
- Nov 11, 2026 · Hugging Face
Hugging Face proposes token pooling-aware training to improve compressed multi-vector retrieval
Trust79 - Nov 11, 2026 · TechCrunch — AI
Fractile raises $720M Series C to develop AI chips and compiler stack
Trust79 - Nov 11, 2026 · The Verge — AI
Google’s Tabletop AI experiment lets users build custom collaborative apps via waitlist
Trust79