Skip to content
Tools · Oct 30, 2026

NVIDIA Nemotron Voice Agent powers open-source Halloween installation with speech, LLM, and 3D animation

A NVIDIA engineer used the Nemotron Voice Agent, Nemotron 3.5 Bolt E4B, Magpie TTS Zeroshot, and Audio2Face-3D to build an interactive, grumpy pirate-themed haunted house projection on a single workstation.

Trust79
HypeLow hype

1 source · cross-referenced

ShareXLinkedInEmail
TL;DR
  • An NVIDIA engineer built an interactive Halloween projection named MonsterHouse using the open Nemotron Voice Agent stack.
  • The installation runs on a single workstation with an NVIDIA RTX PRO 6000 Blackwell Max-Q and an NVIDIA RTX PRO 4000 Blackwell.
  • Components include Nemotron 3 ASR for speech-to-text, Nemotron 3.5 Bolt E4B 4GB for dialogue, Magpie TTS Zeroshot for voice synthesis, and Audio2Face-3D for 3D facial animation.

An NVIDIA engineer built an interactive Halloween installation called MonsterHouse using the NVIDIA Nemotron Voice Agent and related open models. The project runs on a single workstation equipped with an NVIDIA RTX PRO 6000 Blackwell Max-Q and an NVIDIA RTX PRO 4000 Blackwell.

The installation listens to visitors via NVIDIA Nemotron 3 ASR, which converts speech to text. It then generates responses using NVIDIA Nemotron 3.5 Bolt E4B 4GB, a small open language model purpose-built for conversational agents. A LoRA adapter supplies the grumpy pirate personality. Responses are voiced in real time with NVIDIA Magpie TTS Zeroshot, and a 3D wooden mouth is animated using NVIDIA Audio2Face-3D within Unreal Engine 5.

The engineer also used NVIDIA Nemotron 3 Nano Omni to describe costumes and YOLO with ByteTrack to count and track visitors. All components are coordinated by the Nemotron Voice Agent running locally on the workstation.

The project’s character design features a garage door with jagged wooden lips and fiery animations, controlled by a custom rig exported from Autodesk Maya into Unreal Engine. The backstory frames the house as a reincarnated pirate ship haunted by its past, which informed the conversational flow and grumpy responses.

The engineer tested more than a dozen models before selecting Bolt E4B due to its smaller footprint and suitability for local, offline conversational agents. The 4-bit quantized release was used as the default response model, with a BF16 checkpoint available for comparison.

To shape the agent’s behavior, the engineer created a dataset of over 5,000 multi-turn conversations and trained LoRA adapters for each candidate model. Training times varied from about an hour to a couple of hours per model, with a local agent and a large LLM used to generate and judge the dataset.

Sources
  1. 01Hugging Face — nemotron-halloween
Also on Tools

Stories may contain errors. Dispatch is assembled with AI assistance and curated by human editors; despite the trust-score filter, mistakes happen. We correct publicly — every article links to its revision history. Nothing here is financial, legal, or medical advice. Verify before relying on any claim.

© 2026 Dispatch. No ads. No sponsorships. No paid placement. Reader-supported via Ko-fi.

Built by a person who cares about honest AI news.