Hugging Face launches Open TTS Leaderboard for open-source, multilingual text-to-speech evaluation

Hugging Face launched the Open TTS Leaderboard to address the fragmentation and lack of standardization in evaluating open-source, multilingual text-to-speech (TTS) models. As of September 30, 2026, the Hugging Face Hub hosts more than 8,000 TTS models, but evaluation has not kept pace, remaining fragmented and unstandardized.

The new leaderboard uses objective metrics to rank models across three dimensions: intelligibility (word/character error rate via Qwen3 ASR), speed (inverse real-time factor for batched offline inference and time-to-first-audio for streaming latency on H200 GPU and CPU), and speaker similarity (cosine similarity of WavLM speaker embeddings between generated and reference audio).

Models are ranked by macro-average WER on the English splits of Seed TTS Eval and CV3 Eval (zero-shot), with multilingual comparisons available by toggling languages. Chinese, Japanese, and Korean use character error rate (CER), and the average across languages is a macro-average. Voice cloning evaluations add a speaker similarity (SIM) column and additional Pareto plots for SIM versus batched inference and model size.

The leaderboard includes a "Listen" tab where users can compare generated outputs behind the metrics, provide feedback, and vote on preferences. A "Streaming" tab compares time-to-first-audio (TTFA) for streaming and non-streaming models on H200 GPU and CPU, using the same 50 English prompts from CV3-Eval and reporting median TTFA after warm-up runs.

The initiative aims to be shaped by the community, with plans to open-source evaluation scripts and solicit feedback via GitHub Issues and PRs, mirroring the approach of the Open ASR Leaderboard. The goal is to keep pace with the rapid release of open-source TTS models and to highlight models that have been neglected by arena-style evaluations due to hosting and API barriers.