Wayfinder Router offers offline, deterministic routing between local and hosted LLMs

Wayfinder Router is an open-source CLI tool that routes prompts to either a local or a hosted LLM tier without calling another model to make the routing decision. The tool analyzes prompt structure—such as length, headings, lists, and code—and wording—including terms like proofs, math, or hard constraints—to produce a deterministic routing recommendation in microseconds. Because the decision is made offline and does not rely on an external model call, it avoids adding latency, API costs, or network dependencies to the routing step.

The project emphasizes calibration on user-specific data. By default, Wayfinder scores only prompt structure, but it also supports lexical cues (e.g., proofs, math, constraints) as optional features. However, these lexical cues are disabled by default because a double-blind test on independently authored prompts showed they only catch about 20% of unseen hard prompts and underperform a simple word-count baseline. Users are advised to enable and weight these cues only after calibrating them on their own traffic.

Wayfinder provides a benchmark suite ( run via make benchmark ) that compares its routing decisions against honest baselines and a perfect oracle. The tool can be evaluated against RouterBench or RouterArena for graded performance numbers. The project documents where it performs well and where it does not, including cases where it performs no better than random on short-but-hard items from RouterBench.

Two interfaces are available for testing the router without API keys or network access. A terminal-based chat interface ( wayfinder-router chat --dry-run ) shows routing decisions, structural scores, and running savings versus an always-cloud baseline. A browser-based web chat UI ( wayfinder-router webchat --dry-run ) includes a live threshold slider to adjust routing behavior. Both interfaces persist conversations across sessions and display the rationale for each routing decision.

The project contrasts its approach with other routers that rely on model calls—such as trained classifiers, LLM judges, or hosted APIs—to make routing decisions. These model-based routers add latency, cost, and randomness to the step intended to save money. Wayfinder’s offline, deterministic routing aims to avoid these pitfalls while still enabling users to route prompts to the most cost-effective model tier.