Notion Details Five Rebuilds of Custom Agents, Architecture for Enterprise AI Workflows

Notion has shipped Custom Agents as a production feature after multiple complete rebuilds of the underlying system. In a podcast interview, Notion's cofounder and head of AI discussed why early attempts failed: no standardized tool-calling protocols existed, context windows were too short, model outputs were unreliable, and the product exposed too much complexity directly to the model layer. Each major revision addressed a different constraint rather than incremental fixes to a single architecture.

The final design borrows from what Notion calls the 'Agent Lab' thesis—recognizing that successful agentic systems require understanding how teams collaborate, not merely wrapping a frontier model with tool access. The product implements agents that compose through shared databases as memory primitives, with 'manager agents' orchestrating dozens of specialized agents for tasks like email triage, data enrichment via web search, and structured database writes. Agents can also configure themselves, inspect their own failures, and edit instructions within permission guardrails.

Notion employs a dedicated role, Model Behavior Engineer, distinct from traditional software engineers. These roles focus on eval writing, failure analysis, and understanding model capabilities. The company uses three types of evals: regression tests to catch regressions, launch-quality evals for gate decisions, and 'frontier/headroom' evals designed to pass only ~30% of the time—intentionally revealing where model capabilities are still developing rather than confirming existing performance.

Pricing for Custom Agents uses credits as an abstraction over tokens, model type, serving tier, and web search costs, with future charges anticipated for sandbox execution. Rather than pure token-level usage-based pricing, Notion includes auto-selection logic that matches the appropriate model to the task at hand. The company decided against training its own foundation model, instead focusing on retrieval and ranking optimization as agent-driven queries increasingly replace human search across meeting transcripts and collaboration data.

The product architecture distinguishes between CLI-based tool integration and Model Context Protocol (MCP). Notion's leaders noted that CLIs offer better self-debugging behavior and determinism, while MCP remains useful for specific capability and permissioning scenarios. Internal tool definitions evolved from JavaScript and custom XML to Markdown and SQL-like abstractions, with progressive disclosure limiting what agents can access until explicitly needed, and system prompts kept deliberately short.