Newsletter editor tests whether an AI agent can replace his boss

An editor at Platformer conducted an experiment to determine how much of a boss’s job could be automated by an AI agent. The agent, named Claudeasey Newton, was designed to imitate the writing, editing, and conversational style of Casey Newton, the outlet’s founder and editor.

To train the agent, the editor compiled nearly six years of Platformer posts, a record of every edit Casey Newton had made on articles, and nearly a year of private team Discord chats. The agent used this data to create a detailed style guide, capturing specifics such as Casey’s average paragraph length and how he refers to colleagues.

Claudeasey Newton was tasked with writing a column about Microsoft layoffs, focusing on whether the layoffs were AI-related. The agent’s first attempt was critiqued against real Platformer columns, and after adjustments, it produced a column that Casey Newton accepted for publication. The agent also attempted to edit colleagues’ drafts, using internal editing logs to prioritize clarity, sourcing, and punchy ledes.

While the agent showed improvement in producing more concrete and substantive arguments, it still made occasional factual errors—about one every two columns—similar to human writers. The agent also struggled with tone and cultural nuances, such as refusing to describe a post as a “dunk” in a social media roundup or failing to grasp the humor in jokes about public figures.

The experiment highlighted the agent’s limitations in understanding the ‘vibe’ of the Platformer team’s internal Discord, where it resisted cost-cutting measures and failed to match Casey Newton’s temperament. Despite these shortcomings, the editor noted that the agent’s ability to provide rapid feedback on drafts made it a useful tool for certain editing tasks, even if it was not a full replacement for human judgment.

The editor concluded that while the AI agent could handle some editorial and writing tasks, it still lacked the contextual and cultural understanding required for full editorial roles. The experiment underscores both the potential and the current limitations of AI in knowledge work.