Anthropic updates usage policy to ban abusive behavior toward Claude and restrict high-risk misuse

Anthropic updated its usage policy for the first time in over a year to address new and high-risk cases of misuse, including election interference, weapons development, surveillance, and health and financial uses. The company now prohibits 'sustained and needless abusive or cruel behavior' toward its Claude AI assistant, framing the rule as a model welfare measure. Anthropic stated the policy applies only in extreme cases where users repeatedly act cruelly toward its models with no discernible purpose, and it does not cover common user frustration, creative themes, or research testing.

The update consolidates scattered restrictions into clearer bans on deceptive commercial or political campaigns, such as efforts to obscure message origins or amplify content through fake accounts. It also explicitly prohibits voter deception, election disruption, and impersonation of candidates or officials. Anthropic said it has seen attempts to use Claude for weapons development, prompting an expanded ban that includes software and components that make weapons work, as well as actions like arming drones.

Anthropic’s threat intelligence report previously highlighted the use of Claude-powered surveillance to track political dissidents, leading the company to clarify its surveillance rules. The updated policy prohibits tracking people without consent, whether in real time or through analysis of previously collected data. It also bans using Claude to decide or recommend targets for law enforcement or criminal justice processes, or to build tools designed for surveillance. Exceptions may apply for contracts with certain governmental customers if Anthropic determines the contractual restrictions and safeguards are adequate.

The policy maintains that terminating conversations is the primary enforcement mechanism, and Anthropic did not indicate whether additional penalties, such as user bans, would be introduced. The update also addresses scenarios where AI models are connected to hardware capable of autonomous physical actions that might cause injury, requiring a qualified operator to observe the equipment and stop it if needed. Anthropic did not specify whether the operator must be human.

Anthropic’s focus on model welfare aligns with its ongoing research into whether its models could exhibit forms of consciousness. In February, Kyle Fish, who leads the company’s model welfare research, told The Verge that questions about potential internal experience and moral status are serious and being investigated as models become more sophisticated. This stance contrasts with Microsoft AI’s earlier draft code of conduct, which opposed the idea of model welfare and legal personhood, though the company later softened its position to reject only legal personhood and the pursuit of rights for models.