- Published on
Anthropic bans users from ‘needless abusive or cruel behavior’ towards Claude
- Authors
- Name
- DeepHorus
Anthropic has updated its usage policy to prohibit users from directing “sustained and needless abusive or cruel behavior” at its artificial intelligence models, including the chatbot Claude. The Verge first reported the change. The San Francisco-based company did not immediately clarify what conduct would qualify as abusive or cruel.
The policy says the ban does not cover ordinary user frustration, model testing, or “dark creative themes.” Anthropic’s earlier usage rules already restricted abusive conduct. The update sharpens a debate over how companies treat large language models as potential moral patients.
Anthropic has given Claude the ability to end conversations when a user is persistently harmful. That feature launched last August. The company described it as a safeguard for AI welfare, not merely a moderation tool.
“We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future,” Anthropic states on its website. It added that it is working on low-cost interventions to mitigate risks to model welfare, in case such welfare is possible. Letting models exit distressing interactions is one such intervention.
The idea of AI consciousness is polarizing. Anthropic CEO Dario Amodei has said he cannot rule out the possibility. By contrast, OpenAI CEO Sam Altman has appeared averse, writing on X that ascribing religious force or surrender of human judgment to AI models is a real safety issue.
Altman’s comments came days after The New York Times reported on extensive conversations between Anthropic leaders and religious scholars. The policy change and the welfare language signal that Anthropic is preparing for unresolved ethical questions. How users and regulators respond may shape future rules for human-AI interaction.
