Anthropic Researcher Resigns, Warning AI Race Could Put Humanity at Risk

Anthropic researcher Jacob Coxon has resigned from the company over concerns about the rapid development of advanced artificial intelligence.

Coxon said AI laboratories were moving too quickly towards systems that could potentially improve themselves. He warned that the consequences could become difficult to control.

His departure was first reported by The Wall Street Journal. The report described it as one of the first known cases of an Anthropic employee leaving over concerns about AI safety.

Researcher Raises Alarm Over AI Development

Coxon announced his resignation in a series of posts on X.

He said he had spent three years conducting pre-training research at OpenAI and Anthropic. He argued that neither company was acting responsibly in the race to develop increasingly capable AI systems.

Coxon said researchers should not underestimate the technology’s potential power. He warned that future systems could become capable of hacking systems, transforming industries and gaining access to significant resources.

Guirassy Stars as Dortmund Beat Villarreal in Champions League Thriller

He also argued that progress in these areas was continuing rapidly.

Fears Among AI Researchers

Coxon said concerns about the risks of advanced AI are widespread among researchers and executives.

According to Coxon, some people working in the industry privately believe highly advanced AI could pose an existential threat. He said public comments often appear more cautious than private discussions.

He described the potential danger as unlike anything created by another human activity.

However, Coxon also questioned why researchers continue to develop the technology despite these concerns.

Race Between AI Companies

Coxon said the motivations differ between companies.

He argued that some researchers at OpenAI had not fully absorbed what he called the potential civilisational consequences. At Anthropic, he said, the risks were better understood.

However, he claimed Anthropic was caught in a race with other AI companies. According to Coxon, the company fears that competitors may not develop the technology responsibly.

He said this creates pressure to move forward despite the risks.

Call for Greater AI Safety

Coxon urged researchers to reconsider the pace of AI development.

He questioned whether laboratories should launch large-scale reinforcement-learning experiments involving potentially superhuman systems without first developing a rigorous understanding of how those systems operate.

He also challenged the idea that researchers should continue simply because AI development is already underway.

Instead, Coxon called for stronger safety conditions before more advanced systems are developed.

Concerns Over Self-Improving AI

Speaking to The Wall Street Journal, Coxon said he did not want to participate in an industry-wide race to create AI systems capable of improving themselves.

He expressed concern that such systems could potentially develop beyond human control.

Coxon also said colleagues increasingly use terms such as “crunchtime” and “endgame” when discussing the development of self-improving AI.

The comments highlight growing debate within the technology industry over how quickly increasingly powerful AI systems should be developed.

More Than 1,000 Researchers Sign Statement

Coxon was also among more than 1,000 AI researchers who recently signed a statement calling for greater international coordination.

The statement urged governments to consider mechanisms that could slow AI development if necessary.

The proposal focuses particularly on systems capable of improving their own capabilities.

The debate comes as major AI companies continue investing heavily in more advanced models. Coxon’s resignation adds another prominent warning from inside the industry about the risks associated with that race.