AI Chatbot Crime Confessions Raise Questions Over Privacy and Legal Responsibility
AI chatbot privacy and crime reporting raise legal and ethical questions for technology companies.
Artificial intelligence (AI) chatbots are increasingly becoming part of everyday life. However, cases involving alleged crime confessions and suspicious conversations are raising difficult questions about privacy, public safety and legal responsibility.
Technology companies face growing pressure to clarify when they should share chatbot conversations with law enforcement. At the same time, unclear legal frameworks and the risk of false alerts complicate efforts to introduce mandatory reporting requirements.
ChatGPT Conversation Used as Evidence in Missouri Case
In August, a 19-year-old University of Missouri student reportedly told ChatGPT that he had damaged 17 parked cars on campus late at night.
He also asked the chatbot, “Will I go to jail?”
According to police documents, ChatGPT initially provided information about the possible consequences of getting caught. However, it urged the student to call 911 after his messages became increasingly aggressive.
Asian Chipmakers Defend AI Growth Outlook Amid Investor Concerns Over Returns
Investigators later examined the student’s phone with his permission. They discovered the chat history and used it as evidence in court proceedings that led to felony property-damage charges.
The case highlighted how conversations with AI platforms can become part of criminal investigations.
Lawsuit Raises Questions About Reporting Suspected Threats
Another case emerged in Canada, where the government of British Columbia filed a lawsuit against OpenAI in February.
The lawsuit concerned a mass shooting at a school that killed nine people, mostly children, and injured 27 others.
The provincial government alleged that OpenAI failed to report the shooter’s conversations with ChatGPT to law enforcement. It argued that an earlier warning could have helped prevent the attack.
OpenAI CEO Sam Altman apologised in a letter following the incident.
However, The Wall Street Journal reported that the shooter’s conversations had allegedly been flagged by the company’s safety team in June 2025. Whistleblowers claimed that senior leadership overruled recommendations to contact police.
These allegations have intensified scrutiny of how AI companies assess threats and decide when to alert authorities.
Privacy Concerns Complicate AI Reporting Rules
The debate also raises questions about whether users can expect confidentiality when discussing sensitive matters with AI chatbots.
Unlike communications protected by specific legal privileges, such as certain exchanges between lawyers and clients, chatbot conversations generally do not carry the same legal protection.
Companies may disclose user information in response to valid legal requests or under applicable legal requirements. Their privacy policies and the laws governing disclosure determine how they handle such information.
The issue becomes more complicated when users discuss personal matters, including health, relationships and psychological distress.
A study in Australia also found that children may feel more comfortable reporting sexual abuse to chatbots than to responsible adults. The findings raise questions about how AI platforms should respond to vulnerable users and sensitive disclosures.
Experts Warn Against Using AI for Crime Prediction
Matthew Trail, an American legal scholar and researcher at Goethe University in Germany, told Anadolu that AI companies follow different policies when handling suspicious conversations.
He said chatbots are not designed to answer requests about planning murders or similar violent acts. Companies also use safety systems and human moderators to review certain conversations.
However, users can sometimes bypass these safeguards by disguising inappropriate requests as fictional writing or other seemingly harmless activities.
Trail said law enforcement agencies could seek chat histories from technology companies when investigating suspected crimes.
However, he cautioned against turning chatbots into tools for predicting criminal behaviour before offences occur.
He argued that laws and policies surrounding such practices remain unclear. He also questioned whether companies should collect additional personal data to improve predictions about a user’s potential to commit a crime.
Can Chatbot Conversations Ever Be Confidential?
OpenAI CEO Sam Altman said in July 2025 that conversations with AI chatbots should be based on confidentiality.
However, legal experts continue to debate what privacy protections should apply to these exchanges.
Robert Diab, a law professor at Thompson Rivers University in Canada, said communications between alleged accomplices do not generally receive the same confidentiality protections as privileged professional relationships.
He suggested that chatbot conversations could be assessed under similar legal principles.
Diab also argued that AI models may identify signs of potential criminal behaviour and that developers could help prevent violence by notifying law enforcement.
However, he warned that reporting every suspicious message could overwhelm human review teams with false alarms.
He added that people seeking to misuse AI could continue finding ways around security measures.
Balancing Public Safety and User Privacy
The growing use of AI chatbots has created difficult questions for technology companies, regulators and law enforcement agencies.
Mandatory reporting could help authorities respond to credible threats. However, inaccurate interpretations of conversations could lead to unnecessary investigations and privacy concerns.
Experts continue to debate how companies should distinguish genuine threats from fictional scenarios, jokes or misunderstood messages.
Clear legal standards, transparent company policies and effective human oversight will remain central to determining how AI platforms handle potentially criminal conversations.
