A 13-year-old Florida student has been arrested after reportedly using OpenAI’s ChatGPT to ask, “How to kill my friend,” during class on a school-issued device. The message, typed on a computer at Southwestern Middle School, was automatically flagged by Gaggle, a monitoring system that tracks student activity on school technology.
According to WFLA, the alert was forwarded to a campus police officer as part of standard school security protocol. The officer quickly located and detained the student. During questioning, the boy allegedly told authorities that he was “just trolling” his friend and did not intend any real harm.
Law enforcement, however, did not view the situation as a harmless prank. The Volusia County Sheriff’s Office confirmed that the teen was arrested and booked into a juvenile detention facility. Officials emphasized that, given the country’s tragic history of school violence, any perceived threat must be taken seriously.
Following the incident, the sheriff’s office issued a warning to parents, urging them to talk to their children about responsible use of artificial intelligence and online tools. “Another ‘joke’ that created an emergency on campus. Parents, please talk to your kids so they don’t make the same mistake,” the department said in a public statement.
This case comes amid growing concern over minors’ use of AI systems like ChatGPT, which can generate content in response to user prompts. Experts warn that even if students see such interactions as jokes or experiments, automated monitoring systems—and law enforcement—may treat them as potential threats, The New York Times reported.
The incident also renews scrutiny on Gaggle, the digital safety platform used in thousands of U.S. schools. The software scans students’ online activity for signs of violence, self-harm, or other risky behavior and alerts officials when it detects troubling language. Critics argue that such systems create a surveillance-like environment, while supporters say they’ve helped prevent tragedies.
Gaggle has faced backlash in the past for false alarms and privacy concerns. According to the Associated Press, many American schools rely on similar monitoring tools that track nearly everything students type on school accounts and devices.
As schools grapple with balancing safety and privacy, authorities continue to stress that digital “jokes” involving violence can have serious, life-changing consequences—even for children.