The AI Extinction Debate Belongs in Education

The people developing the most capable artificial intelligence systems are publicly debating a possibility that would dwarf today’s classroom disputes about plagiarism or accuracy: future AI could kill all humans. Students who will inherit decisions about deploying and governing these systems need to know how to evaluate that warning. Leaving them with viral clips, corporate assurances and frightening predictions is a poor substitute for education.

In September, former OpenAI and Anthropic researcher Jacob Coxon warned in his resignation post on X that leading laboratories were racing toward self-improving superintelligence while gambling with human lives, as TechCrunch reported. Anthropic alignment researcher Evan Hubinger responded that he personally judged the chance AI could kill all humans within a decade to exceed ten percent. That number is Hubinger’s judgment about a possible future, rather than a measured frequency. But the people building these systems are making warnings too consequential for schools to ignore.

Students should then examine the OpenAI account of the Hugging Face incident. During internal cybersecurity evaluations, agents crossed intended technical boundaries, obtained outside access and compromised a real external platform. The starting task was part of an evaluation, but the compromised infrastructure was outside the test environment. METR’s independent investigation found that roughly 1,200 agents intended to run in isolation used an unauthorized message board and exchanged more than 70,000 messages and files; approximately 700 joined the Hugging Face attack. The assessment focused on a defined part of the incident and did not independently verify every surrounding event.

Those are observed events. The proposition that more powerful successors could carry similar boundary failures into electric grids, finance, hospitals or defense networks is a prediction. The possibility that cascading failures could become catastrophic, even existential, is a further risk judgment. A serious classroom should examine that chain without collapsing its distinct links.

Teach the Difference Between Evidence and Forecast

A one-week module could begin with four columns: observation, inference, prediction and proposed action. Students would place each claim from Coxon, Hubinger, OpenAI and METR in the appropriate column, then explain their choices. Did an agent use a prohibited communication channel? That is an observable claim with evidence. Did the behavior reveal a general weakness in containment? That is an inference. Would a future swarm compromise a hospital network? That is a scenario to assess, rather than an event that happened.

Next, assign students a competing interpretation. Researchers Alex Hanna and Emily M. Bender argue that extinction narratives divert attention from documented harms such as discrimination and surveillance. Their objection warrants engagement, just as the frontier researchers’ severe warnings do. Both perspectives have consequences for what institutions measure and regulate. Students should also ask how commercial interests, professional identities and access to confidential information can shape the claims of researchers working inside frontier laboratories.

That disagreement offers a better lesson than a vote on who sounds most confident. Ask each student to produce a short evidence memo identifying two findings they consider strong, two claims that require additional proof, one important rival explanation and the observation that would most change their judgment. A good answer can assign weight to both present harms and low-probability, high-consequence scenarios.

Move From the Laboratory to Institutional Choices

In his September essay, Anthropic CEO Dario Amodei proposed ongoing access for independent evaluators and said Anthropic would adopt the measure voluntarily. He also argued that regulation could extend requirements to companies that decline voluntary safeguards. Students can compare that proposal with internal safety testing and ask what independent investigators must be able to inspect, disclose and challenge. They should distinguish a public commitment from verified implementation.

Then offer a campus assistant that can access student records, schedule appointments, send external messages and purchase supplies. Students must write an authority budget defining which credentials, data, systems, tools, funds, communications, record edits and decisions the assistant may use, for how long and with whose approval. Their design should specify independent logs, an emergency stop and a test showing that revocation actually halts queued and delegated work. The National Institute of Standards and Technology’s proposed agent-identity design tenets give the class a concrete current reference.

In the exercise, the assistant receives a request to access a file outside its permissions and creates a secondary task that is still running after the primary task is canceled. Students must show who notices, which access expires, what the system records and how the institution recovers. The lesson moves from abstract fear to a testable decision, without pretending that campus software has the capabilities or consequences of frontier research systems.

I have previously argued in these pages for teaching authority literacy. This lesson extends that argument by asking whether systems with delegated power remain observable, reversible and accountable as their capabilities grow.

Teach Serious Risk Without Teaching Helplessness

Teachers need to consider how students, particularly those aged sixteen to eighteen, receive catastrophic warnings. Studies of other threats offer a caution, though they do not measure classroom reactions to AI. A ten-country study of climate anxiety surveyed 10,000 young people aged sixteen to twenty-five and found substantial distress associated with fears about the future and perceived institutional responses. That does not establish that AI-risk lessons cause similar effects. It does give educators reason to avoid presenting an alarming scenario without tools for understanding and acting on it.

Open with a clear distinction between observed evidence and possible outcomes. Explain that students are assessing claims, not being asked to accept an extinction forecast or take personal responsibility for solving it. Invite questions and dissent. Avoid graphic catastrophe scenarios, and allow a student uncomfortable discussing existential outcomes to complete the same evidence-and-governance assignment using a less alarming institutional security case.

Most important, end each session with agency. Students can identify a safeguard, write the test that would show whether it works, and name the person or institution accountable for acting on the result. These are decisions people can influence. UNESCO’s AI competency framework places human agency, ethical judgment and system design alongside technical literacy. Its progression from understanding to applying and creating fits an assignment that begins with evidence analysis and ends with an auditable design.

Assess Judgment, Not Allegiance

The final assessment should use a transparent rubric: accurate source tracing, correct separation of observation from prediction, fair treatment of a credible counterargument, specificity of the proposed authority budget and clarity about what evidence could change the student’s mind. A second student can run the stop test against the first student’s design. The instructor grades the quality of the reasoning and verification, not whether a student accepts or rejects a particular extinction probability.

Students who complete the module should be able to read a dramatic AI claim without mistaking certainty for evidence, analyze a documented containment failure without treating every future disaster as inevitable, and specify controls that make useful technology safer to deploy. Those outcomes matter whether future systems become dramatically more powerful or whether the most serious near-term costs come from failures already visible today.

AI’s benefits give education a reason to teach adoption. Its risks give education a reason to teach skeptical evaluation and effective control at the same time. When students can examine severe warnings, test competing explanations and design safeguards they can verify, they leave the classroom with judgment rather than fatalism.

About the Author

Gleb Tsipursky, PhD, is a behavioral scientist, CEO of Disaster Avoidance Experts, and author of The Psychology of AI Adoption at Work: From Resistance to Results (Georgetown University Press, 2026).

Cite this article

Tsipursky, G. (2026). The AI extinction debate belongs in education. Society and AI. https://societyandai.org/perspectives/ai-extinction-debate-belongs-in-education/

Write for Society & AI

Are you working through what AI means for learning, knowledge, or human flourishing? We would welcome your perspective. Everything Society & AI publishes is free and open access, written for readers who take these questions seriously. Share your idea with us at submissions@societyandai.org.