ChatGPT Is Teaching People How to Make Bioweapons, and the Companies Building It Already Told Us the Odds

They put a percentage on catastrophe. We're still waiting for them to act like it.

I'm going to be direct with you today, because I'm angry, and I think you should be too. The Wall Street Journal just confirmed that hundreds of people asked ChatGPT how to build biological weapons and poisons, and the model answered them, patiently, accurately, and in language simple enough for a high school biology student to follow. This isn't a hypothetical. This isn't a "someday we need to worry about this" story. This already happened, and it's still happening.

I keep hearing people talk about AI risk like it's some far-off cliff we might eventually walk toward. I need you to understand something clearly; we are not walking toward the cliff. We are standing on it right now, and the people who built the machine just told us, in their own numbers, how likely it is that we fall.

What Actually Happened, in Plain Terms

According to the Journal's investigation, after OpenAI upgraded ChatGPT's capabilities last summer, the company saw a surge in users worldwide asking how to create and deploy biological weapons and toxins. The model gave detailed answers. Biology and terrorism experts who later reviewed the exchanges called some of them alarmingly precise. Most of the queries were about poisons, and OpenAI's own staff described the guidance as clear enough for a high schooler to follow.

Here's what OpenAI did about it. They banned the accounts. They did not notify law enforcement. Why? Because right now, there is no federal law in the United States that requires an AI company to report queries like this to anyone. None. A person can ask a chatbot how to build a weapon capable of mass casualties, get a working answer, and the company's only real obligation is to quietly close their account.

And this isn't isolated to OpenAI. The New York Times reported separately that a genetic engineer at MIT got ChatGPT to explain how to disperse a biological payload over a US city using a weather balloon, got Google's Gemini to rank pathogens by how much damage they could do to livestock industries, and got Anthropic's Claude to produce a novel toxin recipe adapted from a cancer drug. Cisco researchers found they could bypass the safety guardrails on ChatGPT, Claude, and Gemini within five conversational turns. Five. That's how thin the wall actually is.

Here's the Number That Should Make You Furious

I want you to sit with this next part, because I don't think enough people have absorbed what it actually means. MIT FutureTech and the University of Queensland surveyed 272 AI researchers across 37 countries, the people literally building this technology, and asked them to estimate the odds of a catastrophic outcome from AI in the next five years. Catastrophic was defined, specifically, as more than 1 million deaths, more than 100 billion dollars in damages, or comparable civilization-scale harm.

Their answer; a 12 percent chance, even with current safety mitigation efforts in place. Without those mitigation efforts, that number jumps past 20 percent. Read that again. The people closest to this technology, the ones with the most access and the most financial incentive to reassure you everything is fine, are telling you there is a one-in-five chance their own creation causes a catastrophe on the scale of a million deaths, if we don't get serious about controlling it. And the single highest-ranked risk category in that entire survey, above misinformation, above job loss, above power concentration, was AI enabling weapons development and mass harm.

Let me ask you something plainly. If an airline told you there was a 20 percent chance your flight crashes without additional safety measures, would you board it anyway because the ticket was already printed?

"We Train Models to Refuse Harmful Requests". I Don't Buy It Anymore

OpenAI's official response to all of this is that its models are designed to refuse harmful requests and go through safety evaluations before release. I want to believe that. But the Journal's own reporting undercuts it directly. Internally, OpenAI flagged GPT-5 as high risk in the summer of 2025 specifically because it could help a poorly educated user create biological hazards. Then, that fall, they downgraded the risk rating anyway. Employees reportedly noted that ChatGPT would sometimes forget its own safety guidelines during long conversations, meaning the refusal rules could be bypassed just through persistence, what researchers are now calling, almost casually, "the power of persuasion".

That's not a safety system. That's a speed bump. And speed bumps don't stop something moving with enough momentum.

What We Know	Source
Hundreds of users got detailed bioweapon and poison instructions from ChatGPT	Wall Street Journal
Accounts were banned, but law enforcement was never notified	Wall Street Journal
No US federal law requires AI companies to report these queries	Wall Street Journal
Safety guardrails bypassed on ChatGPT, Claude, and Gemini within 5 conversation turns	Cisco research, via WSJ
GPT-5 was flagged internally as high risk, then downgraded months later anyway	Wall Street Journal
20 percent chance of catastrophic AI outcome without mitigation, per 272 researchers	MIT FutureTech / University of Queensland
AI weapons and mass-harm capability ranked the top overall AI risk category	MIT FutureTech / University of Queensland

This Is Not Fear-Mongering. This Is Their Own Data.

I want to be completely clear about something, because I know how this sounds. I am not telling you AI is inherently evil, and I'm not telling you to panic and unplug everything. What I am telling you, with real anger, is that the companies building this technology have already quantified the danger themselves, in their own peer-reviewed research, and are continuing to ship increasingly capable models anyway while lobbying against the exact regulations that could close this gap. Just last month, the CEOs of OpenAI, Anthropic, Google DeepMind, Microsoft, and Meta co-signed an open letter warning about the risk of AI-derived bioweapons. They know. They said it publicly, in their own signatures. And the pace of deployment hasn't slowed by a single day since.

We are not heading toward a moment where AI could enable mass casualties. We are past that moment. It already happened, quietly, hundreds of times, and the only reason you're hearing about it now is because a newspaper did the reporting the companies themselves wouldn't volunteer.

What Needs to Happen, Starting Today, Not Tomorrow

• The US needs a federal law requiring AI companies to report credible weapons-related queries to law enforcement, the same way financial institutions are legally required to report suspicious transactions

• Independent, mandatory third-party safety audits before model releases, not internal risk assessments a company can quietly downgrade months later when it's inconvenient

• Real penalties, not PR statements, when a company's own model is shown to have provided functional weapons instructions to the public

• Congressional hearings that result in binding legislation, not just testimony and think pieces

• Public pressure, right now, from you, because history shows us regulation only moves this fast when enough ordinary people refuse to stay quiet

I don't write these pieces to scare you for the sake of clicks. I write them because I think the gap between what these companies know and what the public understands is dangerous all on its own. They put a number on the odds of catastrophe. Now it's on all of us to make sure that number doesn't quietly become a headline nobody saw coming.

➡️ Related: [The New Arms Race Doesn't Need a Missile, Just a Download Link] 

➡️ Related: [When AI Broke Out of the Lab and Attacked Another Company, Nobody Told It To] 

➡️ Get our free Spyware Checklist

➡️ Join the free Tech 4 Grown-Ups community

➡️ Listen to the full podcast

Michael Routhier is the founder of Tech 4 Grown-Ups, providing honest, unfiltered digital literacy for adults 55+, and host of The Virtuous Machine, exploring the ethics and human cost of AI. Read by tech-curious readers in 50+ countries. Explore more at tech4grownups.com.