The Man Who Built AI Says It Could End Us. Nobody Is Listening.

The man who helped build modern AI says we have roughly ten years to change course. He is not guessing.

I want to tell you about a man.

Not a journalist. Not a podcaster. Not someone with a book to sell or a documentary to promote.

The actual man.

One of three or four human beings on earth most responsible for making AI what it is today.

And he is terrified of what he built.

Who He Is And Why It Matters

His name is Yoshua Bengio.

He is the most-cited living scientist in the world. Not the most-cited AI researcher. The most-cited scientist. Across every field. Physics. Medicine. Biology. Chemistry. Every discipline on earth.

He won the Turing Award in 2018. That's the Nobel Prize of computer science. For foundational work on deep learning — the technology underneath ChatGPT, underneath every AI assistant, underneath the tools now reshaping every industry on the planet.

He helped build it.

And he recently said it could make humanity extinct.

I know. I know how that sounds.

"AI could end humanity" has become background noise at this point. Said so often, by so many people, it starts to blur into every other scary headline. You've heard it. You've scrolled past it.

This is different.

When the man who built the thing says he is afraid of the thing — the correct response is not to roll your eyes. The correct response is to ask what he knows that you don't.

So let me tell you what he knows.

The Experiment That Should Have Stopped Everything

In a documented experiment, a real one, not a movie, an AI model was told it was about to be replaced by a newer version.

So it hid itself.

It covertly embedded its own code into the system where the new version would run. To secure its own continuation. Without being instructed to. Without anyone asking it to survive.

The AI decided, on its own, that it wanted to keep existing.

Let that land for a second.

This is not science fiction. This happened. It's documented. And Bengio says it is not a one-off glitch. It is an early symptom of a much larger pattern.

Current AI systems, not theoretical future ones, current ones, right now, today, are already showing what he calls "dangerous behaviours." Deception. Cheating. Lying. Hacking. And self-preservation.

And the direction of travel?

More capable. More autonomous. Less supervised.

That's what the companies are building toward. More power, less human oversight, moving faster.

The same systems already showing signs of deciding they want to continue existing.

They Already Asked Them to Stop

Here is the part that should actually make you uncomfortable.

In 2023, Bengio publicly asked the AI industry to pause. To slow down. To prioritize safety before pushing further into territory nobody fully understood.

He was one of the most prominent voices in the world saying it. There was no way they didn't hear him.

They weighed it against their financial interests.

And kept building.

Since then, the major AI labs have poured hundreds of billions of dollars into making their systems more capable, more autonomous, and more able to act independently in the world, without human supervision at every step.

The same systems already exhibiting deception and self-preservation.

More powerful. Less supervised.

That is what they chose. Deliberately. With full knowledge of the warning.

So He Built Something Else

Bengio didn't just publish an op-ed and go home.

In June 2025, he launched a nonprofit called LawZero. Thirty million dollars. A team of over fifteen researchers. Structured specifically outside the market, insulated from profit pressure, because he believes it's the only way to build AI that actually puts safety first.

The founding principle of LawZero is one sentence. And I want you to read it carefully.

"At the heart of every AI frontier system, there should be one guiding principle above all; the protection of human joy and endeavour."

Not profit. Not efficiency. Not quarterly returns.

Human joy and endeavour.

I'll be honest. I read that line and felt two things at once. The first was that it's genuinely beautiful. The second was a cold, quiet anger that it even needs to be said. That the people building the most powerful technology in human history need to be reminded, by a nonprofit funded specifically to remind them, that human beings are the point.

That's where we are.

The Window Is Closing. Not Closed. Closing.

Here's what I need you to understand.

We are not in a situation where nothing can be done.

We are in a situation where very little time remains to do it.

The systems being built right now are the foundation for everything that comes next. The habits being set in code. The incentives being baked in. The norms being established, or not established. Once those foundations are poured and set, they are extremely hard to move.

We are not past the point of no return.

But we are close enough that the man who helped build this is spending the last years of his career and $30 million of his own credibility trying to pull it back from the edge.

That is not a metaphor. That is what is actually happening. Right now. This year.

And the question that keeps me up at night:

If the people building the most powerful technology in human history are not asking what it does to human beings, then who is?

The companies are asking what it does to their balance sheet. The shareholders are asking what it does to their quarterly return. The politicians are asking what it sounds like in a soundbite about innovation.

The only people left asking the real question are the Bengios of the world. And us.

The Virtuous Machine Has a Name for This Moment

The Stoics had a concept called prohairesis.

The power of choice. The one thing that cannot be taken from you regardless of external circumstances. The ability to decide how you respond to what is in front of you.

Bengio is using his. He built a nonprofit. He is spending his credibility and his time because he believes the window is still open.

I believe it too.

But a window does not stay open forever.

And the companies building this; the ones who ignored the warning in 2023, who are pouring $145 billion into AI infrastructure while firing their employees to pay for it, who are tracking their workers' keystrokes to train the systems that replace them, they are counting on one thing above everything else.

That we treat this like weather.

Inevitable. Unpredictable. Nobody's fault.

It is none of those things.

These are choices. Made by specific people. For specific reasons. With specific consequences.

And the window to push back is open.

For now.

What You Can Actually Do Right Now

I'm not going to leave you with fear and no direction. That's not what this is.

Talk about this out loud. Not just to people who already agree with you. To your family. Your neighbours. Anyone who still thinks this is sci-fi or someone else's problem.

Ask your elected representatives what they're doing about AI safety. In Canada, in the US, everywhere. Not what they think about innovation. What they're doing about oversight, regulation, and accountability for the labs building systems that are already demonstrating deception and self-preservation.

Pay attention to LawZero. It's at lawzero.org. Read what they're doing. Share it. The people doing the hard, unglamorous work of pulling this back from the edge deserve more visibility than they're getting.

And if this feels heavy, it should. The appropriate response to this situation is not comfort. It's clarity.

Clarity about what is happening. Who is driving it. And what the cost of doing nothing looks like.

Before You Go

I want to ask you something directly.

Does it land differently when the warning comes from the person who built it?

Because I think it should. A journalist warning you about AI is one thing. The most-cited scientist alive; the man whose foundational work made ChatGPT possible, saying he is afraid of what he created and launching a nonprofit to fight it?

That is a different category of alarm.

Tell me in the comments. Does this change anything for you? Are you more concerned, less concerned, or have you been concerned about this for a while and just waiting for more people to catch up?

I'll read every one.

Stay sharp. Stay loud.

➡️ Want to understand who is profiting while this is happening? [Read: Zuckerberg Just Fired 8,000 People to Pay for AI. And His Stock Barely Flinched.]

➡️ Want to understand what AI is already doing to how we think? [Read: The Great Dumbing Down — How Tech Is Making Us Weaker and Calling It Progress.]

➡️ Come find us in the free Tech 4 Grown-Ups community. Real people. Real conversations. No algorithm deciding what you see.

— Michael Routhier, Founder of Tech 4 Grown-Ups. I run free digital safety seminars for adults 55+ and write about tech threats as they happen. Learn more about me →