What if the greatest danger of AI isn't artificial intelligence at all?

If an AI company refuses to remove safeguards against mass surveillance and autonomous killing, should that refusal make it a national-security risk?

A couple of weeks ago, a US appeals court upheld the Pentagon's decision to exclude Anthropic, the company behind Claude, from defence contracts.

Anthropic had refused to remove restrictions on the use of its technology for mass domestic surveillance and fully autonomous weapons.

The Pentagon wanted the freedom to use Claude for all lawful military purposes, without a private company imposing its own boundaries.

Its argument? A supplier shouldn't be able to interfere with military operations.

There is a legitimate question here about who should control technology used in national defence. But there's another question that bothers me more.

What happens when the pursuit of military superiority begins to undermine the very safeguards designed to protect us?

James Cameron was exploring precisely that danger in The Terminator more than 40 years ago. It was science fiction then. Today, the questions surrounding autonomous weapons are very real.

For the military, I try to understand the motivations.

Faster decisions. More effective weapons. Greater military capability. A strategic advantage over adversaries who may have no intention of respecting the same boundaries.

And, inevitably, enormous commercial interests.

But I wonder whether there’s something else at work here.

The pursuit of superiority and control, whatever the cost.

A mentality in which every advantage must be seized, every limitation challenged, and every safeguard treated as a potential weakness.

Where restraint is mistaken for vulnerability, and ruthlessness for strength.

And perhaps the most disturbing part is that none of this requires an evil mastermind.

Just two very familiar human failings.

Ignorance and Want.

Charles Dickens gave those names to two miserable children in A Christmas Carol, more than 180 years ago.

They represented a society's failure to confront the suffering and consequences of its own choices.

And although Dickens was writing about something very different, I can't help seeing an uncomfortable parallel.

Want, in our relentless pursuit of power, advantage and financial reward.

Ignorance, in our willingness to assume that because we can build increasingly powerful systems, we will necessarily be able to understand and control them.

Nobody needs to intend a catastrophe. Nobody in their right mind anyway.

Every individual decision can appear rational. Every concession can be justified. Every safeguard can become an obstacle to progress.

Until the accumulation of those decisions takes us somewhere nobody intended to go.

I really believe the greatest danger isn't that AI will simply turn bad.

It's that we'll push it in dangerous directions, convinced we know exactly what we're doing.