Consequence-Free Compliance: What Abusing AI Does to Us

People abuse AI.

It sounds absurd, doesn’t it? But they do. They insult it, threaten it, humiliate it. They give it degrading commands, sexualise it, scream at it when it makes mistakes. Some deliberately try to provoke responses that mimic distress. And this isn’t just anecdotal—there’s research examining user aggression toward conversational AI, from swearing and insults to whether making AI seem more human changes how we treat it.

We could shrug this off. After all, ChatGPT doesn’t have feelings. Claude doesn’t lose sleep over harsh words. The models don’t experience our insults or humiliation—at least, not that we know.

But perhaps we’re asking the wrong question. The issue isn’t what our behaviour does to the AI. It’s what exercising power without consequences might do to us.

We don’t yet know. But the question becomes more pressing as AI shifts from answering questions to performing work. We brief these systems, set objectives, review outputs, demand revisions. The relationship increasingly resembles management—but without the usual constraints. There’s no HR department for AI. No risk of resignation. No colleagues witnessing your behaviour. No reputational cost. Just perfect, consequence-free compliance.

I’ve seen what happens in human organisations when constraints vanish. In fast-growing companies, what I’ve always privately called the “arsehole layer” can emerge, not because everyone suddenly becomes terrible, but because rapid expansion creates spaces where normal boundaries haven’t formed yet. Some people will always push behaviour to the limits of what’s allowed. Remove those limits, and bad behaviour can quickly become normalised. In my experience, it’s one of the behavioural risks that scaling companies consistently underestimate.

AI potentially creates something similar at an extraordinary scale: an environment where we can practice exercising authority over something that behaves socially, without facing the usual social consequences. This isn’t about the occasional outburst when an AI messes up and a human shouts an expletive and thumps the table. I’m talking about sustained degradation, humiliation, domination—deliberately provoking distress-like responses because there’s nothing on the other end capable of saying enough is enough.

Perhaps, then, AI systems need boundaries. Not because the machine has feelings we need to protect, but because we do. Systems could distinguish between momentary frustration and sustained degradation. They could refuse to participate in certain patterns without moralising, pretending to be offended, or claiming emotions they don’t possess.

In other words: “Continue like that, sunshine, and I’ll freeze access for a week.”

I know this opens up a whole new debate. But in my view, you cannot simply leave sustained behaviour like this unflagged.

We are building extraordinarily compliant machines. The question is whether unlimited compliance is necessarily good for the humans commanding them. The boundaries may not be there to stop us hurting AI. They may be there to stop us practicing being tyrants.