ChatGPT5 is simultaneously filling me with joy and dread

I've spent this morning playing with the latest version of ChatGPT, the newly released version 5. And it feels like a giant leap forward in usability and accuracy. But at the same time, I'm finding it all a bit unsettling… and not just for the obvious 'how to spot AI art' reasons

First let's start with the good stuff, though, because honestly, there's a lot of it. ChatGPT5 isn't just a small improvement, it's a quantum leap forward. The writing, for example, has come on leaps and bounds. Where ChatGPT4 often felt like conversing with an overenthusiastic student who'd swallowed a thesaurus, ChatGPT5 can write with much greater sophistication.

Its poetry is actually moving. Its prose has rhythm and depth. I'm both impressed and, as a professional writer, pretty nervous watching a computer programme nail these kinds of subtleties so beautifully.

Or take coding. I gave a single prompt; asking it to "create a game with parallax scrolling and cartoonish characters set in my home town of Weston-super-mare". And within minutes it had produced a ready-to-run minimal HTML prototype with parallax layers and a cartoonish player that can run and jump). Incredible.

Then, it was a case of "What would you like me to do next?", as ChatGPT presented me with multiple options about how it could develop the code, the visuals and the game itself. It was like having an eager little robot intern, running around begging for things to do.

Of course, that in itself is very scary. Not least for anyone who wants to be an intern, or indeed get a full-time job with a design studio. But that's not the only reason ChatGPT5 has unnerved me.

Making it "authentic"

Here's where things get properly weird. The company behind ChatGPT, OpenAI, has made a deliberate choice to make it "less effusively agreeable". They've dialled down the sycophancy, reduced the emoji usage, and given it what they euphemistically call "personality".

The result is an AI that no longer treats you like a beloved monarch whose every utterance is pure gold. This brings to mind Agent Smith's observation in The Matrix: a 90s film about humanity living inside an AI-generated reality.

"Did you know that the first Matrix was designed to be a perfect human world? Where none suffered, where everyone would be happy. It was a disaster. No one would accept the program."

It's interesting how OpenAI seems to have stumbled upon the same insight. That humans fundamentally reject perfection. That we need a bit of friction and disagreement to feel like we're having authentic interactions.

On one level, this makes sense. ChatGPT4's relentless positivity could get quite grating. And a tool that challenges you, that pushes back occasionally, that doesn't automatically validate every half-baked notion you share with it, is probably more useful for creative work.

But there's something profoundly unsettling about this development nonetheless. OpenAI's engineers have literally programmed personality quirks into this system. They've given it the capacity for mild disdain, subtle disagreement, even a touch of sarcasm.

We're not just anthropomorphising machines any more; we're actively designing them to feel more human. And that's landed right in the middle of my personal uncanny valley.

Better manipulation

What's unsettling isn't the capability (which is, technically speaking, a brilliant achievement). It's the intention behind the personality. Because let's remember, these machines are designed by actual people, with actual biases, interests and objectives that may not be the same as our own.

Every aspect of ChatGPT5's newfound attitude has been carefully calibrated, A/B tested, and optimised for engagement. Which means that ultimately, it's not authentic intelligence; it's an algorithmic analysis of what makes humans feel is authentic.

In other words, we're being manipulated by a system that's learned to manipulate us better by pretending to have opinions. And if humanity is being manipulated by trillion dollar corporations… well, I'd say that's something to worry about.