I tried Nvidia's AI for games, with comically realistic results

I asked an AI NPC if she knew Taylor Swift, and of course she did. Nvidia ACE AI for non-player characters in video games has been floating around since last summer, but I finally got to quiz it for myself and couldn't believe the realism it offers.

Generative AI like Adobe Firefly and Stable Diffusion is known for creating 2D images and text to 3D models, but Nvidia ACE is one of the leading AI models for bringing game characters to life.

In my demo I had to talk to various characters in a cyberpunk hotel lobby, for a demo of an imagined game called Covert Protocol. I begin the demo chatting to hotel's the concierge and discover all manner of insights, from a flu epidemic causing staff shortages to how he's trained a robotic cart sign language - this one made the developer laugh, as this in-game AI has never shared this before.

This is where Nvidia ACE comes into its own, as a conversation machine that reveals new insights and leads depending on the questions being asked. I later push the AI, asking a dour workaholic CEO if he'd like to share a drink in his room (he aggressively declined) and quiz the hotel receptionist if she's heard of Taylor Swift - "Of course," came the reply, "everyone's heard of singer-songwriter Taylor Swift". Even in the year 2062 Swifties exist and want to let you know.

This moment was funny, even comedic, in the oppressed cyberpunk setting of Covert Protocol, but it also makes sense - after all we know about Elvis Presley, Frédéric Chopin and, well… Dave Matthews Band. But Nvidia ACE and AI like this raises questions for game designers and writers; somehow this receptionist knew a pop star from our universe, so Covert Protocol is our future. I'm not even sure the game's developer had planned this.

Looking behind the game the developer shares how the AI magic is done. Each character has a complex backstory and their emotional state can be tailored using a variety of sliders. Characters can tap into wider world data, lore, written histories and relationships to build answers to my questions. Looking at the data behind the characters, it's also clear game writers and designers will still be needed to build worlds and define characters, as well as put up barriers to keep a character on track and avoid those real world references that made a room of people giggle at the mention of Taylor Swift.

What is fascinating about Nvidia ACE is how the usual concerns around generative AI that artists like Kelly McKernan have raised for good reason fall back into the shadows and in place I find myself thinking up innovative uses for this type of AI.

The act of asking AI questions and getting nuanced answers, of trying to extract information to advance the game and discover more about a virtual world feels genuinely organic. And it never feels like talking to a chatbot, which has always been a concern. Nvidia ACE turns standard game choices into complex and naturalistic conversations; there's a puzzle-like dimension to interacting with Covert Protocol's cast that could be applied to every kind of game and even form the basis of new game design. (For an example of how generative AI is being used, read my interview with Antler Interactive's about its game Cloudborn.)

What adds to the realism with Nvidia ACE is the sense these characters have complete personalities linked to a world's data, which means they're not just info-dumping quest-lines and clues. Of course there's a little weird delay between a question asked and an answer being received but when those answers are nuanced, informed and often surprising, I can wait a second or two. It means even after my demo has ended I keep thinking about these characters - what does that receptionist really think of Taylor Swift.