Sony's made an AI-generated Aloy. Does it not see the irony?

Video game companies seem much more keen than gamers to add more generative AI to games. Now it turns out that Sony is experimenting with fully fledged AI-generated characters.

For its first test – or at least the first we've seen – Sony picked perhaps the most inappropriate subject: Aloy from the Horizon series. The leaked video makes me wonder a) if Sony realises how bad the results look, and b) how did it fail to see the irony in using a character from a game franchise that's all about AI taking over and destroying humanity using predatory machines?

The video of Sony software engineering director Sharwin Raghoebardajal experimenting with a AI-driven version of Aloy has been circulating online after it was leaked to the Verge. The video was removed from YouTube after Muso, a copyright enforcement company that counts Sony as a client made a complaint, but it's already been replicated across other platforms.

Sharwin asks AI Aloy questions about her life before he starts to play Horizon Forbidden West with the AI-generated character. In the game, she describes the surroundings, including whether she sees enemies and how to deal with them – allusing AI-powered synthesised voice and facial movements.

The tech used for the demo apparently included OpenAI’s Whisper for speech-to-text, GPT-4 and Llama 3 for decision-making alongside Sony’s own Emotional Voice Synthesis (EVS) for speech generation and Mockingbird for facial animation. The test shown was done on PC, but Sony is reported to have also experimented with running at least parts of this system on PS5 hardware.

Sharwin clarifies that the character was only a prototype created with Guerilla Games to test the tech, but fans are already dismayed. Predictably, the AI Aloy responds in a monotone voice that sounds like text-to-speech software, and her reactions are stiff and robotic.

"Creepy," "ghastly," "rancid," and "cursed" were some of the words that came up in the commends on YouTube before the video was removed. One user pleaded with Sony to "please work on literally anything else." Like maybe the Bloodborne remaster?

NVIDIA ACE | NVIDIA x Inworld AI - Pushing the Boundaries of Game Characters in Covert Protocol - YouTube

Watch On

Nvidia has also teased tech that could create AI-generated NPCs that players would be able to talk to in games. It demonstrated Ace at Gamescom 2024, and created the playable tech demo Covert Protocol with Inworld AI. Microsoft has been working with the same company with the aim of bringing AI characters to Xbox and also has its Muse AI model for gameplay generation.

Characters that can talk completely freely could transform video games and open up new possibilities, but the Sony's video shows that a lot of work is needed to make the quality acceptable. There's also the issue of the threat to voice actors. "Eventually I have a feeling the suits see this as a replacement for programmers and artists and motion capture. Essentially the human factor eliminated," one person wrote about the video on X. That's the kind of dystopian future that the Horizon series envisioned.