Meet Neuralangelo, the Nvidia AI that turns 2D video into 3D scenes

Nvidia AI Neuralangelo; Michelangelo statue rendered in 3D
(Image credit: Nvidia)

Just when you thought the wild world of AI couldn't surprise any more, Nvidia unveils its new Neuralangelo AI, which can turn 2D video clips into 3D structures and scenes. It literally generates detailed replicas of buildings, sculptures and real objects from video clips taken on a mobile or camera. 

The idea behind Neuralangelo, which 'sculpts' life-like 3D scenes from video, like a modern-day Michelangelo (geddit), is that these 3D scenes can then be imported into other software. The Neuralangelo assets can be edited for use in video games, arch-viz, web 3 projects, well, anything really, including aiding digital artists to define light sources.

This comes on the back of Nvidia's AI for conversing with game characters and the new text-to-3D generative Ai that's still in development from researchers at HiFA. In a Neuralangelo demo (below) Nvidia showed how its AI could recreate complex objects, such as Michelangelo’s David, as well as something as everyday as a flatbed truck.

Nvidia has released a detailed explanation of how Neuralangelo works, sharing how it can translate intricate details in a video, such as glass and smooth marble, into 3D. The pitch is this AI will help speed-up workflows of creatives and unlock 3D virtual worlds and assets to new industries not replace jobs.

"The 3D reconstruction capabilities Neuralangelo offers will be a huge benefit to creators, helping them recreate the real world in the digital world," said Ming-Yu Liu, senior director of research and co-author on the paper, in a press statement. "This tool will eventually enable developers to import detailed objects – whether small statues or massive buildings – into virtual environments for video games or industrial digital twins."

The best AI generators are the ones that can fit into, and enhance, a workflow rather than replace jobs. New generative AI models like Adobe Firefly, for example, can fit into a concept art workflow and speed the process, not replace the artists doing the work. Read Ken Coleman's tutorial on how to use Adobe Firefly or my interview with Chris Colman from Final Frontier, who is using AI everyday.

Thank you for reading 5 articles this month* Join now for unlimited access

Enjoy your first month for just £1 / $1 / €1

*Read 5 free articles per month without a subscription

Join now for unlimited access

Try first month for just £1 / $1 / €1

Ian Dean
Editor, Digital Arts & 3D

Ian Dean is Editor, Digital Arts & 3D at Creativebloq, and the former editor of many leading magazines. These titles included ImagineFX, 3D World and leading video game title Official PlayStation Magazine. In his early career he wrote for music and film magazines including Uncut and SFX. Ian launched Xbox magazine X360 and edited PlayStation World. For Creative Bloq, Ian combines his experiences to bring the latest news on AI, digital art and video game art and tech, and more to Creative Bloq, and in his spare time he doodles in Procreate, ArtRage, and Rebelle while finding time to play Xbox and PS5. He's also a keen Cricut user and laser cutter fan, and is currently crafting on Glowforge and xTools M1.