The pipeline uses 3D scenes to structure AI images.
(Image credit: Nvidia / Black Forest Labs / AI)
AI image generation can still be very hit and miss. Even with style and structure references, it can take a lot of guesswork with text prompts and many attempts to get the structure you want for an image. The GPU giant Nvidia thinks it has a solution – and it uses Blender, one of the best 3D modelling software options.
The new Nvidia AI Blueprint for 3D-guided Generative AI (a catchy moniker, I know) aims to provide more precise control for AI image generation by allowing uses to create scenes in 3D first. It then takes the 3D scenes into Black Forest Lab’s Flux.1 image generator.
The process behind the Nvidia AI blueprint (Image credit: Nvidia)
The idea is that you can create a draft layout of an image using Blender, placing things like buildings, plants, vehicles and other objects where you want them. Using Blender means you can adjust the camera position etc. The draft 3D scene will then be used as a reference for the AI model to generate a 2D image.
The process provides more control than using text prompts alone because the reference Blender scene can be adjusted visually, reducing the guesswork involved in having to try out multiple prompts to get somewhere near the composition you want. And there's no need for the Blender scene to have a lot of detail or polish because it's only there to provide the composition.
Nvidia says the AI Blueprint for 3D-guided generative AI is a ready-to-use workflow that provides an automatic pipeline for intuitive control over image structure, layout, and perspective directly from Blender.
The blueprint comes with step-by-step instructions, sample files, and a preconfigured environment, including customisable source code and a demo showing how to integrate ComfyUI workflows. Performance is optimised for RTX AI PCs via Nvidia NIM microservices and accelerated by TensorRT and FP4.
Nvidia notes that it's suitable for non-commercial work only.
Get the Creative Bloq Newsletter
Daily design news, reviews, how-tos and more, as picked by the editors.
What are NVIDIA NIM Microservices and AI Blueprints for RTX AI PCs? - YouTube
Joe is a regular freelance journalist and editor at Creative Bloq. He writes news, features and buying guides and keeps track of the best equipment and software for creatives, from video editing programs to monitors and accessories. A veteran news writer and photographer, he now works as a project manager at the London and Buenos Aires-based design, production and branding agency Hermana Creatives. There he manages a team of designers, photographers and video editors who specialise in producing visual content and design assets for the hospitality sector. He also dances Argentine tango.
You must confirm your public display name before commenting
Please logout and then login again, you will then be prompted to enter your display name.
