How to create photo-realistic images from rough sketches
Check out the latest AI-powered art technology; how long until it appears in Photoshop?
It doesn't seem that long since Google unleashed its DeepDream technology and the nightmarish image possibilities of deep learning neural networks, and gave birth to this harrowing designer quiz.
Since then the world of neural network-powered image trickery has been blazing ahead at a terrifying rate; a few months ago we introduced you to Neural Doodles, a way to turn simple doodles into fine art, and now there's a new technique that goes even further.
It doesn't have a catchy name just yet, but the possibilities of Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman and Alexei A. Efros' new image manipulation technology are incredibly exciting.
Their new technique, Generative Visual Manipulation on the Natural Image Manifold, tackles one of the major problems of realistic image manipulation: unless you're really skilled, it's hard to modify an image to any great extent while preserving the realism of the result.
In their terms, it's easy to fall off the natural image manifold while editing, and so they've come up with a way to learn the natural image manifold using a generative adversarial neural network.
We don't really understand that either, but thankfully the results speak for themselves. Using this new tool you can manipulate images in real time and even morph one object into the shape and colour of another, while still getting realistic results. In their video, they demonstrate this by taking shoes and handbags and morphing them into quite different shapes; it's a remarkable process to see in action.
You can use the technology to sketch new details onto existing objects - for examples, a set of laces on a shoe - and see them instantly appear in photorealistic form. And taking the technology further, you can use it to create entirely new photorealist images from rough sketches.
Get top Black Friday deals sent straight to your inbox: Sign up now!
We curate the best offers on creative kit and give our expert recommendations to save you time this Black Friday. Upgrade your setup for less with Creative Bloq.
The video demonstrates how, with a few lines and some dashes of colour you can create a stunning mountain landscape, and then instantly adjust it until it's exactly what you want.
The possibilities are dizzying; the bottom line seems to be that it's only a matter of time before anyone will be able to create amazing artwork without any artistic skill whatsoever. And bear in mind that this project is funded and developed in part by Adobe, so you might see this technology - most likely, a much more refined and powerful version - turn up in a future edition of Photoshop CC.
For now, though, you can try out iGAN (Interactive Image Generation via Generative Adversarial Networks) for yourself, if you're feeling brave. Like so many of these cutting edge image tool, everything you need is available to download on GitHub, but you'll need to be comfortable with using Python and third-party libraries.
Let us know in the comments if you manage to create anything with it!
Thank you for reading 5 articles this month* Join now for unlimited access
Enjoy your first month for just £1 / $1 / €1
*Read 5 free articles per month without a subscription
Join now for unlimited access
Try first month for just £1 / $1 / €1
Jim McCauley is a writer, performer and cat-wrangler who started writing professionally way back in 1995 on PC Format magazine, and has been covering technology-related subjects ever since, whether it's hardware, software or videogames. A chance call in 2005 led to Jim taking charge of Computer Arts' website and developing an interest in the world of graphic design, and eventually led to a move over to the freshly-launched Creative Bloq in 2012. Jim now works as a freelance writer for sites including Creative Bloq, T3 and PetsRadar, specialising in design, technology, wellness and cats, while doing the occasional pantomime and street performance in Bath and designing posters for a local drama group on the side.
Related articles
- This cute new logo animation is the best thing about the Louis Vuitton x Takashi Murakami rehash
- “We knew it would be special”: how design team Fluid created art and ad campaigns for two decades of Final Fantasy
- Incredible weather visualisation in Unreal Engine 5 can show real-time conditions in any city
- 2D art of the week: Vanessa Morales