Amazing AI turns doodles into fine art

Neural Doodles - Renoir

This Renoir-style landscape was generated from a doodle and an AI algorithm

There's a lot of exciting work going on at the interface between art and artificial intelligence at the moment. In the last year we've been both amazed and horrified by Google's DeepDream, which can turn any image into a psychedelic lizard-dog freak-out, and we've loved the results of DeepForger, which uses a similar process but instead turns pictures into more traditional works of art.

The problem with both of these impressive techniques is that the results can be very random; there's no way of telling what you're going to get, or of tweaking the process to steer it in a particular direction.

But now there's a new AI-powered algorithm that offers a lot more control and enables you create stunning artwork based on any painting you like.

Neural Doodles - mockup

Imagine being able to do this in Photoshop. One day...

Neural Doodles is the work of Alex J Champandard, AI expert and the creator of DeepForger, and like DeepForger it'll create new works based on existing paintings. Unlike DeepForger, though, it allows you to direct how the work is created, using a process called Semantic Style Transfer.

Neural Doodles - How it works 1

Start with a source painting and create a two-bit reference image...

As well as a source image, you sketch out a two-bit, MS-Paint-style version; an annotation that Neural Doodles can use to recognise the different elements of the artwork.

Neural Doodles - How it works 2

...then do another sketch and Neural Doodles will turn it into fine art

You then make a second sketch – your doodle – and the algorithm will extract annotated patches from the source image, and then incrementally transfer them into a new target image, based on how closely they match.

The result is a new synthesised image in the style of the original artwork, without the random elements that often crop up when you simply fling an image at a deep neural network and let it do its thing. As you can see from Champandard's early experiments, with a little preparation you can create a passable Renoir or Monet, and if the end result isn't up to scratch you can fine-tune the annotation and try again.

Neural Doodles - Monet

This fabulous fake Monet is all Neural Doodles' own work

Like early DeepDream methods, Neural Doodles isn't for the faint-hearted at the moment. It's only available as a (well-comented) Python script and needs a couple of libraries installed; you can find more details in its GitHub repository.

However Champansard envisions it working as simple image app plugin in the future, and he's put together a neat little mockup of how it might work.

You can find Neural Doodles here, and read more about it in this blog post.

Liked this? Read these!

Thank you for reading 5 articles this month* Join now for unlimited access

Enjoy your first month for just £1 / $1 / €1

*Read 5 free articles per month without a subscription

Join now for unlimited access

Try first month for just £1 / $1 / €1

Jim McCauley

Jim McCauley is a writer, performer and cat-wrangler who started writing professionally way back in 1995 on PC Format magazine, and has been covering technology-related subjects ever since, whether it's hardware, software or videogames. A chance call in 2005 led to Jim taking charge of Computer Arts' website and developing an interest in the world of graphic design, and eventually led to a move over to the freshly-launched Creative Bloq in 2012. Jim now works as a freelance writer for sites including Creative Bloq, T3 and PetsRadar, specialising in design, technology, wellness and cats, while doing the occasional pantomime and street performance in Bath and designing posters for a local drama group on the side.