Amazing AI turns doodles into fine art

The latest breakthrough in artistic artificial intelligence can turn sketches into digital 'oil paintings'.

Neural Doodles - Renoir

This Renoir-style landscape was generated from a doodle and an AI algorithm

There's a lot of exciting work going on at the interface between art and artificial intelligence at the moment. In the last year we've been both amazed and horrified by Google's DeepDream, which can turn any image into a psychedelic lizard-dog freak-out, and we've loved the results of DeepForger, which uses a similar process but instead turns pictures into more traditional works of art.

The problem with both of these impressive techniques is that the results can be very random; there's no way of telling what you're going to get, or of tweaking the process to steer it in a particular direction.

But now there's a new AI-powered algorithm that offers a lot more control and enables you create stunning artwork based on any painting you like.

Neural Doodles - mockup

Imagine being able to do this in Photoshop. One day...

Neural Doodles is the work of Alex J Champandard, AI expert and the creator of DeepForger, and like DeepForger it'll create new works based on existing paintings. Unlike DeepForger, though, it allows you to direct how the work is created, using a process called Semantic Style Transfer.

Neural Doodles - How it works 1

Start with a source painting and create a two-bit reference image...

As well as a source image, you sketch out a two-bit, MS-Paint-style version; an annotation that Neural Doodles can use to recognise the different elements of the artwork.

Neural Doodles - How it works 2

...then do another sketch and Neural Doodles will turn it into fine art

You then make a second sketch – your doodle – and the algorithm will extract annotated patches from the source image, and then incrementally transfer them into a new target image, based on how closely they match.

The result is a new synthesised image in the style of the original artwork, without the random elements that often crop up when you simply fling an image at a deep neural network and let it do its thing. As you can see from Champandard's early experiments, with a little preparation you can create a passable Renoir or Monet, and if the end result isn't up to scratch you can fine-tune the annotation and try again.

Neural Doodles - Monet

This fabulous fake Monet is all Neural Doodles' own work

Like early DeepDream methods, Neural Doodles isn't for the faint-hearted at the moment. It's only available as a (well-comented) Python script and needs a couple of libraries installed; you can find more details in its GitHub repository.

However Champansard envisions it working as simple image app plugin in the future, and he's put together a neat little mockup of how it might work.

You can find Neural Doodles here, and read more about it in this blog post.

Liked this? Read these!


Jim McCauley is a writer, editor and occasional podcaster, and is available for space parties.