Transfer learning. It’s a branch of AI that allows for the style transfer from one image to another. It seems like a straightforward concept: take my selfie and make it look like a Michelangelo painting. However, it is a fairly recent innovation in Deep Neural Networks that has allowed us to separate the content of an image from its style. And in doing so, to combine multiple images in ways that were previously impossible. For example, taking a long-dead artist’s style and applying it to your weekend selfie.
Just to prove that this is pretty cool, I’m going to take my newly built style transfer algorithm and apply it to a ‘selfie’ of my good dog, Lawrence. Here’s the original:
And here’s the image that I’m going to apply the style of:
That’s right, it’s Davinci’s Mona Lisa, one of the most iconic paintings of all time. I’m going to use machine learning to apply Davinci’s characteristic style to my iPhone X photo of my, admittedly very handsome,
If you’re interested, here’s a link to the original paper describing how to use Convolutional Neural Networks or CNNs to accomplish image style transfer. It’s written in relatively understandable language for such a technical paper so I do recommend you check it out, given you’re already reading a fairly technical blog.
So what is image content and style and how can we separate out the two? Well, neural networks are built in many layers, and the way it works out, some of the layers end up being responsible for detecting shapes and lines, as well as the arrangement of objects. These layers are responsible for understanding the ‘content’ of an image. Other layers, further down in the network are responsible for the style, colors and textures
Pretty striking, if I do say so myself.
Using a pre-trained Neural Network called VGG19 and a few lines of my own code to pull the figures and what’s called a Gram Matrix I choose my style weights (how much I want each layer to apply). Then using a simple loss function to push us in the right direction we apply the usual gradient descent algorithm and poof. Lawrence is forever immortalized as a Davinci masterpiece.
Impressed? Not Impressed? Let me know in the comments below. If you have anything to add, or you think I could do better please chime in! This is a learning process for me and I’m just excited to share my newfound knowledge.
Here’s a link to my code in a Google Colab Notebook if you want to try it out for yourself!