Feature Extraction for Style Transferring

After loading the images into memory, we will implement the style transfer. It is necessary to separate the style of the image from its contents to achieve the style transfer. After that, it is also possible to transfer the style elements of one image to the content elements of the second image. This process is done using mainly feature extraction from standard convolutional neural networks.

These features are then manipulated to extract either content information or style information. This process involves three images a style image, a content image and finally a target image. The style of the style image is combined with the content in the content image to create a final target image.

Feature Extraction for Style Transferring

This process begins by selecting a few layers within our model to extract features from. We will get a good idea of how our image is being processed throughout the neural network by selecting a few layers to extract features from. We extract the model features of our style image and content image as well. After that, we extract features from our target image and compare it to our style image feature and our content image feature.

Getting features from the images

# Defining simple method with two arguments, i.e. our image and our model
def get_features(image,model):
#choosing specific layer within our VGG-19 model that we are going to extract features from 
# Defining layers dictionary object which contains the specific layers  
  layers={'0':'conv1_1', #Mapping 0 to conv1_1
          '5':'conv2_1', #Mapping 5 to conv2_1
          '10':'conv3_1', #Mapping 10 to conv3_1
          '19':'conv4_1', #Mapping 19 to conv4_1
          '21':'conv4_2', #Mapping 21 to conv4_2
          '28':'conv5_1',} #Mapping 28 to conv5_1

Now we have six feature extraction layers. In these six feature extraction layer, we will use five of these for style extraction and only one of them for content extraction. We will use conv4_2 for content extraction. This only single layer is sufficient for extracting content. This layer is deeper into our neural network and provide high depth image feature. This is the reason that pre-trained object detection convolutional neural network become very effective in representing content elements.

Getting style features from various features throughout the network allowing for optimal style creation. Extracting style features from numerous layers will allow for the most effective style extraction and recreation.

#Defining an empty dictionary called features to store all the extracted features 
features={}
  #Running a loop which iterates over all of the layers in our model 
  for name, layer in model._modules.items(): 
    #Running our image through the specific layer and store into the new image
    image=layer(image)
    #checking the name of the current layer which we are iterating through is inside layers
    if name in layers:
      #If true then store the output from that specific layer 
      features[layers[name]]=image 
  #returning feature dictionary 
  return features

Once we initialize our get feature method, we have to call it with our content image and our VGG model.

in the same way, we will do it for our style image as:

Next TopicGram Matrix for Style Transferring

← prev next →