Style Transfer via Image Component Analysis

Wei Zhang, Chen Cao, Shifeng Chen, Jianzhuang Liu, and Xiaoou Tang


Abstract

Example-based stylization provides an easy way of making artistic effects for images and videos. However, most existing methods do not consider the content and style separately. In this paper, we propose a style transfer algorithm via a novel component analysis approach, based on various image processing techniques. First, inspired by the steps of drawing a picture, an image is decomposed into three components: draft, paint and edge, which describe the content, main style, and strengthened strokes along the boundaries. Then the style is transferred from the template image to the source image in the paint and edge components. Style transfer is formulated as a global optimization problem by using Markov random fields, and a coarse-to-fine belief propagation algorithm is used to solve the optimization problem. To combine the draft component and the obtained style information, the final artistic result can be achieved via a reconstruction step. Compared to other algorithms, our method not only synthesizes the style, but also preserves the image content well. We also extend our algorithm from single image stylization to video personalization, by maintaining the temporal coherence and identifying faces in video sequences. The results indicate that our approach performs excellently in stylization and personalization for images and videos.


 

Ÿ  Image style transfer (more results)

 

flowchart.png

Figure 1. The framework of image style transfer.

 

pastel.jpg

 "Pastel" style 

rhone.jpg

"Rhone" style

craq.jpg

"Craquelure" style

freud.jpg

"Freud" style

watercolor.jpg

"Watercolor" style

Figure 2. Style templates.

 

flowers.jpg

 Source image

flowers-pastel.jpg

"Pastel" style

flowers-rhone.jpg

"Rhone" style

flowers-craq.jpg

"Craquelure" style

flowers-freud.jpg

"Freud" style

flowers-watercolor.jpg

"Watercolor" style

Figure 3. Style transfer results for the flower image.

 

Ÿ  Video personalization (more results)

 

mattybC1.jpg

Frame 1

watercolor.jpg

"Watercolor" template

mattyb1.jpg

Watercolor style

mattyb3.jpgmattyb5.jpgmattyb7.jpgmattyb11.jpgmattyb13.jpgmattyb17.jpg

mattyb19.jpgmattyb21.jpgmattyb23.jpgmattyb25.jpgmattyb27.jpgmattyb29.jpg

mattyb31.jpgmattyb33.jpgmattyb35.jpgmattyb37.jpgmattyb39.jpgmattyb41.jpg

Figure 4. Video personalization.

 

Ÿ  References:

W. Zhang, C. Cao, S. Chen, J. Liu, and X. Tang, "Style Transfer via Image Component Analysis," IEEE Trans. on Multimedia (TMM), accepted.  [pdf]

C. Cao, S. Chen, W. Zhang, and X. Tang, "Automatic Motion-Guided Video Stylization and Personalization," ACM Multimedia (ACM-MM), 2011. [pdf]