Fast Image Rearrangement via Multi-Scale Patch Copying

Jiayao Hu, Shifeng Chen, Jianzhuang Liu, and Xiaoou Tang


In this paper, we propose a simple interactive way for a novel type of image synthesis called image rearrangement whose goal is to construct a new image based on some objects cropped from source images. The synthesis results are obtained by copying patches from the source images in a globally consistent way. The patch copying problem is formulated with the Markov random field model, and belief propagation is used as the optimization tool. To speed up our algorithm, a two-step belief propagation and a multiscale patch copying scheme are taken. Experimental results indicate that our algorithm obtains satisfactory results in both performance and efficiency.



Figure 1: Illustration of image rearrangement. We try to synthesize a new image with some objects appearing in the given source images. First, the interesting objects are manually cropped and pasted in the canvas. Then multi-scale patch copying synthesizes the image in a globally consistent way. Finally, the result is refined in an interactive way.



Figure 2: An example of multi-scale patch copying in a 3-level pyramid. (a) The source image. (b) Two objects are cropped from the source image and pasted on the canvas. (c) The patch copying result at the top level. (d) The result at the second level. (e) The result at the bottom level (the original scale).



Figure 3: An example of the re
fining step. (a) The structure labeled manually. The red region is synthesized by copying patches from the green region. (b) The result with re
fined structure. (c) The black region is manually labeled for patch re-copying. (d) The
final result.



Figure 4: Example result of object duplicating. (a) The source image. (b) The cropped object are pasted on the canvas. (c) The synthesized image.



Figure 5: Example results of content aware image resize. (a) The source image. (b) The resized image by nearest neighbor. (c) The resized image by seam carving. (d) The cropped objects are pasted on the canvas. (e) Our result.



Figure 6: Example result of our algorithm. (a)-(b) The source images. (c) Two parts cropped from the source images. (d) Our result.




Ÿ  References:

ü J. Hu, S. Chen, J. Liu, and X. Tang, “Fast Image Rearrangement via Multi-Scale Patch Copying,” Proc. ACM Multimedia (ACM-MM), 2010. [pdf]