Image Inpainting by Global Structure and Texture Propagation

Ting Huang, Shifeng Chen, Jianzhuang Liu, and Xiaoou Tang


Image inpainting is a technique to repair damaged images or modify images in a non-detectable form. In this paper, a novel global algorithm for region filling is proposed for image inpainting. After removing objects from an image, our approach fills the regions using patches taken from the image. The filling process is formulated as an energy minimization problem by Markov random fields (MRFs) and the belief propagation (BP) is utilized to solve the problem. Our energy function includes structure and texture information obtained from the image. One challenge in using BP is that its computational complexity is the square of the number of label candidates. To reduce the large number of label candidates, we present a coarse-to-fine scheme where two BPs run with much smaller numbers of label candidates instead of one BP running with a large number of label candidates. Experimental results demonstrate the excellent performance of our algorithm over other related algorithms.



Figure 1: The comparative results on the "bungee" image. (a) The result of the greedy approach. (b) The result of BP without confidence and structure terms. (c) Our result.



Figure 2: More comparative results. The first row contains two pairs of original and masked images. On the second and third rows, from left to right, are the results obtained by the greedy algorithm, BP without confidence and structure terms, and our algorithm.



Figure 3: More results by our algorithm. From left to right: original images, masked images, and the results.



Ÿ  References:

ü  T. Huang, S. Chen, J. Liu, and X. Tang, “Image Inpainting by Global Structure and Texture Propagation,” Proc. ACM Int. Conf. Multimedia (ACM MM), 2007. [pdf]