Video Completion via Motion Guided Spatial-Temporal Global Optimization

Ming Liu, Shifeng Chen, Jianzhuang Liu, and Xiaoou Tang


Abstract

In this paper, a novel global optimization based approach is proposed for video completion whose target is to restore the spatialtemporal missing regions of a video in a visually plausible way. Our algorithm consists of two stages: motion field completion and color completion via global optimization. First, local motions within the missing parts are completed patch-by-patch greedily using precomputed available motions in the video. Then the missing regions are filled by sampling patches from available parts of the video. We formulate the video completion as a global energy minimization problem by Markov random fields (MRFs). Based on the completed motion field of the video, a well-defined energy function involving both spatial and temporal coherence relationship is constructed. A coarse-to-fine Belief Propagation (BP) is proposed to solve the optimization problem. Experimental results have demonstrated the good performance of our algorithm.


 

 

Figure 1: Some results on the “performance” video. The three rows show the original frames, the manually removed regions, and the video completion results by our algorithm, respectively.

 

 

Figure 2: Some results on the “beach” and “running” videos.

 

 

Figure 3: Some results on the “car” video.

 

 

 

Ÿ  References:

ü  M. Liu, S. Chen, J. Liu, and X. Tang, "Video Completion via Motion Guided Spatial-Temporal Global Optimization," Proc. ACM Multimedia (ACM-MM), 2009. [pdf]