This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Space-Time Video Completion Y. Wexler E. Shechtman M. Irani Dept. of Computer Science and Applied Math The Weizmann Institute of Science Rehovot, 76100 Israel Abstract We present a method for space-time completion of large space-time holes in video sequences of complex dy- namic scenes. The missing portions are filled-in by sam- pling spatio-temporal patches from the available parts of the video, while enforcing global spatio-temporal consistency between all patches in and around the hole. This is ob- tained by posing the task of video completion and synthesis as a global optimization problem with a well-defined ob- jective function. The consistent completion of static scene parts simultaneously with dynamic behaviors leads to real- istic looking video sequences. Space-time video completion is useful for a variety of tasks, including, but not limited to: (i) Sophisticated video removal (of undesired static or dynamic objects) by com- pleting the appropriate static or dynamic background infor- mation, (ii) Correction of missing/corrupted video frames in old movies, and (iii) Synthesis of new video frames to add a visual story, modify it, or generate a new one. Some examples of these are shown in the paper. 1. Introduction We present a method for space-time completion of large space-time holes in video sequences of complex dynamic scenes. We follow the spirit of  and use non-parametric sampling, while extending it to handle static and dynamic information simultaneously. The missing video portions are filled-in by sampling spatio-temporal patches from other video portions, while enforcing global spatio-temporal con- sistency between all patches in and around the hole. Global consistency is obtained by posing the problem of video completion/synthesis as a global optimization problem with a well-defined objective function and solving it appropri- ately. The objective function states that the resulting com- pletion should satisfy the two following constraints: (i) Ev- ery local space-time patch of the video sequence should This work was supported in part by the Israeli Science Founda- tion(Grant No. 267/02) and by the Moross Laboratory at the Weizmann Institute of Science. The work of Y. Wexler was supported by E. & L.Kaufmann Postdoctoral Fellowship. be similar to some local space-time patch in the remain- ing parts of the video sequence (the input data-set), while (ii) globally all these patches must be consistent with each other, both spatially and temporally. Solving the above optimization problem is not a simple task, especially due to the large dimensionality of video data. However, we exploit the spatio-temporal relations and redundancies to speed-up and constrain the optimiza- tion process in order to obtain realistic looking video se- quences with complex scene dynamics at reasonable com- putation times....
View Full Document
- Spring '08