Home > Research > Publications & Outputs > How not to be seen
View graph of relations

How not to be seen: object removal from videos of crowded scenes

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published
  • Miguel Granados
  • James Tompkin
  • Kwang In Kim
  • Oliver Grau
  • Jan Kautz
  • Christian Theobalt
Close
<mark>Journal publication date</mark>05/2012
<mark>Journal</mark>Computer Graphics Forum
Issue number2 Pt 1
Volume31
Number of pages10
Pages (from-to)219-228
Publication StatusPublished
<mark>Original language</mark>English

Abstract

Removing dynamic objects from videos is an extremely challenging problem that even visual effects professionals often solve with time-consuming manual frame-by-frame editing. We propose a new approach to video completion that can deal with complex scenes containing dynamic background and non-periodical moving objects. We build upon the idea that the spatio-temporal hole left by a removed object can be filled with data available on other regions of the video where the occluded objects were visible. Video completion is performed by solving a large combinatorial problem that searches for an optimal pattern of pixel offsets from occluded to unoccluded regions. Our contribution includes an energy functional that generalizes well over different scenes with stable parameters, and that has the desirable convergence properties for a graph-cut-based optimization. We provide an interface to guide the completion process that both reduces computation time and allows for efficient correction of small errors in the result. We demonstrate that our approach can effectively complete complex, high-resolution occlusions that are greater in difficulty than what existing methods have shown.