space_time_jrn - IEEE TRANSACTIONS ON PATTERN ANALYSIS AND...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Space-Time Super-Resolution Eli Shechtman, Yaron Caspi, and Michal Irani, Member , IEEE Abstract —We propose a method for constructing a video sequence of high space-time resolution by combining information from multiple low-resolution video sequences of the same dynamic scene. Super-resolution is performed simultaneously in time and in space. By “temporal super-resolution,” we mean recovering rapid dynamic events that occur faster than regular frame-rate. Such dynamic events are not visible (or else are observed incorrectly) in any of the input sequences, even if these are played in “slow- motion.” The spatial and temporal dimensions are very different in nature, yet are interrelated. This leads to interesting visual trade-offs in time and space and to new video applications. These include: 1) treatment of spatial artifacts (e.g., motion-blur) by increasing the temporal resolution and 2) combination of input sequences of different space-time resolutions (e.g., NTSC, PAL, and even high quality still images) to generate a high quality video sequence. We further analyze and compare characteristics of temporal super-resolution to those of spatial super-resolution. These include: How many video cameras are needed to obtain increased resolution? What is the upper bound on resolution improvement via super-resolution? What is the temporal analogue to the spatial “ringing” effect? Index Terms —Super-resolution, space-time analysis, temporal resolution, motion blur, motion aliasing, high-quality video, fast cameras. æ 1I NTRODUCTION A video camera has limited spatial and temporal resolution. The spatial resolution is determined by the spatial density of the detectors in the camera and by their induced blur. These factors limit the minimal size of spatial features or objects that can be visually detected in an image. The temporal resolution is determined by the frame- rate and by the exposure-time of the camera. These limit the maximal speed of dynamic events that can be observed in a video sequence. Methods have been proposed for increasing the spatial resolution of images by combining information from multiple low-resolution images obtained at subpixel displacements (e.g., [1], [2], [3], [6], [7], [11], [13], [14], [15], [16]. See [4] for a comprehensive review). An extension of [15] for increasing the spatial resolution in three-dimensional (x, y, z) medical imagery has been proposed in [12], where MRI data was reconstructed both within image slices (x and y axis) and between the slices (z axis). The above-mentioned methods, however, usually assume static scenes with limited spatial resolution and do not address the limited temporal resolution observed in dynamic scenes. In this paper, we extend the notion of super- resolution to the space-time domain. We propose a unified framework for increasing the resolution both in time and in space by combining information from multiple video sequences of dynamic scenes obtained at (subpixel) spatial and (subframe) temporal misalignments. As will be shown,
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 15

space_time_jrn - IEEE TRANSACTIONS ON PATTERN ANALYSIS AND...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online