Proc. IEEE Multimedia Signal Processing Workshop,
Sep. 17-20, 2012, Banff, Canada.
Probabilistic 3-D Motion Estimation for Rolling Shutter
Video Rectification from Visual and Inertial Measurements
Brian L. Evans
Department of Electrical
and Computer Engineering,
The University of Texas at Austin,
Austin, TX 78712 USA
Fig 1a Distortion -
Fig 1b Correction -
Fig 9a Distortion -
Fig 9b Correction
won IEEE MMSP Top 10% Paper Award
Guitar Oscillations Captured with iPhone 4 showing rolling shutter artifacts
Video acquired by handheld CMOS cameras may suffer from rolling shutter
Rolling shutter artifacts, which are due to the rows in the image sensor
array being exposed sequentially from top to bottom, increase with the
speed of the relative motion between the scene and camera.
To rectify these artifacts, one needs to recover the projection parameters
for each row.
In this paper, we propose a probabilistic method to estimate 3-D camera
rotation by using video and inertial measurements on the handheld platform,
such as a smart phone.
Our contributions are
Experiments indicate that the proposed sensor fusion algorithm produces a
more accurate orientation estimate and better rectifies rolling shutter
- an efficient sensor fusion algorithm using an extended Kalman filter,
- a quality assessment method using vanishing point detection.
After the paper was published, we developed video demonstrations to show
the difference between the original and the original after rolling shutter
artifact rectification had been applied.
The rolling shutter artifacts in these two examples are primarily caused by
panning the camera.
In each video demonstration, the original video is on the left and the rectified
video is on the right.
Video #1 -
In the third video example, we compare the results of video stabilization
with and without rolling shutter artifact rectification.
The rolling shutter artifacts in this example is caused by jitter of a
walking man's hands.
Original video is on the top.
Bottom left is the stabilized video without rolling shutter artifact
rectification, and bottom right is the stabilized video with rolling shutter
COPYRIGHT NOTICE: All the documents on this server
have been submitted by their authors to scholarly journals or conferences
as indicated, for the purpose of non-commercial dissemination of
The manuscripts are put on-line to facilitate this purpose.
These manuscripts are copyrighted by the authors or the journals in which
they were published.
You may copy a manuscript for scholarly, non-commercial purposes, such
as research or instruction, provided that you agree to respect these
Early Ideas for Rolling Shutter Artifact Reduction (March 7, 2012)
Last Updated 02/12/14.