IEEE Transactions on Signal Processing, vol. 62, no. 13, pp. 3293-3304, Jul. 1, 2014.

Constrained 3D Rotation Smoothing via Global Manifold Regression for Video Stabilization

Chao Jia and Brian L. Evans

Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX 78712 USA
kurtjc@gmail.com - bevans@ece.utexas.edu

Paper Draft - Software Release - Project Site

Videos: Urban Street - Apartment Complex

Abstract

We present a novel motion smoothing algorithm for hand-held cameras with application to video stabilization. Video stabilization seeks to remove unwanted frame-to-frame jitter due to camera shake. For video stabilization, we use a pure 3D rotation motion model with known camera projection parameters. The 3D camera rotation can be reliably tracked by a gyroscope as commonly found on a smart phone or tablet. In this paper, we directly smooth the sequence of camera rotation matrices for the video frames by exploiting the Riemannian geometry on a manifold. Our contributions are
  1. formulation of motion smoothing as a geodesic-convex constrained regression problem on a non-linear manifold based on geodesic distance,
  2. computation of gradient and Hessian of the objective function using Riemannian geometry for gradient-related manifold optimization, and
  3. generalization of the two-metric projection algorithm in Euclidean space to manifolds to solve the proposed manifold optimization problem efficiently.
The geodesic-distance-based smoothness metric better exploits the manifold structure of sequences of rotation matrices. The geodesic-convex constraints effectively guarantee that no black borders intrude into the stabilized frames. The proposed manifold optimization algorithm can find the global optimal solution in only a few iterations. Experimental results show that video stabilization based on our motion smoothing algorithm outperforms state-of-the-art methods by generating videos with less jitter and without black borders.

COPYRIGHT NOTICE: All the documents on this server have been submitted by their authors to scholarly journals or conferences as indicated, for the purpose of non-commercial dissemination of scientific work. The manuscripts are put on-line to facilitate this purpose. These manuscripts are copyrighted by the authors or the journals in which they were published. You may copy a manuscript for scholarly, non-commercial purposes, such as research or instruction, provided that you agree to respect these copyrights.

Last Updated 06/22/14.