*Proc. Asilomar Conference on Signals, Systems and Computers*,
Nov. 2-5, 2014, pp. 673-677, Pacific Grove, CA.
# Real-Time 3D Rotation Smoothing for Video Stabilization

*Chao Jia*,
*Zeina Sinno* and
*Brian L. Evans*
Department of Electrical
and Computer Engineering,
Wireless Networking and Communications Group,
The University of Texas at Austin,
Austin, TX 78712 USA

kurtjc@gmail.com -
zeina@utexas.edu -
bevans@ece.utexas.edu

Paper(1 MB) -
PDF Slides (1 MB) -
PowerPoint Slides (160 MB) -
Project Site

Videos:

Figure 1:
(a) Original Video -
(b) Proposed IIR-like smoothing -
(c) Proposed UKF-like smoothing

Figure 2:
(a) Original Video -
(b) Proposed IIR-like smoothing -
(c) Proposed UKF-like smoothing

Original videos are 480 x 720 and stablized videos are 360 x 540

## Abstract

We propose two real-time motion smoothing algorithms for video
stabilization using a pure 3D rotation motion model with known
camera projection parameters.
Both proposed algorithms aim at smoothing 3D rotation matrix
sequences in a causal way.
The first algorithm smooths the 3D rotation sequences in a way
similar to 1st-order IIR filtering.
The second algorithm uses sequential probabilistic estimation
under a constant angular velocity model.
These two algorithms are generalized from classical 2D motion
smoothing algorithms.
We exploit the manifold structure of the rotation matrices so
that the proposed algorithms directly smooth the 3D rotation
sequences on the manifold.
In addition, we introduce a simple projection step in order to
guarantee that no black borders intrude into the stabilized video
frames.
Experimental results show that the proposed algorithms
are able to effectively stabilize video sequences and outperform
their 2D counterparts with less jitter and distortion.
## Questions and Answers

*Why do you consider the methods presented real time?*

Answer: The first method, the IIR-based, executes a frame every
1.57 ms while the second one, the KF based, takes 6.97 ms.
Given that the acquisition is done with a frame rate below
30 frames/s, the delay for the first method would be less than
50 ms and for the second one less than 250 ms.
As a result we can consider the methods to be online.

*The presented videos are subject to very high distortions.
Are the videos always distorted as such?
If you have less distortion would you expect good results?*

Answer: Handheld devices are used by a wide variety of users, so
we would expect a wide range of distortions.
We would expect such distorted videos under some conditions,
e.g if the road is not smooth.
For a proof-of-concept, we present highly distorted cases.
We would expect our algorithm to perform as well when we have
less distortions.

*What is the frame size of the new video?*

Answer: 540x360. Originally it was 720x480 in the two provided examples.

COPYRIGHT NOTICE: All the documents on this server have been submitted
by their authors to scholarly journals or conferences as indicated,
for the purpose of non-commercial dissemination of scientific work.
The manuscripts are put on-line to facilitate this purpose.
These manuscripts are copyrighted by the authors or the journals in
which they were published.
You may copy a manuscript for scholarly, non-commercial purposes,
such as research or instruction, provided that you agree to respect
these copyrights.

Last updated 01/31/16.