15 October 2012
The toolbox is used for rolling shutter video rectification on smartphones. It estimates the camera 3D rotation using gyroscope readings and the video using extended Kalman filtering (EKF). The rotation estimate is then used to rectify the rolling shutter effects of the video. Please run the "rolling_shutter_demo.m" script to see how it works.
To rectify the rolling shutter videos, we need the video file and also the gyroscope readings while the video is recorded. We used the Android app "Sensor Data Logger" to capture the gyroscope readings while recording video. This app will save the gyroscope readings in a text file that can be directly processed by the functions in the toolbox. In addition, we need to extract the framestamps of the video. The framestamps are extracted by a C++ implementation based on OpenCV. The source file "extract_framestamp.cpp" is included in the toolbox release.
The parameters of the camera on different smartphones are different, so we need to estimate the parameters first (calibration step). Please run the "calib_demo.m" script to estimate the camera parameters if necessary.
In the toolbox, there is a example video file, together with its corresponding gyroscope readings and framestamps. The video (with gyroscope readings) is captured by Google Nexus S smartphone. The parameters of the camera are also provided. Please start to use the toolbox with this example.
The toolbox depends on two open source libraries:
This toolbox was programmed by Chao Jia. For questions on the toolbox please write to Chao Jia at cjia@utexas.edu.
Reference
C. Jia and B. L. Evans, "Probabilistic 3-D Motion Estimation for Rolling Shutter Video Rectification from Visual and Inertial Measurements", Proc. IEEE Int. Workshop on Multimedia Signal Processing, Sep. 17-20, 2012, Banff, Canada.