Signal Processing: Image Communication, vol. 61, Feb. 2018, pp. 54-72.

Perceptual Quality Evaluation of Synthetic Pictures Distorted by Compression and Transmission

Debarati Kundu (1), Lark Kwon Choi (2), Alan C. Bovik (2) and Brian L. Evans (1)

(1) Embedded Signal Processing Laboratory, Wireless Networking and Communications Group, The University of Texas at Austin, Austin, TX 78712 USA
debarati@utexas.edu - bevans@ece.utexas.edu

(2) Laboratory for Image and Video Engineering, Wireless Networking and Communications Group, The University of Texas at Austin, Austin, TX 78712 USA
larkkwonchoi@gmail.com - bovik@ece.utexas.edu

Paper

ESPL Synthetic Image Database

Abstract

Measuring visual quality, as perceived by human observers, is becoming increasingly important in a large number of applications where humans are the ultimate consumers of visual information. Many natural image databases have been developed that contain human subjective ratings of the images. Subjective quality evaluation data is less available for synthetic images, such as those commonly encountered in graphics novels, online games or internet ads. A wide variety of powerful full-reference, reduced-reference and no-reference Image Quality Assessment (IQA) algorithms have been proposed for natural images, but their performance has not been evaluated on synthetic images. In this paper we
  1. conduct a series of subjective tests on a new publicly available Embedded Signal Processing Laboratory (ESPL) Synthetic Image Database, which contains 500 distorted images (20 distorted images for each of the 25 original images) in 1920 × 1080 resolution, and
  2. evaluate the performance of more than 50 publicly available IQA algorithms on the new database.
The synthetic images in the database were processed by post acquisition distortions, including those arising from compression and transmission. We collected 26,000 individual ratings from 64 human subjects which can be used to evaluate full-reference, reduced-reference, and no-reference IQA algorithm performance. We find that IQA models based on scene statistics models can successfully predict the perceptual quality of synthetic scenes. The database is available at http://signal.ece.utexas.edu/%7Ebevans/synthetic/.

Expected Contributions

What is the contribution of this paper to the image processing community (a couple of sentences)?

In this paper, the authors have developed a synthetic image database containing different distortions and conducted subjective tests to gauge the perceptual quality of the images. More than 50 state-of-the-art full-reference, reduced-reference and no-reference image quality assessment algorithms have been evaluated on this database and correlated against the subjective test scores.

Why is this contribution significant (What impact will it have)?

The number of synthetic image databases with subjective ratings is relatively less compared to those available for natural images. This works aims to fill in that gap and will enable researchers to evaluate the performance of image quality assessment algorithms on synthetic images.

What are the three papers in the published literature most closely related to this paper?

What is distinctive/new about the current paper relative to these previously published works?

The first and the second references are popular natural image quality assessment databases, but our work emphasizes how users perceive distortions in synthetic images. The third reference is a paper on synthetic image quality assessment, but compared to the database used in this paper, our database deals with a wider class of distortions, especially transmission artifacts such as those arising from JPEG compression and transmission over an wireless channel, which has not been studied before for computer graphics generated images.

COPYRIGHT NOTICE: All the documents on this server have been submitted by their authors to scholarly journals or conferences as indicated, for the purpose of non-commercial dissemination of scientific work. The manuscripts are put on-line to facilitate this purpose. These manuscripts are copyrighted by the authors or the journals in which they were published. You may copy a manuscript for scholarly, non-commercial purposes, such as research or instruction, provided that you agree to respect these copyrights.

Last Updated 11/29/17.