IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2957-2971, Jun. 2017, DOI 10.1109/TIP.2017.2685941.

No-Reference Quality Assessment of Tone-Mapped HDR Pictures

Debarati Kundu (1), Deepti Ghadiyaram (2), Alan C. Bovik (2) and Brian L. Evans (1)

(1) Embedded Signal Processing Laboratory, Wireless Networking and Communications Group, The University of Texas at Austin, Austin, TX 78712 USA
debarati@utexas.edu - bevans@ece.utexas.edu

(2) Laboratory for Image and Video Engineering, Wireless Networking and Communications Group, The University of Texas at Austin, Austin, TX 78712 USA
deepti@cs.utexas.edu - bovik@ece.utexas.edu

Paper Draft

Related Resources: ESPL-LIVE HDR Image Database - Crowdsourced HDR IQA Study - PhD Dissertation on HDR IQA

Abstract

Being able to automatically predict digital picture quality, as perceived by human observers, has become important in many applications where humans are the ultimate consumers of displayed visual information. Standard dynamic range (SDR) images provide 8 bits/color/pixel. High dynamic range (HDR) images, which are usually created from multiple exposures of the same scene, can provide 16 or 32 bits/color/pixel, but must be tonemapped to SDR for display on standard monitors. Multi-exposure fusion (MEF) techniques bypass HDR creation, by fusing the exposure stack directly to SDR format while aiming for aesthetically pleasing luminance and color distributions. Here we describe a new no-reference image quality assessment (NR IQA) model for HDR pictures that is based on standard measurements of the bandpass and on newly conceived differential natural scene statistics (NSS) of HDR pictures. We derive an algorithm from the model which we call the HDR Image GRADient based Evaluator (HIGRADE). NSS models have previously been used to devise NR IQA models that effectively predict the subjective quality of SDR images, but they perform significantly worse on tonemapped HDR content. Towards ameliorating this we make here the following contributions:
  1. We design HDR picture NR IQA models and algorithms using both standard space-domain NSS features as well as novel HDR-specific gradient based features that significantly elevate prediction performance,
  2. We validate the proposed models on a large-scale crowdsourced HDR image database, and
  3. We demonstrate that the proposed models also perform well on legacy natural SDR images.
The software is available at: http://live.ece.utexas.edu/research/Quality/higradeRelease.zip.

Expected Contributions

What is the contribution of this paper to the image processing community?

The primary contributions of this paper are:

  1. No Reference Image Quality Assessment model and algorithm for High Dynamic Range (HDR) pictures using features extracted in the spatial and gradient domains,
  2. validation of the proposed method against existing approaches on a large­scale crowdsourced HDR image database, and
  3. demonstration that the proposed method performs well even on legacy natural standard dynamic range images.

Why is this contribution significant (What impact will it have)?

There is a growing trend of acquiring/creating and displaying high dynamic range (HDR) images that are typically obtained by blending a stack of standard dynamic range (SDR) images at varying exposure levels. Since different fusion/tone­mapping algorithms result in different SDR images, a natural question is how to evaluate the quality of the images obtained. Subjective testing is important for evaluating the visual quality of images produced by different algorithms, but these are time­consuming and expensive. A highly desirable goal is to design objective quality prediction models that automate the process of Image Quality Assessment (IQA). Hence, this contribution is significant.

What is distinctive/new about the current paper relative to these previously published works?

To date, the IQA algorithms developed in HDR images have been full­reference ones. However, when evaluating the quality of images created by HDR processing algorithms, it is hard to assume a ‘reference’ image, since the input to the algorithms is an exposure stack that may have a varying number of images of unknown qualities based on the camera settings used. In these applications, no­reference IQA is the only realistic option. In this paper, we develop NR­IQA algorithms using Natural Scene Statistics (NSS) models. NSS based algorithms have been found to do remarkably well in evaluating the quality of standard dynamic range (SDR) images. To the best of our knowledge, NSS models have not been used by other authors to create IQA models for NR evaluation of HDR image quality. Towards filling this gap, we propose two NSS based NR IQA models that deliver good predictive performance when evaluating quality of HDR­processed images. We also show that the proposed methods show high correlations with human judgment even for legacy SDR images.

What are the three papers in the published literature most closely related to this paper?

  1. Y. Zhang and D. M. Chandler, “No­reference image quality assessment based on log­derivative statistics of natural scenes,” J Electronic Imaging, vol. 22, no. 4, 2013.
  2. A. Mittal, A. K. Moorthy, and A. C. Bovik, “No­reference image quality assessment in the spatial domain,” IEEE Trans. Image Process., vol. 21, no. 12, pp. 4695–4708, Dec 2012.
  3. K. Ma, K. Zeng and Z. Wang, "Perceptual Quality Assessment for Multi­Exposure Image Fusion," IEEE Trans. Image Process., vol. 24, no. 11, pp. 3345­3356, Nov. 2015.

COPYRIGHT NOTICE: All the documents on this server have been submitted by their authors to scholarly journals or conferences as indicated, for the purpose of non-commercial dissemination of scientific work. The manuscripts are put on-line to facilitate this purpose. These manuscripts are copyrighted by the authors or the journals in which they were published. You may copy a manuscript for scholarly, non-commercial purposes, such as research or instruction, provided that you agree to respect these copyrights.

Last Updated 03/20/17.