EI2019 Short Course Description
SC06: Perceptual Metrics for Image and Video Quality in a Broader Context
Sunday 13 January • 8:00 am – 12:15 pm
Course Length: 4 hours
Course Level: Intermediate (Prerequisites: Basic understanding of image compression algorithms; background in digital signal processing and basic statistics: frequency-based representations, filtering, distributions.)
Instructors: Thrasyvoulos N. Pappas, Northwestern University, and Sheila S. Hemami, Draper
Fee*: Member: $290 /
Non-member: $315 /
Student: $95
*after December 18, 2018, members / non-members prices increase by $50, student price increases by $20
The course examines objective criteria for the evaluation of image quality that are based on models of visual perception. The primary emphasis will be on image fidelity, i.e., how close an image is to a given original or reference image, but we will broaden the scope of image fidelity to include structural equivalence. Also discussed is no-reference and limited-reference metrics. An examination of a variety of applications with special emphasis on image and video compression is included. We examine near-threshold perceptual metrics, which explicitly account for human visual system (HVS) sensitivity to noise by estimating thresholds above which the distortion is just-noticeable, and supra-threshold metrics, which attempt to quantify visible distortions encountered in high compression applications or when there are losses due to channel conditions. The course also considers metrics for structural equivalence, whereby the original and the distorted image have visible differences but both look natural and are of equally high visual quality. This short course takes a close look at procedures for evaluating the performance of quality metrics, including database design, models for generating realistic distortions for various applications, and subjective procedures for metric development and testing. Throughout the course we discuss both the state of the art and directions for future research.
Learning Outcomes
- Gain a basic understanding of the properties of the human visual system and how current applications (image and video compression, restoration, retrieval, etc.) attempt to exploit these properties.
- Gain an operational understanding of existing perceptually-based and structural similarity metrics, the types of images/artifacts on which they work, and their failure modes.
- Understand current distortion models for different applications and how they can be used to modify or develop new metrics for specific contexts.
- Understand the differences between sub-threshold and supra-threshold artifacts, the HVS responses to these two paradigms, and the differences in measuring that response.
- Understand criteria by which to select and interpret a particular metric for a particular application.
- Understand the capabilities and limitations of full-reference, limited-reference, and no-reference metrics, and why each might be used in a particular application.
Intended Audience
Image and video compression specialists who wish to gain an understanding of how performance can be quantified. Engineers and scientists who wish to learn about objective image and video quality evaluation.
Managers who wish to gain a solid overview of image and video quality evaluation. Students who wish to pursue a career in digital image processing. Intellectual property and patent attorneys who wish to gain a more fundamental understanding of quality metrics and the underlying technologies. Government laboratory personnel who work in imaging.
Thrasyvoulos N. Pappas received SB, SM, and PhD in electrical engineering and computer science from MIT in 1979, 1982, and 1987, respectively. From 1987 until 1999, he was a member of the technical staff at Bell Laboratories, Murray Hill, NJ. He is currently a professor in the department of electrical and computer engineering at Northwestern University, which he joined in 1999. His research interests are in image and video quality and compression, image and video analysis, content-based retrieval, perceptual models for multimedia processing, model-based halftoning, and tactile and multimodal interfaces. Pappas has served as co-chair of the 2005 SPIE/IS&T Electronic Imaging (EI) Symposium, and since 1997 he has been co-chair of the EI Conference on Human Vision and Electronic Imaging. Pappas is a Fellow of IEEE and SPIE. He is currently serving as Vice President-Publications for the Signal Processing Society of IEEE. He has also served as Editor-in-Chief of the IEEE Transactions on Image Processing (2010-12), elected member of the Board of Governors of the Signal Processing Society of IEEE (2004-06), chair of the IEEE Image and Multidimensional Signal Processing (now IVMSP) Technical Committee, and technical program co-chair of ICIP-01 and ICIP-09.
Sheila S. Hemami received a BSEE from the University of Michigan (1990), MSEE and PhD from Stanford University (1992 and 1994). She was most recently at Northeastern University as professor and chair of the electrical engineering and computer science department at the College of Engineering; with Hewlett-Packard Laboratories in Palo Alto, California in 1994; and with the School of Electrical Engineering at Cornell University from 1995-2013. She is currently Director, Strategic Technical Opportunities, at Draper, Cambridge, MA. Her research interests broadly concern communication of visual information from the perspectives of both signal processing and psychophysics. She was elected a Fellow of the IEEE in 2009 for contributions to robust and perceptual image and video communications. Hemami has held various visiting positions, most recently at the University of Nantes, France and at Ecole Polytechnique Fédérale de Lausanne, Switzerland. She has received numerous university and national teaching awards, including Eta Kappa Nu's C. Holmes MacDonald Award. She was a Distinguished Lecturer for the IEEE Signal Processing Society in 2010-2011, was editor-in-chief for the IEEE Transactions on Multimedia from 2008-2010. She has held various technical leadership positions in the IEEE.
Related EI Conferences