Color Imaging XXVIII: Displaying, Processing, Hardcopy, and Applications
Monday 16 January 2023
10:20 – 10:50 AM Coffee Break
Vision 1 (M2)
Session Chair:
Reiner Eschbach, Norwegian University of Science and Technology (Norway) and Monroe Community College (United States)
11:05 AM – 12:30 PM
Mission II/III
11:05
Conference Welcome
11:10COLOR-183
Pseudocolor visualizations of light patterns on retinal receptors after glare (Invited), John J. McCann, McCann Imaging (United States) [view abstract]
Glare introduces a complex scene-depended transformation of the array of “All Scene Luminances” making a different spatial light pattern on all receptors, called “Retinal Contrast”. The convolution of “All Scene Luminances” with Vos and van den Berg’s CIE 1999 Glare Spread Function calculates high-resolution arrays of “Retinal Contrasts”. The results show that uniform-luminance scene segments become low-slope gradients that are nearly invisible, or invisible. Visual inspection of these arrays is misleading. Pseudocolor Look-up Tables (LUT)s are very helpful in visualizing the complexity of glare’s spatial transformation that controls the amount of light falling on rods and cones. This article studies Lightness Illusions that contain two identical scene-luminance segments. Following receptor responses, neural spatial processes generate a second spatial-image transformation that leads to appearances. Contrast, Assimilation, and Natural Scene Illusions demonstrate [Appearance ≠ scene luminance]. Analysis of Illusion’s patterns of light on receptors shows that: Contrast Illusions, Edwin Land’s B&W Mondrian, Adelson’s Checkershadow all exhibit Glare’s Paradox. Namely, that vision’s second neural transformation overcompensates the effects of glare. Illusions’ Grays appear darker despite large amounts of glare light on receptors. Grays that appear lighter have smaller amounts of glare.
11:50COLOR-184
Color blindness and modern board games, Alessandro Rizzi1 and Matteo Sassi2; 1Università degli Studi di Milano and 2consultant (Italy) [view abstract]
Board game industry is experiencing a strong renewed interest. In the last few years, about 4000 new board games have been designed and distributed each year. Board game players gender balance is reaching the equality, but nowadays the male component is a slight majority. This means that (at least) around 10% of board game players are color blind. How does the board game industry deal with this ? Recently, a raising of awareness in the board game design has started but so far there is a big gap compared with (e.g.) the computer game industry. This paper presents some data about the actual situation, discussing exemplary cases of successful board games.
12:10COLOR-185
Testing the role of vision spatial processing in color deficiency, Alice Plutino1, Reiner Eschbach2, Luca Armellin1, Andrea Mazzoni3, Roberta Marcucci3, and Alessandro Rizzi1; 1Università degli Studi di Milano (Italy), 2Norwegian University of Science and Technology (Norway) and Monroe Community College (United States), and 3Aerospace Medical Institute - Italian Airforce (Italy) [view abstract]
In the last 80 years, the role of spatial processing in the visual system has been analyzed and demonstrated from many studies and experiments. Starting from the first studies of Young, Helmholtz and Hering, color vision models have developed, and several biological and physiological research paper proved the importance of spatial processing in color vision. In this paper, we present some of the studies which have explored the role of spatial processing to study color vision deficiency. Main scope of this work is to increase the awareness of the scientific community on the importance to include spatial processing not only in color vision models, but also in developing color deficiency aids and tests.
12:30 – 2:00 PM Lunch
Monday 16 January PLENARY: Neural Operators for Solving PDEs
Session Chair: Robin Jenkin, NVIDIA Corporation (United States)
2:00 PM – 3:00 PM
Cyril Magnin I/II/III
Deep learning surrogate models have shown promise in modeling complex physical phenomena such as fluid flows, molecular dynamics, and material properties. However, standard neural networks assume finite-dimensional inputs and outputs, and hence, cannot withstand a change in resolution or discretization between training and testing. We introduce Fourier neural operators that can learn operators, which are mappings between infinite dimensional spaces. They are independent of the resolution or grid of training data and allow for zero-shot generalization to higher resolution evaluations. When applied to weather forecasting, neural operators capture fine-scale phenomena and have similar skill as gold-standard numerical weather models for predictions up to a week or longer, while being 4-5 orders of magnitude faster.
Anima Anandkumar, Bren professor, California Institute of Technology, and senior director of AI Research, NVIDIA Corporation (United States)
Anima Anandkumar is a Bren Professor at Caltech and Senior Director of AI Research at NVIDIA. She is passionate about designing principled AI algorithms and applying them to interdisciplinary domains. She has received several honors such as the IEEE fellowship, Alfred. P. Sloan Fellowship, NSF Career Award, and Faculty Fellowships from Microsoft, Google, Facebook, and Adobe. She is part of the World Economic Forum's Expert Network. Anandkumar received her BTech from Indian Institute of Technology Madras, her PhD from Cornell University, and did her postdoctoral research at MIT and assistant professorship at University of California Irvine.
3:00 – 3:30 PM Coffee Break
Vision 2 (M3)
Session Chair:
John McCann, McCann Imaging (United States)
3:30 – 5:10 PM
Mission II/III
3:30COLOR-186
Heterochromatic brightness matching experiments to evaluate brightness prediction model including Helmholtz-Kohlrausch effect, Garam Seong1, Youngshin Kwak1, and Hyosun Kim2; 1Ulsan National Institute of Science and Technology and 2Samsung Display Co., Ltd (Republic of Korea) [view abstract]
In this study, the brightness matching experiment was conducted to obtain the equivalent luminance between chromatic and achromatic colors. Observers adjusted the luminance of achromatic colors until achromatic colors were perceived as having the same brightness with chromatic colors. A total of 285 chromatic colors having three different luminance levels, 30cd/m<sup>2</sup>, 95cd/m<sup>2</sup>, and 300cd/m<sup>2</sup> were used as the test colors. Twenty observers participated in this experiment repeating three times. The results showed that the brightness-to-luminance (B/L) ratio, where brightness means the luminance of achromatic color, increases as CIE 1976 saturation increases in all luminance levels indicating the Helmholtz-Kohlrausch effect. Also, as the luminance level of chromatic color increases, B/L ratio decreases. It is found that the existing color appearance models predicting the H-K effect overestimate the brightness increment by chroma compared to our new heterochromatic brightness matching data set.
3:50COLOR-187
HyperspectrACE: A human vision inspired hyperspectral color and contrast adjustment, Beatrice Sarti, Alice Plutino, and Alessandro Rizzi, Università degli Studi di Milano (Italy) [view abstract]
The Automatic Color Equalization (ACE) algorithm, part of the Retinex-derived family of Spatial Color Algorithms (SCAs), is an image enhancement algorithm that mimic the adjustment behavior of the human vision system. It is commonly used on a unique channel for black\&white images or independently on the three channels for color images. In this work, we introduce a novel application of ACE on hyperspectral images, that we called HyperspectrACE. Here the goal is not to introduce the most performing hyperspectral image enhancer in the field, but to discuss the performance of a qualitative model of human color sensation when applied on more than the standard RGB channels. For this reason, we present the test of the proposed approach compared with classic ACE and other classic methods, to assess the differences in image dynamic stretching and global and local filtering contrast adjustment.
4:10COLOR-188
Spatiochromatic natural image statistics modelling: Applications from display analysis to neural networks, Scott Daly, Timo Kunkel, Guan-Ming Su, and Anustup Choudhury, Dolby Laboratories, Inc. (United States) [view abstract]
Natural image statistics are well known to have a spatial frequency power spectra that has a 1/f^a behavior, with a typically stated as between 2 and 4. This indicates an invariance to scale. Further work has theorized how the visual system is tuned for such statistics in visual cortex (V1) [1]. Color image statistics also show an invariance to scale [2]. The luminance histogram is typically understood to be log normal with respect to luminance, although for HDR images, a subcomponent with skew toward much higher luminances is observed. Color statistics were initially described at the simplest level via the gray world hypothesis [3], but more details are now available, even at the hyperspectral [4]. The a power function for HDR was found to increase from the lower values of 2 to more typical values of 4 and 5 [5]. For temporal statistics, the data tends to be measured primarily for media, with a 1/f^a for scene cut statistics [6], and temporal frequency and temporal frequency for media with a focus on the motion statistics via optical flow [7]. Statistics for purely natural as well as human made environments (e.g., buildings and the resulting perspective geometry) have been studied, each having different orientation statistics [8]. The use of image statistics for standardized assessment of television power consumption was used to replace test targets, which were often detected and used to lower TV power consumption in well known cheating schemes. To prevent this, a short test video that had luminance statistics matching 48 hours of broadcast content was generated and used for TV power testing [9]. The highly adaptative nature of current TVs (power limiting, dual modulation, dynamic response) has motivated researchers to incorporate complex noise fields following natural image statistics into measurement targets [10,11]. One particular natural image statistic-based still image test target (dead leaves) is widely used in camera optics and sensor development. Algorithm development and testing for image and video processing has almost always been ad hoc, with a mixture of geometric test targets and hand selected test images, sometimes aiming to be corner cases, sometimes not. More recently, large data sets of images have been used to train various neural network models for tasks such as super resolution, bit rate compression, and dynamic range mapping. However, images are not ergodic, and possibly not even wide-sense stationary. We propose the use of imagery based on noise following the natural image statistics for spatio-chromatic (and temporal) to compactly probe the wide variety of image possibilities for algorithmic development, in addition to the existing uses for image capture and display analysis. While we don’t suggest replacing actual practical imagery, we believe such noise fields can augment image algorithm analysis. To address the problem of non-ergodicity, we allow the basic power term a in the natural image statistic model to vary over a large range in a video, such that it includes the extremes of white noise and low frequency gradients. We use color image statistic models that include decorrelated colors to generate the RGB video. We will present results for traditional adaptive data compression (with chromatic subsampling), as well as a more contemporary neural network approach (Neural Fields [12]) as applied to upscaling and denoising. We analyze the results both visually and through several recent color image quality models. Field DJ. Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A, 1987; 4:2379-2394 C. Parraga, T. Troscianko, and D.J. Tolhurst (2002) spatiochromatic properties of natural images and human vision. Current Biology V 12 R. M. Evans, Method for correcting photographic color prints, US Patent 2,571,697 (1951) A. Chakrabarti and T. Zickler (2011) Statistics of real-world hyperspectral images CVPR R. Dror, A. Willsky, and E. Adelson (2004) statistical characterization of real-world illumination. JOV V4 J. Cutting (2019) Sequences in popular cinema generate inconsistent event segmentation. Attn. Percept. And Psycho. V 81. D. Lee, H. Ko, J. Kim, and A. Bovik (2021) On the space-time statistics of motion pictures. JOSA A V 38 #7 A. Torralba and A. Oliva (2003) Statistics of natural image categories, Network: Computational Neural Systems 14 391-412 International Electrotechnical Commission, IEC 62087:2008(E), “Methods of measurement for the power consumption of audio, video, and related Equipment. Kunkel T, Daly S. 57-1: Spatiotemporal Noise Targets Inspired by Natural Imagery Statistics. SID Symposium Digest of Technical Papers, 2020, 51:842-845. Kunkel, T, Friedrich, F. Utilizing advanced spatio-temporal backgrounds with dynamic test signals for high dynamic range display metrology. J Soc Inf Display. 2022; 30( 5): 423– 432. https://doi.org/10.1002/jsid.1125 Yiheng Xie1, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, Srinath Sridhar1, "Neural Fields in Visual Computing and Beyond", Eurographics / CGF State-of-the-Art Report, 2022.
4:30COLOR-189
Lessons from research in color science on the bleeding edge (Invited), Giordano B. Beretta, consultant (United States) [view abstract]
The end of life is a good time to look back to what I have learned in the past 50 years and share my lessons. Except for stints in engineering and marketing, I have worked mostly in research labs. Although I was mostly in an imaging lab, de facto my research has been primarily in color science. I have worked in industry, but on the side I have volunteered for national science foundations, learned societies, and patent offices. The main take-away is that life in research is not smooth, you have to be resilient to set-backs, and be well connected.
EI 2023 Highlights Session
Session Chair: Robin Jenkin, NVIDIA Corporation (United States)
3:30 – 5:00 PM
Cyril Magnin II
Join us for a session that celebrates the breadth of what EI has to offer with short papers selected from EI conferences.
NOTE: The EI-wide "EI 2023 Highlights" session is concurrent with Monday afternoon COIMG, COLOR, IMAGE, and IQSP conference sessions.
IQSP-309
Evaluation of image quality metrics designed for DRI tasks with automotive cameras, Valentine Klein, Yiqi LI, Claudio Greco, Laurent Chanas, and Frédéric Guichard, DXOMARK (France) [view abstract]
Driving assistance is increasingly used in new car models. Most driving assistance systems are based on automotive cameras and computer vision. Computer Vision, regardless of the underlying algorithms and technology, requires the images to have good image quality, defined according to the task. This notion of good image quality is still to be defined in the case of computer vision as it has very different criteria than human vision: humans have a better contrast detection ability than image chains. The aim of this article is to compare three different metrics designed for detection of objects with computer vision: the Contrast Detection Probability (CDP) [1, 2, 3, 4], the Contrast Signal to Noise Ratio (CSNR) [5] and the Frequency of Correct Resolution (FCR) [6]. For this purpose, the computer vision task of reading the characters on a license plate will be used as a benchmark. The objective is to check the correlation between the objective metric and the ability of a neural network to perform this task. Thus, a protocol to test these metrics and compare them to the output of the neural network has been designed and the pros and cons of each of these three metrics have been noted.
SD&A-224
Human performance using stereo 3D in a helmet mounted display and association with individual stereo acuity, Bonnie Posselt, RAF Centre of Aviation Medicine (United Kingdom) [view abstract]
Binocular Helmet Mounted Displays (HMDs) are a critical part of the aircraft system, allowing information to be presented to the aviator with stereoscopic 3D (S3D) depth, potentially enhancing situational awareness and improving performance. The utility of S3D in an HMD may be linked to an individual’s ability to perceive changes in binocular disparity (stereo acuity). Though minimum stereo acuity standards exist for most military aviators, current test methods may be unable to characterise this relationship. This presentation will investigate the effect of S3D on performance when used in a warning alert displayed in an HMD. Furthermore, any effect on performance, ocular symptoms, and cognitive workload shall be evaluated in regard to individual stereo acuity measured with a variety of paper-based and digital stereo tests.
IMAGE-281
Smartphone-enabled point-of-care blood hemoglobin testing with color accuracy-assisted spectral learning, Sang Mok Park1, Yuhyun Ji1, Semin Kwon1, Andrew R. O’Brien2, Ying Wang2, and Young L. Kim1; 1Purdue University and 2Indiana University School of Medicine (United States) [view abstract]
We develop an mHealth technology for noninvasively measuring blood Hgb levels in patients with sickle cell anemia, using the photos of peripheral tissue acquired by the built-in camera of a smartphone. As an easily accessible sensing site, the inner eyelid (i.e., palpebral conjunctiva) is used because of the relatively uniform microvasculature and the absence of skin pigments. Color correction (color reproduction) and spectral learning (spectral super-resolution spectroscopy) algorithms are integrated for accurate and precise mHealth blood Hgb testing. First, color correction using a color reference chart with multiple color patches extracts absolute color information of the inner eyelid, compensating for smartphone models, ambient light conditions, and data formats during photo acquisition. Second, spectral learning virtually transforms the smartphone camera into a hyperspectral imaging system, mathematically reconstructing high-resolution spectra from color-corrected eyelid images. Third, color correction and spectral learning algorithms are combined with a spectroscopic model for blood Hgb quantification among sickle cell patients. Importantly, single-shot photo acquisition of the inner eyelid using the color reference chart allows straightforward, real-time, and instantaneous reading of blood Hgb levels. Overall, our mHealth blood Hgb tests could potentially be scalable, robust, and sustainable in resource-limited and homecare settings.
AVM-118
Designing scenes to quantify the performance of automotive perception systems, Zhenyi Liu1, Devesh Shah2, Alireza Rahimpour2, Joyce Farrell1, and Brian Wandell1; 1Stanford University and 2Ford Motor Company (United States) [view abstract]
We implemented an end-to-end simulation for perception systems, based on cameras, that are used in automotive applications. The open-source software creates complex driving scenes and simulates cameras that acquire images of these scenes. The camera images are then used by a neural network in the perception system to identify the locations of scene objects, providing the results as input to the decision system. In this paper, we design collections of test scenes that can be used to quantify the perception system’s performance under a range of (a) environmental conditions (object distance, occlusion ratio, lighting levels), and (b) camera parameters (pixel size, lens type, color filter array). We are designing scene collections to analyze performance for detecting vehicles, traffic signs and vulnerable road users in a range of environmental conditions and for a range of camera parameters. With experience, such scene collections may serve a role similar to that of standardized test targets that are used to quantify camera image quality (e.g., acuity, color).
VDA-403
Visualizing and monitoring the process of injection molding, Christian A. Steinparz1, Thomas Mitterlehner2, Bernhard Praher2, Klaus Straka1,2, Holger Stitz1,3, and Marc Streit1,3; 1Johannes Kepler University, 2Moldsonics GmbH, and 3datavisyn GmbH (Austria) [view abstract]
In injection molding machines the molds are rarely equipped with sensor systems. The availability of non-invasive ultrasound-based in-mold sensors provides better means for guiding operators of injection molding machines throughout the production process. However, existing visualizations are mostly limited to plots of temperature and pressure over time. In this work, we present the result of a design study created in collaboration with domain experts. The resulting prototypical application uses real-world data taken from live ultrasound sensor measurements for injection molding cavities captured over multiple cycles during the injection process. Our contribution includes a definition of tasks for setting up and monitoring the machines during the process, and the corresponding web-based visual analysis tool addressing these tasks. The interface consists of a multi-view display with various levels of data aggregation that is updated live for newly streamed data of ongoing injection cycles.
COIMG-155
Commissioning the James Webb Space Telescope, Joseph M. Howard, NASA Goddard Space Flight Center (United States) [view abstract]
Astronomy is arguably in a golden age, where current and future NASA space telescopes are expected to contribute to this rapid growth in understanding of our universe. The most recent addition to our space-based telescopes dedicated to astronomy and astrophysics is the James Webb Space Telescope (JWST), which launched on 25 December 2021. This talk will discuss the first six months in space for JWST, which were spent commissioning the observatory with many deployments, alignments, and system and instrumentation checks. These engineering activities help verify the proper working of the telescope prior to commencing full science operations. For the session: Computational Imaging using Fourier Ptychography and Phase Retrieval.
HVEI-223
Critical flicker frequency (CFF) at high luminance levels, Alexandre Chapiro1, Nathan Matsuda1, Maliha Ashraf2, and Rafal Mantiuk3; 1Meta (United States), 2University of Liverpool (United Kingdom), and 3University of Cambridge (United Kingdom) [view abstract]
The critical flicker fusion (CFF) is the frequency of changes at which a temporally periodic light will begin to appear completely steady to an observer. This value is affected by several visual factors, such as the luminance of the stimulus or its location on the retina. With new high dynamic range (HDR) displays, operating at higher luminance levels, and virtual reality (VR) displays, presenting at wide fields-of-view, the effective CFF may change significantly from values expected for traditional presentation. In this work we use a prototype HDR VR display capable of luminances up to 20,000 cd/m^2 to gather a novel set of CFF measurements for never before examined levels of luminance, eccentricity, and size. Our data is useful to study the temporal behavior of the visual system at high luminance levels, as well as setting useful thresholds for display engineering.
HPCI-228
Physics guided machine learning for image-based material decomposition of tissues from simulated breast models with calcifications, Muralikrishnan Gopalakrishnan Meena1, Amir K. Ziabari1, Singanallur Venkatakrishnan1, Isaac R. Lyngaas1, Matthew R. Norman1, Balint Joo1, Thomas L. Beck1, Charles A. Bouman2, Anuj Kapadia1, and Xiao Wang1; 1Oak Ridge National Laboratory and 2Purdue University (United States) [view abstract]
Material decomposition of Computed Tomography (CT) scans using projection-based approaches, while highly accurate, poses a challenge for medical imaging researchers and clinicians due to limited or no access to projection data. We introduce a deep learning image-based material decomposition method guided by physics and requiring no access to projection data. The method is demonstrated to decompose tissues from simulated dual-energy X-ray CT scans of virtual human phantoms containing four materials - adipose, fibroglandular, calcification, and air. The method uses a hybrid unsupervised and supervised learning technique to tackle the material decomposition problem. We take advantage of the unique X-ray absorption rate of calcium compared to body tissues to perform a preliminary segmentation of calcification from the images using unsupervised learning. We then perform supervised material decomposition using a deep learned UNET model which is trained using GPUs in the high-performant systems at the Oak Ridge Leadership Computing Facility. The method is demonstrated on simulated breast models to decompose calcification, adipose, fibroglandular, and air.
3DIA-104
Layered view synthesis for general images, Loïc Dehan, Wiebe Van Ranst, and Patrick Vandewalle, Katholieke University Leuven (Belgium) [view abstract]
We describe a novel method for monocular view synthesis. The goal of our work is to create a visually pleasing set of horizontally spaced views based on a single image. This can be applied in view synthesis for virtual reality and glasses-free 3D displays. Previous methods produce realistic results on images that show a clear distinction between a foreground object and the background. We aim to create novel views in more general, crowded scenes in which there is no clear distinction. Our main contributions are a computationally efficient method for realistic occlusion inpainting and blending, especially in complex scenes. Our method can be effectively applied to any image, which is shown both qualitatively and quantitatively on a large dataset of stereo images. Our method performs natural disocclusion inpainting and maintains the shape and edge quality of foreground objects.
ISS-329
A self-powered asynchronous image sensor with independent in-pixel harvesting and sensing operations, Ruben Gomez-Merchan, Juan Antonio Leñero-Bardallo, and Ángel Rodríguez-Vázquez, University of Seville (Spain) [view abstract]
A new self-powered asynchronous sensor with a novel pixel architecture is presented. Pixels are autonomous and can harvest or sense energy independently. During the image acquisition, pixels toggle to a harvesting operation mode once they have sensed their local illumination level. With the proposed pixel architecture, most illuminated pixels provide an early contribution to power the sensor, while low illuminated ones spend more time sensing their local illumination. Thus, the equivalent frame rate is higher than the offered by conventional self-powered sensors that harvest and sense illumination in independient phases. The proposed sensor uses a Time-to-First-Spike readout that allows trading between image quality and data and bandwidth consumption. The sensor has HDR operation with a dynamic range of 80 dB. Pixel power consumption is only 70 pW. In the article, we describe the sensor’s and pixel’s architectures in detail. Experimental results are provided and discussed. Sensor specifications are benchmarked against the art.
COLOR-184
Color blindness and modern board games, Alessandro Rizzi1 and Matteo Sassi2; 1Università degli Studi di Milano and 2consultant (Italy) [view abstract]
Board game industry is experiencing a strong renewed interest. In the last few years, about 4000 new board games have been designed and distributed each year. Board game players gender balance is reaching the equality, but nowadays the male component is a slight majority. This means that (at least) around 10% of board game players are color blind. How does the board game industry deal with this ? Recently, a raising of awareness in the board game design has started but so far there is a big gap compared with (e.g.) the computer game industry. This paper presents some data about the actual situation, discussing exemplary cases of successful board games.
5:00 – 6:15 PM EI 2023 All-Conference Welcome Reception (in the Cyril Magnin Foyer)
Tuesday 17 January 2023
Applications 1 (T1)
Session Chair:
John McCann, McCann Imaging (United States)
9:10 – 10:30 AM
Mission II/III
9:10COLOR-190
Influence of fluorescence on the color prediction of translucent samples of dental resin composites, Vincent Duveiller1, Raphael Clerc1, Anthony Cazier1, Jean-Pierre Salomon2,3,4, and Mathieu Hebert1; 1University Jean Monnet Saint-Etienne (France), 2Faculté d'Odontologie de Nancy (France), 3Institut de Science des Matériaux de Mulhouse IMR 7361 CNRS (France), and 4Oregon Health and Science University (United States) [view abstract]
The fluorescence property of human teeth under UV light has long been studied in dentistry and is now used in the diagnosis of anomalies, such as dental decays. Its role in the appearance of teeth and dental restorations has also been demonstrated, and fluorescence, even under daylight, may sensibly modify the color of dental restorations. As such, dental resin composites used in aesthetic restorative dentistry include fluorescent agents which aim to reproduce the natural fluorescence of teeth. While several studies have measured the fluorescence properties of dental biomaterials and a few other studies have focused on predicting the color of samples, the influence of fluorescence on color prediction models remains to be assessed. In this paper, we propose a prediction model for the spectral emission of slices of a dental biomaterial as a function of their thicknesses, in reflection and in transmission modes, in order to improve color prediction models for these materials.
9:30COLOR-191
Can image cues explain the impact of translucency on perceived gloss?, Davit Gigilashvili and Akib J. Islam, Norwegian University of Science and Technology (Norway) [view abstract]
Gloss perception is a complex psychovisual phenomenon that is not yet fully explained. In addition to surface reflectance, the state-of-the-art studies demonstrate that object shape and illumination geometry also affect the magnitude of gloss perceived by the human visual system (HVS). Recent studies attribute this to image cues – specific regularities in image statistics that are generated by a combination of these physical properties, and that, in their part, are proposedly used by the HVS for assessing gloss. Another study has recently demonstrated that subsurface scattering of light is an additional factor that can play the role in perceived gloss but provides limited explanation of this phenomenon. In this work, we aim to shed more light to this observation and explain why translucency impacts perceived gloss, and why this impact varies among shapes. We are conducting four psychophysical experiments in order to explore whether image cues typical for opaque objects also explain the variation of perceived gloss in translucent objects and to quantify how these cues are modulated by the subsurface scattering properties.
9:50COLOR-192
A cross-polarization as a possible cause for color shift in illumination, Tarek Abu Haila1,2 and Davit Gigilashvili3; 1Fraunhofer IGD, 2Technical university Darmstadt (Germany), and 3presenter only (Norway) [view abstract]
A cross polarization could be indispensable in certain applications when scanning and digitizing highly reflective materials or when certain applications couldn’t afford following the recommended imaging geometry 0<sup>0</sup>/45<sup>0</sup> | 45<sup>o</sup>/0<sup>o</sup> for some technical reasons. However, that puts very much color fidelity in question, to which extent a cross polarization may impact the source illuminant in the first place that is consequently impacting the color appearance during the imaging and the color correction procedures. In this research we show how certain cross polarization setups are adding a chroma tint to the light source, D50 in this study, causing by that undesirable color shift of the color of the light source. Consequently, a shift in its color correlated temperature moving, in worst case scenario, from ~5000K to ~4500K and resulting in an increased DE00 as a result of the added chroma when compared against a standard D50; nearly doubled in best case scenario and nearly tripled in worst case scenario.
10:10COLOR-193
Image color-based preset light matching algorithm for an electric vitrine, Byeongjin Kim1, Ye Jin Kim2, Myoung Suk Kim2, Hong Seung Do2, and Hyeon-Jeong Suk1; 1Korea Advanced Institute of Science and Technology (KAIST) and 2LG Electronics (Republic of Korea) [view abstract]
Recent advanced light systems offer light presets to enable users to navigate a proper light. Often, the presets are labeled with target ambiance or mood, yet ambiguous for users to predict the light. This study proposes an algorithm that matches the light presets based on the image color characteristics derived from a photograph taken by users. In particular, we developed an automatic match of a light preset for an electric vitrine in which 15 RGB LEDs were linearly arrayed beneath the cover. We conducted a creativity workshop with eight light designers and composed a pool of 22 light presets for the electric vitrine. The presets attempted to cover five standard illuminants, eleven chromatic lights, and six kinds of the color spectrum. The algorithm enables users to receive the optimally matched light presets in order of hue similarities between displayed objects and the light preset. Based on stakeholders’ feedback, the algorithm UX is designed to provide three alternatives made up of the best chromatic light color, the best standard illuminant, and the best color spectrum. We expect the algorithm to be applied to different contexts, where light needs to be optimally tuned by being aware of the context characteristics.
10:00 AM – 7:30 PM Industry Exhibition - Tuesday (in the Cyril Magnin Foyer)
10:20 – 10:50 AM Coffee Break
Applications 2 (T2)
Session Chair:
Gabriel Marcu, consultant (United States)
11:10 AM – 12:10 PM
Mission II/III
11:10COLOR-194
Active learning approaches to analysis of thin-film printed sensors for determining nitrate levels in soil, Xihui Wang, Bruno Ribeiro, Ali Shakouri, and Jan P. Allebach, Purdue University (United States) [view abstract]
No further details about this work can be provided at this time, since a patent may be filed prior to the start of the conference on 15 January 2023 to protect the technology.
11:30COLOR-195
Simulation and estimation of printer media velocity variation, Runzhe Zhang1,2, Yeri Nam3, Yousun Bang3, Ki-Youn Lee3, Mark Shaw3, and Jan P. Allebach4; 1Purdue University (United States), 2Apple (United States), and 3HP (Republic of Korea) [view abstract]
No further details about this work can be provided at this time, since a patent may be filed prior to the start of the conference on 15 January 2023 to protect the technology.
11:50COLOR-196
Analysis of food crystal images, Qiyue Liang, Ali Shakouri, and Jan P. Allebach, Purdue University (United States) [view abstract]
No further details about this work can be provided at this time, since a patent application may be filed to prior to the start of the conference on 15 January 2023 to protect the technology.” Please let me know if you have any questions!
12:30 – 2:00 PM Lunch
Tuesday 17 January PLENARY: Embedded Gain Maps for Adaptive Display of High Dynamic Range Images
Session Chair: Robin Jenkin, NVIDIA Corporation (United States)
2:00 PM – 3:00 PM
Cyril Magnin I/II/III
Images optimized for High Dynamic Range (HDR) displays have brighter highlights and more detailed shadows, resulting in an increased sense of realism and greater impact. However, a major issue with HDR content is the lack of consistency in appearance across different devices and viewing environments. There are several reasons, including varying capabilities of HDR displays and the different tone mapping methods implemented across software and platforms. Consequently, HDR content authors can neither control nor predict how their images will appear in other apps.
We present a flexible system that provides consistent and adaptive display of HDR images. Conceptually, the method combines both SDR and HDR renditions within a single image and interpolates between the two dynamically at display time. We compute a Gain Map that represents the difference between the two renditions. In the file, we store a Base rendition (either SDR or HDR), the Gain Map, and some associated metadata. At display time, we combine the Base image with a scaled version of the Gain Map, where the scale factor depends on the image metadata, the HDR capacity of the display, and the viewing environment.
Eric Chan, Fellow, Adobe Inc. (United States)
Eric Chan is a Fellow at Adobe, where he develops software for editing photographs. Current projects include Photoshop, Lightroom, Camera Raw, and Digital Negative (DNG). When not writing software, Chan enjoys spending time at his other keyboard, the piano. He is an enthusiastic nature photographer and often combines his photo activities with travel and hiking.
Paul M. Hubel, director of Image Quality in Software Engineering, Apple Inc. (United States)
Paul M. Hubel is director of Image Quality in Software Engineering at Apple. He has worked on computational photography and image quality of photographic systems for many years on all aspects of the imaging chain, particularly for iPhone. He trained in optical engineering at University of Rochester, Oxford University, and MIT, and has more than 50 patents on color imaging and camera technology. Hubel is active on the ISO-TC42 committee Digital Photography, where this work is under discussion, and is currently a VP on the IS&T Board. Outside work he enjoys photography, travel, cycling, coffee roasting, and plays trumpet in several bay area ensembles.
3:00 – 3:30 PM Coffee Break
DISCUSSION: Dark Side of Color (T3)
Session Chair:
Alessandro Rizzi, Università degli Studi di Milano (Italy)
3:30 – 4:30 PM
Mission II/III
A session for unexpected topics, including: "Music and Color and Noise with a splash of Synaesthesia", "A view from the dark side", and "What you see is what you get and beyond".
COLOR-460
Music and color and noise, with a splash of synaesthesia (Invited), Scott Daly, Dolby Laboratories, Inc. (United States) [view abstract]
For the "Dark Side of Color" discussion session.
COLOR-461
A view from the dark side (Invited), Alessandro Rizzi, Università degli Studi di Milano (Italy) [view abstract]
For the "Dark Side of Color" discussion session.
COLOR-462
What you see is what you get and beyond (Invited), Gabriel Marcu, consultant (United States) [view abstract]
For the "Dark Side of Color" discussion session.
5:30 – 7:00 PM EI 2023 Symposium Demonstration Session (in the Cyril Magnin Foyer)
Wednesday 18 January 2023
Processing (W1)
Session Chair:
Gabriel Marcu, consultant (United States)
9:10 – 10:10 AM
Mission II/III
9:10COLOR-198
Hue-preserving color enhancement under a cylindrical model without geometric deformation of the RGB color cube, Tieling Chen and Onan Chew, University of South Carolina Aiken (United States) [view abstract]
Some commonly used color models have their associated cylindrical representations that severely deform the RGB color cube. In this paper, color image processing techniques are studied under a color model that uses the distance from a color point to the gray diagonal of the RGB color cube as a component, thereby forming a cylindrical model that does not deform the color cube. This distance component is closely related to color vividness and can be used as a color processing channel by itself. The space of this cylindrical model is larger than the RGB color cube, so the gamut problem may occur when color processing is not well controlled. This paper introduces a method to solve the gamut problem by compressing color overflow near the boundary of the color cube. This allows free application of conventional processing methods of redistributing the values of a component. With the proposed compressing method, the usual techniques of adjusting the distribution can be used in two components including the intensity component and the distance component without worrying about the gamut problem, providing an effect that is difficult to achieve by other pseudo cylindrical models geometrically deforming the RGB color cube.
9:30COLOR-199
Machine learning estimation of camera spectral sensitivity functions with non-RGB color filters, Abraham Sachs1,2 and Ramakrishna Kakarala1; 1Omnivision and 2UC Davis (United States) [view abstract]
The spectral sensitivity functions of a digital image sensor determine the sensor’s color response to scene-radiated light. Knowing these spectral sensitivity functions is very important for applications that require accurate color, such as computer vision. Traditional measurements of these functions are time consuming, and require expensive lab equipment to generate narrow-band monochromatic light. Previous works have shown that sensitivity curves can be estimated using images of a color checker chart with known spectral reflectances, using either numerical optimization or machine learning. However, previous works in the literature have not considered sensitivity functions for CFAs (color filter arrays) other than RGB, such as RCCB (Red Clear Blue) or RYYCy (Red Yellow Cyan). Non-RGB CFAs have been shown to be useful for automotive and security camera applications, especially in low light situations. We propose a machine learning method to estimate the sensitivity curves of sensors with non-RGB filters, in addition to the RGB filters addressed previously in the literature, using a single image of a color chart under unknown illumination. Including non-RGB filters makes the estimation problem much more challenging, since the resulting space of color filters is no longer modelled by simple Gaussian shapes.
9:50COLOR-201
Towards a colorimetric camera, Tripurari Singh and Mritunjay Singh, Image Algorithmics (United States) [view abstract]
Modern color cameras employ sensors that do not mimic human cone spectral sensitivities, and more generally do not meet the Luther Condition since the accompanying color correction substantially amplifies noise in the red channel. This begs the question: if cone spectral sensitivities yield low SNR, why has the Human Visual System so evolved? We answer the above question by noting that since modern ISPs remove virtually all chrominance noise, chrominance denoising artifacts rather than the chrominance noise itself should be considered. While sensor green, blue are reasonable analogs of human M, S cones respectively, the spectral sensitivity of L is much wider than that of red and overlaps with green. An imager employing L instead of red suffers from increased red noise but is also more sensitive. This allows a high SNR L+green guide image to be reconstructed and used for denoising. Modeling the color filter array on the human retina, with a higher density of L pixels at the expense of blue pixels, further improves the red SNR without the accompanying loss of blue quality being perceptible. The resulting camera outperforms conventional cameras in color accuracy and luminance SNR while being competitive in chrominance denosing artifacts.
10:00 AM – 3:30 PM Industry Exhibition - Wednesday (in the Cyril Magnin Foyer)
10:20 – 10:50 AM Coffee Break
Halftoning 1 (W2)
Session Chair:
Reiner Eschbach, Norwegian University of Science and Technology (Norway) and Monroe Community College (United States)
11:10 AM – 12:30 PM
Mission II/III
11:10COLOR-202
A career retrospective and lessons learned: From digital holography and digital halftoning to printed thin film sensors (Invited), Jan P. Allebach, Purdue University (United States) [view abstract]
In this talk, I will review my 50-year career since starting my graduate studies. My Ph.D. studies were evenly divided between digital holography and digital halftoning. After completing my Ph.D., I continued work on digital holography and also investigated non-uniform sampling. However, I also continued work on digital halftoning, and together with printing that became the mainstay of my career, especially after I started working with HP Inc. in 1992. In recent years, printing has evolved the capability to manufacture functional devices including packaging, rather than just things to look at. My research has also followed this evolution, although I have always kept one foot in the field of traditional printing. It goes without saying, although I will say it anyway, that none of this would have been possible without the dedicated effort of my graduate students – too numerous to individually name here. Along the way, there were some lessons learned. And I will discuss these, as well. For individuals starting a career, especially a career in academia, this may be of greater interest than the facts of my research efforts.
11:50COLOR-203
Descreening of halftone images using generative adversarial network, Baekdu Choi and Jan P. Allebach, Purdue University (United States) [view abstract]
Halftoning a continuous-tone image inherently results in loss of information, which makes the inverse process, descreening, a challenging problem. Current state-of-the-art descreening algorithms have two issues: first, they mostly are PSNR-oriented reconstruction algorithms, which tend to generate piecewise smooth images that do not appear realistic due to their lack of texture. Furthermore, these algorithms are typically trained with halftone images generated from the Floyd-Steinberg error diffusion algorithm, which is not an optimal choice since the algorithm is known to generate visible artifacts in the halftone image. We address these issues by the following: first, we propose a new descreening algorithm based on conditional generative adversarial networks (cGAN) that generate descreened images with abundant texture resulting in more realistic appearance. Next, we propose using the direct binary search (DBS) algorithm instead of Floyd-Steinberg error diffusion for generating the halftone images, since it is known to generate halftone images without visible artifacts. Both qualitative and quantitative comparisons show that our algorithm outperforms state-of-the-art descreening algorithms significantly.
12:10COLOR-204
Simulation of the impact of a coating layer on the appearance of various halftone patterns., Fanny Dailliez1,2, Mathieu Hebert2, Lionel Chagas1, Thierry Fournel2, and Anne Blayo1; 1LGP2 and 2Université Jean Monnet de Saint Etienne (France) [view abstract]
In the printing industry, prints are often overlaid with a transparent layer, i.e. varnish or lamination layer, which modifies the colors of the print. Recent investigations have shown that the color variations are related to the position-dependent multiple reflections of light between the printed support and the surface of the coating, and depend on the coating thickness. This effect can be rather accurately predicted thanks to an optical model that we have developed, by means of microscopic multispectral pictures of the prints of periodic line patterns before coating. The overall color difference between the predictions and the measurements was ΔE*<sub>00 </sub>= 0.92. In this paper, we analyze the influence of the halftone patterns on the color changes. In theory, the effect is stronger in case of line halftones than any other halftone pattern shapes, but we also observe in practice an influence of the real pattern size, due to the influence of the dot gain effect related to the diffusion of light and inks within the substrate. Simulations were then conducted on the reflectance profiles, showing that prints without dot gain are more impacted by the halo effect than the ones already subject to dot gain.
12:30 – 2:00 PM Lunch
Wednesday 18 January PLENARY: Bringing Vision Science to Electronic Imaging: The Pyramid of Visibility
Session Chair: Andreas Savakis, Rochester Institute of Technology (United States)
2:00 PM – 3:00 PM
Cyril Magnin I/II/III
Electronic imaging depends fundamentally on the capabilities and limitations of human vision. The challenge for the vision scientist is to describe these limitations to the engineer in a comprehensive, computable, and elegant formulation. Primary among these limitations are visibility of variations in light intensity over space and time, of variations in color over space and time, and of all of these patterns with position in the visual field. Lastly, we must describe how all these sensitivities vary with adapting light level. We have recently developed a structural description of human visual sensitivity that we call the Pyramid of Visibility, that accomplishes this synthesis. This talk shows how this structure accommodates all the dimensions described above, and how it can be used to solve a wide variety of problems in display engineering.
Andrew B. Watson, chief vision scientist, Apple Inc. (United States)
Andrew Watson is Chief Vision Scientist at Apple, where he leads the application of vision science to technologies, applications, and displays. His research focuses on computational models of early vision. He is the author of more than 100 scientific papers and 8 patents. He has 21,180 citations and an h-index of 63. Watson founded the Journal of Vision, and served as editor-in-chief 2001-2013 and 2018-2022. Watson has received numerous awards including the Presidential Rank Award from the President of the United States.
3:00 – 3:30 PM Coffee Break
Halftoning 2 (W3)
Session Chair:
Reiner Eschbach, Norwegian University of Science and Technology (Norway) and Monroe Community College (United States)
3:30 – 4:50 PM
Mission II/III
3:30COLOR-205
Structure-aware color halftoning with adaptive sharpness control (JIST-first), Fereshteh Abedini1, Sasan Gooran1, and Abigail Trujillo-Vazquez2; 1Linköping University (Sweden) and 2presenter only (United States) [view abstract]
Structure-aware halftoning algorithms aim at improving their non-structure-aware version in terms of preserving high-frequency details, structures, and tones by employing additional information from the input image content. The recently proposed achromatic structure-aware IMCDP halftoning algorithm uses the angle of the dominant line in each pixel's neighborhood as a supplementary information to align halftone structures with the dominant orientation in each region which results in sharper halftones, gives more three-dimensional impression, and improves the structural similarity and tone preservation. However, the method is only developed for monochrome halftoning, the degree of sharpness enhancement is constant for the entire image, and the algorithm is prohibitively expensive for large images. In this paper, we present a faster and more flexible approach for representing the image structure using a Gabor-based orientation extraction technique which improves the computational performance of the structure-aware IMCDP by an order of magnitude. In addition, we extend it to color halftoning and study the impact of orientation information in different color channels on improving sharpness enhancement, preserving structural similarity, and decreasing color reproduction error. Furthermore, we propose a dynamic sharpness enhancement approach, which adaptively varies the local sharpness of the halftone image based on different textures across the image. Our contributions in the present work, enable the algorithm to adaptively work on large and diverse images that have multiple regions with different textures.
3:50COLOR-206
Effect of halftones on printing iridescent colors, Fereshteh Abedini1, Abigail Trujillo-Vazquez2, Sasan Gooran1, and Susanne Klein2; 1Linköping University (Sweden) and 2University of the West of England (United Kingdom) [view abstract]
The iridescence effect, produced by structural color, is difficult (if not impossible) to capture and print using traditional CMYK pigments. RGB pigments, nonetheless, generate structural colors by light interference. The layered surface structure generated by pigments’ particles reflects different wavelengths of light in different viewing angles. In printed media, pigments’ particles will collectively influence the optical response of the surface, depending on their size, orientation, structure, and dimensions, ultimately, affecting the visual characteristics of the image perceived by the observer. In this work, we have studied the influence of different halftones’ structures on printed images, produced with RGB inks via screen printing. We investigated the influence of different halftones’ structures in creating different spatial combinations of inks on the printed surface that reproduce the characteristics of iridescent effect of a headdress made of quetzal feathers. We applied first-order, second-order, and structure-aware FM halftones to compare how they influence the reproduction of the material qualities of the object represented in the image. The results show that the structure-ware halftones improve the representation of the image structures and details and, therefore, it could better convey the 3D surface features that produce iridescence effect in the original feathers of the headdress.
4:10COLOR-207
Three-dimensional adaptive digital halftoning (JIST-first), Sasan Gooran1, Fereshteh Abedini1, and Abigail Trujillo-Vazquez2; 1Linköping University (Sweden) and 2presenter only (United States) [view abstract]
Two and a half and 3D printing are becoming increasingly popular, and consequently the demand for high quality surface reproduction is also increasing. Halftoning plays an important role on the quality of the surface reproduction. Three dimensional halftoning methods, that adapt the halftone structures to the geometrical structure of 3D surfaces or to the viewing direction could further improve surface reproduction quality. In this paper, a 3D adaptive halftoning method is proposed, that incorporates different halftone structures on the same 3D surface. The halftone structures are firstly adapted to the 3D geometrical structure of the surface. Secondly, the halftone structures are adapted based on the normal vector to the surface at a specific voxel. Two simple approaches to approximate the normal vector are also proposed. The problem of edge artefact that might occur in the previously proposed 3D IMCDP halftoning method is discussed and a solution to reduce this artefact is given. The results show that the proposed adaptive halftoning can combine different halftone structures on the same 3D surface with no transition artefacts between different halftone structures. It is also shown that using second-order FM halftone, in comparison to first-order FM, can result in more homogeneous appearance of 3D surfaces with undesirable structures on them.
4:30COLOR-208
Dot profile model-based direct binary search, Yafei Mao1, Utpal Sarkar2, Isabel Borrell2, Lluis Abello2, and Jan P. Allebach3; 1Purdue University (United States) and 2HP Inc (Spain) [view abstract]
A dot profile model to compensate dot shape irregularity errors of inkjet printers is proposed. Previous tabular approaches for parameterizing the printer model rely on the measurements of the gray level of various printed halftone patterns. However, lots of patterns need to be printed and scanned if the printer generates large drops of colorant. To solve this problem, we propose to simulate the appearance of the rendered patterns so that the model parameters can be computed analytically. The simulation uses the mean dot as the printer dot profile and saturated addition to resolve dot overlap. Besides, we incorporate a standard definition (SD) and a high definition (HD) equivalent gray-scale representation of the printed halftone image produced by the dot profile model into the direct binary search (DBS) algorithm. Experimental results show great improvement in the mid-tone and shadow regions over the printed image halftoned by the original DBS. The HD model further enhances details in the shadows.
5:30 – 7:00 PM EI 2023 Symposium Interactive (Poster) Paper Session (in the Cyril Magnin Foyer)
5:30 – 7:00 PM EI 2023 Meet the Future: A Showcase of Student and Young Professionals Research (in the Cyril Magnin Foyer)