LIM 2023 Summer School

The interaction between light and the materials that make up objects plays an essential role in our perception of them. Depending on their composition and structure, various optical phenomena can occur: absorption, which attenuates the intensity of light rays depending on the wavelength, scattering, which redirects its trajectories, and reflections at smooth or rough interfaces.

This course proposes a review of these elementary optical phenomena and the basic models allowing to describe and combine them, through various characteristic examples such as glass, metals and paints.

The measurement of appearance of objects seen by individuals is a necessity to answer to industrial and academic needs (quality control at the end of production lines, validation of rendering models, development of new product). Yesterday, simple 0/45 or 0/Diffuse spectrophotometric measurements were enough to characterize the colour of basic product. But today, one wants to measure iridescence, goniochromatism, glossiness, sparkle, translucency or reflectance functionalities. To characterize and quantify these visual effect, metrologists need to move to bidirectional reflectance or transmittance measurements. At the highest level, National Metrological Institutes are working in order to develop new references, new transfer artefacts, new measurement protocols to provide solution. This course will propose an overview of bidirectional spectrophotometry, starting from basic concepts and moving to ongoing research on this topic.

Photorealistic rendering of real-world materials is important to a range of applications, including visual effects, computer games, cultural heritage, architectural modelling, and virtual reality.  Visual perception of material appearance depends on how lighting is reflected, scattered, and absorbed by a surface. In order to faithfully reproduce appearance, we must therefore understand how light interacts with materials, and how to best acquire and represent material properties in computer graphics applications.

In this course, we first describe the state of the art in material appearance acquisition and modelling, ranging from mathematical BSDFs for surface reflectance representation to volumetric/thread models for patterned fabrics.

We then present a case study in the entertainment industry: the creation of digital doubles of real subjects. Such a task proves to be particularly challenging, since besides acquiring a faithful geometry of the face, recreating the appearance of human skin, a complex, multi-layered heterogeneous material, is extremely important.

Quantification of material perception requires adequate control of test stimuli and measurement of observers’ behavioural responses to these stimuli. The complexity of light-material interactions and the high number of parameters required to specify the range of possible physical stimuli has necessitated compromises in approach. For example, computer-graphics generated stimuli afford high levels of experimental control and replicability, but are limited by the fidelity of material models and rendering. Conversely, real stimulus samples offer the richness of light-material interactions, but experiments are limited by manufacturing processes or quantification of the samples’ physical properties, and experiments can be laborious when automation of stimulus presentation is limited. Interactions between stimulus parameters and viewing conditions can have profound effects on visual appearance, producing combinatorial explosion in the potential number of stimulus conditions to be tested. In addition, there can be large individual differences, which present further challenges to extracting general principles of material perception. We present a series of case studies of psychophysical experiments – using real, photographed and computer-generated stimuli; bespoke displays; lab-based and online testing; and a range of psychophysical methods and data modelling approaches. Through a convergence of results from different experiments, we hint at some general principles, such as (i) the benefits of stimulus richness for perceptual extraction of physical properties; (ii) the non-static nature of perception, with an emphasis on dynamic sampling of stimuli; and (iii) the rejection of ‘inverse optics’ models in favour of heuristics-based perceptual decisions that work ‘well enough’ to support successful behavioural interaction with the material world.