Human Vision and Electronic Imaging (HVEI)

How human vision science shapes future displays

Qi Sun, New York University (US)
March 2, 8:30 am

Abstract: Wearable display systems offer groundbreaking opportunities for rendering virtual content, sensing physical environments, and precisely tracking human behavior. With rapid advances in display optics, sensing, and machine learning, the hardware–software pipeline of displays is undergoing a fundamental transformation. However, this transformation also demands a new understanding of how human perception and cognition interact with visual computing systems. In this talk, I will present our recent research on modeling visual perception and behavior to develop efficient and user-centric display systems. We explore how perceptual constraints can guide the optimization of rendering and sensing pipelines, enabling XR systems that allocate computation only where the human visual system is most sensitive. By combining psychophysical modeling, computational imaging, and generative methods, our work aims to align machine efficiency with human perception—paving the way toward adaptive, perceptually grounded wearable displays that enhance realism, comfort, and performance. In particular, I will discuss how human visual acuity in color and luminance connect to the power consumption of near eye displays to answer the fundamental question "what's the visual quality gain per Watt of power usage?", and how perceptually guided algorithms may enable significant battery life extension. Additionally, I will share new insights on the necessity to further understand the consequential human behaviors --- such as how fast do we react to visual targets --- to avoid negative effects and risks for users using wearable displays in daily life.

Qi Sun is an associate professor at New York University. Before joining NYU, he was a research scientist at Adobe Research. He received his PhD at Stony Brook University. His research interests lie in VR/AR, perceptual computer graphics, computational display, and applied perception. He is a recipient of the IEEE Virtual Reality Best Dissertation Award. With colleagues, his research has been recognized as several best paper and honorable mention awards at ACM SIGGRAPH, IEEE ISMAR, IEEE VR, IEEE VIS, and ACM SAP.

Imaging and Multimedia Analytics at the Edge (IMAGE)

Advancing visual perception for embodied intelligence: From 2D scenes to dynamic 3D worlds

Zenglin Xu, Fudan University (China)
March 2, 8:30 am

Abstract: Recent advances in AI and deep learning have significantly improved perceptual, cognitive, and generative capabilities. As a result, embodied intelligence, which tightly integrates perception, cognition, decision-making, and action, has gained increasing attention. However, visual perception—the cornerstone of embodied systems—still faces critical challenges due to the real world's complexity, three-dimensionality, and temporal dynamics.

This talk presents a progression of visual perception research from 2D images to dynamic 3D scenes, addressing key limitations in accuracy, robustness, and efficiency. First, we introduce MixingMask, a novel contour-focused method that enhances 2D object detection and segmentation by capturing fine-grained boundary information. Second, we propose a boundary-optimized approach for monocular 3D plane reconstruction, improving geometric accuracy through precise segmentation and centerness filtering. Third, we present an octree-based semantic occupancy framework for multi-view panoramic scenes, significantly reducing computational cost by exploiting the sparsity of 3D space. Lastly, we develop LinkOcc, a temporally-aware 3D occupancy method that uses a sparse query mechanism and contrastive learning to improve temporal consistency.

Together, these contributions push the boundaries of visual perception towards practical, resource-efficient deployment in real-world embodied systems. Experimental results across multiple benchmarks validate the effectiveness of our methods and provide promising directions for future research in perception for embodied AI.

Zenglin Xu, PhD, is a distinguished professor, Artificial Intelligence Innovation and Incubation (AI^3) Institute, at Fudan University in Shanghai, China. He is editor-in-chief for Academia AI and Applications and senior action editor for Neural Networks. He is also VP for education and BoG member for the International Neural Network Society as well as chair for the Section on AI for Education for the International Neural Network Society.

Stereoscopic Displays and Applications

Embodying the Machine: Strategies and Frameworks for Ethical XR

Michelle Cortese, Meta (US)
March 2, 4:20 pm

Abstract: This presentation compiles a decade's worth of strategies and subject-matter that have proven successful in teaching augmented, virtual and extended reality (AR, VR, XR) concepts and practices. The methodologies in this talk have been successfully applied: in academia at NYU's Interactive Telecommunications program (in the form of a 14-week graduate class); in the tech industry at Meta Reality Labs' Metaverse department (in the form of an internal upskilling program); in summary form at various conferences; and in as an upcoming book (to be published via Intellect Books in 2027). The material argues that to build and use XR experiences, it's not enough to learn the hard skills -- it's also our responsibility to prime ourselves for the human impact of the medium. As a means to develop XR experiences that are both enjoyable and accountable, this learning approach proposes we borrow structures and principles from Hedonomics, a branch of ergonomic science that facilitates pleasurable human-technology interaction. Through the Hedonomic Pyramid, we're able to section our thinking off into regions (Safety, Function, Usability, Pleasure and Individuation) and map out industry-tested XR concepts for each. The result is a hierarchical checklist of proven principles, specifications and practices built to serve as a quickstart guide for implementing inclusive and holistic XR interfaces and systems.

Michelle Cortese is an XR designer, educator and author. She splits her professional time between design leadership at Meta Reality Labs and teaching VR design at NYU. Her work explores immersive interaction systems; the ethical implications of embodied technology on end users; and the transmutation of human expression across new technologies and formats. Michelle has authored AR and VR design research published via Bloomsbury, Meta, IEEE, OneZero, MIT's Immerse Journal and more; she has also exhibited work at CES, Tribeca Film Festival, SXSW, and Sundance.

Autonomous Vehicles and Machines (AVM)

Introduction to the latest mmWave automotive imaging radar technologies

Ryan Wu, NXP (San Jose, CA, US) 
March 3, 11:00 am

Abstract: In this talk, an overview of the latest automotive radar technologies will be provided. Topics include the latest system architecture, high-resolution imaging radar processing, massive MIMO waveform design, and radar-to-radar interference mitigation concepts. Field test results of a state-of-the-art 24 Tx-by-24 Rx imaging radar achieving lidar-like point clouds in challenging environments will also be presented in the end.

Ryan Wu has over 20 years of R&D experience and over 50 granted and pending patents in radar, surveillance, and navigation technologies. He is currently serving as an NXP Fellow leading the development of advanced signal processing and machine learning solutions for next-gen automotive radar systems.

Image Quality and System Performance (IQSP)

Preferred skin color rendition for self-representative faces and avatars in augmented reality

Dara Dimoff, Rochester Institute of Technology (US)
March 3, 11:40 am

Abstract: Augmented reality (AR) is a technology that enables humans to superimpose visual elements over the real world. One contemporary approach to AR technology is optical see-through AR (OST-AR) in which the physical world is viewed through a transparent medium that displays graphical elements. This model faces a major challenge in color appearance: its additive color mixing creates a bleed-through effect in which the environment blends with the appearance of the graphical elements—especially in environments with high luminance or with dark graphical elements—creating a translucent appearance. Notably, graphical human faces having darker skin tones appear more transparent than those with lighter skin tones, introducing both perceptual and social challenges.

In this work, a psychophysical study assesses observers’ preferred renderings of skin tones in OST-AR. Observers are given a task to adjust the lightness and chroma of faces superimposed by AR glasses in various illumination conditions. This study focuses on the color adjustments made to pictures of real faces vs. their corresponding digital avatars, and stimuli representing the observer vs. zero-acquaintance targets.

Human Vision and Electronic Imaging (HVEI)

How gaze direction and dynamics affect visual resolution

Martin Banks, UC Berkeley School of Optometry (US)
March 4, 8:30 am

Abstract: Humans exhibit machine-like eye movements in space and time while performing demanding acuity tasks. To investigate these, we used an adaptive-optics imaging and display system to present ultra-sharp Vernier acuity stimuli briefly every two seconds while simultaneously measuring eye movements, including precisely where on the retina each stimulus fell. We found that drifts and microsaccades combined to confine the landing location of the anticipated stimulus to a tiny retinal region centered on the preferred retinal locus (PRL). The variance of landing location was smallest at the time of stimulus presentation and a few hundred milliseconds after. We correlated where the stimulus fell in space and time with correct or incorrect responses. The PRL and a small area around it, including the anatomical fovea, conferred the best acuity. Acuity declined consistently in the rare events in which the stimulus fell more than 7–10minarc from the PRL. We also found that acuity was best when the last microsaccade occurred sufficiently prior to stimulus presentation. Our findings reveal a highly evolved oculomotor system where gaze direction during fixation is rarely far enough from the PRL to compromise visual resolution when a person makes natural fixational eye movements.

Martin Banks is a distinguished Professor Emeritus at the UC Berkeley School of Optometry, renowned for his pioneering research in human visual development, visual space perception, and multi-sensory integration, particularly how we perceive depth and motion, with significant applied work in virtual reality (VR) and advanced displays, earning him election to the American National Academy of Sciences. He received a B.A. in Psychology from Occidental College in 1970, M.S. in Experimental Psychology from UC San Diego in 1973, and PhD in Developmental Psychology from the University of Minnesota in 1976. Martin Banks received a number of notable awards, among them the McCandless Award, American Psychological Association; Koffka Medal, Giessen University; Holgate Fellow, Durham University; Fellow, American Association for the Advancement of Science; Fellow, American Psychological Society; Prentice Award, American Academy of Optometry; Honorary Professor, University of Wales; Borish Scholar, Indiana Univ; Schade Prize, Society for Information Display, and Tillyer Award from the Optical Society of America.

Engineering Reality of Virtual Reality

The Eureka VR Environment for Mining Engineering Education

Sergiu Dascalu, University of Nevada (US)
March 4, 9:20 am

Abstract: Neutron Resonance Imaging (NRI) is a non-destructive method that uses isotope-specific variations in neutron cross-sections to map isotopic distributions within materials. Distinct resonance features act as spectral fingerprints, allowing 2D and 3D imaging of isotopic composition without damaging samples. Event-mode neutron imaging has advanced NRI into a quantitative tool by recording individual neutron interactions in time and space. This enables energy-resolved measurements, background discrimination, and improved signal quality, enhancing spectral fitting accuracy. These advances extend NRI into the Neutron Resonance Spectroscopy (NRS) regime, providing spatially resolved insights into temperature and chemical environments through Doppler broadening and resonance line-shape analysis.

Sergiu Dascalu is a professor in the Department of Computer Science and Engineering (CSE) at the University of Nevada, Reno (UNR). He received a PhD degree in computer science (with a focus on software engineering) from Dalhousie University, Canada, and a Master’s degree in automatic control and computers from the Polytechnic University of Bucharest, Romania. His main interests are in human-computer interaction, data science, and software engineering. Co-director of the Software Systems Lab at UNR, Dr. Dascalu has worked on numerous research projects funded by federal agencies (NSF, NASA, DoD-ONR) as well as the industry. He published over 60 journal articles and over 240 peer-reviewed conference papers and has advised 13 PhD and 65 MS in CSE students who graduated. Dr. Dascalu contributed in various roles to the organization of more than 150 conferences, symposia, and workshops. He is a senior member of the ACM.

Image Quality and System Performance (IQSP)

Acceptance levels for image quality factors

Dietmar Wueller, Image Engineering GmbH & Co. KG (Germany)
March 4, 11:20 am

Abstract: To determine the lowest light level for which a digital camera still delivers images that are acceptable requires acceptance thresholds for all related image quality factors. ISO 19093 [1] describes these factors and how they can be measured. The acceptance thresholds however may depend on the application for which the images were captured and on peoples‘ individual tolerance for the degradation of the different image quality factors. In order to generate a standard set of tolerance levels for photographic applications a psychophysical experiment was performed as described in this paper.

Computational Imaging (COIMG)

Advanced materials characterization via neutron resonance imaging (NRI) and spectroscopy (NRS) at LANSCE

Alexander Long, Los Alamos National Laboratory (US)
March 4, 3:30 pm

Abstract: Neutron Resonance Imaging (NRI) is a non-destructive method that uses isotope-specific variations in neutron cross-sections to map isotopic distributions within materials. Distinct resonance features act as spectral fingerprints, allowing 2D and 3D imaging of isotopic composition without damaging samples. Event-mode neutron imaging has advanced NRI into a quantitative tool by recording individual neutron interactions in time and space. This enables energy-resolved measurements, background discrimination, and improved signal quality, enhancing spectral fitting accuracy. These advances extend NRI into the Neutron Resonance Spectroscopy (NRS) regime, providing spatially resolved insights into temperature and chemical environments through Doppler broadening and resonance line-shape analysis.

Neutron Resonance Imaging (NRI) is a non-destructive method that uses isotope-specific variations in neutron cross-sections to map isotopic distributions within materials. Distinct resonance features act as spectral fingerprints, allowing 2D and 3D imaging of isotopic composition without damaging samples. Event-mode neutron imaging has advanced NRI into a quantitative tool by recording individual neutron interactions in time and space.

At the Los Alamos Neutron Science Center (LANSCE), optically based event-mode detectors known as LumaCams have been developed to further improve NRI performance. Using Timepix3 sensors coupled to scintillators, LumaCams provide precise spatial and temporal resolution while enabling discrimination between neutron and gamma-ray events. This capability significantly reduces background noise, improves signal quality, and enhances the reliability of energy-resolved transmission spectra, supporting more accurate spectral fitting and isotope quantification.

These advances extend NRI into the Neutron Resonance Spectroscopy (NRS) regime, providing spatially resolved insights into temperature and chemical environments through Doppler broadening and resonance line-shape analysis. Together with ongoing developments in analysis tools and multi-modal integration, NRI and NRS now offer a robust framework for non-destructive, isotope-specific characterization of complex materials under both ambient and extreme conditions.

Human Vision and Electronic Imaging (HVEI)

What we've learned about visual attention

Ruth Rosenholtz, NVIDIA Research (US)
March 4, 3:30 pm

Abstract: Early in the study of visual attention, it appeared promising that understanding of preattentive and attentional processes could provide a unifying explanation of a wide range of visual phenomena, by elucidating a critical capacity limit faced by visual processing. However, researchers have uncovered significant anomalies, frustrating hopes of a single predictive mechanism. This state of affairs requires rethinking visual attention from the ground up. This talk provides my take on the critical phenomena to consider in search of a unifying theory. Commonalities between these phenomena suggest not only new ways of thinking of capacity limits and the mechanisms for dealing with those limits, but also visual perception itself and the contents of visual awareness. This rethinking of visual attention points to a new possibility of a unifying theory, in which all perception results from performing a task, and tasks face a limit on complexity.

Ruth Rosenholtz joined NVIDIA Research in 2023, after a year as visiting scientist. Her research interests include behavioral experiments and computational modeling of human visual perception, and its applications. Particular vision topics include peripheral vision, visual attention, perceptual organization, material perception, and shape/depth perception. Applications include image quality, HCI, and vision for driving. Ruth received her Ph.D. (1994) and MS (1991) in EECS from the University of California, Berkeley, and her BS (1988) in Engineering from Swarthmore College. Previously she was research faculty at MIT's Department of Brain & Cognitive Sciences and CSAIL and has worked at Xerox PARC and NASA Ames.

Image Quality and System Performance (IQSP)

The influence of image semantic complexity on the performance of image quality metrics

Marius Pedersen, Norwegian University of Science and Technology (Norway)
March 5, 8:50 am

Abstract: Image quality assessment has been a longstanding area of research, with significant efforts dedicated to developing objective metrics that can reliably predict perceived image quality. While numerous image quality metrics have been proposed, ranging from traditional handcrafted approaches to modern machine learning-based models, their evaluation typically relies on statistical comparisons with subjective human ratings where correlation is reported as the main way to assess their performance. In this study, we explore an additional dimension in image quality evaluation: the impact of image semantic complexity on metric performance. Specifically, we hypothesize that the number of distinct semantic categories within an image influences the accuracy of image quality metrics. We evaluate 8 state-of-the-art image quality metrics across 2 benchmark datasets, analyzing performance variations with respect to image semantic complexity (category count), based on two vision-language models. Our findings reveal that for some image quality metrics there is an impact of semantic complexity and outliers. This could suggest that content-aware evaluation may be crucial for future image quality research.