Stereoscopic Displays and Applications XXXIV
Monday 16 January 2023
Stereoscopic Displays (M1)
Session Chair:
Bjorn Sommer, Royal College of Art (United Kingdom)
8:45 – 9:50 AM
Cyril Magnin II
8:45
Conference Welcome
8:50SD&A-383
Evaluating the angular resolution of a simulated light field display in regards to three-dimensionality, motion parallax and viewing experience, Sophie Kergaßner and Jan Fröhlich, Stuttgart Media University (Germany) [view abstract]
This paper investigates how the angular resolution of a simulated light field display affects the viewing experience of a 3D rendered scene. The effect of three-dimensionality, helpfulness of motion parallax, and overall viewing experience are evaluated by mean of a quantitative user study. A total of 32 results are recorded and evaluated. The quality metrics show a logarithmic trend, improving with an increasing number of views. A plateau at 0.25° per view can be found in two of the three evaluated questions. Moreover, the participants preferred the viewing experience of a conventional 2D-display over the simulated light field displays with more than 0.5° per view (less than 360 views at 180° FOV). This shows, that light field displays have to have a certain minimum angular resolution to create a pleasant viewing experience.
9:10SD&A-384
CubicSpace for stereo cards: Enabling universal 3D playback on today's display devices, Nicholas Routhier, Mindtrick Innovations Inc. (Canada) [view abstract]
Historical stereo cards represent a massive source of stereoscopic content which is currently dormant in collections throughout the world. They have proven very difficult to repurpose outside of their original viewing devices (stereoscopes), causing namely headaches, eye strain, double vision (diplopia) and general discomfort, This has limited their distribution and use for the general public, depriving users and manufacturers of valuable 3D content. CubicSpace, in collaboration with CLEM Patrimoine and Archéovision, has developped a novel method for repurposing stereo cards and make them available to the general public. To achieve this, the method consisted of working from digitized stereo cards to identify specific features in stereo cards, determine viewing conditions (users and displays) and using the CubicSpace technology to create geometries and define virtual camera parameters providing a comfortable 3D experience. The method further comprised the ability to zoom in and out and to pan within the picture. Lasty, the method comprised an efficient archiving solution to make it simple, light and robust. With this method, it has been demonstrated that stereo cards can be repurposed efficiently and provide an added-value user experience on existing 2D and 3D display devices with minimal processing and storage requirements. In this presentation, we will discuss the solution, a current implementation with the Stéréothèque, potential areas of improvement and potential future use cases.
9:30SD&A-385
Upgrading the Curtin HIVE’s stereoscopic projection facilities, Andrew J. Woods, Wesley Lamont, and Daniel Adams, Curtin University (Australia) [view abstract]
The HIVE (Hub for Immersive Visualisation and eResearch) is Curtin University’s visualisation facility. The HIVE was originally established in November 2013 and consists of both a physical facility with four large-scale immersive display systems with a small team of staff to operate the facility and support visualisation research projects. In 2019 the team commenced the process of renewing the projection hardware in the HIVE. The Cylinder display in the HIVE is the headline display in the facility with a projection surface 3m high, 180° Field of view, and 8m in diameter. Originally equipped with three projectionDesign F35 1920x1200 DLP projectors, in upgrading the display the ideal was to upgrade to 4K projectors, increase brightness, and maintain stereoscopic 3D capability, within budget. Many different options were considered, and the selected projection system was the Digital Projection “Satellite Modular Laser System (MLS)”. This system was delivered. installed and commissioned during the worldwide COVID lockdowns, and is now fully operational. The HIVE was the very first installation worldwide of this new system. The presentation will consider the pro’s and con’s of the wide range of options considered and discuss some of the applications the new systems are being used for.
10:20 – 10:50 AM Coffee Break
KEYNOTE: SD&A 1 (M2.1)
Session Chair: Andrew Woods, Curtin University (Australia)
10:50 – 11:50 AM
Cyril Magnin II
SD&A-386
KEYNOTE: The long-awaited arrival of holographic interfaces, Shawn Frayne, Looking Glass Factory (United States) [view abstract]
Inspired by movies in the 80s and 90s, Shawn Frayne has been reaching towards the dream of the hologram for over 20 years. Frayne got his start with a classic laser interference pattern holographic studio he built in high school, followed by training in advanced holographic film techniques at MIT. He has been awarded dozens of patents around the world and serves as co-founder and CEO of Looking Glass Factory, based in Brooklyn and Hong Kong.
Holographic or light field generating devices that could enable groups of people to see and interact with genuinely three-dimensional content have long been held as a “holy grail” by those inventors and engineers that work in the field of perfecting the human-computer interface. Now after decades of work, real-time holographic interfaces are at last commercially available. The driving forces behind the emergence of holographic interfaces, the advantages of these headset-free systems compared to head-mounted displays, and the unique characteristics of the first commercially-viable approaches to come to market will be covered here.
Stereoscopic Simulation (M2.2)
Session Chair:
Andrew Woods, Curtin University (Australia)
11:50 AM – 12:30 PM
Cyril Magnin II
11:50SD&A-387
Estimation of motion sickness in automated vehicles using stereoscopic visual simulation (JIST-first), Yoshihiro Banchi and Takashi Kawai, Waseda University (Japan) [view abstract]
Automation of driving leads to a decrease in driver agency, and there are concerns about motion sickness in automated vehicles. The automated driving agencies are closely related to virtual reality technology, which has been confirmed in relation to simulator sickness. Such motion sickness has a similar mechanism as sensory conflict. In this study, we investigated the use of deep learning for predicting motion. We conducted experiments using an actual vehicle and a stereoscopic image simulation. For each experiment, we predicted the occurrences of motion sickness by comparing the data from the stereoscopic simulation to an experiment with actual vehicles. Based on the results of the motion sickness prediction, we were able to extend the data on a stereoscopic simulation in improving the accuracy of predicting motion sickness in an actual vehicle. Through the performance of stereoscopic visual simulation, it is considered possible to utilize the data in deep learning.
12:10
Lenny Lipton - In Memoriam
12:30 – 2:00 PM Lunch
Monday 16 January PLENARY: Neural Operators for Solving PDEs
Session Chair: Robin Jenkin, NVIDIA Corporation (United States)
2:00 PM – 3:00 PM
Cyril Magnin I/II/III
Deep learning surrogate models have shown promise in modeling complex physical phenomena such as fluid flows, molecular dynamics, and material properties. However, standard neural networks assume finite-dimensional inputs and outputs, and hence, cannot withstand a change in resolution or discretization between training and testing. We introduce Fourier neural operators that can learn operators, which are mappings between infinite dimensional spaces. They are independent of the resolution or grid of training data and allow for zero-shot generalization to higher resolution evaluations. When applied to weather forecasting, neural operators capture fine-scale phenomena and have similar skill as gold-standard numerical weather models for predictions up to a week or longer, while being 4-5 orders of magnitude faster.
Anima Anandkumar, Bren professor, California Institute of Technology, and senior director of AI Research, NVIDIA Corporation (United States)
Anima Anandkumar is a Bren Professor at Caltech and Senior Director of AI Research at NVIDIA. She is passionate about designing principled AI algorithms and applying them to interdisciplinary domains. She has received several honors such as the IEEE fellowship, Alfred. P. Sloan Fellowship, NSF Career Award, and Faculty Fellowships from Microsoft, Google, Facebook, and Adobe. She is part of the World Economic Forum's Expert Network. Anandkumar received her BTech from Indian Institute of Technology Madras, her PhD from Cornell University, and did her postdoctoral research at MIT and assistant professorship at University of California Irvine.
3:00 – 3:30 PM Coffee Break
EI 2023 Highlights Session
Session Chair: Robin Jenkin, NVIDIA Corporation (United States)
3:30 – 5:00 PM
Cyril Magnin II
Join us for a session that celebrates the breadth of what EI has to offer with short papers selected from EI conferences.
NOTE: The EI-wide "EI 2023 Highlights" session is concurrent with Monday afternoon COIMG, COLOR, IMAGE, and IQSP conference sessions.
IQSP-309
Evaluation of image quality metrics designed for DRI tasks with automotive cameras, Valentine Klein, Yiqi LI, Claudio Greco, Laurent Chanas, and Frédéric Guichard, DXOMARK (France) [view abstract]
Driving assistance is increasingly used in new car models. Most driving assistance systems are based on automotive cameras and computer vision. Computer Vision, regardless of the underlying algorithms and technology, requires the images to have good image quality, defined according to the task. This notion of good image quality is still to be defined in the case of computer vision as it has very different criteria than human vision: humans have a better contrast detection ability than image chains. The aim of this article is to compare three different metrics designed for detection of objects with computer vision: the Contrast Detection Probability (CDP) [1, 2, 3, 4], the Contrast Signal to Noise Ratio (CSNR) [5] and the Frequency of Correct Resolution (FCR) [6]. For this purpose, the computer vision task of reading the characters on a license plate will be used as a benchmark. The objective is to check the correlation between the objective metric and the ability of a neural network to perform this task. Thus, a protocol to test these metrics and compare them to the output of the neural network has been designed and the pros and cons of each of these three metrics have been noted.
SD&A-224
Human performance using stereo 3D in a helmet mounted display and association with individual stereo acuity, Bonnie Posselt, RAF Centre of Aviation Medicine (United Kingdom) [view abstract]
Binocular Helmet Mounted Displays (HMDs) are a critical part of the aircraft system, allowing information to be presented to the aviator with stereoscopic 3D (S3D) depth, potentially enhancing situational awareness and improving performance. The utility of S3D in an HMD may be linked to an individual’s ability to perceive changes in binocular disparity (stereo acuity). Though minimum stereo acuity standards exist for most military aviators, current test methods may be unable to characterise this relationship. This presentation will investigate the effect of S3D on performance when used in a warning alert displayed in an HMD. Furthermore, any effect on performance, ocular symptoms, and cognitive workload shall be evaluated in regard to individual stereo acuity measured with a variety of paper-based and digital stereo tests.
IMAGE-281
Smartphone-enabled point-of-care blood hemoglobin testing with color accuracy-assisted spectral learning, Sang Mok Park1, Yuhyun Ji1, Semin Kwon1, Andrew R. O’Brien2, Ying Wang2, and Young L. Kim1; 1Purdue University and 2Indiana University School of Medicine (United States) [view abstract]
We develop an mHealth technology for noninvasively measuring blood Hgb levels in patients with sickle cell anemia, using the photos of peripheral tissue acquired by the built-in camera of a smartphone. As an easily accessible sensing site, the inner eyelid (i.e., palpebral conjunctiva) is used because of the relatively uniform microvasculature and the absence of skin pigments. Color correction (color reproduction) and spectral learning (spectral super-resolution spectroscopy) algorithms are integrated for accurate and precise mHealth blood Hgb testing. First, color correction using a color reference chart with multiple color patches extracts absolute color information of the inner eyelid, compensating for smartphone models, ambient light conditions, and data formats during photo acquisition. Second, spectral learning virtually transforms the smartphone camera into a hyperspectral imaging system, mathematically reconstructing high-resolution spectra from color-corrected eyelid images. Third, color correction and spectral learning algorithms are combined with a spectroscopic model for blood Hgb quantification among sickle cell patients. Importantly, single-shot photo acquisition of the inner eyelid using the color reference chart allows straightforward, real-time, and instantaneous reading of blood Hgb levels. Overall, our mHealth blood Hgb tests could potentially be scalable, robust, and sustainable in resource-limited and homecare settings.
AVM-118
Designing scenes to quantify the performance of automotive perception systems, Zhenyi Liu1, Devesh Shah2, Alireza Rahimpour2, Joyce Farrell1, and Brian Wandell1; 1Stanford University and 2Ford Motor Company (United States) [view abstract]
We implemented an end-to-end simulation for perception systems, based on cameras, that are used in automotive applications. The open-source software creates complex driving scenes and simulates cameras that acquire images of these scenes. The camera images are then used by a neural network in the perception system to identify the locations of scene objects, providing the results as input to the decision system. In this paper, we design collections of test scenes that can be used to quantify the perception system’s performance under a range of (a) environmental conditions (object distance, occlusion ratio, lighting levels), and (b) camera parameters (pixel size, lens type, color filter array). We are designing scene collections to analyze performance for detecting vehicles, traffic signs and vulnerable road users in a range of environmental conditions and for a range of camera parameters. With experience, such scene collections may serve a role similar to that of standardized test targets that are used to quantify camera image quality (e.g., acuity, color).
VDA-403
Visualizing and monitoring the process of injection molding, Christian A. Steinparz1, Thomas Mitterlehner2, Bernhard Praher2, Klaus Straka1,2, Holger Stitz1,3, and Marc Streit1,3; 1Johannes Kepler University, 2Moldsonics GmbH, and 3datavisyn GmbH (Austria) [view abstract]
In injection molding machines the molds are rarely equipped with sensor systems. The availability of non-invasive ultrasound-based in-mold sensors provides better means for guiding operators of injection molding machines throughout the production process. However, existing visualizations are mostly limited to plots of temperature and pressure over time. In this work, we present the result of a design study created in collaboration with domain experts. The resulting prototypical application uses real-world data taken from live ultrasound sensor measurements for injection molding cavities captured over multiple cycles during the injection process. Our contribution includes a definition of tasks for setting up and monitoring the machines during the process, and the corresponding web-based visual analysis tool addressing these tasks. The interface consists of a multi-view display with various levels of data aggregation that is updated live for newly streamed data of ongoing injection cycles.
COIMG-155
Commissioning the James Webb Space Telescope, Joseph M. Howard, NASA Goddard Space Flight Center (United States) [view abstract]
Astronomy is arguably in a golden age, where current and future NASA space telescopes are expected to contribute to this rapid growth in understanding of our universe. The most recent addition to our space-based telescopes dedicated to astronomy and astrophysics is the James Webb Space Telescope (JWST), which launched on 25 December 2021. This talk will discuss the first six months in space for JWST, which were spent commissioning the observatory with many deployments, alignments, and system and instrumentation checks. These engineering activities help verify the proper working of the telescope prior to commencing full science operations. For the session: Computational Imaging using Fourier Ptychography and Phase Retrieval.
HVEI-223
Critical flicker frequency (CFF) at high luminance levels, Alexandre Chapiro1, Nathan Matsuda1, Maliha Ashraf2, and Rafal Mantiuk3; 1Meta (United States), 2University of Liverpool (United Kingdom), and 3University of Cambridge (United Kingdom) [view abstract]
The critical flicker fusion (CFF) is the frequency of changes at which a temporally periodic light will begin to appear completely steady to an observer. This value is affected by several visual factors, such as the luminance of the stimulus or its location on the retina. With new high dynamic range (HDR) displays, operating at higher luminance levels, and virtual reality (VR) displays, presenting at wide fields-of-view, the effective CFF may change significantly from values expected for traditional presentation. In this work we use a prototype HDR VR display capable of luminances up to 20,000 cd/m^2 to gather a novel set of CFF measurements for never before examined levels of luminance, eccentricity, and size. Our data is useful to study the temporal behavior of the visual system at high luminance levels, as well as setting useful thresholds for display engineering.
HPCI-228
Physics guided machine learning for image-based material decomposition of tissues from simulated breast models with calcifications, Muralikrishnan Gopalakrishnan Meena1, Amir K. Ziabari1, Singanallur Venkatakrishnan1, Isaac R. Lyngaas1, Matthew R. Norman1, Balint Joo1, Thomas L. Beck1, Charles A. Bouman2, Anuj Kapadia1, and Xiao Wang1; 1Oak Ridge National Laboratory and 2Purdue University (United States) [view abstract]
Material decomposition of Computed Tomography (CT) scans using projection-based approaches, while highly accurate, poses a challenge for medical imaging researchers and clinicians due to limited or no access to projection data. We introduce a deep learning image-based material decomposition method guided by physics and requiring no access to projection data. The method is demonstrated to decompose tissues from simulated dual-energy X-ray CT scans of virtual human phantoms containing four materials - adipose, fibroglandular, calcification, and air. The method uses a hybrid unsupervised and supervised learning technique to tackle the material decomposition problem. We take advantage of the unique X-ray absorption rate of calcium compared to body tissues to perform a preliminary segmentation of calcification from the images using unsupervised learning. We then perform supervised material decomposition using a deep learned UNET model which is trained using GPUs in the high-performant systems at the Oak Ridge Leadership Computing Facility. The method is demonstrated on simulated breast models to decompose calcification, adipose, fibroglandular, and air.
3DIA-104
Layered view synthesis for general images, Loïc Dehan, Wiebe Van Ranst, and Patrick Vandewalle, Katholieke University Leuven (Belgium) [view abstract]
We describe a novel method for monocular view synthesis. The goal of our work is to create a visually pleasing set of horizontally spaced views based on a single image. This can be applied in view synthesis for virtual reality and glasses-free 3D displays. Previous methods produce realistic results on images that show a clear distinction between a foreground object and the background. We aim to create novel views in more general, crowded scenes in which there is no clear distinction. Our main contributions are a computationally efficient method for realistic occlusion inpainting and blending, especially in complex scenes. Our method can be effectively applied to any image, which is shown both qualitatively and quantitatively on a large dataset of stereo images. Our method performs natural disocclusion inpainting and maintains the shape and edge quality of foreground objects.
ISS-329
A self-powered asynchronous image sensor with independent in-pixel harvesting and sensing operations, Ruben Gomez-Merchan, Juan Antonio Leñero-Bardallo, and Ángel Rodríguez-Vázquez, University of Seville (Spain) [view abstract]
A new self-powered asynchronous sensor with a novel pixel architecture is presented. Pixels are autonomous and can harvest or sense energy independently. During the image acquisition, pixels toggle to a harvesting operation mode once they have sensed their local illumination level. With the proposed pixel architecture, most illuminated pixels provide an early contribution to power the sensor, while low illuminated ones spend more time sensing their local illumination. Thus, the equivalent frame rate is higher than the offered by conventional self-powered sensors that harvest and sense illumination in independient phases. The proposed sensor uses a Time-to-First-Spike readout that allows trading between image quality and data and bandwidth consumption. The sensor has HDR operation with a dynamic range of 80 dB. Pixel power consumption is only 70 pW. In the article, we describe the sensor’s and pixel’s architectures in detail. Experimental results are provided and discussed. Sensor specifications are benchmarked against the art.
COLOR-184
Color blindness and modern board games, Alessandro Rizzi1 and Matteo Sassi2; 1Università degli Studi di Milano and 2consultant (Italy) [view abstract]
Board game industry is experiencing a strong renewed interest. In the last few years, about 4000 new board games have been designed and distributed each year. Board game players gender balance is reaching the equality, but nowadays the male component is a slight majority. This means that (at least) around 10% of board game players are color blind. How does the board game industry deal with this ? Recently, a raising of awareness in the board game design has started but so far there is a big gap compared with (e.g.) the computer game industry. This paper presents some data about the actual situation, discussing exemplary cases of successful board games.
5:00 – 6:15 PM EI 2023 All-Conference Welcome Reception (in the Cyril Magnin Foyer)
SD&A Conference 3D Theater (M4)
Hosts:
Eric Kurland, 3-D SPACE (United States); Dan Lawrence, Lightspeed Design, Inc. (United States); John Stern, retired (United States); and Andrew Woods, Curtin University (Australia)
6:00 – 7:30 PM
Cyril Magnin II
This ever-popular session of each year's Stereoscopic Displays and Applications Conference showcases the wide variety of 3D content that is being produced and exhibited around the world. All 3D footage screened in the 3D Theatre Session is shown in high-quality polarized 3D on a large screen. The final program will be announced at the conference and 3D glasses will be provided.
Tuesday 17 January 2023
Stereoscopy in Education and Vergence Accommodation (T1)
Session Chair:
Takashi Kawai, Waseda University (Japan)
8:50 – 10:10 AM
Cyril Magnin II
8:50SD&A-388
A review of the VR workshop at the Grand Challenge 2022, Bjorn Sommer, Kam Raoofi, Robert Daniel Philips, Laura Ferrarello, Ashley Hall, and Paul Anderson, Royal College of Art (United Kingdom) [view abstract]
Since 2016 the RCA Grand Challenge is an interdisciplinary event that runs each year across the entire School of Design at the Royal College of Art. Around 400 Master and PhD students with design background participate in this event, focusing on tackling key global challenges through collaboration. In 2022, the Grand Challenge was focusing on elaborating topics in the context of Ocean-driven activities. In this context, a VR workshop was conducted with around 40 students with mostly no background in using VR platforms. In the end – after a 4-week design sprint – 10 UNREAL engine-based projects were presented of which we will discuss a selection here. One of the projects managed to be among the overall winning teams of the Grand Challenge 2022.
9:10SD&A-389
Use of a multi-display, mobile 2D / 3D video wall in surgical operating theatres, Justus F. Ilgner, Thien A. Duong Dinh, Martin Westhofen, and Stephan Hackenberg, RWTH Aachen University (Germany) [view abstract]
Following the trend towards endoscopic and microscopic surgery, the use of display technology in the operating room extends over many devices the surgeon uses even at the same time. Information from the operating microscope, endoscope, MRI and CT scans, navigation technology is mostly displayed from a variety of screens placed in various positions on the OR floor, thus increasing stress level for the surgeon, contributing to professional hazards due to cables on the floor, obstruction of passageways etc. The paper presents a low-cost, mobile, easy to operate display wall that can be retrofitted to existing OR theatres. It contains one HD stereo display for 3D microscopic images and three UHD mono displays for showing 2D endoscopic images alongside CT and MRI scans from the hospital's database and from intra-operative image-guided navigation. The system allows one 3D stream to be recorded on two solid-state disk recorders in full HD per side as well as one 2D stream on SD card independently. Moreover, one 3D stream in frame-compatible HD side-by-side format plus one full HD 2D stream can be delivered to any point in the hospital at the same time.
9:30SD&A-390
Investigating the use of spectacle lenses to alleviate vergence-accommodation mismatch in a stereoscopic remote vision system, Eleanor O'Keefe1, Kirk Moffitt2, Eric Seemiller3, Marc Winterbottom4, and Steven Hadley4; 1KBRwyle, 2Moffitt Consulting, 3EIS, and 4United States Air Force (United States) [view abstract]
Vergence-accommodation (VA) mismatch is a component of stereoscopic 3D remote vision system (RVS) design linked to depth misperception and visual discomfort. VA mismatch is caused by an unnatural conflict between the focal distance of the image (and thus accommodative demand) and the binocular vergence demand. A possible solution to mitigate VA mismatch is to change the accommodative demand with an optical correction, reducing the mismatch with the vergence demand. This experiment investigated the effect of low-add spectacle lenses (eyewear) on RVS performance and visual comfort. While previous research showed a positive effect of decreasing VA mismatch with the use of switchable lenses to adjust focal distance, the optical changes in this investigation were insufficient to make a difference. We conclude that the use of eyewear with a small dioptric add is not an effective solution to improve stereoscopic RVS performance or viewing comfort.
9:50SD&A-391
Observed optical resolution of light field display: Empirical and simulation results, Yuta Miyanishi, Qi Zhang, Erdem Sahin, and Atanas Gotchev, Tampere University (Finland) [view abstract]
Light field (LF) displays are a promising 3D display technology to mitigate the vergence-accommodation conflict. Recently, we have proposed a simulation framework to model an LF display system. It has predicted that the in-focus optical resolution on the retina would drop as the relative depth of a rendered image to the display-specific optical reference depth grows. In this study, we examine the empirical optical resolution of a near-eye LF display prototype by capturing rendered test images and compare it to simulation results based on the previously developed computational model. We use an LF display prototype that employs a time-multiplexing technique and achieves a high angular resolution of 6-by-6 viewpoints in the eyebox. The test image is rendered at various depths ranging 0–3 diopters, and the optical resolution of the best-focus images is analyzed from images captured by a camera. Additionally, we compare the measurement results to the simulation results, discussing theoretical and practical limitations of LF displays.
10:00 AM – 7:30 PM Industry Exhibition - Tuesday (in the Cyril Magnin Foyer)
10:20 – 10:50 AM Coffee Break
KEYNOTE: SD&A 2 (T2.1)
Session Chair: Nicolas Holliman, King's College London (United Kingdom)
10:50 – 11:50 AM
Cyril Magnin II
This session is jointly sponsored by: Engineering Reality of Virtual Reality 2023, and Stereoscopic Displays and Applications XXXIV.
SD&A-224
KEYNOTE: Human performance using stereo 3D in a helmet mounted display and association with individual stereo acuity, Bonnie Posselt, RAF Centre of Aviation Medicine (United Kingdom) [view abstract]
Wing Commander (Dr) Bonnie Posselt is a medical officer in the RAF (UK) specialising in Aviation and Space Medicine. Bonnie is currently based at the RAF Centre of Aviation Medicine in Bedfordshire, UK, having recently returned from a 3.5year exchange tour to Wright-Patterson Air Force Base in Ohio. While working with the 711th Human Performance Wing and the Air Force Research Laboratory (AFRL) in Ohio, Bonnie undertook a PhD in Helmet Mounted Displays and vision standards in collaboration with the University of Birmingham (UK). Bonnie is a graduate of the University of Manchester, King’s College London, and the International Space University. She is an associate fellow of the Aerospace Medical Association and elected member of the Royal Aeronautical Society.
Binocular Helmet Mounted Displays (HMDs) are a critical part of the aircraft system, allowing information to be presented to the aviator with stereoscopic 3D (S3D) depth, potentially enhancing situational awareness and improving performance. The utility of S3D in an HMD may be linked to an individual’s ability to perceive changes in binocular disparity (stereo acuity). Though minimum stereo acuity standards exist for most military aviators, current test methods may be unable to characterise this relationship. This presentation will investigate the effect of S3D on performance when used in a warning alert displayed in an HMD. Furthermore, any effect on performance, ocular symptoms, and cognitive workload shall be evaluated in regard to individual stereo acuity measured with a variety of paper-based and digital stereo tests.
Stereoscopy in VR (T2.2)
Session Chair:
Nicolas Holliman, King's College London (United Kingdom)
11:50 AM – 12:30 PM
Cyril Magnin II
This session is jointly sponsored by: Engineering Reality of Virtual Reality 2023, and Stereoscopic Displays and Applications XXXIV.
11:50SD&A-225
Incidence of stereo-blindness in a recent VR distance perception user study, Michael Wiebrands, Andrew J. Woods, and Hugh Riddell, Curtin University (Australia) [view abstract]
Estimates of stereo-blindness, the inability to see in 3D using stereopsis, often sit in the 5-10% range. At the Curtin University HIVE visualization facility, we regularly show stereoscopic content. During those demonstrations we invariably show a test random dot stereogram and it’s been our casual observation that the incidence of stereo-blindness amongst visits has been much lower than the 5-10% figure, perhaps as low as 2%. Our thought was that perhaps eye care has improved since the time that the original stereo-blindness studies were performed. A VR based user study was recently run in the HIVE, as part of a PhD project, with an aim to study distance perception in underwater virtual heritage experiences. Distance perception is facilitated by a range of visual cues, including stereoscopic vision, and as a result we screened participants for stereo-blindness. Using a standardized stereo test, we found approximately 5% of participants reported as stereo blind. This presentation will provide some background on stereo-blindness, and discuss issues related to measuring stereo-blindness and its likely prevalence in the general population. It will also provide an early look at the VR based user study investigating distance perception in underwater virtual heritage experiences.
12:10SD&A-226
Evaluating requirements for design education in a virtual studio environment, Bjorn Sommer1, Ayn Sayuti2, Zidong Lin1, Shefali Bohra1, Emre Kayganaci1, Caroline Yan Zheng1, Chang Hee Lee3, Ashley Hall1, and Paul Anderson1; 1Royal College of Art (United Kingdom), 2Universiti Teknologi MARA (UiTM) (Malaysia), and 3Korea Advanced Institute of Science and Technology (KAIST) (Republic of Korea) [view abstract]
Design education has special requirements on using virtual studio setups. The Zoom fatigue during the world-wide COVID pandemic let our institution to test new tools which can help improve digital design education by supporting features like idea sharing, collaborative making, serendipitous discussion and group forming. For this purpose, we tested different tools during our Master program Innovation Design Engineering and collected student feedback. We found that immersion is a key factor impacting the effectiveness of group work in distance learning. This paper presents our applications and analysis of different platforms and contributes insights on how to build a virtual studio environment for an interdisciplinary master programme in design engineering. In this work, we will focus on our studies on Gather.town, Meta Horizon Workrooms and Spatial.
12:30 – 2:00 PM Lunch
Tuesday 17 January PLENARY: Embedded Gain Maps for Adaptive Display of High Dynamic Range Images
Session Chair: Robin Jenkin, NVIDIA Corporation (United States)
2:00 PM – 3:00 PM
Cyril Magnin I/II/III
Images optimized for High Dynamic Range (HDR) displays have brighter highlights and more detailed shadows, resulting in an increased sense of realism and greater impact. However, a major issue with HDR content is the lack of consistency in appearance across different devices and viewing environments. There are several reasons, including varying capabilities of HDR displays and the different tone mapping methods implemented across software and platforms. Consequently, HDR content authors can neither control nor predict how their images will appear in other apps.
We present a flexible system that provides consistent and adaptive display of HDR images. Conceptually, the method combines both SDR and HDR renditions within a single image and interpolates between the two dynamically at display time. We compute a Gain Map that represents the difference between the two renditions. In the file, we store a Base rendition (either SDR or HDR), the Gain Map, and some associated metadata. At display time, we combine the Base image with a scaled version of the Gain Map, where the scale factor depends on the image metadata, the HDR capacity of the display, and the viewing environment.
Eric Chan, Fellow, Adobe Inc. (United States)
Eric Chan is a Fellow at Adobe, where he develops software for editing photographs. Current projects include Photoshop, Lightroom, Camera Raw, and Digital Negative (DNG). When not writing software, Chan enjoys spending time at his other keyboard, the piano. He is an enthusiastic nature photographer and often combines his photo activities with travel and hiking.
Paul M. Hubel, director of Image Quality in Software Engineering, Apple Inc. (United States)
Paul M. Hubel is director of Image Quality in Software Engineering at Apple. He has worked on computational photography and image quality of photographic systems for many years on all aspects of the imaging chain, particularly for iPhone. He trained in optical engineering at University of Rochester, Oxford University, and MIT, and has more than 50 patents on color imaging and camera technology. Hubel is active on the ISO-TC42 committee Digital Photography, where this work is under discussion, and is currently a VP on the IS&T Board. Outside work he enjoys photography, travel, cycling, coffee roasting, and plays trumpet in several bay area ensembles.
3:00 – 3:30 PM Coffee Break
VR Systems and Immersion (T3)
Session Chairs:
Margaret Dolinsky, Indiana University (United States) and Ian McDowall, Intuitive Surgical / Fakespace Labs (United States)
3:35 – 5:45 PM
Cyril Magnin II
This session is jointly sponsored by: Engineering Reality of Virtual Reality 2023, and Stereoscopic Displays and Applications XXXIV.
3:35ERVR-213
Display system sharpness modeling and requirement in VR and AR applications, Jiawei Lu, Trisha Lian, and Jerry Jia, Meta (United States) [view abstract]
Pixels per degree (PPD) alone is not a reliable predictor for high-resolution experience in VR and AR. This is because "high-resolution experience" not only depends on PPD but also display fill factor, pixel arrangement, graphics rendering, and other factors. This complicates architecture decisions and design comparisons. Is there a simple way to capture all the contributors and match user experience? In this paper, we present a system level model, system MTF, to predict perceptual quality considering all the key VR/AR dimensions: pixel shape (display), pixel per degree (display), fill factor (display), optical blur (Optics), and image processing (graphics pipeline). The metric can be defined in much the same way of traditional MTF for imaging systems by examining image formation of a point source and then performing Fourier transform over the response function, with special mathematical treatment. One application is presented on perceived text quality, where two weight functions depending on text orientation and frequency incorporated into the above model. A perceptual study about text quality was performed to validate the system MTF model.
4:05ERVR-214
Tangible extended reality with sensor fusion, Yang Cai, CMU (United States) [view abstract]
Many extended reality systems use controllers, e.g. near-infrared motion trackers or magnetic coil-based hand-tracking devices for users to interact with virtual objects. These interfaces lack tangible sensation, especially during walking, running, crawling, and manipulating an object. Special devices such as the Tesla suit and omnidirectional treadmills can improve tangible interaction. However, they are not flexible for broader applications, builky, and expensive. In this study, we developed a configurable multi-modal sensor fusion interface for extended reality applications. The system includes wearable IMU motion sensors, gait classification, gesture tracking, and data streaming interfaces to AR/VR systems. This system has several advantages: First, it is reconfigurable for multiple dynamic tangible interactions such as walking, running, crawling, and operating with an actual physical object without any controllers. Second, it fuses multi-modal sensor data from the IMU and sensors on the AR/VR headset such as floor detection. And third, it is more affordable than many existing solutions. We have prototyped tangible extended reality in several applications, including medical helicopter preflight walking around checkups, firefighter search and rescue training, and tool tracking for airway intubation training with haptic interaction with a physical mannequin.
4:25ERVR-216
Immersion, presence and behavioral validity in virtual and augmented environments, Daniel R. Mestre; CNRS and Aix-Marseille University (France) [view abstract]
Virtual (or Mixed) Reality, as opposed to previous experimental situations, has promoted realistic (also called ecological) research conditions, while enabling researchers to have precise control of these conditions. However, precisely because more realistic and complex situations can be experimentally investigated, important questions arise concerning the validity and representativeness of observed behavior, with reference to a real situation. This problem involves many aspects, such as the physical fidelity of the setup (including sensorial rendering and interfaces), but also human and contextual factors. Fundamental and applied research are both concerned. We will review past and ongoing research, aiming at evaluating to what extent users’ behavior in an immersive interactive context can be generalized to real situations.
4:45ERVR-217
Immersive security personnel training module for active shooter events, Sharad Sharma and Nishith Mannuru, University of North Texas (United States) [view abstract]
There is a need to prepare for emergencies such as active shooter events. Emergency response training drills and exercises are necessary to train for such events as we are unable to predict when emergencies do occur. There has been progress in understanding human behavior, unpredictability, human motion synthesis, crowd dynamics, and their relationships with active shooter events, but challenges remain. This paper presents an immersive security personnel training module for active shooter events in an indoor building. We have created an experimental platform for conducting active shooter drills for training that gives a fully immersive feel of the situation and allow one to perform virtual evacuation drills. The security personnel training module also incorporates four sub-modules namely 1) Situational assessment module, 2) Individual officer intervention module, 3) Team Response Module, and 4) Rescue Task Force module. We have developed an immersive virtual reality training module for active shooter events using an Oculus for course of action, visualization, and situational awareness for active shooter events as shown in Fig.1. The immersive security personnel training module aims to get information about the emergency situation inside the building. The dispatched officer will verify the active shooter situation in the building. The security personnel should find a safe zone in the building and secure the people in that area. The security personnel should also find the number and location of persons in possible jeopardy. Upon completion of the initial assessment, the first security personnel shall advise communications and request resources as deemed necessary. This will allow determining whether to take immediate action alone or with another officer or wait until additional resources are available. After successfully gathering the information, the personnel needs to update the info to their officer through a communication device.
5:05ERVR-218
Mobile augmented reality system for object detection, alert, and safety, Sharad Sharma1, Nishith Mannuru1, and Don Engel2; 1University of North Texas and 2University of Maryland, Baltimore County (United States) [view abstract]
The usefulness of mobile devices has increased greatly in recent years allowing users to perform more tasks in daily life. Mobile devices and applications provide many benefits for users, perhaps most significantly is the increased access to point-of-use tools, navigation, and alert systems. This paper presents a prototype of a cross-platform mobile augmented reality (AR) system with the core purpose of finding a better means to keep the campus community secure and connected. The mobile AR System consists of four core functionalities – an events system, a policing system, a directory system, and a notification system. The events system keeps the community up-to-date on current events that are happening or will be happening on campus. The policing system allows the community to stay in arms reach of campus resources that will allow them to stay secure. The directory system serves as a one-stop-shop for campus resources, ensuring that staff, faculty, and students will have a convenient and efficient means of accessing pertinent information on the campus departments. The mobile augmented reality system includes integrated guided navigation system that users can use to get directions to various destinations on campus. The various destinations are different buildings and departments on campus. This mobile augmented reality application will assist the students and visitors on campus to efficiently navigate the campus as well as send alert and notifications in case of emergencies. This will allow campus police to respond to the emergencies in a quick and timely manner. The mobile AR system was designed using Unity Game Engine and Vuforia Engine for object detection and classification. Google Map API was integrated for GPS integration in order to provide location-based services. Our contribution lies in our approach to create a user specific customizable navigational and alert system in order to improve the safety of the users at their workplace. Specifically, the paper describes the design and implementation of the proposed mobile AR system and reports the results of the pilot study conducted to evaluate their perceived ease-of-use, and usability.
5:30 – 7:00 PM EI 2023 Symposium Demonstration Session (in the Cyril Magnin Foyer)
Wednesday 18 January 2023
KEYNOTE: AR/VR Special Session 1 (W1)
Session Chair: Alexandre Chapiro, Meta (United States)
9:05 – 10:10 AM
Cyril Magnin II
This session is jointly sponsored by: Engineering Reality of Virtual Reality 2023, Human Vision and Electronic Imaging 2023, and Stereoscopic Displays and Applications XXXIV.
Joint Conference Welcome
HVEI-219
KEYNOTE: Display consideration for AR/VR systems, Ajit Ninan, Reality Labs at Meta (United States) [view abstract]
Ajit Ninan is a display industry veteran and led the way to the industry adopting HDR. His inventions & innovations are manifest in millions of shipped HDR TV’s and consumer electronics from multiple companies. He holds 400+ granted patents in imaging and display technology and now works in imaging related to AR/VR at Meta as Senior Director of Applied Perceptual Science and Image Quality. His work spans multiple subjects ranging from Displays, Imaging, Color, Video, Compression, Audio and Networking. His career spans early start-ups to public companies. Ninan is the inventor of the local dimmed quantum dot TV and led the way to the industry adoption of quantum dot displays by working with Vizio, Nanosys and 3M to release the first of its kind R-series QD TV with HDR. He also led the effort with the JPEG committee to standardize JPEG-XT to enable JPEG HDR images. Ninan was inducted as a SMPTE Fellow for his contributions to imaging and standards. The display that caused the world to adopt HDR called the “Pulsar” capable of 4000nits down to .005nits with P3 color in 2010, built by Ninan and his team, has received many awards including the Advanced Imaging Society’s Lumiere award which enabled the development of Dolby Vision and earned Ninan an Emmy.
AR and VR displays must take into consideration human perception and image quality factors that are required for a product. At Meta, we study these perceptual factors and determine what quality targets and requirements are needed. This talk will discuss some of these aspects and highlight examples of our process that help us set direction. The presenter, Ajit Ninan, is the director of Engineering, Display and Optics, at Meta.
10:00 AM – 3:30 PM Industry Exhibition - Wednesday (in the Cyril Magnin Foyer)
10:20 – 10:50 AM Coffee Break
AR/VR Special Session 2 (W2)
Session Chairs:
Nicko Caluya, Ritsumeikan University (Japan) and Alexandre Chapiro, Meta (United States)
10:50 AM – 12:30 PM
Cyril Magnin II
This session is jointly sponsored by: Engineering Reality of Virtual Reality 2023, Human Vision and Electronic Imaging 2023, and Stereoscopic Displays and Applications XXXIV.
10:50HVEI-220
Comparison of AR and VR memory palace quality in second-language vocabulary acquisition (Invited), Xiaoyang Tian, Nicko Caluya, and Damon M. Chandler, Ritsumeikan University (Japan) [view abstract]
The method of loci (memory palace technique) is a learning strategy that uses visualizations of spatial environments to enhance memory. One particularly popular use of the method of loci is for language learning, in which the method can help long-term memory of vocabulary by allowing users to associate location and other spatial information with particular words/concepts, thus making use of spatial memory to assist memory typically associated with language. Augmented reality (AR) and virtual reality (VR) have been shown to potentially provide even better memory enhancement due to their superior visualization abilities. However, a direct comparison of the two techniques in terms of language-learning enhancement has not yet been investigated. In this presentation, we present the results of a study designed to compare AR and VR when using the method of loci for learning vocabulary from a second language.
11:10HVEI-221
Projection mapping for enhancing the perceived deliciousness of food (Invited), Yuichiro Fujimoto, Nara Institute of Science and Technology (Japan) [view abstract]
The perceived deliciousness of a food item is highly related to its appearance. Image processing has been widely used to make food images more appealing to the public, such as when capturing and posting images on social networking sites. In this research, I propose a system to enhance the degree of subjective deliciousness of food visually perceived by a person by automatically changing its appearance with spatial augmented reality (SAR) technique in a real environment. The relationship between the degree of subjective deliciousness and four appearance features for each food category is modeled using data gathered via a crowdsourcing-based questionnaire. Using this model, the system generates the appropriate projection image to increase the deliciousness of the food. Experiments verify that the system can actually change and improve the impression of the target food’s deliciousness.
11:30HVEI-222
Real-time imaging processing for low-vision users, Yang Cai, CMU (United States) [view abstract]
We have developed an assistive technology for people with vision disabilities of central field loss (CFL) and low contrast sensitivity (LCS). Our technology includes a pair of holographic AR glasses with enhanced image magnification and contrast, for example, highlighting objects, and detecting signs, and words. In contrast to prevailing AR technologies which project either mixed reality objects or virtual objects to the glasses, Our solution fuses real-time sensory information and enhances images from reality. The AR glasses technology has two advantages: it’s relatively ‘fail-safe.” If the battery dies or the processor crashes, the glasses can still function because it is transparent. The AR glasses can also be transformed into a VR or AR simulator when it overlays virtual objects such as pedestrians or vehicles onto the glasses for simulation. The real-time visual enhancement and alert information are overlaid on the transparent glasses. The visual enhancement modules include zooming, Fourier filters, contrast enhancement, and contour overlay. Our preliminary tests with low-vision patients show that the AR glass indeed improved patients' vision and mobility, for example, from 20/80 to 20/25 or 20/30.
11:50HVEI-223
Critical flicker frequency (CFF) at high luminance levels, Alexandre Chapiro1, Nathan Matsuda1, Maliha Ashraf2, and Rafal Mantiuk3; 1Meta (United States), 2University of Liverpool (United Kingdom), and 3University of Cambridge (United Kingdom) [view abstract]
The critical flicker fusion (CFF) is the frequency of changes at which a temporally periodic light will begin to appear completely steady to an observer. This value is affected by several visual factors, such as the luminance of the stimulus or its location on the retina. With new high dynamic range (HDR) displays, operating at higher luminance levels, and virtual reality (VR) displays, presenting at wide fields-of-view, the effective CFF may change significantly from values expected for traditional presentation. In this work we use a prototype HDR VR display capable of luminances up to 20,000 cd/m^2 to gather a novel set of CFF measurements for never before examined levels of luminance, eccentricity, and size. Our data is useful to study the temporal behavior of the visual system at high luminance levels, as well as setting useful thresholds for display engineering.
12:10HVEI-253
A multichannel LED-based lighting approach to improve color discrimination for low vision people, Linna Yang1, Éric Dinet1, Pichayada Katemake2, Alain Trémeau1, and Philippe Colantoni1; 1University Jean Monnet Saint-Etienne (France) and 2Chulalongkorn University (Thailand) [view abstract]
The population of low vision people increases continuously with the acceleration of aging society. As reported by WHO, most of this population is over the age of 50 years and 81% were not concerned by any visual problem before. A visual deficiency can dramatically affect the quality of life and challenge the preservation of a safe independent existence. This study presents a LED-based lighting approach to assist people facing an age-related visual impairment. The research procedure is based on psychophysical experiments consisting in the ordering of standard color samples. Volunteers wearing low vision simulation goggles performed such an ordering under different illumination conditions produced by a 24-channel multispectral lighting system. A filtering technique using color rendering indices coupled with color measurements allowed to objectively determine the lighting conditions providing the best scores in terms of color discrimination. Experimental results were used to combine 3 channels to produce white light inducing a stronger color perception in a low vision context than white LEDs nowadays available for general lighting. Even if further studies will be required, these first results give hope for the design of smart lighting devices that adapt to the visual needs of the visually impaired.
12:30 – 2:00 PM Lunch
Wednesday 18 January PLENARY: Bringing Vision Science to Electronic Imaging: The Pyramid of Visibility
Session Chair: Andreas Savakis, Rochester Institute of Technology (United States)
2:00 PM – 3:00 PM
Cyril Magnin I/II/III
Electronic imaging depends fundamentally on the capabilities and limitations of human vision. The challenge for the vision scientist is to describe these limitations to the engineer in a comprehensive, computable, and elegant formulation. Primary among these limitations are visibility of variations in light intensity over space and time, of variations in color over space and time, and of all of these patterns with position in the visual field. Lastly, we must describe how all these sensitivities vary with adapting light level. We have recently developed a structural description of human visual sensitivity that we call the Pyramid of Visibility, that accomplishes this synthesis. This talk shows how this structure accommodates all the dimensions described above, and how it can be used to solve a wide variety of problems in display engineering.
Andrew B. Watson, chief vision scientist, Apple Inc. (United States)
Andrew Watson is Chief Vision Scientist at Apple, where he leads the application of vision science to technologies, applications, and displays. His research focuses on computational models of early vision. He is the author of more than 100 scientific papers and 8 patents. He has 21,180 citations and an h-index of 63. Watson founded the Journal of Vision, and served as editor-in-chief 2001-2013 and 2018-2022. Watson has received numerous awards including the Presidential Rank Award from the President of the United States.
3:00 – 3:30 PM Coffee Break
PANEL: AR/VR Special Session (W3.1)
Session Chairs: Nicko Caluya, Ritsumeikan University (Japan) and Alexandre Chapiro, Meta (United States)
Panelists: Alexandre Chapiro, Meta (United States); Yuichiro Fujimoto, Nara Institute of Science and Technology (Japan); Nicolas Holliman, King's College London (United Kingdom); and Ajit Ninan, Reality Labs at Meta (United States)
3:30 – 4:50 PM
Cyril Magnin II
This session is jointly sponsored by: Engineering Reality of Virtual Reality 2023, Human Vision and Electronic Imaging 2023, and Stereoscopic Displays and Applications XXXIV.
DISCUSSION: Wednesday End of Joint Sessions (W3.2)
Session Chair:
Damon Chandler, Ritsumeikan University (Japan)
4:50 – 5:30 PM
Cyril Magnin II
This session is jointly sponsored by: Engineering Reality of Virtual Reality 2023, Human Vision and Electronic Imaging 2023, and Stereoscopic Displays and Applications XXXIV.
Please join us for a lively discussion of today's presentations. Participate in an interactive, moderated discussion, where key topics and questions are discussed from many perspectives, reflecting the diverse HVEI community.
5:30 – 7:00 PM EI 2023 Symposium Interactive (Poster) Paper Session (in the Cyril Magnin Foyer)
5:30 – 7:00 PM EI 2023 Meet the Future: A Showcase of Student and Young Professionals Research (in the Cyril Magnin Foyer)
Thursday 19 January 2023
Imaging for XR Workshop 2023
8:45 AM – 6:00 PM
Cyril Magnin I/II
Engage with Others at Imaging for XR.
Held in-person during the Electronic Imaging Symposium and scheduled as the final day for SD&A attendees.
More at
XR Imaging Workshop.
6:00 AM – 10:50 PM Coffee Break