Monday 17 January 2022
IS&T Welcome & PLENARY: Quanta Image Sensors: Counting Photons Is the New Game in Town
07:00 – 08:10
The Quanta Image Sensor (QIS) was conceived as a different image sensor—one that counts photoelectrons one at a time using millions or billions of specialized pixels read out at high frame rate with computation imaging used to create gray scale images. QIS devices have been implemented in a CMOS image sensor (CIS) baseline room-temperature technology without using avalanche multiplication, and also with SPAD arrays. This plenary details the QIS concept, how it has been implemented in CIS and in SPADs, and what the major differences are. Applications that can be disrupted or enabled by this technology are also discussed, including smartphone, where CIS-QIS technology could even be employed in just a few years.
Eric R. Fossum, Dartmouth College (United States)
Eric R. Fossum is best known for the invention of the CMOS image sensor “camera-on-a-chip” used in billions of cameras. He is a solid-state image sensor device physicist and engineer, and his career has included academic and government research, and entrepreneurial leadership. At Dartmouth he is a professor of engineering and vice provost for entrepreneurship and technology transfer. Fossum received the 2017 Queen Elizabeth Prize from HRH Prince Charles, considered by many as the Nobel Prize of Engineering “for the creation of digital imaging sensors,” along with three others. He was inducted into the National Inventors Hall of Fame, and elected to the National Academy of Engineering among other honors including a recent Emmy Award. He has published more than 300 technical papers and holds more than 175 US patents. He co-founded several startups and co-founded the International Image Sensor Society (IISS), serving as its first president. He is a Fellow of IEEE and OSA.
08:10 – 08:40 EI 2022 Welcome Reception
Wednesday 19 January 2022
IS&T Awards & PLENARY: In situ Mobility for Planetary Exploration: Progress and Challenges
07:00 – 08:15
This year saw exciting milestones in planetary exploration with the successful landing of the Perseverance Mars rover, followed by its operation and the successful technology demonstration of the Ingenuity helicopter, the first heavier-than-air aircraft ever to fly on another planetary body. This plenary highlights new technologies used in this mission, including precision landing for Perseverance, a vision coprocessor, new algorithms for faster rover traverse, and the ingredients of the helicopter. It concludes with a survey of challenges for future planetary mobility systems, particularly for Mars, Earth’s moon, and Saturn’s moon, Titan.
Larry Matthies, Jet Propulsion Laboratory (United States)
Larry Matthies received his PhD in computer science from Carnegie Mellon University (1989), before joining JPL, where he has supervised the Computer Vision Group for 21 years, the past two coordinating internal technology investments in the Mars office. His research interests include 3-D perception, state estimation, terrain classification, and dynamic scene analysis for autonomous navigation of unmanned vehicles on Earth and in space. He has been a principal investigator in many programs involving robot vision and has initiated new technology developments that impacted every US Mars surface mission since 1997, including visual navigation algorithms for rovers, map matching algorithms for precision landers, and autonomous navigation hardware and software architectures for rotorcraft. He is a Fellow of the IEEE and was a joint winner in 2008 of the IEEE’s Robotics and Automation Award for his contributions to robotic space exploration.
EI 2022 Interactive Poster Session
08:20 – 09:20
EI Symposium
Poster interactive session for all conferences authors and attendees.
Tuesday 25 January 2022
IS&T Awards & PLENARY: Physics-based Image Systems Simulation
07:00 – 08:00
Three quarters of a century ago, visionaries in academia and industry saw the need for a new field called photographic engineering and formed what would become the Society for Imaging Science and Technology (IS&T). Thirty-five years ago, IS&T recognized the massive transition from analog to digital imaging and created the Symposium on Electronic Imaging (EI). IS&T and EI continue to evolve by cross-pollinating electronic imaging in the fields of computer graphics, computer vision, machine learning, and visual perception, among others. This talk describes open-source software and applications that build on this vision. The software combines quantitative computer graphics with models of optics and image sensors to generate physically accurate synthetic image data for devices that are being prototyped. These simulations can be a powerful tool in the design and evaluation of novel imaging systems, as well as for the production of synthetic data for machine learning applications.
Joyce Farrell, Stanford Center for Image Systems Engineering, Stanford University, CEO and Co-founder, ImagEval Consulting (United States)
Joyce Farrell is a senior research associate and lecturer in the Stanford School of Engineering and the executive director of the Stanford Center for Image Systems Engineering (SCIEN). Joyce received her BS from the University of California at San Diego and her PhD from Stanford University. She was a postdoctoral fellow at NASA Ames Research Center, New York University, and Xerox PARC, before joining the research staff at Hewlett Packard in 1985. In 2000 Joyce joined Shutterfly, a startup company specializing in online digital photofinishing, and in 2001 she formed ImagEval Consulting, LLC, a company specializing in the development of software and design tools for image systems simulation. In 2003, Joyce returned to Stanford University to develop the SCIEN Industry Affiliates Program.
PANEL: The Brave New World of Virtual Reality
08:00 – 09:00
Advances in electronic imaging, computer graphics, and machine learning have made it possible to create photorealistic images and videos. In the future, one can imagine that it will be possible to create a virtual reality that is indistinguishable from real-world experiences. This panel discusses the benefits of this brave new world of virtual reality and how we can mitigate the risks that it poses. The goal of the panel discussion is to showcase state-of-the art synthetic imagery, learn how this progress benefits society, and discuss how we can mitigate the risks that the technology also poses. After brief demos of the state-of-their-art, the panelists will discuss: creating photorealistic avatars, Project Shoah, and digital forensics.
Panel Moderator: Joyce Farrell, Stanford Center for Image Systems Engineering, Stanford University, CEO and Co-founder, ImagEval Consulting (United States)
Panelist: Matthias Neissner, Technical University of Munich (Germany)
Panelist: Paul Debevec, Netflix, Inc. (United States)
Panelist: Hany Farid, University of California, Berkeley (United States)
Wednesday 26 January 2022
KEYNOTE: Exascale Visualization
Session Chair: Thomas Wischgoll, Wright State University (United States)
10:00 – 11:05
Red Room
10:00
Conference Introduction
10:05VDA-403
KEYNOTE: The big changes behind exascale visualization [PRESENTATION-ONLY], Hank Childs, University of Oregon (United States)
Exascale computers, i.e., supercomputers that can perform one billion billion floating point operations per seconds, will arrive this year. To meet constraints in total cost and power usage, these computers have significantly different designs than the supercomputers from one decade ago. For the scientific visualization community, the two most important challenges from these new designs are the presence of accelerators with massive parallelism and the relative decrease in I/O bandwidth when compared to compute power. The accelerators challenge has led to the usage of data-parallel primitives to achieve both performance and portability. The I/O bandwidth challenge has led to a shift towards in situ processing, i.e., visualizing data as it is computed, which in turn has required new approaches for automation, data reduction, and software delivery. In this talk, I will describe the challenges the exascale visualization community has faced and the solutions we will deploy as exascale computers come online.
Hank Childs is a professor in the department of computer and information science at the University of Oregon. He received his PhD in computer science from the University of California at Davis (2006). Childs' research focuses on scientific visualization, high performance computing, and the intersection of the two. In July of 2012, Childs received the Department of Energy's Early Career Award to research visualization with exascale computers (i.e., computers that can do 1018 floating operations per second). Childs spent more than a dozen years at Lawrence Berkeley and Lawrence Livermore National Laboratories, directing research in big data visualization. Outside of his research, Childs is best known as the architect of the VisIt project, a visualization application for very large data that is used around the world.
10:45VDA-404
Visualizing and slicing topological surfaces in four dimensions (JIST-first), Hui Zhang and Huan Liu, University of Louisville (United States) [view abstract]
Smooth topological surfaces embedded in 4D create complex internal structures in their projected 3D figures. Often these 3D figures twist, turn, and fold back on themselves, leaving important properties behind the surface sheets. Triangle meshes are not well suited for illustrating such internal structures and their topological features. In this paper, we propose a new approach to visualize these internal structures by slicing the 4D surfaces in our dimensions and revealing the underlying 4D structures using their cross-sectional diagrams. We think of a 4D-embedded surface as a collection of 3D curves stacked and evolved in time, very much like a 3D movie in a time-elapse form; and our new approach is to translate a surface in 4-space into such a movie --- a sequence of time-lapse frames where successive terms in the sequence differ at most by a critical change. The visualization interface presented in this paper allows us to interactively define the longitudinal axis, and the automatic algorithms can partition the 4D surface into parallel slices and expose its internal structure by generating a time-lapse movie consisting of topologically meaningful cross-sectional diagrams from the representative slices. We have extracted movies from a range of known 4D mathematical surfaces with our approach. The results of the usability study show that the proposed slicing interface allows a mathematically true user experience with surfaces in four dimensions.
AR/VR Visualization
Session Chair:
David Kao, NASA Ames Research Ctr. (United States)
15:00 – 16:00
Red Room
15:00VDA-407
On the suitability of current augmented reality head-mounted devices, Sadan Suneesh Menon and Thomas Wischgoll, Wright State University (United States) [view abstract]
Simulation is a recognized and much-appreciated tool in healthcare and education. Advances in simulation have led to the burgeoning of various technologies. In recent years, one such technological advancement has been Augmented Reality (AR). Augmented Reality simulations have been implemented in healthcare on various fronts with the help of a plethora of devices including cellphones, tablets, and wearable AR headsets. AR headsets offer the most immersive experience of the AR simulation as they are head-mounted and offer a stereoscopic view of the superimposed 3D models through the attached goggles overlaid on real-world surfaces. To this effect, it is important to understand the performance capabilities of the AR headsets based on workload. In this paper, our objective is to compare the performances of two prominent AR headsets of today, the Microsoft Hololens and the Magic Leap One. We use surgical AR software that allows the surgeons to show internal structures, such as the rib cage, to assist in the surgery as a reference application to obtain performance numbers for those AR devices. Based on our research, there are no performance measurements and recommendations available for these types of devices in general yet.
15:20VDA-408
AR visualization for coastal water navigation, Randy Herritt and Stephen Brooks, Dalhousie University (Canada) [view abstract]
When conducting Coastal Water Navigation, a ship's Navigating Officer (NavO) has multiple sources of data to consider. To obtain the information required to safely manoeuvre the ship, they make use of specialized equipment. The time spent interacting with the equipment is a risk, as it prevents them from visually monitoring the ever-changing maritime environment. Data visualization through Augmented Reality (AR) offers a way to obtain the information while maintaining a proper and effective lookout. Additionally, our research suggests that the information can be presented in new ways. We created a simulator that allows testing and evaluation of AR Navigation Aids (ARNAs). These visualizations were evaluated by subject matter experts through a user study. The user study suggests that ARNAs can improve maritime safety and assist in the conduct of navigation.
15:40VDA-409
Digital reconstruction of Elmina Castle for mobile virtual reality via point-based detail transfer, Sifan Ye1, Ting Wu2, Michael Jarvis3, and Yuhao Zhu3; 1Stanford University, 2eBay Inc., and 3University of Rochester (United States) [view abstract]
Reconstructing 3D models from large, dense point clouds is critical to enable Virtual Reality (VR) as a platform for entertainment, education, and heritage preservation. Existing 3D reconstruction systems inevitably make trade-offs between three conflicting goals: the efficiency of reconstruction (e.g., time and memory requirements), the visual quality of the constructed scene, and the rendering speed on the VR device. This paper proposes a reconstruction system that simultaneously meets all three goals. The key idea is to avoid the resource-demanding process of reconstructing a high-polygon mesh altogether. Instead, we propose to directly transfer details from the original point cloud to a low polygon mesh, which significantly reduces the reconstruction time and cost, preserves the scene details, and enables real-time rendering on mobile VR devices. While our technique is general, we demonstrate it in reconstructing cultural heritage sites. We for the first time digitally reconstruct the Elmina Castle, a UNESCO world heritage site at Ghana, from billions of laser-scanned points. The reconstruction process executes on low-end desktop systems without requiring high processing power, making it accessible to the broad community. The reconstructed scenes render on Oculus Go in 60 FPS, providing a real-time VR experience with high visual quality.
Information Visualization and Analytics Tools
Session Chair:
Yi-Jen Chiang, New York University (United States)
16:15 – 17:15
Red Room
16:15VDA-413
CoursePathVis: Course path visualization using flexible grouping and funnel-augmented Sankey diagram, Brendan J. O'Handley, Morgan K. Ludwig, Samantha R. Allison, Michael T. Niemier, Shreya Kumar, Ramzi Bualuan, and Chaoli Wang, University of Notre Dame (United States) [view abstract]
We present CoursePathVis, a visual analytics tool for exploring and analyzing students’ progress through a college curriculum using a Sankey diagram. Focusing on four student cohorts in a department, we group students in multiple ways (by their AP courses, term courses, and a user specified funnel course) to comprehensively understand the data. CoursePathVis helps us identify patterns or outliers that affect student success with these flexible grouping techniques and the funnel-augmented Sankey diagram. Three stakeholders from the same department formulate design requirements and provide an ad-hoc evaluation.
16:35VDA-414
Visualizing semantic 3D object clouds, Bola Okesanjo and Stephen Brooks, Dalhousie University (Canada) [view abstract]
3D object clouds, first introduced by Hong and Brooks, visualize the pairwise similarity between a set of objects and a central object of interest. This similarity is used to determine the position of each object within the cloud. However, this does not capture the semantic relationship of all the objects and the lack of consistently may reduce the expectation of finding an object when performing visual search. To generate a semantic 3D object cloud, we define and subsequently minimize an energy function that captures the pairwise similarity amongst all the objects within the cloud. The energy is minimized using several statistical machine learning techniques and we show that the generated layouts from such techniques outperform those of other algorithms on a variety of metrics for evaluating layouts.
16:55VDA-415
Nirmaan: Dataset generation for multiclass scatterplot studies, Allison Wong1, Alark Joshi2, and Sophie Engle2; 1Geico and 2University of San Francisco (United States) [view abstract]
This paper presents Nirmaan, an open-source web-based tool for generating synthetic datasets of multiclass blobs for use in research related to scatterplots. We demonstrate how to use Nirmaan to generate datasets in the context of a user study where users must determine the centers of each class, but this tool can be used to generate datasets for other scatterplot tasks as well.