Conference Keynotes: Broaden Your Horizons 

A number of the conferences invite prominent individuals to become keynote speakers. Attendees make a point of listening to all of these talks to gain a broader understanding of the current state of Electronic Imaging advances. Following is the EI2022 conference keynote line-up.

Autonomous Vehicles and Machines 2022

Aerial Navigation

Deep drone navigation and advances in vision-based navigation, Matthias Müller, Embodied AI Lab at Intel (Germany)

This talk will be divided into two parts. In the first part, I will present our recent line of work on deep drone navigation in collaboration with the University of Zurich. We have developed vision-based navigation algorithms that can be trained entirely in simulation via privileged learning and then transferred to a real drone that performs acrobatic maneuvers or flies through complex indoor and outdoor environments at high speeds. This is achieved by using appropriate abstractions of the visual input and relying on an end-to-end pipeline instead of a modular system. Our approach works with only onboard sensing and computation. In the second part, I will present some interesting advances in graphics, computer vision and robotics from our lab with an outlook of their application to vision-based navigation.

Matthias Müller holds a BSc in Electrical Engineering and Math Minor from Texas A&M University. Early in his career, he worked at P+Z Engineering as an electrical engineer developing mild-hybrid electric machines for BMW. Later, he obtained a MSc and PhD in electrical engineering from KAUST with focus on persistent aerial tracking and sim-to-real transfer for autonomous navigation. Matthias has contributed to more than 15 publications published in top tier conferences and journals such as CVPR, ECCV (best paper ECCV'18 UAVision workshop), and ICCV. He has extensive experience in object tracking and autonomous navigation of embodied agents such as cars and UAVs.

Deep Learning and Video Restoration

Deep learning for image and video restoration/super-resolution, Ahmet Murat Tekalp, Koç University (Turkey)

Recent advances in neural architectures and training methods led to significant improvements in the performance of learned image/video restoration and SR. We can consider learned image restoration and SR as learning either a mapping from the space of degraded images to ideal images based on the universal approximation theorem, or a generative model that captures the probability distribution of ideal images. An important benefit of data-driven deep learning approach is that neural models can be optimized for any differentiable loss function, including visual perceptual loss functions, leading to perceptual video restoration and SR, which cannot be easily handled by traditional model-based approaches. I will discuss loss functions and evaluation criteria for image/video restoration and SR, including fidelity and perceptual criteria, and the relation between them, where we briefly review the perception vs. fidelity (distortion) trade-off. We then discuss practical problems in applying supervised training to real-life restoration and SR, including overfitting image priors and overfitting the degradation model and some possible ways to deal with these problems.

Ahmet Murat Tekalp holds BS degrees in electrical engineering and mathematics from Bogazici University, and a PhD in electrical, computer, and systems engineering from Rensselaer Polytechnic Institute. After working at Eastman Kodak Company, then the University of Rochester, since June 2001, he has been a professor at Koc University in Istanbul, where he also served as dean of Engineering for some time. His research interests are in the area of digital image and video processing, including video compression and streaming, motion-compensated filtering, super-resolution, video segmentation, object tracking, content-based video analysis and summarization, 3D video processing, deep learning for image and video processing, video streaming and real-time video communications services, and software-defined networking. He is a Fellow of IEEE and member of Turkish Academy of Sciences and Academia Europaea. He was named as 1998 Distinguished Lecturer by IEEE Signal Processing Society, awarded a Fulbright Senior Scholarship in 1999, and received the TUBITAK Science Award (highest scientific award in Turkey) in 2004. Ahmet holds eight US patents and his group contributed technology to the ISO/IEC MPEG-4 and MPEG-7 standards

Computational Imaging XX

Computational Imaging - Phase Retrieval

Computational imaging by phase retrieval: From astronomical speckle to x-ray coherent diffractive imaging, James R. Fienup, University of Rochester (United States)

Researchers used phase retrieval for reconstructing electron density functions of crystalline structures from scattered x-ray data (x-ray crystallography) for many decades. In the 1970's, algorithms were finally developed that worked for non-crystalline (non-periodic), general objects and were applied to astronomical imaging, overcoming the blurring effects of atmospheric turbulence, using data from Labeyrie's stellar speckle interferometry approach. In 1999, the algorithms developed for astronomy began to be used for reconstructing images of non-crystalline objects illuminated with coherent x-rays. As high-brightness, highly coherent x-ray sources were developed, the field of x-ray coherent diffractive imaging grew. The development of alternative data-collection approaches, ptychography in particular, allows for a very robust reconstruction of images of small objects on the nanometer scale. This presentation will describe the development of those phase retrieval algorithms.

James R. Fienup received an AB in physics and mathematics from Holy Cross College (Worcester, MA), and a PhD in applied physics from Stanford University, where he was a National Science Foundation graduate fellow. He performed research for 27 years at the Environmental Research Institute of Michigan and Veridian Systems, where he was a senior scientist. He joined the faculty at the University of Rochester in 2002 as the Robert E. Hopkins Professor of Optics. Fienup is a fellow of OSA and SPIE, and a senior member of IEEE. He was awarded the SPIE 1979 Rudolf Kingslake Medal and Prize, the 1983 International Prize in Optics fInternational Commission for Optics), the OSA 2013 Emmett N. Leith Medal, and became a member of the National Academy of Engineering in 2012. He was a distinguished visiting scientist at JPL in 2009, was editor-in-chief of JOSA (1997-2003), served as division editor of Applied Optics - Information Processing, and associate editor of Optics Letters. One of his papers [J.R. Fienup, "Phase Retrieval Algorithms: a Comparison," Appl. Opt. 21, 2758-2769 (1982)] has received over 4,600 citations (Google Scholar) and is the most highly cited paper (out of more than 50,000) in the journal Applied Optics.

Human Vision and Electronic Imaging 2022

High Dynamic Range

HDR arcana, Scott Daly, Dolby Laboratories, Inc. (United States)

Consumers seeing the high-end versions of these displays for the first time typically comment that the imagery shows more depth ("looks like 3D"), or looks more realistic, ("feels like you're there"), or has stronger affectivity ("it's visceral") or has a wow effect ("#!@*&% amazing"). Prior to their introduction to the consumer market, such displays were being demonstrated to the technical community. This motivated detailed discussions of the need for an ecosystem (capture, signal format, and display) which were fruitful, but at the same time often led to widely stated common misunderstandings. These often boiled down HDR to a single issue with statements like "HDR is all about the explosions" referring to its capability to convey strong transients in luminance. Another misconception was "HDR causes headaches" referring to effects caused by poor creative choices or sloppy automatic processing. Other simplifying terms such as brightness, bit-depth, contrast ratio, image capture f-stops, display capability, have all been used to describe "the key" aspect of HDR. One misunderstanding circa 2010 that permeated photography hobbyists was "HDR makes images look like paintings", often meant as a derision. While the technical community has moved beyond such oversimplifications, there still are key perceptual phenomenon involved with HDR displayed imagery that are either poorly understood or rarely mentioned. The field of applied vision science is at a mature enough state to have enabled engineering design for signal formats, image capture and display capabilities needed to create both consumer and professional HDR ecosystems. Light-adaptive CSF models, optical PSF and glare, LMS cone capture, opponent colors, and color volume are examples used in the ecosystem design. However, we don't have a similar level of quantitative understanding of why HDR triggers the kinds of expressions mentioned at the beginning of this paragraph. This talk will give a survey of the apparently mysterious perceptual issues of HDR being explored by a handful of researchers often unaware of each other's work. Coupled with several hypotheses and speculation, this focus on the arcane aspects of HDR perception is hoping to motivate more in-depth experiments and understanding.

Scott Daly is an applied perception scientist at Dolby Laboratories with specialties in spatial, temporal, and chromatic vision. He has significant experience in applications toward display engineering, image processing, and video compression with more than 100 technical papers. Current focal areas include high dynamic range, auditory-visual interactions, physiological assessment, and preserving artistic intent. He has a BS in bioengineering from North Carolina State University and an MS in bioengineering from the University of Utah. Past accomplishments led to the Otto Schade award from the Society for Information Display (SID) in 2011, and a team technical Emmy in 1990. He is a member of the IEEE, SID, and SMPTE. He recently completed the 100-patent dash in just under 30 years.

Quality of Experience

Two aspects of quality of experience: Augmented reality for the industry and for the hearing impaired & Current research at the Video Quality Experts Group (VQEG), Kjell Brunnström^1,2; ¹RISE Research Institutes of Sweden AB and ²Mid Sweden University (Sweden)

This presentation will be divided into two parts. (1) Focus on Quality of Experience (QoE) of Augmented Reality (AR) for industrial applications and for aids for the hearing impaired. Examples will be given from research done at RISE Research Institutes of Sweden and Mid Sweden University on remote control of machines and speech-to-text presentations in AR. (2) An overview of current work of the Video Quality Experts Group (VQEG), an international organization of video experts from both industry and academia. At the beginning VQEG was focused around measuring perceived video quality. Over the last 20 years from the formation, it has shifted the expertise from the visual quality of video to QoE (not involving audio), taking a more holistic view on the visual quality perceived by the user in contemporary video based services and applications.

Kjell Brunnström, PhD, is a Senior Scientist at RISE (Digital System, Dep. Industrial Systems, Unit Networks), leading Visual Media Quality and Adjunct Professor at Mid Sweden University. He is co-chair of the Video Quality Experts Group (VQEG). Brunnström's research interests are in Quality of Experience (QoE) for video and display quality assessment (2D/3D, VR/AR, immersive). He is associate editor of the Elsevier journal Signal Processing: Image Communication and has written more than hundred articles in international peer-reviewed scientific journals and conferences.

Imaging and Multimedia Analytics at the Edge 2022

25 Years of Photography

Analogue – Digital – Mobile – Social: How photography has changed in the last 25 years, Reiner Fageth, CEWE Stiftung & Co. KGAA (Germany)

This paper will describe how digital photography has evolved from being a niche product for digital experts and IT freaks, to total domination of the mass-market and the disruption of analogue photography. This rapid, industry-altering process will be put into relationship to conferences and presentations given at Electronic Imaging in the last three decades. Developments and influence on quality were driven by imaging sensors and printing technologies. A review of the battle for attainting the high quality of silver-halide prints from negatives will be presented. The development of image enhancement technologies and printing technologies will be analyzed. The classical one-hour in-store photo order process in North America and Asia based on mini labs, and the logistics systems in Europe (collecting the film from the points of sale in the evening and returning the prints, processed by huge photofinishing plants, the next day) were substituted by digital kiosk systems in stores, and software applications in the browser or downloadable software. The development of the technologies involved will also be presented, as well as the efforts in supporting the selection process of the most suitable images. New digital printers based on liquid ink and toner offered new products, personalized photobooks and calendars allowed for story-telling and emotionalized gifting via tangible photo products and raised the value of every printed image. A review will be provided of the improvements there, as well as classical silver-halide based printing systems. The introduction of smartphones disrupted the new digital imaging ecosystems once more. The camera was now a constant companion in nearly everybody's pocket. The resulting increase in the number of images taken complicated the selection process (convenience photos, images "only" for social communication, ...) and the image quality discussion was once more raised and addressed in these conference. All of these challenges are addressed by actual imaging eco-systems. They include ordering possibilities over all devices (classical digital cameras and smartphones) and retail locations, as well as providing home delivery options. Selling these products became more of a marketing than a technological challenge. These systems utilize AI based solutions (on device and utilizing edge computing) combined with experience/heuristics gathered in the last 25 years. Some very good approaches will be presented at the end.

Reiner Fageth is head-Technology, Research & Development at CEWE Stiftung & Co. KGaA and chairman-Supervisory Board of CEWE Color as (a subsidiary of CEWE Stiftung & Co. KGaA). He is also on the board of CeWe Color, Inc. and Member-Management Board at Neumüller CEWE COLOR Stiftung. Fageth studied electronic engineering at the Fachhochschule Heilbronn, Germany and received a PhD from the University of Northumbria at Newcastle, UK in split research with Telefunken Microelectronic and the Steinbeis Transferzentrum Image Processing. His major research topics there and also for the following years were industrial image processing systems based on classification using fuzzy logic and neural networks. In 1998 he joined CeWe Color with the charge to drive the analog photo business into digital. First he was responsible for R&D and the production of consumers digital files on silver halide paper. He is a member of the German DIN Normenausschuss Bild und Film NA 049-00-04 AA and has published more than 30 technical papers.

Image Processing: Algorithms and Systems XX

Perception and Image Quality

Perception-guided image quality measurements: Principles and future trends, Sos S. Agaian, College of Staten Island and the Graduate Center, CUNY (United States)

Bio-inspired image processing is about learning image algorithms from computational neuroscience, cognitive science, and biology and applying them to the design of real-world image processing-based systems. More specifically, this field is giving computers the ability to "see" just as humans do. Recently, many useful image processing algorithms developed with varying degrees of correspondence with biological vision studies. This is natural since a biological system can provide a source of inspiration for new computational efficient/robust vision models and measurements. Simultaneously, the image processing tools may give new insights for understanding biological visual systems. Digital images are subject to various distortions during acquisition, processing, transmission, compression, storage, and reproduction. How can we automatically predict quantitatively or perceived image quality? In this talk, we present originating in visual perception studies: Visual perception-driven image quality measurements: principles, future trends, applications. We will also give our recent research works and a synopsis of the current state-of-the-art results in image quality measurements and discuss future trends in these technologies and the associated commercial impact and opportunities.

Sos S. Agaian is a distinguished professor of computer science at CSI and the Graduate Center, CUNY. Prior, he was a Peter T. Flawn professor of the University of Texas at San Antonio. His research sponsors include DARPA, NSF, US Departments of Transportation and Energy, NIJ, and private industry. Agaian's research interests are in big and small data analytics, computational vision and sensing, machine learning and urban computing, multimodal biometric and digital forensics, information processing and fusion, and fast algorithms. He has special interests in finding meaning in visual content-examine images for faces, text, objects, action, sciences, and other contents; and in the development of scientific systems and architectures in the theory and practice of engineering and computer sciences.Agaian has developed applications in multiple application areas; has published 750 articles, 10 books, and 19 book chapters; and holds more than 56 issued or pending patents/ disclosures, several of which are commercially licensed. He is an associate editor for several journals, including the Image processing Transaction (IEEE) and IEEE Transaction of Cybernetics. He is a fellow of IS&T, SPIE, AAAS, IEEE, and AAI.

Image Quality and System Performance XIX

Quality and Perception

Towards neural representations of perceived visual quality, Sebastian Bosse, Fraunhofer Heinrich Hertz Institute (Germany)

Accurate computational estimation of visual quality as it is perceived by humans is crucial for any visual communication or computing system that has humans as the ultimate receivers. But most importantly besides the practical importance, there is a certain fascination to it: While it is so easy, almost effortless, to assess the visual quality of an image or a video, it is astonishingly difficult to predict it computationally. Consequently, the problem of quality estimation touches on a wide range of disciplines like engineering, psychology, neuroscience, statistics, computer vision, and, since a couple of years now, on machine learning. In this talk, Bosse gives an overview of recent advances in neural network-based-approaches to perceptual quality prediction. He examines and compares different concepts of quality prediction with a special focus on the feature extraction and representation. Through this, Bosse revises the underlying principles and assumptions, the algorithmic details and some quantitative results. Based on a survey of the limitations of the state of the art, Bosse discusses challenges, novel approaches and promising future research directions that might pave the way towards a general representation of visual quality.

Sebastian Bosse is head of the Interactive & Cognitive Systems group at Fraunhofer Heinrich Hertz Institute (HHI), Berlin, Germany. He studied electrical engineering and information technology at RWTH Aachen University, Germany, and Polytechnic University of Catalonia, Barcelona, Spain. Sebastian received the Dr.-Ing. degree in computer science (with highest distinction) from the Technical University Berlin in 2018. During his studies he was a visiting researcher at Siemens Corporate Research, Princeton, USA. In 2014, Sebastian was a guest scientist in the Stanford Vision and Neuro-Development Lab (SVNDL) at Stanford University, USA. After 10 years as a research engineer working in the Image & Video Compression group and later in the Machine Learning group, he founded the research group on Interactive & Cognitive Systems at Fraunhofer HHI in 2020 that he has headed since. Sebastian is a lecturer at the German University in Cairo. He is on the board of the Video Quality Expert Group (VQEG) and on the advisory board of the Interational AIQT Foundation. Sebastian is an affiliate member of VISTA, York University, Toronto, and serves as an associate editor for the IEEE Transactions on Image Processing. Since 2021 he has been appointed a chair for the ITU focus group on Artificial Intelligence for Agriculture. His current research interests include the modelling of perception and cognition, machine learning, computer vision, and human-machine interaction over a wide field of applications ranging from multimedia and augmented reality, through medicine to agriculture and industrial production.

Imaging Sensors and Systems 2022

Sensing for Autonomous Driving

Recent developments in GatedVision imaging - Seeing the unseen, Ofer David, BrightWay Vision (Israel)

Imaging is the basic building block for automotive autonomous driving. Any computer vision system will require a good image as an input at all driving conditions. GatedVision provides an extra layer on top of the regular RGB/RCCB sensor to augment these sensors at nighttime and harsh weather conditions. GatedVision images in darkness and different weather conditions will be shared. Imagine that you could detect a small target laying on the road having the same reflectivity as the back ground meaning no contrast, GatedVision can manipulate the way an image is captured so that contrast can be extracted. Additional imaging capabilities of GatedVision will be presented.

Ofer David has been BrightWay Vision CEO since 2010. He has more than 20 years experience in the area of active imaging systems and laser detection, and has produced various publications and patents. Other solutions in which he is involved include fog penetrating day/night imaging systems and visibility measurement systems. David received his BSc and MSc from the Technion – Israel Institute of Technology and his PhD in electro-optics from Ben-Gurion University.

AR/VR Sensors

Sensing and computing technologies for AR/VR, Chiao Liu, Meta Reality Labs Research (United States)

Augmented and Virtual Reality (AR/VR) will be the next great wave of human oriented computing, dominating our relationship with the digital world for the next 50 years, much as personal computing has dominated the last 50. AR glasses require multiple cameras to enable all the computer vision (CV) and AI functions while operating under stringent weight, power, and socially acceptable form factor constraints. The AR sensors need to be small, ultra-low power, with wide dynamic range (DR) and excellent low light sensitivity to support day/night, indoor/outdoor, all day wearable use cases. The combination of lowest power, best performance, and minimal form factor makes AR sensors the new frontier in the image sensors field. In this talk, we will first introduce some CV and AI functions to be supported by AR sensors and their associated camera sensor requirements. We will then present a new ultra-low power, ultra-wide dynamic range Digital Pixel sensor (DPS) designed to meet above specific challenges. Finally, we will discuss some system level tradeoffs and architecture directions.

Chiao Liu received his PhD in EE from Stanford University. He was a senior scientist at Canesta Inc (now part of Microsoft), developing the very first CMOS time-of-flight (ToF) depth sensors. He was a technical fellow at Fairchild Imaging (now part of BAE Systems), and worked on a wide range of scientific and medical imaging systems. In 2012, he joined Microsoft as a principal architect and was part of the 1st generation Microsoft AR Hololens team. Currently he is the director of research at Meta Reality Labs Research, leading the Sensors and Systems Research team. Liu is a member of the IEEE International Electron Devices Meeting (IEDM) technical committee. He also served as guest reviewer for Nature and IEEE Transactions on Electron Devices.

Machine Learning for Scientific Imaging 2022

Computational Imaging Pipelines

Learning to image the invisible, Katherine L. Bouman, California Institute of Technology (United States)

As imaging requirements become more demanding, we must rely on increasingly sparse and/or noisy measurements that fail to paint a complete picture. Computational imaging pipelines, which replace optics with computation, have enabled image formation in situations that are impossible for conventional optical imaging. For instance, seismic and black hole imaging have only been made possible through the development of computational imaging pipelines. However, these computational "cameras" often suffer from (a) being difficult to analyze for image uncertainties, and (b) forward model mismatch. This talk will discuss how we are leveraging and building upon recent advances in machine learning in order to achieve more efficient uncertainty quantification and to better recover solutions in the presence of unknown model mismatch. The proposed approaches will be presented in the context of discussing the methods and procedures used to capture the first and future images of a black hole from the Event Horizon Telescope, as well as seismic localization and tomography.

Katherine L. Bouman is a Rosenberg Scholar and assistant professor of Computing and Mathematical Sciences (CMS) and by courtesy in Electrical Engineering and Astronomy at Caltech. Bouman's research focuses on computational imaging includes designing systems that tightly integrate algorithm and sensor design, making it possible to observe phenomena previously difficult or impossible to measure with traditional approaches. Her group at Caltech combines ideas from signal processing, computer vision, machine learning, and physics to find and exploit hidden signals for both scientific discovery and technological innovation. Prior to starting at Caltech, Bouman was a postdoctoral fellow with the Event Horizon Telescope, which published the first picture of a black hole in April of 2019. She received her BSE in electrical engineering from University of Michigan and her SM and PhD in electrical engineering and computer science from MIT. Bouman is a recipient of an NSF CAREER Award, the 2020 Electronic Imaging Scientist of the Year Award, an Okawa Research Grant, a Caltech faculty teaching award, a finalist for the AAAS Early Career Award for Public Engagement with Science, and is a co-recipient of the Breakthrough Prize.

Machine Learning for Scientific Imaging 2022

Inverse Problems in Imaging

Tackling tough inverse problems in imaging using PINNs and DeepOnets, George E. Karniadakis, Brown University (United States)

We will review physics-informed neural networks (PINNs) and operator regression networks (DeepOnets) with emphasis on discovering missing physics and system identification in diverse applications in fluid mechanics, solid mechanics, bioengineering, and beyond. We will demonstrate that we can use multimodality inputs from images and point measurements to discover effects in materials, obtain three-dimension fields, and improve greatly existing techniques such as particle tracking in fluid mechanics. The diverse problems we consider are ill-posed and cannot be solved with any traditional methods. For example, in one application in aortic dissections we identify from mechanical measurements the genotype that corresponds to the specific mouse tested among five different classes.

George E. Karniadakis is the Charles Pitts Robinson and John Palmer Barstow Professor of Applied Mathematics and Engineering at Brown University  in the Center for Fluid Mechanics. He  received his SM and PhD from MIT. After serving as a lecturer in the Department of Mechanical Engineering at MIT, he joined the Center for Turbulence Research at Stanford / Nasa Ames, before becoming assistant professor at Princeton University in the Department of Mechanical and Aerospace Engineering and associate faculty in the Program of Applied and Computational Mathematics. He has also been a visiting professor at Caltech in the Aeronautics Department; a visiting professor and senior lecturer of Ocean/Mechanical Engineering at MIT since 2000, and twice a visiting professor at Peking University. He has had a joint appointment with PNNL since 2013. Karniadakis is a Fellow of SIAM, APS, ASME, and AIAA He received the 2021 SIAM CSE/ACM prize, the 2015 SIAM Ralf E Kleinman Award, the (inaugural) 2013 J. Tinsley Oden Medal, and the 2007 CFD award from the US Association in Computational Mechanics. His h-index is 112 and he has been cited more than 59,500 times.

Machine Learning - High Data Volume

Physics based machine learning, David Rousseau, Université Paris-Saclay (France)

High Energy Physics experiments aim at establishing fundamental laws of physics by measuring the product of high energy particle collision with increasingly complex detectors. For example, tracking detectors deliver point clouds of micron precision over cubic meters, calorimeter detectors provide energy deposits in irregularly shapes voxels. Dedicated Machine Learning techniques are developed to deal with the specificities of the data, with the constraints of: dealing with large volume of data (many PetaBytes) within resources, maximise sensitivity to rare phenomenons, master unknowns along the full pipe line to be able to quote uncertainties. In addition, a long history of building accurate complex simulators is being supplemented by ML generators orders of magnitude faster but which have to demonstrate their ability to reproduce all details of the feature space.

David Rousseau is a high energy physicist at IJCLab, CNRS/IN2P3, and Université Paris-Saclay, currently working for the ATLAS experiment at CERN on the Large Hadron Collider. After a dozen years designing and implementing many pieces of the ATLAS experiment software, a chance meeting in 2013 with a machine learning (what was this?) computer scientist decided a new path in his career: develop the interface between high energy physics (and science in general) and machine learning (or artificial intelligence). Current research topics include: simulation based inference, machine learning for fast particle tracking at the large hadron collider, generator models for fast detector simulation, using an OPU for event classification, and uncertainty aware training. Rousseau received his PhD in high energy physics from the Université Aix-Marseille II. Since 2009, Rousseau has been senior researcher (directeur de recherche) at CNRS/IN2P3 LAL-Orsay, which merged into IJCLab-Orsay in 2020. He began his career at CERN as a CERN Fellow in 1997.

2D Unknown View Tomography

An adversarial learning approach for 2D unknown view tomography, Mona Zehni and Zhizhen J. Zhao, University of Illinois at Urbana-Champaign (United States)

The goal of 2D tomographic reconstruction is to recover an image given its projection lines from various views. It is often presumed that projection angles associated with the projections are known in advance. Under certain situations, however, these angles are known only approximately or are completely unknown, when imaging moving objects. It becomes challenging to reconstruct the image from a collection of random projection. We introduce an adversarial learning based approach to recover the image and the projection angle distribution by matching the empirical distribution of the measurements with the generated data. Fitting the distributions is achieved through solving a min-max game between a generator and a critic based on Wasserstein generative adversarial network structure. To accommodate the update of the projection angle distribution through gradient back propagation, we approximate the loss using the Gumbel-Softmax reparameterization of samples from discrete distributions. Our theoretical analysis verifies the unique recovery of the image and the projection distribution up to a rotation and reflection upon convergence. Our numerical experiments showcase the potential of the method to accurately recover the image and the projection angle distribution under noise contamination.

Zhizhen J. Zhao is an assistant professor in the Department of Electrical and Computer Engineering at the University of Illinois, Urbana-Champaign, with affiliation to the Coordinated Science Laboratory and the National Center Supercomputing Applications. She is also an affiliate assistant professor in the Department of Mathematics and the Department of Statistics. Her areas of research include geometric data analysis, dimensionality reduction, mathematical signal processing, scientific computing, and machine learning, with applications to imaging sciences and inverse problems, including cryo-electron microscopy image processing, data-driven methods for dynamical systems, and uncertainty quantification. Prior to joining ECE Illinois in 2016, she was a Courant Instructor at the Courant Institute of Mathematical Sciences, New York University. She received her PhD in physics from Princeton University working with Amit Singer and graduated from Trinity College, Cambridge University, with a BS and MS in physics.

Stereoscopic Displays and Applications XXXIII

Making 3D Magic

Tasks, traps, and tricks of a minion making 3D magic, John R. Benson, Illumination Entertainment (France)

"Um, this movie is going to be in stereo, like, 3D? Do we have to wear the glasses? How do we do that? How expensive is it going to be? And more importantly, if I buy that tool you wanted, can you finish the movie a week faster? No, ok, then figure it out for yourself. Go on, you can do it. We have faith…" And so it begins. From Coraline to Sing2, with Despicable Me, Minions, Pets, and a few Dr. Suess films, John RA Benson has designed the look and developed processes for making the stereo films of Illumination Entertainment both cost efficient and beneficial to the final film, whether as 2D or 3D presentations. He will discuss his workflow and design thoughts, as well as the philosophy of how he uses stereo visuals as a story telling device and definitely not a gimmick.

John R. Benson began his professional career in the camera department, shooting motion control and animation for "Pee-wee's Playhouse" in the mid 80s. He's been a visual effect supervisor for commercials in New York and San Francisco, managed the CG commercials division for Industrial Light and Magic, and was compositor for several films, including the Matrix sequels and Peter Jackson's "King Kong". After "Kong", he helped design the motion control systems and stereo pipeline for Laika's "Coraline". Since 2009, he has been working for Illumination Entertainment in Paris, France as the Stereographic Supervisor for the "Despicable Me" series, "Minions", several Dr. Seuss adaptations, the "Secret Life of Pets" series and both "Sing" films. Together, the Illumination projects have grossed over $6.7 billion worldwide.

Visualization and Data Analysis 2022

Visualization 2022

Visualization and Data Analysis 2022, Hank Childs, University of Oregon (United States)

Hank Childs is a professor in the Department of Computer and Information Science at the University of Oregon. He received his PhD in computer science from the University of California at Davis. His research focuses on scientific visualization, high performance computing, and the intersection of the two. In July of 2012, Childs received the Department of Energy's Early Career Award to research visualization with exascale computers (i.e., computers that can do 1018 floating operations per second). Childs spent more than a dozen years at Lawrence Berkeley and Lawrence Livermore National Laboratories, directing research in big data visualization. Outside of his research, Childs is best known as the architect of the VisIt project, a visualization application for very large data that is used around the world.