Journal-first submissions deadline
8 Aug
Priority submissions deadline 30 Jul
Final abstract submissions deadline 15 Oct
Manuscripts due for FastTrack publication
30 Nov

Early registration ends 31 Dec

Short Courses
11-14 Jan
Symposium begins
17 Jan
All proceedings manuscripts due
31 Jan

No content found

Machine Learning for Scientific Imaging 2022

  • Below is the the program in San Francisco time.
  • Talks are to be presented live during the times noted and will be recorded. The recordings may be viewed at your convenience, as often as you like, until 15 May 2022.

Monday 17 January 2022

IS&T Welcome & PLENARY: Quanta Image Sensors: Counting Photons Is the New Game in Town

07:00 – 08:10

The Quanta Image Sensor (QIS) was conceived as a different image sensor—one that counts photoelectrons one at a time using millions or billions of specialized pixels read out at high frame rate with computation imaging used to create gray scale images. QIS devices have been implemented in a CMOS image sensor (CIS) baseline room-temperature technology without using avalanche multiplication, and also with SPAD arrays. This plenary details the QIS concept, how it has been implemented in CIS and in SPADs, and what the major differences are. Applications that can be disrupted or enabled by this technology are also discussed, including smartphone, where CIS-QIS technology could even be employed in just a few years.

Eric R. Fossum, Dartmouth College (United States)

Eric R. Fossum is best known for the invention of the CMOS image sensor “camera-on-a-chip” used in billions of cameras. He is a solid-state image sensor device physicist and engineer, and his career has included academic and government research, and entrepreneurial leadership. At Dartmouth he is a professor of engineering and vice provost for entrepreneurship and technology transfer. Fossum received the 2017 Queen Elizabeth Prize from HRH Prince Charles, considered by many as the Nobel Prize of Engineering “for the creation of digital imaging sensors,” along with three others. He was inducted into the National Inventors Hall of Fame, and elected to the National Academy of Engineering among other honors including a recent Emmy Award. He has published more than 300 technical papers and holds more than 175 US patents. He co-founded several startups and co-founded the International Image Sensor Society (IISS), serving as its first president. He is a Fellow of IEEE and OSA.

08:10 – 08:40 EI 2022 Welcome Reception

Wednesday 19 January 2022

IS&T Awards & PLENARY: In situ Mobility for Planetary Exploration: Progress and Challenges

07:00 – 08:15

This year saw exciting milestones in planetary exploration with the successful landing of the Perseverance Mars rover, followed by its operation and the successful technology demonstration of the Ingenuity helicopter, the first heavier-than-air aircraft ever to fly on another planetary body. This plenary highlights new technologies used in this mission, including precision landing for Perseverance, a vision coprocessor, new algorithms for faster rover traverse, and the ingredients of the helicopter. It concludes with a survey of challenges for future planetary mobility systems, particularly for Mars, Earth’s moon, and Saturn’s moon, Titan.

Larry Matthies, Jet Propulsion Laboratory (United States)

Larry Matthies received his PhD in computer science from Carnegie Mellon University (1989), before joining JPL, where he has supervised the Computer Vision Group for 21 years, the past two coordinating internal technology investments in the Mars office. His research interests include 3-D perception, state estimation, terrain classification, and dynamic scene analysis for autonomous navigation of unmanned vehicles on Earth and in space. He has been a principal investigator in many programs involving robot vision and has initiated new technology developments that impacted every US Mars surface mission since 1997, including visual navigation algorithms for rovers, map matching algorithms for precision landers, and autonomous navigation hardware and software architectures for rotorcraft. He is a Fellow of the IEEE and was a joint winner in 2008 of the IEEE’s Robotics and Automation Award for his contributions to robotic space exploration.

Machine Learning for Scientific Imaging 2022 Posters

08:20 – 09:20
EI Symposium

Poster interactive session for all conferences authors and attendees.

P-17: Advantage of machine learning over maximum likelihood in limited-angle low-photon x-ray tomography, Zhen Guo1, Jung Ki Song1, George Barbastathis1, Michael A. Glinsky2, Courtenay T. Vaughan2, Kurt W. Larson2, Bradley K. Alpert3, and Zachary H. Levine4; 1Massachusetts Institute of Technology, 2Sandia National Laboratory, 3Applied and Computational Mathematics Division, National Institute of Standards and Technology, and 4Quantum Measurement Division, National Institute of Standards and Technology (United States) [view abstract]


P-18: CNN to mitigate atmospheric turbulence effect on Shack-Hartmann Wavefront Sensing: A case study on the Magdalena Ridge Observatory Interferometer, Siavash Norouzi1, James J. Luis2, Ramyaa Ramyaa1, John S. Young2, Eugene B. Seneta2, Morteza Darvish Morshedi Hosseini3, and Edgar R. Ligon4; 1New Mexico Institute of Mining and Technology (United States), 2University of Cambridge (United Kingdom), 3Binghamton University (United States), and 4CHARA Array (United States) [view abstract]


P-19: ISP distillation, Eli Schwartz1, Alex Bronstein2, and Raja Giryes1; 1Tel Aviv University and 2Technion (Israel) [view abstract]


Monday 24 January 2022

KEYNOTE: Inverse Problems in Imaging

Session Chair: Marc Klasky, Los Alamos National Laboratory (United States)
07:00 – 08:05
Green Room

Conference Introduction

KEYNOTE: Tackling tough inverse problems in imaging using PINNs and DeepOnets, George E. Karniadakis, Brown University (United States)

We will review physics-informed neural networks (PINNs) and operator regression networks (DeepOnets) with emphasis on discovering missing physics and system identification in diverse applications in fluid mechanics, solid mechanics, bioengineering, and beyond. We will demonstrate that we can use multimodality inputs from images and point measurements to discover effects in materials, obtain three-dimension fields, and improve greatly existing techniques such as particle tracking in fluid mechanics. The diverse problems we consider are ill-posed and cannot be solved with any traditional methods. For example, in one application in aortic dissections we identify from mechanical measurements the genotype that corresponds to the specific mouse tested among five different classes.

George E. Karniadakis received his SM (1984) and PhD (1987) from Massachusetts Institute of Technology. He was appointed Lecturer in the department of mechanical engineering at MIT in 1987 and subsequently he joined the Center for Turbulence Research at Stanford / Nasa Ames. He joined Princeton University as assistant professor in the department of mechanical and aerospace engineering and as associate faculty in the lrogram of applied and computational mathematics. He was a Visiting Professor at Caltech (1993) in the Aeronautics department. He joined Brown University as associate professor of applied mathematics in the Center for Fluid Mechanics on January 1, 1994. He became a full professor on July 1, 1996. He has been a Visiting Professor and Senior Lecturer of Ocean/Mechanical Engineering at MIT since September 1, 2000. He was Visiting Professor at Peking University (Fall 2007 & 2013). He has a joing appointment with PNNL since 2013. He is a Fellow of the Society for Industrial and Applied Mathematics (SIAM, 2010-), Fellow of the American Physical Society (APS, 2004-), Fellow of the American Society of Mechanical Engineers (ASME, 2003-) and Associate Fellow of the American Institute of Aeronautics and Astronautics (AIAA, 2006-). He received the SIAM CSE/ACM prize (2021), the SIAM Ralf E Kleinman award (2015), the (inaugural) J. Tinsley Oden Medal (2013), and the CFD award (2007) by the US Association in Computational Mechanics. His h-index is 112 and he has been cited more than 59,500 times.

Signal reconstruction of sparse, nano-scale metrology data using neural networks, Eva Natinsky1 and Remi Dingreville2; 1The University of Texas at Austin and 2Sandia National Laboratories (United States) [view abstract]


Video -- Intelligent Manufacturing -- Dynamic Tomography

Session Chair: Marc Klasky, Los Alamos National Laboratory (United States)
08:30 – 09:30
Green Room

Video from coded motion blur using dynamic phase coding, Erez Yosef, Shay Elmalem, and Raja Giryes, Tel Aviv University (Israel) [view abstract]


Feature anomaly detection system (FADS) for intelligent manufacturing, Anthony Garland, Sandia National Laboratories (United States) [view abstract]


Learning from hydrodynamics simulations with mass constraints for density reconstruction in dynamic tomography, Zhishen Huang1, Michael T. McCann2, Jennie Disterhaupt2, Marc Klasky2, and Saiprasad Ravishankar1; 1Michigan State University and 2Los Alamos National Laboratory (United States) [view abstract]


KEYNOTE: Machine Learning - High Data Volume

Session Chair: Marc Klasky, Los Alamos National Laboratory (United States)
10:00 – 11:00
Green Room

KEYNOTE: Physics based machine learning, David Rousseau, Université Paris-Saclay (France)

High Energy Physics experiments aim at establishing fundamental laws of physics by measuring the product of high energy particle collision with increasingly complex detectors. For example, tracking detectors deliver point clouds of micron precision over cubic meters, calorimeter detectors provide energy deposits in irregularly shapes voxels. Dedicated Machine Learning techniques are developed to deal with the specificities of the data, with the constraints of: dealing with large volume of data (many PetaBytes) within resources, maximise sensitivity to rare phenomenons, master unknowns along the full pipe line to be able to quote uncertainties. In addition, a long history of building accurate complex simulators is being supplemented by ML generators orders of magnitude faster but which have to demonstrate their ability to reproduce all details of the feature space.

David Rousseau is a High Energy Physicist at IJCLab, CNRS/IN2P3 and Université Paris-Saclay, currently working for the ATLAS experiment at CERN on the Large Hadron Collider. After a dozen years designing and implementing many pieces of the ATLAS experiment software, a chance meeting in 2013 with a machine learning computer scientist decided a new path in his career: develop the interface between high energy physics (and science in general) and machine learning (or artificial intelligence). Current research topics include: simulation based inference, machine learning for fast particle tracking at the large hadron collider, generator models for fast detector simulation, using an OPU for event classification, and uncertainty aware training. Rousseau received his PhD (1992) in high energy physics from the Université Aix-Marseille II. Since 2009, Rousseau has been senior researcher (Directeur de Recherche) at CNRS/IN2P3 LAL-Orsay, which merged into IJCLab-Orsay in 2020. He began his career at CERN as a CERN Fellow in 1997.

Learning optimal wavefront shaping for multi-channel imaging, Elias Nehme1, Boris Ferdman1, Lucien E. Weiss1, Tal Naor1, Daniel Freedman2, Tomer Michaeli1, and Yoav Shechtman1; 1Technion -- Israel Institute of Technology and 2Google Machine Perception (Israel) [view abstract]


Super Resolution -- Toroidal Magnetic Fields -- Nonlocal Kernel Network

Session Chairs: Marc Klasky, Los Alamos National Laboratory (United States) and Jong Chul Ye, Korea Advanced Institute of Science and Technology (KAIST) (Republic of Korea)
15:00 – 16:00
Green Room

Fast neural Poincare maps for toroidal magnetic fields, Joshua W. Burby, Los Alamos National Laboratory (United States) [view abstract]


Nonlocal Kernel Network (NKN): A stable and resolution-independent deep neural network, Marta D'Elia, Sandia National Laboratories (United States) [view abstract]


KEYNOTE: Computational Imaging Pipelines

Session Chairs: Marc Klasky, Los Alamos National Laboratory (United States) and Jong Chul Ye, Korea Advanced Institute of Science and Technology (KAIST) (Republic of Korea)
16:15 – 17:15
Green Room

KEYNOTE: Learning to image the invisible, Katherine L. Bouman, California Institute of Technology (United States)

As imaging requirements become more demanding, we must rely on increasingly sparse and/or noisy measurements that fail to paint a complete picture. Computational imaging pipelines, which replace optics with computation, have enabled image formation in situations that are impossible for conventional optical imaging. For instance, seismic and black hole imaging have only been made possible through the development of computational imaging pipelines. However, these computational "cameras" often suffer from (a) being difficult to analyze for image uncertainties, and (b) forward model mismatch. This talk will discuss how we are leveraging and building upon recent advances in machine learning in order to achieve more efficient uncertainty quantification and to better recover solutions in the presence of unknown model mismatch. The proposed approaches will be presented in the context of discussing the methods and procedures used to capture the first and future images of a black hole from the Event Horizon Telescope, as well as seismic localization and tomography.

Katherine L. Bouman is a Rosenberg Scholar and assistant professor of computing and mathematical sciences (CMS) and by courtesy in Electrical Engineering and Astronomy at Caltech in Pasadena, California. Bouman's research focuses on computational imaging: designing systems that tightly integrate algorithm and sensor design, making it possible to observe phenomena previously difficult or impossible to measure with traditional approaches. Her group at Caltech combines ideas from signal processing, computer vision, machine learning, and physics to find and exploit hidden signals for both scientific discovery and technological innovation. Prior to starting at Caltech, Bouman was a postdoctoral fellow with the Event Horizon Telescope, which published the first picture of a black hole in April of 2019. She received her PhD (2017) in electrical engineering and computer science from the Massachusetts Institute of Technology (MIT) and previously received a BSE (2011) in electrical engineering from the University of Michigan, Ann Arbor, MI, and an SM (2013) in electrical engineering and computer science from MIT. Bouman is a recipient of an NSF CAREER Award, the Electronic Imaging Scientist of the Year Award, an Okawa Research Grant, a Caltech faculty teaching award, a finalist for the AAAS Early Career Award for Public Engagement with Science, and is a co-recipient of the Breakthrough Prize.

Limited-angle CT with deep physics and image priors, Semih Barutcu1, Selin Aslan2, Aggelos Katsaggelos1, and Doga Gursoy1,2; 1Northwestern University and 2Argonne National Laboratory (United States) [view abstract]


Tuesday 25 January 2022

IS&T Awards & PLENARY: Physics-based Image Systems Simulation

07:00 – 08:15

Three quarters of a century ago, visionaries in academia and industry saw the need for a new field called photographic engineering and formed what would become the Society for Imaging Science and Technology (IS&T). Thirty-five years ago, IS&T recognized the massive transition from analog to digital imaging and created the Symposium on Electronic Imaging (EI). IS&T and EI continue to evolve by cross-pollinating electronic imaging in the fields of computer graphics, computer vision, machine learning, and visual perception, among others. This talk describes open-source software and applications that build on this vision. The software combines quantitative computer graphics with models of optics and image sensors to generate physically accurate synthetic image data for devices that are being prototyped. These simulations can be a powerful tool in the design and evaluation of novel imaging systems, as well as for the production of synthetic data for machine learning applications.

Joyce Farrell, Stanford Center for Image Systems Engineering, Stanford University, CEO and Co-founder, ImagEval Consulting (United States)

Joyce Farrell is a senior research associate and lecturer in the Stanford School of Engineering and the executive director of the Stanford Center for Image Systems Engineering (SCIEN). Joyce received her BS from the University of California at San Diego and her PhD from Stanford University. She was a postdoctoral fellow at NASA Ames Research Center, New York University, and Xerox PARC, before joining the research staff at Hewlett Packard in 1985. In 2000 Joyce joined Shutterfly, a startup company specializing in online digital photofinishing, and in 2001 she formed ImagEval Consulting, LLC, a company specializing in the development of software and design tools for image systems simulation. In 2003, Joyce returned to Stanford University to develop the SCIEN Industry Affiliates Program.

PANEL: The Brave New World of Virtual Reality

08:15 – 09:00

Advances in electronic imaging, computer graphics, and machine learning have made it possible to create photorealistic images and videos. In the future, one can imagine that it will be possible to create a virtual reality that is indistinguishable from real-world experiences. This panel discusses the benefits of this brave new world of virtual reality and how we can mitigate the risks that it poses. The goal of the panel discussion is to showcase state-of-the art synthetic imagery, learn how this progress benefits society, and discuss how we can mitigate the risks that the technology also poses. After brief demos of the state-of-their-art, the panelists will discuss: creating photorealistic avatars, Project Shoah, and digital forensics.

Panel Moderator: Joyce Farrell, Stanford Center for Image Systems Engineering, Stanford University, CEO and Co-founder, ImagEval Consulting (United States)
Panelist: Matthias Neissner, Technical University of Munich (Germany)
Panelist: Paul Debevec, Netflix, Inc. (United States)
Panelist: Hany Farid, University of California, Berkeley (United States)

CT Reconstruction -- Decomposition

Session Chairs: Marc Klasky, Los Alamos National Laboratory (United States) and Jong Chul Ye, Korea Advanced Institute of Science and Technology (KAIST) (Republic of Korea)
16:15 – 17:15
Red Room

Limited-view cone beam CT reconstruction using 3D patch-based supervised and adversarial learning: Validation using hydrodynamic simulations and experimental tomographic data, Marc Klasky1, Anish Lahiri2, Jeff Fessler2, Saiprasad Ravishankar3, Michelle Espy1, Michael T. McCann1, Trevor Wilcox1, and Ajeeta Khatiwada1; 1Los Alamos National Laboratory, 2University of Michigan, and 3Michigan State University (United States) [view abstract]


Machine learning and spatial decomposition for large CT scans, Gary J. Saavedra and Eric C. Cyr, Sandia National Laboratories (United States) [view abstract]


X-ray CT reconstruction leveraging CAD models, physics-based information and GANs, Amir K. Ziabari1, Abhishek Dubey2, Singanallur Venkatakrishnan1, Paul Brackman3, Curtis Frederick3, Philip Bingham1, Ryan Dehoff1, and Vincent Paquit1; 1Oak Ridge National Laboratory, 2Center of Cancer Research (NCI-NIH), and 3Carl Zeiss Metrology LLC (United States) (United States) [view abstract]


Wednesday 26 January 2022

KEYNOTE: 2D Unknown View Tomography -- Phase Imaging and Artificial Intelligence

Session Chairs: Marc Klasky, Los Alamos National Laboratory (United States) and Jong Chul Ye, Korea Advanced Institute of Science and Technology (KAIST) (Republic of Korea)
15:00 – 16:00
Green Room

KEYNOTE: An adversarial learning approach for 2D unknown view tomography, Mona Zehni and Zhizhen J. Zhao, University of Illinois at Urbana-Champaign (United States)

The goal of 2D tomographic reconstruction is to recover an image given its projection lines from various views. It is often presumed that projection angles associated with the projections are known in advance. Under certain situations, however, these angles are known only approximately or are completely unknown, when imaging moving objects. It becomes challenging to reconstruct the image from a collection of random projection. We introduce an adversarial learning based approach to recover the image and the projection angle distribution by matching the empirical distribution of the measurements with the generated data. Fitting the distributions is achieved through solving a min-max game between a generator and a critic based on Wasserstein generative adversarial network structure. To accommodate the update of the projection angle distribution through gradient back propagation, we approximate the loss using the Gumbel-Softmax reparameterization of samples from discrete distributions. Our theoretical analysis verifies the unique recovery of the image and the projection distribution up to a rotation and reflection upon convergence. Our numerical experiments showcase the potential of the method to accurately recover the image and the projection angle distribution under noise contamination.

Zhizhen J. Zhao is an assistant professor in the department of electrical and computer engineering at the University of Illinois, Urbana-Champaign, with affiliation to the Coordinated Science Laboratory and the National Center Supercomputing Applications. She is also an affiliate assistant professor in the department of mathematics and the department of statistics. Her areas of research include geometric data analysis, dimensionality reduction, mathematical signal processing, scientific computing, and machine learning. Applications to imaging sciences and inverse problems, including cryo-electron microscopy image processing, data-driven methods for dynamical systems, and uncertainty quantification. Prior to joining ECE Illinois in 2016, she was a Courant Instructor at the Courant Institute of Mathematical Sciences, New York University. She received her PhD in physics from Princeton University (2013) working with Amit Singer and graduated from Trinity College, Cambridge University, with a BS and MS in physics (2008).

Quantitative phase imaging and artificial intelligence: Label-free 3D imaging, classification, and inference, Yongkeun Park1,2; 1Korea Advanced Institute of Science and Technology (KAIST) and 2Tomocube, Inc. (Republic of Korea) [view abstract]


Imaging Through Scattering Medium -- Hydrodynamics Simulations

Session Chairs: Marc Klasky, Los Alamos National Laboratory (United States) and Jong Chul Ye, Korea Advanced Institute of Science and Technology (KAIST) (Republic of Korea)
16:15 – 17:15
Green Room

Physics-embedded deep learning for diffuser-based funduscope, Yunzhe Li and Lei Tian, Boston University (United States) [view abstract]


Imaging through scattering medium with deep phase retrieval, Mooseok Jang, Hyungjin Chung, Hyeonggeon Kim, Gookho Song, and Jong Chul Ye, Korea Advanced Institute of Science and Technology (KAIST) (Republic of Korea) [view abstract]


Physics based hydrodynamic learning from hydrodynamics simulations with mass constraints for density reconstruction in dynamic tomography, Marc Klasky1, Balu Nadiga1, Soumi De1, Oleg Korobkin1, Jennie Disterhaupt1, Trevor Wilcox1, Mallha Hossain1,2, and Charles A. Bouman2; 1Los Alamos National Laboratory and 2Purdue University (United States) [view abstract]


No content found

No content found

No content found

No content found

No content found