Machine Learning for Scientific Imaging 2023 (ON SABBATICAL)

Here are highlights from EI 2022 for consideration in developing your EI 2024 contribution.

The inverse problems keynote, "Tackling tough inverse problems in imaging using PINNs and DeepOnets," presented by George E. Karniadakis, Brown University (United States), reviewed physics-informed neural networks (PINNs) and operator regression networks (DeepOnets) with emphasis on discovering missing physics and system identification in diverse applications in fluid mechanics, solid mechanics, bioengineering, and beyond. We will demonstrate that we can use multimodality inputs from images and point measurements to discover effects in materials, obtain three-dimension fields, and improve greatly existing techniques such as particle tracking in fluid mechanics. The diverse problems we consider are ill-posed and cannot be solved with any traditional methods. For example, in one application in aortic dissections we identify from mechanical measurements the genotype that corresponds to the specific mouse tested among five different classes.

The machine learning keynote, "Physics based machine learning," presented by David Rousseau, Université Paris-Saclay (France), dealt with high energy physics experiments aimed at establishing fundamental laws of physics by measuring the product of high energy particle collision with increasingly complex detectors. For example, tracking detectors deliver point clouds of micron precision over cubic meters, calorimeter detectors provide energy deposits in irregularly shapes voxels. Dedicated Machine Learning techniques have been developed to deal with the specificities of the data, with the constraints of: dealing with large volume of data (many PetaBytes) within resources, to maximize sensitivity to rare phenomena, and to master unknowns along the full pipe line to be able to quote uncertainties. In addition, a long history of building accurate complex simulators is being supplemented by ML generators orders of magnitude faster but which have to demonstrate their ability to reproduce all details of the feature space.

The computational imaging pipelines keynote, "Learning to image the invisible," presented by Katherine L. Bouman, California Institute of Technology (United States), dealt with managing increasingly demanding imaging requirements that rely on increasingly sparse and/or noisy measurements that fail to paint a complete picture. Computational imaging pipelines, which replace optics with computation, have enabled image formation in situations that are impossible for conventional optical imaging. For instance, seismic and black hole imaging have only been made possible through the development of computational imaging pipelines. However, these computational "cameras" often suffer from (a) being difficult to analyze for image uncertainties, and (b) forward model mismatch. This talk discussed how we are leveraging and building upon recent advances in machine learning in order to achieve more efficient uncertainty quantification and to better recover solutions in the presence of unknown model mismatch. The proposed approaches will be presented in the context of discussing the methods and procedures used to capture the first and future images of a black hole from the Event Horizon Telescope, as well as seismic localization and tomography.

The unknown view tomography keynote, "An adversarial learning approach for 2D unknown view tomography," presented by Mona Zehni and Zhizhen J. Zhao, University of Illinois at Urbana-Champaign (United States), reviewed the goals and challenges of 2D tomographic reconstruction to recover an image given its projection lines from various views. It is often presumed that projection angles associated with the projections are known in advance. Under certain situations, however, these angles are known only approximately or are completely unknown, when imaging moving objects. It becomes challenging to reconstruct the image from a collection of random projection. An adversarial learning based approach can be used to recover the image and the projection angle distribution by matching the empirical distribution of the measurements with the generated data. Fitting the distributions is achieved through solving a min-max game between a generator and a critic based on Wasserstein generative adversarial network structure. To accommodate the update of the projection angle distribution through gradient back propagation, we approximate the loss using the Gumbel-Softmax reparameterization of samples from discrete distributions. Our theoretical analysis verifies the unique recovery of the image and the projection distribution up to a rotation and reflection upon convergence. Our numerical experiments showcase the potential of the method to accurately recover the image and the projection angle distribution under noise contamination.

MLSI sessions included:

• Inverse Problems in Imaging
• Video -- Intelligent Manufacturing -- Dynamic Tomography
• Machine Learning - High Data Volume
• Super Resolution -- Toroidal Magnetic Fields -- Nonlocal Kernel Network
• Computational Imaging Pipelines
• CT Reconstruction -- Decomposition
• 2D Unknown View Tomography -- Phase Imaging and Artificial Intelligence
• Imaging Through Scattering Medium -- Hydrodynamics Simulations