Machine Learning for Scientific Imaging 2025 Program

MLSI 2025 Program

TUESDAY 4 FEBRUARY 2025

Astronomy & Geophysics

Session Chair: He Sun, Peking University
09:30 - 10:30
Regency C

09:30MLSI-289
Invited: AI as a lens: Expanding scientific vision in biomedical and astronomical imaging, Brandon Feng, Massachusetts Institute of Technology (US) [view abstract]

Scientific discovery in both biomedicine and astronomy hinges on the ability to capture subtle, often hidden visual information. This talk presents AI-driven approaches that extend imaging capabilities by learning the physics of light rather than relying on human-labeled datasets. In astronomical imaging, I introduce a differentiable telescope model designed for exoplanet detection, which enhances sensitivity by directly integrating wavefront aberration data into real-time optimization, recently demonstrated using JWST data. For biomedical applications, I discuss Neural Wavefront Shaping, an imaging technique that corrects for light scattering through tissue, enabling high-resolution, non-invasive imaging. These techniques illustrate how AI can function as an active "lens," advancing our capacity to acquire and interpret visual data that could enable transformative discoveries across scientific domains.

09:50MLSI-290
Invited: Automatic differentiation for seismic inversion, Weiqiang Zhu, UC Berkeley (US); Kailai Xu, Stanford University (US); Eric Darve, Stanford University (US); Biondo Biondi, Stanford University (US); Gregory Beroza, Stanford University (US) [view abstract]

Seismic inversion is widely used to constrain earthquake source parameters and subsurface velocity structures, which cannot be measured directly and must instead be inferred from observational data. Unlike traditional geophysical inversion methods, such as the adjoint-state technique, modern deep learning frameworks (e.g., PyTorch and TensorFlow) can automatically optimize neural networks without requiring manual derivation and implementation for each inversion problem. To fill the gap between these two fields, we developed a general seismic inversion framework using automatic differentiation, which enables simultaneous optimization of both neural networks and partial differential equations (PDEs). We demonstrate this framework's effectiveness in solving seismic inversion problems, such as velocity model inversion, rupture imaging, and earthquake location within a unified approach. By integrating generative neural networks with PDEs, this method combines the feature-learning capabilities of deep neural networks with the accuracy of PDE solvers, improving geophysical inversion processes. Additionally, it introduces a new regularization method based on deep image priors and an uncertainty quantification method using Monte Carlo dropout, improving robustness against local minima (cycle-skipping) and increasing stability under noisy conditions.

10:10MLSI-291
Invited: Simplifying full waveform inversion via domain-independent self-supervised learning, [view abstract]

Deep learning has marked a significant advancement in geophysics, especially in tackling the intricate challenge of full waveform inversion (FWI). This breakthrough has enabled the effective prediction of subsurface velocity maps from seismic data. The process involves transforming seismic data into subsurface velocity maps, a task we have approached as a sophisticated form of image translation. This paper discusses a remarkable discovery: when encoders and decoders are trained independently within their specific domains through self-supervised learning, a linear relationship emerges in their latent spaces, transcending domain boundaries. This finding not only sheds light on the underlying mechanics of FWI but also elegantly unites multiple FWI datasets. These datasets can efficiently utilize a shared set of self-trained encoders and decoders, each adapted with distinct linear mappings. Building upon these insights, we introduce SimFWI, a novel methodology comprising two key steps: firstly, the independent learning of seismic encoders and velocity decoders across various datasets using masked image modeling, and secondly, the customization of a linear mapping for each dataset. Our experiments reveal that SimFWI matches the performance of traditionally trained models, which rely on paired seismic data and velocity maps, thereby opening new avenues in geophysical research.

Optical Computing & Generative AI

Session Chair: Yi Xue, University of California, Davis
11:00 - 12:20
Regency C

11:00MLSI-292
Invited: Cameras as nanophotonic optical computers, Felix Heide, Princeton University (US) [view abstract]

Camerashavebecomeaubiquitousinterfacebetweentherealworldandcomputers,withapplicationsacrossdomainsinfundamentalscience,robotics,health,andcommunication.Althoughtheirapplicationsarediverse,today'scamerasacquireinformationinthesamewaytheydidinthe19thcentury:theyfocuslightfromthesceneonasensingplaneusingasetofrefractivelensesthatminimizedeviationsfromGauss'slinearmodelofoptics.Inthisparadigm,increasinglycomplexlensandsensorstacksaredesignedtorecordanidealimageandperformcomputationonlyafterthecapture.Forexample,theopticalstackintheiPhone15containsmorethansevenelementsat7mminlength.Inthistalk,Iwilldiscusscomputationalcamerasthatlearntomanipulatethewavefrontofincidentlightwithwavelength-scalestructuring,previouslyimpracticaltodesignwithexistingelectromagneticwavesimulationmethods.Theseneuralnanophotoniccamerasmayenableunprecedentedcapabilitiesinopticaldesign,imaging,andcomputervision.Asexamples,Iwilldescribeanultra-smallcameraatafewhundredmicronsinsizethatmatchesthequalityachievedwithcm-sizecompoundlenses.Iwillalsopresentultra-thinnanophotoniccamerasthatperform99.9%ofneuralnetworkcompute-typicallyexecutedinelectronicsafterthecapture-intheopticsbeforesensing,atthespeedoflight.

11:20MLSI-293
Invited: Enhance efficiency of diffusion-based generative models for solving inverse problems via posterior sampling, Liyue Shen, University of Michigan (US) [view abstract]

Diffusion models have recently emerged as powerful generative priors for solving inverse problems. However, training diffusion models is data-intensive and computationally demanding, which restricts their applicability for high-dimensional and high-resolution data such as medical and scientific imaging. In this talk, I will introduce our recent works on how to improve the efficiency (data, time, and memory efficiency) of diffusion-based generative models for solving general inverse problems through posterior sampling. Particularly, I will introduce two plausible solutions we propose to enable learning diffusion priors for solving high-dimensional inverse problems through latent diffusion and patch diffusion models. The results are demonstrated in solving various inverse problems for both natural and medical images including 3D medical image reconstruction, showing the effectiveness of our proposed methods in both model efficiency and model performance. These research open the door to leverage diffusion-based generative models in tackling complex real-world data for addressing various crucial problems in many scientific disciplines.

11:40MLSI-294
Diffusion posterior sampling with inaccurate priors or physics, Weimin Bai; Yifei Wang; Siyi Chen; Wenzheng Chen; He Sun [view abstract]

Deep generativemodels have revolutionized the field of computational due to their exceptional ability to model complex prior distributions. However, their reliance on extensive, clean datasets for learning or accurate physics model for inference limits their practical use where clean data is scarce or physics model is inaccurate, respectively. In this paper, we propose an expectation-maximization (EM) approach for diffusion posterior sampling with inaccurate priors or physics. Our method alternates between reconstructing clean medical images from corrupted clinic data using a known diffusion prior and physical model (E-step) and refining the prior or physical model based on these reconstructions (M-step). This iterative process gradually guides the learned image prior or physical model to converge to true clean distribution. We validate our method through extensive experiments on diverse computational imaging tasks, including inpainting, deblurring and denoising, achieving new state-of-the-art performance.

Biomedicine & Nano Science

Session Chair: Shagan Sah, NVIDIA
15:30 - 17:30
Regency C

15:30MLSI-295
Supervised deep learning for phase retrieval of highly strained Bragg coherent diffraction patterns, Ewen Bellec, ESRF (France); Steven Leake, ESRF (France); Tobias Schulli, ESRF (France); Marie-Ingrid Richard, CEA - Grenoble (France); Vincent Favre-Nicolin, ESRF (France) [view abstract]

The reconstruction of highly strained particles from the Bragg coherent diffraction pattern is often impossible with standard iterative algorithm due to the strong distortions induced in the measured intensity. Here we propose a convolutional neural network for the prediction of the reciprocal space phase of the diffracted amplitude, from which the particle reconstruction is obtained by the inverse Fourier transform. The adopted model is a UNet and the its full potential is exploited by the intrinsic structural similarity between input and output. The training is performed in a supervised fashion on simulated data with the use of a custom loss function that accounts for the phase symmetries. The model demonstrates effective performance on experimental data, particularly as an initial guess for standard iterative phase retrieval, enhancing the likelihood of successful convergence.

15:50MLSI-296
A nondestructive image-based microstructural characterization of solid oral dosage forms, Nicole Tin, Gilead Sciences (US); Tiffany Pham, Gilead Sciences (US); Adedayo Sanni, Gilead Sciences (US); Remus O?an, Gilead Sciences (US) [view abstract]

Optimizing the performance of drug products requires a strong understanding of the interplay between material properties, manufacturing process parameters, and tablet performance. Microstructure is a crucial intrinsic property that details the spatial and material arrangement that can be examined in a rapid nondestructive scan. Here, we aim to demonstrate the development of a nondestructive, image-based microstructural characterization with potential to rapidly troubleshoot tablet performance with custom analysis. X-ray computed microtomography (micro-CT) is used alongside 3D advanced imaging techniques to examine the spatial distribution of tablet coating and porosity to then model the effect of porosity under simulated dissolution. The benefits of a developed microstructural analysis cannot be overstated, as it has the potential to routinely provide a mechanistic basis to understand, predict, and troubleshoot tablet performance.

16:10MLSI-297
Invited: Motion-induced contrast and neural space-time model for dynamic multi-shot imaging, Ruiming Cao, UC Berkeley (US); Guanghan Meng; Nikita S. Divekar; James K. Nu ez; Srigokul Upadhyayula; Laura Waller, [view abstract]

Motion is considered as a primary source of imaging artifacts, especially during the observation of fast-moving live samples. Although most imaging systems assume static scenes during image acquisition, there are scenarios where motion can be useful for the image formation and/or unknown motion can be estimated. I will discuss how sample motion can serve as a contrast mechanism for super-resolution microscopy and how leveraging spatiotemporal correlations can enable dynamics imaging free from motion artifacts.

THURSDAY 6 FEBRUARY 2025

Computational Camera II

Session Chair: Greg Buzzard, Purdue University
11:00 - 12:20
Regency B

11:40MLSI-288
Invited: Optics, sensors and AI: synergic computational imaging to go beyond the limits imposed by conventional imaging, Ashok Veeraraghavan, Rice University (US) [view abstract]

Inthistalk,Iwilldiscussaboutseveralprojectsinmylabattheconfluenceofoptics,sensorsandartificialintelligence.Inparticular,Iwillprovideexamplesofhowco-designingsensors,opticsandAIalgorithmsresultsinsuperiorperformancecapabilitiesforimagingsystems.Iwillprovideafewexampleprojects:(1)howco-designingimagingopticsalongwithAIalgorithmscanenablehigh-throughput3Dimaging,andmicroscopy,(2)hownoveldiffractiveandmeta-opticalelementsallowustorealizeimagingsystemswithnovelfunctionalitiesandform-factorsandfinallytime-permitting,(3)howemergingneuralrepresentationsalongwithhighresolutionspatiallightmodulatorscanallowustoimagethroughthickscatteringmediawithouttheneedforguidestars.Iwillusetheseprojectstoarguethatweshouldlookatthethreecomputationalblockswithinanimagingsystem,optics,sensorsandalgorithmstogetherandthatco-designingthemcanresultinsignificantperformanceimprovementsoverthestateofart.