Image Sensors and Systems 2025 Program

ISS 2025 Program

WEDNESDAY 5 FEBRUARY 2025

Sensor Design and Optimization

Session Chair: Hari Tagat, Samsung
09:30 - 10:30
Grand Peninsula C

09:30ISS-276
A simple model for characterizing and simulating dark current in CMOS sensors, Steve Wang, Omnivision Technologies, Inc. (US); Eiichi Funatsu, OmniVision Technologies Inc (US); Boyd Fowler, OmniVision Technologies Inc. (US) [view abstract]

A normal distribution is typically used for characterizing and modeling dark current in CMOS sensors. Although it provides simplicity and speeds needed in real-time applications, it is usually not a very good representation of the real dark current characteristics observed in real devices. The statistical distribution of CMOS sensor dark noise is typically right-skewed with a long tail, i.e. with more �hot� pixels than described in a normal distribution. Furthermore, the spatial distribution in real devices typically exhibit a 1/f-like power spectrum instead of a flat spectrum from a simple Gaussian distributions model. When simulating sensor images, for example generating images and videos for training and testing image processing algorithms, it is important to reproduce both characteristics accurately. We propose a simple convolution-type algorithm using seed images with a log-normal distribution and randomized kernels that is able to reproduce both the correct statistical and spatial distributions, and can be easily matched to measured noise data. It can also be parallelized with CUDA acceleration to support real-time execution.

09:50ISS-277
A 6-tap 720x488-pixel short-pulse indirect time of flight image sensor with 30-phase demodulation for 100m outdoor operation, Koji Itaba, Shizuoka University (Japan); Keita Yasutomi, Shizuoka University (Japan); Keiichiro Kagawa, Shizuoka University (Japan); Kamel Mars, Shizuoka Institute of Science and Technology (Japan); Shoji Kawahito, Shizuoka University (Japan) [view abstract]

Long-range measurement with ambient light tolerance has been achieved using a 720x488-resolution short-pulse-based indirect time-of-flight (SP-iToF) image sensor, featuring 6-tap, single-drain pixels, fabricated with a 110nm FSI process. The sensor performs 30-phase light-pulse delay measurements in both direct and indirect modes, utilizing the 6-tap pixels and five range-shifted sub-frame readouts, each with a readout time of 4.4ms. By leveraging multi-phase SP-iToF measurements with reduced parasitic light sensitivity and high demodulation contrast in 6-tap pixels, the sensor successfully reduces ambient light charge from direct sunlight, enabling accurate 100m distance measurements at 10 fps under 100 klux illumination, with depth noise of less than 1%.

10:10ISS-278
RGBC-IR: A CFA for single exposure dark flash, Tripurari Singh, Image Algorithmics (US); Mritunjay Singh, Image Algorithmics (US) [view abstract]

Modern RGB-IR cameras have evolved to capture accurate colors and NIR from a single sensor. While these cameras can employ their RGB images to effectively denoise IR, they contain too few IR pixels to do the reverse: denoise RGB with IR.Improving low light RGB with an IR illuminator is an important feature for upcoming automotive applications where cabins have to be kept dark at night so as not to distract the driver. Current solutions to this problem either discard the IR cut filter and take separate RGB and IR exposures and suffer from poor colors. Or employ a bulky beam splitter architecture with separate RGB and IR sensors.We propose a camera with a novel RGBC-IR color filter array containing clear pixels that are sensitive to both visible light and IR. Its RGB pixels feature an IR attenuating coating while its IR pixels contain a black filter that blocks visible light.Mulitspectral demosaicking techniques are used to reconstruct RGB and IR images, as well as a high SNR luminance image containing the Clear, RGB and IR signals. Fusion techniques developed for beam splitter RGB-IR cameras are used to denoise RGB and IR using the luminance.

Imaging Applications

Session Chair: Francisco Imai, Rochester Institute of Technology
11:00 - 12:20
Grand Peninsula C

11:00ISS-279
Infinite-ISP: An Open Source Hardware Image Signal Processor Platform for all Imaging Needs, Taimur Bilal; Bakhtawar Amjad; Talha Iqbal; Bilal Zafar; Khurram Usman, 10xEngineers (US) [view abstract]

While traditionally not available in the public domain, access to the complete image signal processing (ISP) algorithmic pipeline and its hardware implementation are necessary to enable new imaging use cases and to improve the performance of high level deep learning vision networks. In this paper, we present Infinite-ISP: A complete hardware ISP development suite, comprising of an ISP algorithm development platform, a bit-accurate fixed-point reference model, an ISP register transfer level development platform, an FPGA development workflow and an ISP tuning tool. To aid the hardware development process, we develop end-to-end reference designs for the KV260 vision AI starter kit (AMD Xilinx Kria SOM) and Efinix Ti180J484 kit, with support for 3 image sensors (Sony IMX219, Onsemi AR1335 and Omnivision OV5647), using the Infinite-ISP framework. These ISP reference designs support 10-bit 2592x1536 4 Megapixel (MP) Bayer image sensors with a maximum pixel throughput of 125 MP/s or 30 frames per second. We demonstrate that the image quality of Infinite-ISP is comparable to commercial ISPs found in Skype Certified Cameras and also performs competitively with digital still cameras for most imaging scenarios in terms of the perceptual IQ metrics BRISQUE, PIQE, and NIQE. We envision the Infinite-ISP under the permissive open-source license to streamline the ISP development from algorithmic design to hardware implementation and to foster community building and further research around it for both academia and industry.

11:20ISS-280
Open-source software ecosystem development for hyperspectral imaging, Bai Xue, Yale University (US); Bowen Duanmu, Yale University (US); Holly Rushmeier, Yale University (US) [view abstract]

Hyperspectral imaging (HSI) has the potential to revolutionize various scientific fields by enabling detailed analysis of materials and phenomena beyond the capabilities of traditional RGB imaging systems. However, the adoption of HSI has been hindered by the lack of an integrated software and data-sharing infrastructure. This project aims to address these challenges by developing a sustainable, open-source software ecosystem (HSI-OSE) for HSI, which facilitates cross-disciplinary collaboration by providing a unified platform for sharing datasets, software tools, hardware resources, and literature. The project aims to overcome key issues such as the fragmentation of HSI resources, unnecessary duplication of efforts, and limited access to comprehensive datasets for machine learning (ML) and artificial intelligence (AI) applications. An online platform the HSI Resource Portal was developed with a user interface and a MySQL-based database, allows users to contribute and access resources efficiently. The portal also includes a glossary to bridge terminological gaps between different disciplines and is managed through a user-role system that encourages secure and active participation. Interviews were also conducted to identify the challenges in HIS for OSE development.

11:40ISS-281
KAMF: An interchangeable-lens mirrorless camera made from a kid's camera, Henry Dietz, University of Kentucky (US) [view abstract]

Very low cost consumer-oriented commodity products often can be modified to support more sophisticated uses. The current work explores a variety of ways in which any of a family of small rechargeable-battery stand-alone cameras, intended for use of kids as young as three years old, can be modified to support more sophisticated use. Unfortunately, the image quality and exposure controls available are similar to what would be found in a cheap webcam; to be precise, the camera contains two separate camera modules each comparable to a webcam. However, simple modifications convert these toys into interchangeable-lens mirrorless cameras accepting lenses in a variety of standard mounts. Cameras so modified can capture full spectrum images or can employ a filter providing any desired spectral sensitivity profile. One also has limited access to the internals of the camera, easily allowing options like wiring to an external exposure trigger. The cameras record on a TF card, and typically a 32GB or 64GB card is included with the camera at a total cost virtually identical to the cost of the TF card alone. The current work can be considered as both a study revealing the internal construction of a kids camera and a guide to adapting it for more serious uses as the KAMF mirrorless camera.

12:00ISS-348
GoProUP: A tool to optimise GoPro cameras for underwater photogrammetry, Andrew Woods, Curtin University HIVE (Australia); David Newman, GoPro (US); Daniel Adams, Curtin University HIVE (Australia) [view abstract]

GoPro cameras are widely used for maritime archaeology survey purposes conducted by divers and ROVs. We have developed a set of techniques to optimise the image capture properties of recent GoPro cameras for photogrammetry purposes. The technique makes extensive use of GoPro Labs, a custom firmware for recent GoPro cameras, which allows the GoPro cameras to be configured and programmed using a series of QR codes. GoPro cameras are primarily marketed as action cameras and hence their operation is optimised for action scenes. In action scenes with fast motion it is common to prioritise high shutter speeds. Unfortunately, to achieve high shutter speeds it is often necessary to use high ISO settings, which in turn incurs more image noise which compromises photogrammetry processing quality. For underwater archaeology surveys we load a custom exposure profile onto the camera which prioritises a shutter speed of 1/50 second and an ISO of 100 if light levels allow it. This technique has already been used on several diver and ROV surveys. Once this technique is validated on more surveys, the instructions for this process will be made available more widely. Results to date indicate technique allows the generation of higher quality digital 3D models of underwater sites.

Imaging Sensors and Systems Posters (with lunch)

12:20 - 14:00
The Grove

12:20ISS-282
High aspect ratio nanostructure for anti-refection function in CMOS image sensor devices, Hao-Wei Liu (Taiwan (Greater China)); Yu-Shen Lu (Taiwan (Greater China)); Wei-Lung Tsai (Taiwan (Greater China)); Ping-Hsu Chen (Taiwan (Greater China)); Chung-Hsuan Yu (Taiwan (Greater China)); Chi-Jiun Huang (Taiwan (Greater China)); Shin-Hong Kuo (Taiwan (Greater China)); Huang-Jen Chen (Taiwan (Greater China)); Ching-Chiang Wu (Taiwan (Greater China)); Ken Wu (Taiwan (Greater China)); Hung-Jen Tsai (Taiwan (Greater China)) [view abstract]

Cover glass on the CMOS image sensor chip plays a critical role in image quality. To improve cover glass optical properties, the double sided nanostructure was introduced onto cover glass surfaces as an anti-reflection means in this work. By DUV photolithography and etching process, such nanostructure with high aspect ratio patterning enhanced transmittance to 98~99% and had better angular response than the double sided multi-film anti-reflection coating. In addition, the petal flare phenomenon of double sided nanostructure was comparable to the double sided multi-film anti-reflection coating. Base on this result, the double sided nanostructure technique was achievable and able to mass production for advance CMOS image sensor devices.

12:20ISS-283
Image sensor with low refractive index grid for angular response improvement, Shin-Hong Kuo (Taiwan (Greater China)); Hao-Wei Liu (Taiwan (Greater China)); Po-Hsiang Wang (Taiwan (Greater China)); Huang-Jen Chen (Taiwan (Greater China)); Ching-Chiang Wu (Taiwan (Greater China)); Ken Wu (Taiwan (Greater China)); Hung-Jen Tsai (Taiwan (Greater China)) [view abstract]

???????????????????? 0.8um ? CMOS ????? (CIS) ?????,??????????????????? (LRIG)??? 500nm ?? LRIG ? CIS ?????????????????????????????????? 94.5%?90.1% ? 93.5%???????????(CMG)????,CIS???????????(-15�?15�)?????8.5%???9.5%???8.6%???,?????CIS?????????????????

Advanced Imaging Techniques and Technologies

Session Chair: Hari Tagat, Samsung
15:30 - 17:30
Grand Peninsula C

15:30ISS-284
ISETHDR: A physics-based synthetic radiance dataset for evaluating high dynamic range sensors, Zhenyi Liu, Stanford University (US); Brian Wandell, Stanford University (US) [view abstract]

This paper describes a physics-based end-to-end software simulation for image systems. We explore sensors designed to enhance performance in high dynamic range (HDR) environments, such as driving through daytime tunnels and under nighttime conditions. We synthesize physically realistic HDR spectral radiance images and use them as the input to digital twins that model the optics and sensors of different systems. This papermakes three main contributions: (a) We create a labeled (instance segmentation and depth), synthetic radiance datasetof HDR driving scenes. (b) We describe the development and validation of the end-to-end simulation framework. (c) We present a comparative analysis of two single-shot sensors designed for HDR. We open-source both the dataset and the software.

15:50ISS-285
Lens simulation for high resolution and multispectral image using distributed multi-GPU computing system, SeongHyeon Kang, Samsung Electronics Co., Ltd. (Republic of Korea); Jeongyoung Shin, Samsung Electronics Co., Ltd. (Republic of Korea); Sangmin Kim, Samsung Electronics Co., Ltd. (Republic of Korea); Jeongwook Lee, Samsung Electronics Co., Ltd. (Republic of Korea); Sung-Su Kim, Samsung Electronics Co., Ltd. (Republic of Korea); Yitae Kim, Samsung Electronics Co., Ltd. (Republic of Korea) [view abstract]

In a lens camera, light from an object is projected onto an image sensor through a lens. This projection involves various effects such as point spread function (PSF), distortion, and relative illumination (RI), which can be provided by lens design software. To evaluate the lens, a simulation system can be implemented to apply these lens effects to a scene image. However, it can be computationally inefficient to handle high-resolution multispectral images. We propose a simulation system that can simulate images with 200 million pixels, which is the resolution offered by the latest mobile phones. Our method optimizes the spatially variant PSF algorithm and utilizes a distributed Multi-GPU system for massive parallel computing. We use this system to create a 200M multispectral image with 31 wavelengths ranging from 400 nm to 700 nm. Edge spread function, RI, and distortion are compared with Zemax, showing a similarity of around 99%. The entire process of computing the 200M and 31 wavelength image takes less than 1 hour. Therefore, our simulation system with distributed computing can efficiently and accurately apply lens effects to high-resolution multispectral images. This system can be used for pre-evaluation of the performance of a lens system before manufacturing

16:10ISS-286
Optimization of polarization image sensor for electric field imaging system based on electro-optic effect, Kiyotaka Sasagawa, Nara Institute of Science and Technology (Japan); Ryoma Okada, Nara Institute of Science and Technology (Japan); Katsuya Hyodo, Nara Insistute of Science and Technology (Japan); Hironari Takehara, Nara Institute of Scence and Technlogy (Japan); Makito Haruta, Chitose Institute of Science and Technology (Japan); Hiroyuki Tashiro, Kyushu University (Japan); Jun Ohta (Japan) [view abstract]

Polarization image sensors for field imaging using electro-optical effects need to detect weak polarization changes at a specific wavelength with high sensitivity. This paper describes the optimization of pixel design using a 0.35-�m CMOS process. Factors related to performance include the configuration of the polarizer on the pixel using a metal interconnect layer, light uptake efficiency, and the effect of inter-pixel crosstalk. The intensity of irradiated light and frame rate must also be optimized, since the pixel must operate under photon shot noise limit conditions due to the need to detect minute changes. We show the improvement of pixel performance based on these factors and the improvement of field imaging performance using this system.

16:30ISS-287
A novel multimodal 3D depth sensing device, Jian Ma, Qualcomm Technologies, Inc. (US); Shenge Wang, Qualcomm Technologies, Inc. (US); Matthieu Dupre, Qualcomm Technologies, Inc. (US); Ioannis Nousias, Qualcomm Technologies, Inc. (US); Sergio Goma, Qualcomm Technologies, Inc. (US) [view abstract]

Indirect time-of-flight (iToF) depth sensing technology is widely acclaimed for its ability to capture high-resolution 3D images at remarkable speeds. However, its performance is often compromised by multipath interference (MPI), which significantly diminishes depth accuracy.We present an innovative 3D depth sensing scheme that seamlessly integrates multiple depth sensing modalities and technologies into a single, compact device. Our approach dynamically switches between depth sensing modes, include iToF and structured light, to enable real-time data fusion of depth images.Our solution successfully demonstrates iToF depth imaging without MPI, achieving high image resolution (VGA) and exceptional depth accuracy at a frame rate of 30 fps.