Monday 17 January 2022
IS&T Welcome & PLENARY: Quanta Image Sensors: Counting Photons Is the New Game in Town
07:00 – 08:10
The Quanta Image Sensor (QIS) was conceived as a different image sensor—one that counts photoelectrons one at a time using millions or billions of specialized pixels read out at high frame rate with computation imaging used to create gray scale images. QIS devices have been implemented in a CMOS image sensor (CIS) baseline room-temperature technology without using avalanche multiplication, and also with SPAD arrays. This plenary details the QIS concept, how it has been implemented in CIS and in SPADs, and what the major differences are. Applications that can be disrupted or enabled by this technology are also discussed, including smartphone, where CIS-QIS technology could even be employed in just a few years.
Eric R. Fossum, Dartmouth College (United States)
Eric R. Fossum is best known for the invention of the CMOS image sensor “camera-on-a-chip” used in billions of cameras. He is a solid-state image sensor device physicist and engineer, and his career has included academic and government research, and entrepreneurial leadership. At Dartmouth he is a professor of engineering and vice provost for entrepreneurship and technology transfer. Fossum received the 2017 Queen Elizabeth Prize from HRH Prince Charles, considered by many as the Nobel Prize of Engineering “for the creation of digital imaging sensors,” along with three others. He was inducted into the National Inventors Hall of Fame, and elected to the National Academy of Engineering among other honors including a recent Emmy Award. He has published more than 300 technical papers and holds more than 175 US patents. He co-founded several startups and co-founded the International Image Sensor Society (IISS), serving as its first president. He is a Fellow of IEEE and OSA.
08:10 – 08:40 EI 2022 Welcome Reception
Tuesday 18 January 2022
Image Sensing I
Session Chairs:
Jon McElvain, Dolby Laboratories (United States) and Arnaud Peizerat, CEA (France)
08:30 – 09:35
Red Room
08:30
Conference Introduction
08:35ISS-153
Time domain noise analysis of oversampled CMOS image sensors [PRESENTATION-ONLY], Andreas Suess, Mathias Wilhelmsen, Liang Zuo, and Boyd Fowler, OmniVision (United States) [view abstract]
In this talk we analyze the effectiveness of four oversampled techniques on the read noise performance of CMOS image sensors. We compare correlated multiple sampling (CMS), noise optimized correlated multiple sampling (NOCMS), skipper multiple sampling (SMS) and noise optimized skipper multiple sampling (NOSMS) using model parameters from two sensors. We verify the presented model against CMS measurements.We point out that floating diffusion (FD) leakage current becomes a dominant noise factor in CMS, which NOCMS can greatly reduce. Finally, we conclude that SMS and NOSMS have more potential to further reduce read noise than CMS or NOCMS.
08:55ISS-154
A 40/22nm 200MP stacked CMOS image sensor with 0.61µm pixel [PRESENTATION-ONLY], Masayuki Uchiyama1, Geunsook Park1, Sangjoo Lee1, Tomoyasu Tate1, Masashi Minagawa2, Shino Shimoyamada2, Zhiqiang Lin1, King Yeung1, Lien Tu1, Wu-Zang Yang3, Alan Hsiung1, Vincent Venezia1, and Lindsay Grant1; 1OmniVision Technologies, Inc. (United States), 2OmniVision Technologies Japan (Japan), and 3OmniVision Technologies Taiwan (Taiwan) [view abstract]
We developed a new 40/22nm stacked 200 mega-pixel CMOS image sensor (CIS) with a 0.61µm pixel. By using a 22nm logic wafer process node, compared to 40 nm, digital power consumption was reduced by half while keeping the same clock frequency and the full high definition (FHD) frame rate was increased from 240fps to 480fps. We demonstrate a new source follower (SF) transistor architecture having 63% higher SF transconductance (Gm) compared with our former 0.7µm pixel. A full well capacity (FWC) of 5.0ke- was achieved, with good white pixel (WP) performance. We demonstrate 0.61µm quad photodiode (QPD) structure and achieved comparable quantum efficiency (QE) performance to 0.7µm QPD.
09:15ISS-155
An offset calibration technique for CIS column-parallel SAR ADC using memory, Jaekyum Lee1 and Albert Theuwissen1,2; 1TU Delft (the Netherlands) and 2Harvest Imaging (Belgium) [view abstract]
A column-parallel 10bit SAR ADC for a high-speed image sensor has been implemented. A fast offset calibration that is using memory is proposed to compensate for the offset mismatch, accompanied by an ADC designed in a narrow space of a single column pitch size. The memory accumulates the variation of the offset to track the offset within two cycles. After applying the offset calibration technique, the measured offset variation of the ADC in each column is improved from 4.27LSB to 0.39LSB. The fixed-pattern noise (FPN) is also improved from 4.14LSB to 0.34LSB. This calibration method covers an offset range of ±32LSB. The implemented ADC achieves 500kS/s as a maximum speed, The maximum frame rate of the sensor is 3000fps. The power consumption of the sensor, except for the LVDS interface, is 71mW. This sensor is designed in TowerJazz CIS 180nm process with one poly four metal. The supply voltage of the analog and digital domains is 3.3V and 1.8V, respectively.
KEYNOTE: Sensing for Autonomous Driving
Session Chairs: Patrick Denny, Valeo Vision Systems (Ireland) and Hari Tagat, Casix (United States)
10:00 – 11:00
Green Room
This session is hosted jointly by the Autonomous Vehicles and Machines 2022 and Imaging Sensors and Systems 2022 conferences.
10:00ISS-160
KEYNOTE: Recent developments in GatedVision imaging - Seeing the unseen [PRESENTATION-ONLY], Ofer David, BrightWay Vision (Israel)
Imaging is the basic building block for automotive autonomous driving. Any computer vision system will require a good image as an input at all driving conditions. GatedVision provides an extra layer on top of the regular RGB/RCCB sensor to augment these sensors at nighttime and harsh weather conditions. GatedVision images in darkness and different weather conditions will be shared. Imagine that you could detect a small target laying on the road having the same reflectivity as the back ground meaning no contrast, GatedVision can manipulate the way an image is captured so that contrast can be extracted. Additional imaging capabilities of GatedVision will be presented.
Ofer David has been BrightWay Vision CEO since 2010. David has more than 20 years’ experience in the area of active imaging systems and laser detection, and has produced various publications and patents. Other solutions in which David is involved with, include fog penetrating day/night imaging systems and visibility measurement systems. David received his BSc and MSc from the Technion – Israel Institute of Technology and his PhD in electro-optics from Ben-Gurion University.
10:40AVM-161
Potentials of combined visible light and near infrared imaging for driving automation, Korbinian Weikl1,2, Damien Schroeder1, and Walter Stechele2; 1Bayerische Motoren Werke AG and 2Technical University of Munich (Germany) [view abstract]
Automated driving functions of the highest levels of automation require camera and computer vision (CV) systems which enable them to operate at a safety level that exceeds human driving. To date, the information content of the cameras’ image data does not suffice to reach those performance levels. One degree of freedom to increase the image information content is to extend the spectral range of the cameras. Near infrared (NIR) imaging on CMOS imagers is a promising candidate technology in this research direction. To assess the potentials of combined visible light (VIS) and NIR imaging for the driving automation application, we extend our camera simulation and optimization framework for camera models that include a VIS-NIR CMOS imager. We also adapt our image processing and CV models to process the additional NIR image information. We evaluate the vision system performance for our VIS-NIR camera models, in reference to an automotive VIS-only camera model. We use a data set of synthetic automotive scenes, and a neural network-based object detection system as benchmark CV. Our results give an indication for the performance increase potentials of combined VIS-NIR imaging in driving automation and highlight critical scenarios that can be perceived correctly using VIS-NIR imaging.
[view abstract]
Automated driving functions of the highest levels of automation require camera and computer vision (CV) systems which enable them to operate at a safety level that exceeds human driving. To date, the information content of the cameras’ image data does not suffice to reach those performance levels. One degree of freedom to increase the image information content is to extend the spectral range of the cameras. Near infrared (NIR) imaging on CMOS imagers is a promising candidate technology in this research direction. To assess the potentials of combined visible light (VIS) and NIR imaging for the driving automation application, we extend our camera simulation and optimization framework for camera models that include a VIS-NIR CMOS imager. We also adapt our image processing and CV models to process the additional NIR image information. We evaluate the vision system performance for our VIS-NIR camera models, in reference to an automotive VIS-only camera model. We use a data set of synthetic automotive scenes, and a neural network-based object detection system as benchmark CV. Our results give an indication for the performance increase potentials of combined VIS-NIR imaging in driving automation and highlight critical scenarios that can be perceived correctly using VIS-NIR imaging.
LIDAR and Sensing
Session Chairs:
Robin Jenkin, NVIDIA Corporation (United States) and Min-Woong Seo, Samsung Electronics (Republic of Korea)
15:00 – 16:00
Red Room
This session is hosted jointly by the Autonomous Vehicles and Machines 2022 and Imaging Sensors and Systems 2022 conferences.
15:00AVM-172
Real-time LIDAR imaging by solid-state single chip beam scanner, Jisan Lee, Kyunghyun Son, Changbum Lee, Inoh Hwang, Bongyong Jang, Eunkyung Lee, Dongshik Shim, Hyunil Byun, Changgyun Shin, Dongjae Shin, Otsuka Tatsuhiro, Yongchul Cho, Kyoungho Ha, and Hyuck Choo, Samsung Electronics Co., Ltd. (Republic of Korea) [view abstract]
We present a real-time light detection and ranging (LIDAR) imaging by developing a single-chip solid-state beam scanner. The beam scanner is integrated with a fully functional 32-channel optical phased array, 36 optical amplifiers, and a tunable laser at central wavelength ~1310 nm, all on a 7.5 x 3 mm^2 single chip fabricated with III-V on silicon processes. The phased array is calibrated with self-evolving genetic algorithm to enable beam forming and steering in two dimensions. Distance measurement is performed with a digital signal processing that measures the time of flight (TOF) of pulsed light with a system consisting of an avalanche photodiode (APD), trans-impedance amplifier (TIA), analog-digital converter (ADC), and a processor. The LIDAR module utilizing this system can acquire point cloud images with 120 x 20 resolution with a speed of 20 frames per seconds at a distance up to 20 meters. This work presents the first demonstration of a chip-scale LIDAR solution without any moving part or bulk external light source or amplifier, making an ultra-low cost and compact LIDAR technology a reality.
[view abstract]
We present a real-time light detection and ranging (LIDAR) imaging by developing a single-chip solid-state beam scanner. The beam scanner is integrated with a fully functional 32-channel optical phased array, 36 optical amplifiers, and a tunable laser at central wavelength ~1310 nm, all on a 7.5 x 3 mm^2 single chip fabricated with III-V on silicon processes. The phased array is calibrated with self-evolving genetic algorithm to enable beam forming and steering in two dimensions. Distance measurement is performed with a digital signal processing that measures the time of flight (TOF) of pulsed light with a system consisting of an avalanche photodiode (APD), trans-impedance amplifier (TIA), analog-digital converter (ADC), and a processor. The LIDAR module utilizing this system can acquire point cloud images with 120 x 20 resolution with a speed of 20 frames per seconds at a distance up to 20 meters. This work presents the first demonstration of a chip-scale LIDAR solution without any moving part or bulk external light source or amplifier, making an ultra-low cost and compact LIDAR technology a reality.
15:20ISS-173
A back-illuminated SOI-based 4-tap lock-in pixel with high NIR sensitivity for TOF range image sensors [PRESENTATION-ONLY], Naoki Takada1, Keita Yasutomi1, Hodaka Kawanishi1, Kazuki Tada1, Tatsuya Kobayashi1, Atsushi Yabata2, Hiroki Kasai2, Noriyuki Miura2, Masao Okihara2, and Shoji Kawahito1; 1Shizuoka University and 2LAPIS Semiconductor Co., Ltd. (Japan) [view abstract]
In this study, we present a backside-illuminated (BSI) 4-tap SOI-based lock-in pixel having a high near-infrared sensitivity. Owing to the customized process for lock-in pixels, the size of floating diffusion is greatly reduced using a self-aligned process. The lock-in pixel including 4-tap readout circuits is successfully integrated into the single modulator size of 18 × 18um. The prototype chip is implemented in 0.2um SOI technology. The chip demonstrates a high QE of 65% at 950-nm wavelength and 40ns gate/light pulse modulation. The distance up to 30 m was successfully measured using the prototype chip.
[view abstract]
In this study, we present a backside-illuminated (BSI) 4-tap SOI-based lock-in pixel having a high near-infrared sensitivity. Owing to the customized process for lock-in pixels, the size of floating diffusion is greatly reduced using a self-aligned process. The lock-in pixel including 4-tap readout circuits is successfully integrated into the single modulator size of 18 × 18um. The prototype chip is implemented in 0.2um SOI technology. The chip demonstrates a high QE of 65% at 950-nm wavelength and 40ns gate/light pulse modulation. The distance up to 30 m was successfully measured using the prototype chip.
15:40ISS-174
An 8-tap image sensor using tapped PN-junction diode demodulation pixels for short-pulse time-of-flight measurements [PRESENTATION-ONLY], Ryosuke Miyazawa1, Yuya Shirakawa1, Kamel Mars1, Keita Yasutomi1, Keiichiro Kagawa1, Satoshi Aoyama2, and Shoji Kawahito1; 1Shizuoka University and 2Brookman Technology, Inc. (Japan) [view abstract]
A novel 8-tap short pulse (SP)-based indirect TOF (iTOF) image sensor is presented. This SP-based iTOF image sensor using 8-tap pixels with a drain is suitable for outdoor long-range (>10m) applications. The designed and implemented image sensor uses a novel multi-tap demodulation pixel based on Tapped PN-junction Diode (TPD) structure which uses divided p+ hole-pinning areas of the pinned photodiode (p+/n/p- structure) as multiple electrodes of the multi-tap demodulator for the dynamic control of the buried channel potential. The TPD demodulator can directly modulate the channel potential of the photo-receiving area using multiple p+ electrodes, and hence a large photo receiving area or high fill-factor of the pixel and high-speed photo-carrier transfer in the channel can be realized, leading to simultaneously meeting high sensitivity and high demodulation speed of the multi-tap TOF pixel. Using the eight consecutive time-gating windows, each of which has a width of 10 ns, prepared by the 8-tap SP-based pixel, 10m-range high ambient light TOF measurements have been carried out. The measurement results show that the maximum non-linearity error of 1.32 %FS for the range of 1.0–11.5 m and the depth resolution of maximally 16.4 cm have been attained under sunlight level background light.
[view abstract]
A novel 8-tap short pulse (SP)-based indirect TOF (iTOF) image sensor is presented. This SP-based iTOF image sensor using 8-tap pixels with a drain is suitable for outdoor long-range (>10m) applications. The designed and implemented image sensor uses a novel multi-tap demodulation pixel based on Tapped PN-junction Diode (TPD) structure which uses divided p+ hole-pinning areas of the pinned photodiode (p+/n/p- structure) as multiple electrodes of the multi-tap demodulator for the dynamic control of the buried channel potential. The TPD demodulator can directly modulate the channel potential of the photo-receiving area using multiple p+ electrodes, and hence a large photo receiving area or high fill-factor of the pixel and high-speed photo-carrier transfer in the channel can be realized, leading to simultaneously meeting high sensitivity and high demodulation speed of the multi-tap TOF pixel. Using the eight consecutive time-gating windows, each of which has a width of 10 ns, prepared by the 8-tap SP-based pixel, 10m-range high ambient light TOF measurements have been carried out. The measurement results show that the maximum non-linearity error of 1.32 %FS for the range of 1.0–11.5 m and the depth resolution of maximally 16.4 cm have been attained under sunlight level background light.
KEYNOTE: Processing and AR/VR
Session Chairs: Jon McElvain, Dolby Laboratories (United States) and Jackson Roland, Apple Inc. (United States)
16:15 – 17:15
Red Room
16:15ISS-182
KEYNOTE: Sensing and computing technologies for AR/VR [PRESENTATION-ONLY], Chiao Liu, Meta Reality Labs Research (United States)
Augmented and Virtual Reality (AR/VR) will be the next great wave of human oriented computing, dominating our relationship with the digital world for the next 50 years, much as personal computing has dominated the last 50. AR glasses require multiple cameras to enable all the computer vision (CV) and AI functions while operating under stringent weight, power, and socially acceptable form factor constraints. The AR sensors need to be small, ultra-low power, with wide dynamic range (DR) and excellent low light sensitivity to support day/night, indoor/outdoor, all day wearable use cases. The combination of lowest power, best performance, and minimal form factor makes AR sensors the new frontier in the image sensors field. In this talk, we will first introduce some CV and AI functions to be supported by AR sensors and their associated camera sensor requirements. We will then present a new ultra-low power, ultra-wide dynamic range Digital Pixel sensor (DPS) designed to meet above specific challenges. Finally, we will discuss some system level tradeoffs and architecture directions.
Chiao Liu received his PhD in EE from Stanford University. He was a Senior Scientist at Canesta Inc (now part of Microsoft), developing the very first CMOS time-of-flight (ToF) depth sensors. He was a Technical Fellow at Fairchild Imaging (now part of BAE Systems), and worked on a wide range of scientific and medical imaging systems. In 2012, he joined Microsoft as a Principal Architect and was part of the 1st generation Microsoft AR Hololens team. Currently he is the director of research at Meta Reality Labs Research, leading the Sensors and Systems Research team. Liu is a member of the IEEE International Electron Devices Meeting (IEDM) technical committee. He also served as guest reviewer for Nature and IEEE Transactions on Electron Devices.
16:55ISS-183
On quantization of convolutional neural networks for image restoration, Youngil Seo, Irina Kim, Jeongguk Lee, Wooseok Choi, and Seongwook Song, Samsung Electronics Co., Ltd. (Republic of Korea) [view abstract]
Recently, commercial vision sensors hit the mobile market. To achieve that, computer vision networks had to be quantized. However, this topic was not studied well for Image Signal processor (ISP) challenging image restoration tasks, being crucially important for hardware implementation, as well as for deployment on hardware accelerators, e.g., Neural Processors Units (NPU). In this paper, we studied the effect of the quantization of deep learning network on image quality. We tried various quantization on raw RGBW image demosaicing. Experimental results show that 12 bit weight quantization can sustain image quality at the similar level with floating-point network. 10 bit quantized network shows slight degradation in objective image quality and mild visual artifacts. If network weight’s bit-depth can be significantly reduced for computer vision tasks, our finding shows that it is not true for raw image restoration tasks: at least 10 bit weights are required to provide sufficient image quality. However, we can save some memory on feature maps bit-depth. We can conclude that network bit depth is critical for raw image restoration.
Wednesday 19 January 2022
IS&T Awards & PLENARY: In situ Mobility for Planetary Exploration: Progress and Challenges
07:00 – 08:15
This year saw exciting milestones in planetary exploration with the successful landing of the Perseverance Mars rover, followed by its operation and the successful technology demonstration of the Ingenuity helicopter, the first heavier-than-air aircraft ever to fly on another planetary body. This plenary highlights new technologies used in this mission, including precision landing for Perseverance, a vision coprocessor, new algorithms for faster rover traverse, and the ingredients of the helicopter. It concludes with a survey of challenges for future planetary mobility systems, particularly for Mars, Earth’s moon, and Saturn’s moon, Titan.
Larry Matthies, Jet Propulsion Laboratory (United States)
Larry Matthies received his PhD in computer science from Carnegie Mellon University (1989), before joining JPL, where he has supervised the Computer Vision Group for 21 years, the past two coordinating internal technology investments in the Mars office. His research interests include 3-D perception, state estimation, terrain classification, and dynamic scene analysis for autonomous navigation of unmanned vehicles on Earth and in space. He has been a principal investigator in many programs involving robot vision and has initiated new technology developments that impacted every US Mars surface mission since 1997, including visual navigation algorithms for rovers, map matching algorithms for precision landers, and autonomous navigation hardware and software architectures for rotorcraft. He is a Fellow of the IEEE and was a joint winner in 2008 of the IEEE’s Robotics and Automation Award for his contributions to robotic space exploration.
Imaging Sensors and Systems 2022 Posters
08:20 – 09:20
EI Symposium
Poster interactive session for all conferences authors and attendees. ISS posters on display in this morning EI 2022 poster session will be presented by the authors during the Imaging Sensors and Systems 2022 Evening Interactive Poster Session.
Processing II
Session Chairs:
Jackson Roland, Apple Inc. (United States) and Nitin Sampat, Edmund Optics, Inc. (United States)
10:50 – 11:50
Green Room
10:50ISS-230
Equivalent ray optics model to enable imaging system simulation of 3D scenes [PRESENTATION-ONLY], Thomas Goossens1, Zheng Lyu1, Jamyuen Ko2, Gordon Wan2, Ricardo Motta2, Joyce Farrell1, and Brian Wandell1; 1Stanford University and 2Google Inc. (United States) [view abstract]
Combining image sensor simulation tools (e.g., ISETCam) with physically based renderers (e.g., PBRT) offers many possibilities for designing and optimizing novel imaging systems. The ability to synthesize physically accurate scene radiance images and to calculate sensor responses enables experimentation with novel designs and evaluation of the impact of image systems components. Synthesized data can also have accurate pixel-level labels, making them particularly valuable for machine learning vision applications. One limitation in simulation has been information about the optics: Lens manufacturers rarely disclose details of the lens design, making simulation in PBRT impossible. We present a pragmatic solution to this problem by using a blackbox lens in ZEMAX; such models are often made available by the manufacturer. These models describe the performance of the lens without disclosing its construction. From the blackbox model, we construct an equivalent ray-transfer function that characterizes how spectral rays at different field heights are transformed by the optics. We have implemented methods to use data from the blackbox model as part of the ray tracing calculation. We confirm that the ray transfer calculations match the Zemax lens characterization for point spread functions, chromatic aberration and f-stop dependent vignetting for rays from full 3D scenes.
11:10ISS-231
Using images of partially visible chart for multi-camera system calibration, Radka Tezaur, Gazi Ali, and Oscar Nestares, Intel Corporation (United States) [view abstract]
Incorporating in the geometric calibration images in which the calibration chart is only partially visible can help to make calibration process much more efficient and more accurate. This is particularly true in the case of systems involving a larger number of cameras. A calibration tool developed by us that makes it possible to utilize also images in which only a part of the chart can be detected is described and the benefits of using such a tool compared to the traditional checkerboard chart calibration are demonstrated. Examples illustrating the implications of the requirement that all chart points must be detected in the image on the spatial distribution of the corner points used for calibration are shown and the impact of the resulting distribution on the accuracy of the calibration is analyzed, using both synthetic and real data sets.
11:30ISS-232
ESP32-CAM as a programmable camera research platform, Henry G. Dietz, Dillon Abney, Paul Eberhart, Nick Santini, William Davis, Elisabeth Wilson, and Michael McKenzie, University of Kentucky (United States) [view abstract]
Experimenting with custom-programming of cameras can be difficult. Most consumer cameras are protected to prevent users from reprogramming them. Industrial cameras can be flexibly controlled by an external computer, but are generally not stand-alone programmable devices. However, various inexpensive camera modules, designed largely to be used for building IoT (Internet of Things) devices, combine extensive programmability with a camera in a compact, low-power, module. One of the smallest and least expensive, the ESP32-CAM module, combines a 2MP Omnivision OV2640 camera with a dual-core 32-bit processor, 802.11 WiFi and BlueTooth as well as wired I/O interfaces, a microSD slot, low power modes, etc., all supported by the Arduino programming environment and a rich collection of open source libraries. Why not use it for programmable camera research? This paper describes how the ESP32-CAM had to be adapted to enable use in a variety of experimental cameras. For example, most of these cameras do not use the lens screwed and glued onto the OV2640, and replacing this lens revealed a number of issues ranging from spectral response to adjustment of lens corrections. There are numerous strange interactions between different functions that end-up sharing the same I/O pins, so work-arounds were needed. It also was necessary to devise ways to handle various higher-level issues such as implementation of a live view and synchronization across cameras. However, the key problems have been resolved with open source software and hardware designs described here.
Image Sensing II
Session Chairs:
Boyd Fowler, OmniVision Technologies, Inc. (United States) and Francisco Imai, Apple Inc. (United States)
15:00 – 16:00
Green Room
15:00ISS-242
Accurate event simulation using high-speed video, Xiaozheng Mou, Kaijun Feng, Alex Yi, Steve Wang, Huan Chen, Xiaoqin Hu, Menghan Guo, Shoushun Chen, and Andreas Suess, OmniVision (United States) [view abstract]
Event sensing is a novel modality which is solely sensitive to changes of information. This redundancy reduction can be utilized to achieve high temporal resolution, reduce power consumption, simplify algorithms etc. The hardware-software co-design of event sensors and algorithms requires early simulation of the sensor system. It has been shown that high-speed video is well suited to derive such event data for temporal contrast based event sensors, but the simulators published so far neglect phenomena such as readout latency or refractory period. This paper presents ongoing modeling activities at OmniVision Technologies.
15:20ISS-243
Perfect RGB color routers for sub-wavelength size CMOS image sensor pixels [PRESENTATION-ONLY], Peter B. Catrysse, Nathan Zhao, and Shanhui Fan, Stanford University (United States) [view abstract]
We present a conceptually novel approach to achieve color functionality in solid-state image sensors. Instead of filtering, which absorbs most of the incident light, a color router routes all incident light based on spectral content (color) to the photodetectors. Our color router approach is lossless, photon efficient and scalable to the sub-wavelength pixel sizes. We show here that a red-green-blue (RGB) color router can achieve perfect routing (~100%) of incident light to sub-wavelength pixels while simultaneously achieving near-perfect spectral shape matching to a specific spectral response for the R, G, and B color channels.
15:40ISS-244
An anti-UV organic material integrated microlens for automotive CIS [PRESENTATION-ONLY], William Tsai, Chia-Chien Hsieh, Yuan-Shuo Chang, Sheng-Chuan Cheng, Ching-Chiang Wu, and Ken Wu, VisEra (Taiwan) [view abstract]
CMOS image sensor (CIS) for automotive applications has vast growth in recent years. Much attention has been focused on the reliability requirement of CIS against strict environmental conditions. UV resistance is one important property of cameras used in long term exposure of sun light, since the short wavelength part of solar radiation would damage components like plastic lenses and color filter array, which reduce the optical efficiency and lifetime. The typical method to prevent UV rays from entering CIS is a reflective or absorptive UV cutting layer, however, additional processing steps and cost may apply.
Imaging Sensors and Systems 2022 Evening Interactive Poster Session
16:00 – 16:30
EI Symposium
ISS posters on display in the EI 2022 Posters session in the morning will be presented by the authors during this evening ISS poster session.
ISS-199
P-14: Capture optimization for composite images, Henry G. Dietz and Dillon Abney, University of Kentucky (United States) [view abstract]
A composite image is an image created by combining portions of multiple separately-captured images. Stitching of captures of tiled portions of a larger scene can be used to produce a single composite image (a panorama) with a wider view angle and higher total resolution. Image stacking is a different type of compositing, in which the scene is not changing significantly across captures, but camera parameters might be systematically varied. Focus stacking can extend the depth of field, aperture stacking can implement apodization shaping the out-of-focus point spread function, and noise and motion reduction can be accomplished even using the same camera parameters for each capture to be stacked. These and other compositing methods are well known and commonly used, but the same fixed pattern is commonly used for ordering of captures and choice of capture parameters. This paper examines the problem of static, pseudo-static, or dynamic determination of the optimal capture parameters and ordering.
ISS-200
P-15: DePhaseNet: A deep convolutional network using phase differentiated layers and frequency based custom loss for RGBW image sensor demosaicing, Irina Kim, Youngil Seo, Dongpan Lim, Jeongguk Lee, Wooseok Choi, and Seongwook Song, Samsung Electronics Co., Ltd. (Republic of Korea) [view abstract]
Panchromatic Color Filter Arrays with white signal were introduced a while ago, such as RGBW Kodak (CFA2.0) array, assuming to have better resolution in lowlight due to panchromatic signal. However, there is no successful RGBW image sensor in the industry targeting mobile cameras until now. In this work, we introduce a novel Samsung RGBW image sensor and we study its performance in a popular remosaic scenario. We propose a DePhaseNet - a deep fully convolutional network to solve RGBW remosaicing or demosaicing problem. We propose to have 3 layers of phase differentiated inputs and custom frequency based loss function for each layer. Proposed method successfully suppress False Colors inherent to RGBW sensor due to heavily under-sampled colors. By using this method, we were able not only to increase details preservation, but also increased color reproduction by 2% over conventional method. We found that RGBW sensor is beneficial not only in low light scenarios, but also in widely spread remosaic scenarios. Experiments show improvement in image quality, yielding CPSNR of 42dB for Kodak dataset, reaching the bar of Bayer CFA demosaicing result. Proposed method advances state-of-the-art in RGBW demosaicing, by 6dB in CPSNR.
ISS-201
P-16: The study and analysis of using CMY color filter arrays for 0.8 um CMOS image sensors, Pohsiang Wang, An-Li Kuo, Ta-Yung Ni, Hao-Wei Liu, Yu C. Chang, Ching-Chiang Wu, and Ken Wu, VisEra Technologies (Taiwan) [view abstract]
Digital color cameras detect scenes through color filter array (CFA) of mosaic patterns. Among existing filter arrays in commercial CMOS image sensors (CIS), the CMY color filter set is one that is a more desirable choice for low-illuminating conditions. However, these color filters generally suffered from lower color fidelity than their counterpart RGB color filters. Nevertheless, the overall CIS performance were affected not only by the pigments used for color filters but also by the detailed structure designs and CFA arrangement of the pixel itself, we tends to explore the best combination for both the color filter materials and the pixel designs that will improve CIS performance for low light conditions that will have applications in astrophotography, low phototoxicity bioimaging, as well as general photography during dawn and dusk.
Image Sensing III
Session Chairs:
Boyd Fowler, OmniVision Technologies, Inc. (United States) and Nitin Sampat, Edmund Optics, Inc. (United States)
16:30 – 17:30
Green Room
16:30ISS-256
Design and analysis on low-power and low-noise single slope ADC for digital pixel sensors, Hyun-Yong Jung, Myonglae Chu, Min-Woong Seo, Suksan Kim, Jiyoun Song, Sang-Gwon Lee, Sung-Jae Byun, Minkyung Kim, Daehee Bae, Junan Lee, Sung-Yong Lee, Jongyeon Lee, Jonghyun Go, Jae-kyu Lee, Chang-Rok Moon, and Hyoung-Sub Kim, Samsung Electronics Co., Ltd. (Republic of Korea) [view abstract]
A Low-power and low-noise digital pixel sensor (DPS) is presented in this paper. To design and analyze the random noise (RN) of the developed DPS, especially, we utilize a novel simulationmethod called transient-based AC noise simulation (TBAS) which can effectively help to estimate the noise components of the lowpowered single-slope (SS) analog-to-digital converter (ADC). Based on this noise analysis, the high performance DPS has been successfully designed and demonstrated.
16:50ISS-257
World's first 16:4:1 triple conversion gain sensor with all-pixel AF for 82.4dB single exposure HDR, ChangHyun Park, HongSuk Lee, EunSub Shim, JungBin Yun, KyungHo Lee, Yunhwan Jung, Sukki Yoon, Ilyun Jeong, JungChak Ahn, and Duckhyun Chang, Samsung Electronics Co., Ltd. (Republic of Korea) [view abstract]
A Triple Conversion Gain(TCG) sensor with all-pixel auto focus based on 2PD of 1.4 um-pitch has been demonstrated for mobile applications. TCG was implemented by sharing adjacent Floating Diffusion(FD) without adding other capacitor. TCG helps to reduce the noise gap or slow the noise increase as user gain increases. An image with a Dynamic Range(DR) of 82.4 dB through a single exposure can be obtained through intra-scene TCG(i-TCG). Through this, a wider range of illuminance environments can be captured in the image. In addition, a more natural image can be obtained by reducing the SNR dip in one image by using TCG.
17:10ISS-258
3-Layer stacked pixel-parallel CMOS image sensors using hybrid bonding of SOI wafers, Masahide Goto1, Yuki Honda1, Masakazu Nanba1, Yoshinori Iguchi1, Takuya Saraya2, Masaharu Kobayashi2, Eiji Higurashi3, Hiroshi Toshiyoshi2, and Toshiro Hiramoto2; 1NHK Science & Technology Research Laboratories, 2The University of Tokyo, and 3National Institute of Advanced Industrial Science and Technology (Japan) [view abstract]
We report 3-layer stacked pixel-parallel CMOS image sensors developed for the first time. The hybrid bonding of silicon-on-insulator wafers through damascened Au electrodes in a SiO2 insulator on the front and backside realizes both face-to-face and face-to-back bonding, developing a multi-layer stacked device. A 3-layered pixel circuit is developed to confirm the linear response of 16-bit digital signal output. A prototype sensor with 160 × 120 pixels successfully captures video images, demonstrating the feasibility of multi-layered sensors of high performance as well as multi-functions including signal processing, memory, and computing for applications such as high-quality video cameras, measurements, recognition, robots, and various IoT devices.