Image Processing: Algorithms and Systems XXIII Program

IPAS 2025 Program

TUESDAY 4 FEBRUARY 2025

Image Formation

Session Chair: Robert Bregovic, Tampere University of Applied Sciences
09:30 - 10:30
Harbour B

09:30IPAS-225
Construction, quality assessment, and applications of pixel value error PDF models, Henry Dietz, University of Kentucky (US) [view abstract]

Increasingly sophisticated algorithms, including trained artificial intelligence methods, are now widely employed to enhance image quality. Unfortunately, these algorithms often produce somewhat hallucinatory results, showing details that do not correspond to the actual scene content. It is not possible to avoid all hallucination, but by modeling pixel value error, it becomes feasible to recognize when a potential enhancement would generate image content that is statistically inconsistent with the image as captured. An image enhancement algorithm should never give a pixel a value that is outside of the error bounds for the value obtained from the sensor. More precisely, the repaired pixel values should have a high probability of accurately reflecting the true scene content. The current work investigates computation methods and properties of a class of pixel value error model that empirically maps a probability density function (PDF). The accuracy of maps created by various practical single-shot algorithms is compared to that obtained by analysis of many images captured under controlled circumstances. In addition to applications discussed in earlier work, the use of these PDFs to constrain AI-suggested modifications to an image is explored and evaluated.

09:50IPAS-226
Computationally efficient hue-preserving gamut mapping in RGB and YUV, Touraj Tajbakhsh, Meta (US); Hamid Mirzaei, Meta (US) [view abstract]

In this paper, we present a computationally-efficient gamut mapping algorithm designed for tone-mapped images, focusing on preserving hue and adjusting luma and saturation to effectively map color into the RGB gamut. The algorithm operates in both RGB and YUV color spaces, enabling a practical implementation in both hardware and software real-time systems. We show that the algorithm maintains color fidelity during gamut mapping, offering a viable alternative to more computationally demanding methods operating in perceptually uniform spaces.

10:10IPAS-227
Multi-scale feature matching for image denoising using residual swin transformers, Muqudas Rafiq (Pakistan); Ahsan Jalil; Khurram Usman; Muhammad Abdullah (US); Bilal Zafar, [view abstract]

Image denoising is a crucial task in image processing, aiming to enhance image quality by effectively eliminating noise while preserving essential structural and textural details. In this paper, we introduce a novel denoising algorithm that integrates residual Swin transformer blocks (RSTB) with the concept of the classical non-local means (NLM) filtering. The proposed solution is aimed at striking a balance between performance and computation complexity and is structured into three main components: (1) Feature extraction utilizing a multi-scale approach to capture diverse image features using RSTB (2) Multi-scale feature matching inspired by NLM that computes pixel similarity through learned embeddings enabling accurate noise reduction even in high-noise scenarios, and (3) Residual detail enhancement using the swin transformer block that recovers high-frequency details lost during denoising. Our extensive experiments demonstrate that the proposed model with 743k parameters achieves the best or competitive performance amongst the state-of-the-art models with comparable number of parameters. This makes the proposed solution a preferred option for applications prioritizing detail preservation with limited compute resources. Furthermore, the proposed solution is flexible enough to adapt to other image restoration problems like deblurring and super-resolution.

10:30IPAS-228
JIST 1889: Tensor nuclear norm minus Frobenius norm minimization for color image denoising, Kaito Hosono, Kanagawa University (Japan); Takamichi Miyata, Chiba Institute of Technology (Japan); Hirotsugu Kinoshita (Japan) [view abstract]

The multi-channel methods have attracted much attention in the color image denoising. These methods are image denoising methods that combine the low-rankness of a matrix with the non-local self-similarity of the natural image. It applies to color images with noise of different intensities in each color channel. Denoising methods based on the low-rankness of tensors, and extensions of matrices, have also attracted attention in recent years. Many tensor-based methods have been proposed as extensions of matrix-based methods and achieve higher denoising performance than matrix-based methods. Tensor-based methods perform denoising using an approximate function of the tensor rank. However, unlike multi-channel methods, tensor-based methods do not assume different noise intensities for each channel. On the other hand, tensor nuclear norm minus Frobenius norm (TNNFN) has been proposed in the domain of traffic data completion. TNNFN is one of the tensor rank approximation functions and is known to have high performance in traffic data completion, but it has not been applied to image restorations. In this paper, we propose MC-TNNFN as a tensor-based multi-channel method. It is a TNNFN-based multi-channel method that uses TNNFN to remove noise from a tensor contracted from similar patches and then estimates the original image. Experimental results using natural images show that the proposed method outperforms existing methods objectively and subjectively.

Image Analysis and Recognition

11:00 - 12:20
Harbour B

11:00IPAS-229
Improving version detection for JAB codes, Simon Bugert, Fraunhofer SIT (Germany); Marco Fruehwein, Fraunhofer SIT (Germany); Waldemar Berchtold, Fraunhofer SIT (Germany); Ann-Katrin Riedel, Fraunhofer SIT (Germany); Julian Heeger [view abstract]

The focus of the work is to improve the reading performance of JAB Codes. JAB Code is a polychrome barcode that is ISO standardized. The weakness of the standardized decoding algorithm is the very low reading performance of under 10% for very large and rectangular codes. In many IT security applications, however, large and rectangular codes are required for the huge payload. In this paper, we present three different methods to improve the decoder. These methods aim at determining the version size of the JAB Code to be read. This is the step after the JAB Code has been located by the finder patterns and before the decoding can take place. The three methods have their advantages and disadvantages in their accuracy and performance. In the evaluation, we will point out these differences and give a recommendation.

11:20IPAS-230
PatchNet: A hybrid model for fine segmentation of MRI images across multiple organs, Farah Naaman, University of the Basque Country (Spain); Jinan Charafeddine, Leonard de Vinci Pole Universitaire, Research Center (France); Fadi Dornaika, University of the Basque Country (Spain) [view abstract]

In this paper, we introduce PatchNet, a novel hybrid model tailored for fine segmentation tasks in medical imaging, particularly focusing on MRI scans of various organs. Fine segmentation is essential for accurately distinguishing between different tissue types, identifying small abnormalities, and delineating intricate anatomical structures key in applications requiring high precision. PatchNet integrates the strengths of Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), enabling highly precise segmentation of complex anatomical structures. The model was tested across three prominent datasets ACDC 2017, CHAOS 2019, and BraTS 2020 demonstrating its versatility across different organ systems, including the heart, abdominal organs (liver, kidneys, spleen), and brain tumors Our results show that PatchNet consistently surpasses traditional U-Net models and other state-of-the-art segmentation techniques in dice score, intersection over union (IoU), and other key metrics. The hybrid architecture's ability to combine local feature extraction (CNNs) with global context awareness (ViTs) significantly enhances its performance in capturing fine details in MRI images. These results highlight PatchNet's potential as a robust tool for medical image segmentation, with promising applications across various organ systems and disease conditions. PatchNet represents an advancement in the field and holds the potential to enhance diagnostic accuracy and facilitate personalized treatment planning.

11:40IPAS-231
A novel post-processing method for convolutional neural networks in character recognition, Jarmo Koponen, University of Eastern Finland (Finland); Keijo Haataja, University of Eastern Finland (Finland); Pekka Toivanen, University of Eastern Finland (Finland) [view abstract]

This research presents a novel post-processing method for convolutional neural networks (CNNs) in character recognition, specifically designed to handle inconsistencies and irregularities in character shapes. Convolutional Neural Networks (CNNs) are powerful tools for recognizing and learning character shapes directly from source images, making them well-suited for recognition of characters that contain inconsistencies in their shapes. However, when applied to multi-object detection for character recognition, CNNs require post-processing to convert the recognized characters into code sequences, which has so far limited their applicability. The developed method solves this problem by directly post-processing the inconsistent characters identified by the convolutional neural model into labels corresponding to the source image. An experiment with real pharmaceutical packaging images demonstrates the functionality of the method, showing that it can handle different numbers of characters and labels effectively. As a scientific contribution to the fields of imaging and deep learning, this research opens new possibilities for future studies, particularly in the development of more accurate and efficient multi-object character recognition with post-processing and their application to new areas.

12:00IPAS-347
Automatic calibration of multiple fisheye cameras using recovered human body mesh, Chih-Hsien Chou, Futurewei Technologies, Inc. (US); Lin-Hsi Tsao, Futurewei Technologies, Inc. (US) [view abstract]

Human pose and shape estimation (HPSE) is a crucial function for human-centric applications, while the accuracy of deep learning-based monocular 3D HPSE may suffer due to depth ambiguity. Multicamera systems with wide baselines can solve the problem but accurate and robust multi-camera calibration is a prerequisite. Fisheye cameras providing omnidirectional vision with up to 360 field-ofview (FoV) can cover a given space with fewer cameras. The main objective for the project is to develop fast and accurate algorithms for automatic calibration of multiple fisheye cameras which fully utilize human semantic information without using predetermined calibration patterns or objects. The proposed automatic calibration method detects humans from each fisheye camera in equirectangular or spherical images. For each detected human, the portion of image defined by the bounding box will be cropped and converted to an image patch with normal field-of-view (FoV) by perspective mapping for the corresponding view angle. 3D human body meshes are then estimated by pretrained Human Mesh Recovery (HMR) model and the vertices of each 3D human body mesh are projected onto the 2D image plane for each corresponding image patch. Structure-from-Motion (SfM) algorithm is used to reconstruct 3D shapes from a pair of cameras, using iterative RANSAC algorithm to remove outliers when calculating the essential matrix. Relative camera extrinsic parameters (i.e., the rotation and translation matrices) can be calculated from the estimated essential matrix accordingly. By assuming one main camera's pose in the world coordinate is known, the poses of all other cameras in the multicamera system can be readily calculated. Four use cases with different fisheye camera configurations are simulated using (1) average 2D reprojection error and (2) rotation and translation errors as performance metrics. The proposed method is shown to perform more accurate calibration than methods using appearance-based feature extractors, e.g., Scale-Invariant Feature Transform (SIFT), and deep learning-based 2D human joint estimators, e.g., OpenPose. The proposed method is also less susceptible to partially visible human bodies due to occlusion and out-of-view truncation.

Video Processing

Session Chair: Atanas Gotchev, Tampere University of Applied Sciences
15:30 - 17:30
Harbour B

15:30IPAS-232
Pain recognition using time-space single composite image (TSSCI) technology, Ofer Hadar, Ben Gurion University (Israel); Rami Segal, Ben Gurion University (Israel); Haim Fellner Cohen, Ben Gurion University (Israel); Gal Zohar, Ben Gurion University (Israel) [view abstract]

We have developed an innovative tool for objective pain assessment based on Time-Space Single Composite Image (TSSCI) technology. This tool converts video sequences of facial expressions into a single image representing movement patterns over time, enabling machine learning models to detect and quantify pain levels. By integrating OpenPose for keypoint extraction and a Variational Autoencoder (VAE) for synthetic data generation, the system can train on real and synthetic facial expressions to classify pain intensity. Initial tests have demonstrated high accuracy in detecting pain levels across diverse scenarios, including patients with limited facial movement. This tool is precious when verbal communication is impaired, such as in pediatric care, intensive care units, and telemedicine. With further development, it has the potential to transform pain management by providing real-time, non-invasive pain assessment.

15:50IPAS-233
HEVC compressed video fingerprinting, Mohamed Allouche, Institut Polytechnique de Paris (France); Carl De Sousa Trias, Institut Polytechnique de Paris (France); Mihai Mitrea, Institut Polytechnique de Paris (France) [view abstract]

While conventional video fingerprinting methods act in the uncompressed domain (pixels and/or directly derived representations from pixels), the present paper establishes the proof of concepts for compressed domain video fingerprinting. Thus, visual content is processed at the level of compressed stream syntax elements (luma/chroma coefficients, and intra prediction modes) by a homemade NN-based solution backboned by conventional CNN models (ResNet and MobileNet). The experimental validations are obtained out of processing a state of the art and a homemade HEVC compressed video databases, and bring forthAccuracy,PrecisionandRecallvalues larger than 0.9.

16:10IPAS-234
Green video encoder identification, Mohamed Allouche, Institut Polytechnique de Paris (France); Elliot Cole, Institut Polytechnique de Paris (France); Mateo Zoughebi, Institut Polytechnique de Paris (France); Carl De Sousa Trias, Institut Polytechnique de Paris (France); Mihai Mitrea, Institut Polytechnique de Paris (France) [view abstract]

Video streaming hits more than 80% of the carbon emissions generated by worldwide digital technologies consumption that, in their turn, account for 5% of worldwide carbon emissions. Hence, green video encoding emerges as a research field devoted to reducing the size and the complexity of the video streams, while keeping a preestablished visual quality. The present paper takes the challenge of identifying whether a video stream was produced by a green-encoder or not. To this end, classification solutions backboned by the VGG, ResNet and MobileNet families are considered to discriminate MPEG-4 AVC encoded video streams based on their syntax elements, such as luma/chroma coefficients or intra prediction modes. The original video content sums-up to 46 minutes and is structured in two databases. Three encoders are alternatively studied, namely a proprietary green-encoder solution, and the two by-default encoders available on a large video sharing platform and on a popular social media, respectively. The quantitative results show classification accuracy ranging between 75% to 100%, according to the specific architecture, sub-set of classified elements, and dataset.

16:30IPAS-352
Bi-directional loop closure for visual SLAM, Sari Peltonen, Tampere University (Finland) [view abstract]

Loop closure detection is crucial for the subsequent relocalization in visual navigation systems. State-of-the-art methods typically address the problem of loop closure detection unidirectionally, following the direction of previous motion. Consequently, these methods often fail when there is no significant overlap in perspectives. We propose a bidirectional loop closure approach, enabling relocalization even when traveling in the opposite direction. Our aim is to reduce the long-term odometry drift in the absence of a direct loop. After selecting training data from large datasets suitable for the bidirectional problem, we train and validate two distinct machine learning models for loop closure detection and subsequent regression of 6-DOF camera poses in an end-to-end manner. We provide a rigorous empirical comparison against established methods, evaluating our approach on both outdoor and indoor data from the FinnForest and PennCOSYVIO datasets.

WEDNESDAY 5 FEBRUARY 2025

Image Processing: Algorithms and Systems Posters (with lunch)

12:20 - 14:00
The Grove

12:20IPAS-235
CIS image reconstruction network for real time performance on mobile device, SeokHyeon Lee, Samsung Electronics Co., Ltd (Republic of Korea); YooJeong Seo, Samsung Electronics Co., Ltd (Republic of Korea); KunDong Kim, Samsung Electronics Co., Ltd (Republic of Korea); Sung-Su Kim, Samsung Electronics Co., Ltd (Republic of Korea); Yitae Kim, Samsung Electronics Co., Ltd (Republic of Korea) [view abstract]

With the emergence of 200 mega pixel QxQ Bayer pattern image sensors, the remosaic technology that rearranges color filter arrays (CFAs) into Bayer patterns has become increasingly important. However, the limitations of the remosaic algorithm in the sensor often result in artifacts that degrade the details and textures of the images. In this paper, we propose a deep learning-based artifact correction method to enhance image quality within a mobile environment while minimizing shutter lag. We generated a dataset for training by utilizing a high-performance remosaic algorithm and trained a lightweight U-Net based network. The proposed network effectively removes these artifacts, thereby improving the overall image quality. Additionally, it only takes about 15ms to process a 4000x3000 image on a Galaxy S22 Ultra, making it suitable for real-time applications.

12:20IPAS-236
Use of sharp image content to enhance sharpness of other image areas, Hunter Durkee, University of Kentucky (US); Henry Dietz, University of Kentucky (US) [view abstract]

Many lenses have significantly poorer sharpness in the corners of the image than they have at the center due to optical defects such as coma, astigmatism, and field curvature. In some circumstances, such blur is not problematic. It could even be beneficial by helping to isolate the subject from the background. However, if there exists similar content in the scene that is not blurry, as happens commonly in landscapes or other scenes that have large textured regions, this type of defect can be extremely undesirable. The current work suggests that, in the problematic circumstances where there exists visually similar sharp content, it should be possible to use that sharp content to synthesize detail to enhance the defectively blurry areas by overpainting.The new process is conceptually very similar to inpainting, but is overpainting in the same sense that the term is used in art restoration: it is attempting to enhance the underlying image by creating new content that is congruous with details seen in similar, uncorrupted, portions of the image. The kongsub (Kentucky's congruity substitution) software tool was created to explore this new approach. The algorithms used and various examples are presented, leading to a preliminary evaluation of the merits of this approach. The most obvious limitation is that this approach does not sharpen blurry regions for which there is no similar sharp content in the image.

12:20IPAS-237
Facial image feature analysis and its specialization for Frechet distance and neighborhoods, Doruk Cetin, Align Technology (Switzerland); Benedikt Schesch, ETH Zurich Media Technology Center (Switzerland); Petar Stamenkovic, ETH Zurich Media Technology Center (Switzerland); Majed El Helou, ETH Zurich Media Technology Center (Switzerland) [view abstract]

Assessing distances between images and image datasets is a fundamental task in vision-based research. It is a challenging open problem in the literature and despite the criticism it receives, the most ubiquitous method remains the Fr chet Inception Distance. The Inception network is trained on a specific labeled dataset, ImageNet, which has caused the core of its criticism in the most recent research. Improvements were shown by moving to self-supervision learning over ImageNet, leaving the training data domain as an open question. We make that last leap and provide the first analysis on domain-specific feature training and its effects on feature distance, on the widely-researched facial image domain. We provide our findings and insights on this domain specialization for Fr chet distance and image neighborhoods, supported by extensive experiments and in-depth user studies.