AVM 2025 Program
Excel to HTML
MONDAY 3 FEBRUARY 2025
Objective Imaging Performance and Simulation I
Session Chair: Peter Burns, Burns Digital Imaging LLC
08:30 - 10:30
Grand Peninsula A
09:10AVM-100
IEEE P2020 standard for automotive image quality, Maraget Belska [view abstract]
The long-awaited IEEE P2020 Standard for Automotive Image Quality is slated to be published early this year. The Standard will be available for purchase through IEEE's website.The IEEE P2020Automotive Image Quality Working Groupwas established in 2016 in order to fill a gap in image quality evaluation created by the unique challenges of automotive imaging. These challenges include both external factors such as the environment the cameras must perform in: extreme weather conditions, high dynamic range scenes, quickly changing lighting, low light, etc.As well as the camera systems themselves, which often include fisheye or wide-angle lenses, HDR sensors, and multi-camera systems.Although other image quality standards exist and are extensively leveraged in the development work of P2020, they are not able to fully address the demands of an automotive system.The P2020 Standard contains assessment criteria for seven critical image quality factors (IQFs) including: flare,geometriccalibration validation, noise, dynamic range,spatial frequency response(SFR), flicker, and contrast performance indicators. These criteria were developed through deep collaboration between industry and academia, and through cooperation across multiple industry segments. Competitors came together to work as a team to ensure a solid and robust test framework was developed. The P2020 Standard will increase transparency and efficiency in communication between OEMs, Tier1s, test labs, and component vendors and allow for faster selection and validation of the best possible camera systems for vision applications. By ensuring the quality of the cameras being selected, application of the Standard can contribute to increased on-the-road safety by ensuring that vehicles will see the objects and details that control algorithms expect.The test methodologies presented in the Standard are already being adopted by key OEMs, Tier1s, and test labs. The Standard is poised to become a key element in automotive camera selection and verification across the automotive industry for both external and in-cabin applications.Hear about the development of the standard, why it matters to the industry, who is already using it, and how it will impact your future work with automotive cameras.
10:30 – 11:00 and 15:00 – 15:30 Coffee Breaks
Objective Imaging Performance and Simulation II
Session Chair: Patrick Denny, University of LImerick
15:30 - 17:30
Grand Peninsula A
15:30AVM-101
SOLAS: Superpositioning an optical lens in automotive simulation, Daniel Jakab, University of Limerick, (Ireland); Julian Barthel, University of Applied Sciences, (Germany); Alexander Braun, University of Applied Sciences, (Germany); Reenu Mohandas, University of Limerick, (Ireland); Brian Deegan, University of Galway, (Ireland); Mahendar Kumbham, Valeo Vision Systems, (Ireland); Dara Molloy, Valeo Vision Systems, (Ireland); Fiachra Collins, Valeo Vision Systems, (Ireland); Anthony Scanlan, University of Limerick, (Ireland); Ciaran Eising, University of Limerick, (Ireland) [view abstract]
Automotive Simulation is a potentially cost-effective strategy to identify and test corner case scenarios in automotive perception. Recent work has shown a significant shift in creating realistic synthetic data for road traffic scenarios using a video graphics engine. However, a gap exists in modeling realistic optical aberrations associated with cameras in automotive simulation. This paper builds on the concept from existing literature to model optical degradations in simulated environments using the Python-based ray-tracing library KrakenOS. As a novel pipeline, we degrade automotive fisheye simulation using an optical doublet with +/-2? Field of View(FOV), introducing realistic optical artifacts into two simulation images taken from SynWoodscape and Parallel Domain Woodscape. We evaluate KrakenOS by calculating the Root Mean Square Error (RMSE) which averaged around 0.023 across the RGB spectrum of light compared to Ansys Zemax OpticStudio, an industrial benchmark for optical design and simulation. Lastly, we measure the image sharpness of the degraded simulation using the ISO12233:2023 Slanted Edge Method and show how both qualitative and measured results indicate the extent of the spatial variation in image sharpness from the periphery to the center of the degradations.
15:50AVM-102
The challenge of moving to smaller pixels for autonomous vehicles, Robin Jenkin, NVIDIA, (US) [view abstract]
The 8Mp 2.1um imaging sensor node hasbecome popular for the current generation of autonomous vehicles. As the market further matures, there will likelybe demand for increasing resolution in the luxurysegmentof the market and, concurrently, cost-down measures applied to the existing feature set. Additionally, there is a persistent and permanent desire to reduce the size of camera modules.One potential solution to address these pressures is to reduce pixel size.Withthis reduction thereis a decrease in pixel sensitivityandalong witha need to open the aperture of lenses to support the resolution and provide increased image luminance to the sensor. The change in aperture reduces the depth of focus of the lensesused,and in-turn narrows manufacturing tolerances needed to maintain module performance.This paper examines the relationshipbetween key performance parameters, such as modulation transfer function (MTF), vibration, effective resolution, sensitivity, SNR 1, data-rates and lens placement with respect to pixel size. An envelope is calculated to maintain current camera module performance as compared to the current 8MP 2.1um node and assess the effect on manufacturing tolerances and the effective information capacity of camera systems.
16:50AVM-103
Extending the utility of noise equivalent quanta (NEQ) for dynamic range measurement in imaging systems, Uwe Artmann, Image Engineering GmbH & Co KG, (Germany) [view abstract]
This paper investigates the application of Noise Equivalent Quanta (NEQ) as a comprehensive metric for assessing dynamic range in imaging systems. Building on previous work that demonstrated NEQ's utility in characterizing noise and resolution trade-offs in imaging systems using the Dead Leaves technique, this study seeks to validate the use of NEQ for dynamic range characterization, especially in high-dynamic-range (HDR) systems where conventional metrics may fall short.
TUESDAY 4 FEBRUARY 2025
Scene Understanding Using Deep Learning for Autonomous Vehicles
Session Chair: Patrick Denny, University of LImerick
08:30 - 10:30
Grand Peninsula A
08:30AVM-104
Image segmentation: Inducing graph based learning, Aryan Singh, University of Limerick, (Ireland); Pepijn de Ven, University of Limerick, (Ireland); Ciaran Eising, University of Limerick, (Ireland); Patrick Denny, University of Limerick, (Ireland) [view abstract]
This study explores how graph neural networks (GNNs) can improve semantic segmentation across different types of images. We test their effectiveness on two datasets: the well-known Pascal VOC Segmentation dataset and the WoodScape dataset, which features challenging fisheye images often used in autonomous driving. Unlike typical CNNs like U-Net, U-Net++, and SwinUNet, which we use for comparison, GNNs utilize the connections between pixels to potentially identify object boundaries more accurately. By applying GNNs to both standard images and those with fisheye distortion, we can see how well they handle typical segmentation tasks and adapt to the unique geometry of fisheye lenses. This analysis highlights the flexibility of GNNs in managing difficult imaging situations and emphasizes their potential to enhance semantic segmentation accuracy in various applications, including self-driving technology.
08:50AVM-105
Enhancing robotic navigation with large language models, Xunyu Pan, Frostburg State University, (US); Jeremy Perando, Frostburg State University [view abstract]
This work investigates the integration of large language models (LLMs) like GPT-4 into robotic navigation systems to improve autonomous decision-making. By leveraging the power of LLMs, the robot can analyze its environment using a camera, eliminating the need for extensive pre-programmed responses. Experiments conducted in diverse environments with various objects demonstrated the robot's adaptability and efficiency in real-world scenarios, showcasing its potential for more advanced applications. Future developments seek to enhance this adaptability by enabling LLMs to autonomously generate motion profiles for tasks based on verbal instructions, further improving the robot's ability to perform specific actions without human intervention. This approach promises to create smarter, more flexible robotic systems capable of managing a wider variety of tasks and environments, offering a significant leap in autonomy and versatility for robotics.
09:10AVM-106
An annotated database for pedestrian temporal action recognition, Itsaso Rodriguez-Moreno, University of the Basque Country (UPV/EHU), (Spain); Brian Deegan, University of Galway, (Ireland); Dara Molloy, University of Galway, (Ireland); Jose Maria Martinez-Otzeta, University of the Basque Country (UPV/EHU), (Spain); Martin Glavin, University of Galway, (Ireland); Edward Jones, University of Galway, (Ireland); Basilio Sierra, University of the Basque Country, (Spain) [view abstract]
In this paper, we present a database consisting of the annotations of videos showing a number of people performing several actions in a parking lot. The chosen actions represent situations in which the pedestrian could be distracted and not fully aware of her surroundings. Those are "looking behind", "on a call", and "texting", with another one labeled as "no_action" when none of the previous actions is performed by the person. In addition to actions, also the speed of the person is labeled. There are three possible values for such speed: "standing", "walking" and "running". Bounding boxes of people present in each frame are also provided, along with a unique identifier for each person. The main goal is to provide the research community with examples of actions that can be of interest for surveillance or safe autonomous driving. The addition of the speed of the person when performing the action can also be of interest, as it can be treated as a more dangerous behavior "running" than "waking", when "on a call" or "looking behind", for example, providing the researchers with richer information.
09:30AVM-107
Enhanced industrial defect detection for edge devices using modified YOLOv8, Necdet Sonmez, Middle East Technical University, (Turkey) [view abstract]
The demand for a lightweight model for industrial defect detection from the industry is increasing each year. This work focuses on advancements in industrial defect detection, specifically on improvements to the YOLOv8 model. The study builds on existing literature that incorporates attention mechanisms to improve defect detection performance for small and visually subtle defects. Architectural modifications such as integrating a ResNet beckbone into the YOLOv8 to enhance the feature extraction process. The VISION dataset for industrial defects, containing various defect types across different product types, is used for training and evaluation. The research addresses challenges in detecting small defects, which are hard to detect in a real time setting reliably. Experimental results indicate that the original YOLOv8 model's performance can be improved with these modifications.
10:30 – 11:00 Coffee Break
Efficient Perception and Compute for Automotive
Session Chair: Patrick Denny, University of LImerick
11:00 - 12:20
Grand Peninsula A
11:00AVM-108
Low-power object detection for automotive vision with hybrid cameras, Kamal Rana, Omnivision Technologies Inc, (US); Sean Fausz, Omnivision Technologies Inc, (US); Hiroaki Inaba, Omnivision Technologies Inc, (US); Stiger He, Omnivision Technologies Inc, (US); Bo Mu, OmniVision Technologies Inc, (US) [view abstract]
Automotive vision is a key component of advanced driver assistance systems (ADAS), whose performance improves with higher CIS frame rates. While high CIS frame rates enable faster decision-making, they come at the cost of increased data rates and power consumption. In contrast, lower CIS frame rates lead to low data rate and low power consumption but longer blind intervals between frames, delaying decisions, which could be dangerous in critical situations. This trade-off between data rate and decision-making speed and performance can be mitigated using hybrid sensors relying on event pixels. Previous research on automotive vision has demonstrated that hybrid sensors can achieve performance and latency comparable to a traditional CIS camera with two orders of magnitude higher data rate. However, previous research has not explored the evaluation of models' power consumption on hardware. In this work, we investigate the power consumption of automotive vision models using hybrid sensor data on Omnivision's edge AI platform. We demonstrated that automotive vision using hybrid sensors achieves low data rate with low power consumption and low latency allowing them to be feasible at the edge.
11:20AVM-109
FlexEye - application specific quality-scalable ISP tuning, Sumbal Akram, (Pakistan); Muhammad Abdullah; Khuzaeymah Nasir; Shaharyar Yaqub; Rehan Ahmed; Rehan Hafiz [view abstract]
As AI becomes more prevalent, edge devices face challenges due to limited resources and the high demands of deep learning (DL) applications. In such cases, quality scalability can offer significant benefits by adjusting computational load based on available resources. Traditional ISPtuning methods prioritize maximizing intelligence performance, such as classification accuracy, while neglecting critical system constraints like latency and power dissipation. To address this gap, we introduce FlexEye, an application-specific, quality-scalable ISP tuning framework that leverages ISP parameters as a control knob for quality of service (QoS), enabling trade-off between quality and performance. Experimental results demonstrate up to 6% improvement in Object Detection accuracy and a 22.5% reduction in ISP latency compared to state of the art. In addition, we also evaluate Instance Segmentation task, where 1.2% accuracy improvement is attained with a 73% latency reduction.
11:40AVM-110
Data optimization strategies for collaborative perception, Besma Abdali, Institut VEDECOM, (France); Quentin Picard, Institut VEDECOM, (France); Maryem Fadili, Institut VEDECOM, (France) [view abstract]
Collaborative perception for autonomous vehicles aims to overcome the limitations of individual perception. Sharing information between multiple agents resolve multiple problems, such as occlusion, sensor range limitations, and blind spots. One of the biggest challenge is to find the right trade-off between perception performance and communication bandwidth. This article proposes a new cooperative perception pipeline based on the Where2comm algorithm with optimization strategies to reduce the amount of transmitted data between several agents. Those strategies involve a data reduction module in the encoder part for efficient selection of the most important features and a new representation of messages to be exchanged in a V2X manner that takes into account a vector of information and its positions instead of a high-dimensional feature map. Our approach is evaluated on two simulated datasets, OPV2V and V2XSet. The accuracy is increased by around 7% with AP@50 on both datasets and the communication volume is reduced by 89.77% and 92.19% on V2XSet and OPV2V respectively.
12:00AVM-111
TIOVX Apps - A new approach to development with OpenVX, Rahul Ravikumar, Texas Instruments, (India); Abhay Chirania, Texas Instruments, (India); Shyam Jagannathan, Texas Instruments, (India); Jesse Villarreal, Texas Instruments, (US) [view abstract]
OpenVX is an open, royalty-free standard for cross platform acceleration of computer vision applications. OpenVX is accepted by Automotive industry as a go-to framework for developing performance, power-optimized and safety compliant computer vision processing pipelines on real-time heterogeneous embedded SoCs. Optimizing OpenVX development flow becomes a necessity with every growing demand for variety of vision applications required in both Automotive and Industrial market. Although OpenVX works great when all the elements in the pipeline is implemented with OpenVX, it lacks utilities to effectively interact with other frameworks. We propose a software design to make OpenVX development faster by adding a thin layer on top of OpenVX which simplifies construction of an OpenVX pipeline and exposes simple interface to enable seamless interaction with other frameworks like v4l2, OpenMAX, DRM etc...
15:00 – 15:30 Coffee Break
Multi-View and Multi-Sensor 3D Perception for Automotive
Session Chair: Patrick Denny, University of LImerick
15:30 - 17:30
Grand Peninsula A
15:30AVM-112
Revisiting birds eye view perception models with frozen foundation models: DINOv2 and Metric3Dv2, Seamie Hayes, Me, (Ireland); Ciaran Eising, Supervisor, (Ireland); Ganesh Sistu, Supervisor, (Ireland) [view abstract]
Birds Eye View perception models require extensive data to perform and generalize effectively. While traditional datasets often provide abundant driving scenes from diverse locations, this is not always the case. It is crucial to maximize the utility of the available training data. With the advent of large foundation models such as DINOv2 and Metric3Dv2, a pertinent question arises: can these models be integrated into existing model architectures to not only reduce the required training data but surpass the performance of current models? We choose two model architectures in the vehicle segmentation domain to alter: Lift-Splat-Shoot, and Simple-BEV. For Lift-Splat-Shoot, we explore the implementation of frozen DINOv2 for feature extraction and Metric3Dv2 for depth estimation, where we greatly exceed the baseline results by 7.4 IoU while utilizing only half the training data and iterations. Furthermore, we introduce an innovative application of Metric3Dv2's depth information as a PseudoLiDAR point cloud incorporated into the Simple-BEV architecture, replacing traditional LiDAR. This integration results in a +3 IoU improvement compared to the Camera-only model.
15:50AVM-113
Minimizing occlusion effect on multi-view camera perception in BEV with multi-sensor fusion, Sanjay Kumar, University of Limerick, (Ireland); Hiep Truong, University of Limerick, DSW, Valeo Kronach, Germany, (Ireland); Sushil Sharma, University of Limerick, (Ireland); Ganesh Sistu, University of Limerick, Valeo Vision Systems, Ireland, (Ireland); Tony Scanlan, University of Limerick, (Ireland); Eoin Grua, University of Limerick, (Ireland); Ciaran Eising, University of Limerick, (Ireland) [view abstract]
Autonomous driving technology is rapidly evolving, offering the potential for safer and more efficient transportation. However, the performance of these systems can be significantly compromised by the occlusion on sensors due to environmental factors like dirt, dust, rain, and fog. These occlusions severely affect vision-based tasks such as object detection, vehicle segmentation, and lane recognition. In this paper, we investigate the impact of various kinds of occlusions on camera sensor by projecting their effects from multi-view camera images of the nuScenes dataset into the Bird's-Eye View (BEV) domain. This approach allows us to analyze how occlusions spatially distribute and influence vehicle segmentation accuracy within the BEV domain. Despite significant advances in sensor technology and multi-sensor fusion, a gap remains in the existing literature regarding the specific effects of camera occlusions on BEV-based perception systems. To address this gap, we use a multi-sensor fusion technique that integrates LiDAR and radar sensor data to mitigate the performance degradation caused by occluded cameras. Our findings demonstrate that this approach significantly enhances the accuracy and robustness of vehicle segmentation tasks, leading to more reliable autonomous driving systems.
16:10AVM-114
A comparative study of NeRF and 3D Gaussian splatting for automotive and edge applications, Mary Raymond, Valeo Brain Division, (Ireland); Ganesh Sistu, Valeo Brain Division, (Ireland) [view abstract]
This paper presents a comparative study of Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) models within the context of automotive and edge applications. Both models demonstrate potential for novel view synthesis but encounter challenges related to real-time rendering, memory limitations, and adapting to changing scenes. We assess their performance across key metrics, including rendering rate, training time, memory usage, image quality for novel viewpoints, and compatibility with fisheye data. While neither model fully meets all automotive requirements, this study identifies the gaps that need to be addressed for each model to achieve broader applicability in these environments.
16:30AVM-115
LiDAR panoptic segmentation for autonomous driving: A survey, Aditya Dusi, Stanford University, (US); Bassam Helou, Motional AD Inc., (US) [view abstract]
This survey provides a comprehensive overview of LiDAR-based panoptic segmentation methods for autonomous driving.We motivate the importance of panoptic segmentation inautonomous vehicle perception, emphasizing its advantages overtraditional 3D object detection in capturing a more detailed andcomprehensive understanding of the environment. We summarizeand categorize 42 panoptic segmentation methods based ontheir architectural approaches, with a focus on the kind ofclustering utilized- machine learned or non-learned heuristicclustering. We discuss direct methods, most of which use single-stage architectures to predict binary masks for each instance,and clustering-based methods, most of which predict offsets toobject centers for efficient clustering. We also highlight relevantdatasets, evaluation metrics, and compile performance results onSemanticKITTI and panoptic nuScenes benchmarks. Our analysisreveals trends in the field, including the effectiveness of attentionmechanisms, the competitiveness of center-based approaches,and the benefits of multi-modal sensor fusion. This survey aimsto guide practitioners in selecting suitable architectures and toinspire researchers in identifying promising directions for futurework in LiDAR-based panoptic segmentation for autonomousdriving.
16:50AVM-116
Flex-PLACE: Flexible Robot Base Placement for Industrial Inspection, Vanessa Staderini, Austrian Institute of Technology GmbH, (Austria); Tobias Glueck, Austrian Institute of Technology GmbH, (Austria); Andreas Kugi, Austrian Institute of Technology GmbH; Automation and Control Institute, TU Wien, (Austria) [view abstract]
Automatic visual quality inspection is a cornerstone of modern manufacturing, leveraging advancements in computer vision and robotics to enhance speed and efficiency. While numerous inspection planning methodologies exist, they often neglect the critical challenge of designing the inspection cell specifically, determining the optimal placement of the robot relative to the inspected objects. This placement is pivotal for maximizing inspection performance and minimizing the inspection time.In this work, we present a flexible framework to determine the robot base placement via an optimization routine to facilitate the inspection of diverse objects. This eliminates the need to reprogram the inspection cell whenever the object changes, significantly simplifying and streamlining the process. Extensive simulations validate the effectiveness of our method, demonstrating significant improvements in achieving high coverage and reducing the time compared to a brute force approach.
17:10AVM-117
Evaluation of the geometric accuracy of LiDAR and camera sensors in narrow underground structures, Raimund Edlinger, University of Applied Sciences Upper Austria, (Austria); Kurt Niel, University of Applied Sciences Upper Austria, (Austria) [view abstract]
This study investigates the geometric accuracy and coloring of LiDAR and camera sensors within the confined and complex environments of narrow underground structures, such as Erdstall facilities. Accurate spatial data acquisition in these settings is crucial for mobile navigation, structural monitoring, and geological surveying applications. We collected and analyzed data from LiDAR and camera systems through controlled experiments, comparing their outputs against high-precision ground truth measurements. Key performance metrics such as geometry, spatial resolution, 3D wall and color structure, error rates, and data completeness were assessed.