Physical Annotation for Automated Optical Inspection: A Concept for In-Situ, Pointer-Based Trainingdata Generation
1 st Oliver Krumpek, 2 nd Oliver Heimann, 3 rd Prof. Jörg Krüger · 2025
Read Physical Annotation for Automated Optical Inspection: A Concept for In-Situ, Pointer-Based Trainingdata Generation by 1 st Oliver Krumpek, 2 nd Oliver Heimann, 3 rd Prof. Jörg Krüger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-05026.txt.
Introduction
Oliver Krumpek, Oliver Heimann, and Jorg Kruger propose a hands on way to label flaws for machine vision training. Skilled checkers hold know how that screen clicks capture poorly. Their rig lets a checker point and trace right on the part itself with a tracked pointer.
A projector throws guide marks on the part to keep labels true. Traces turn into standard label files that feed known training tools. Early tests show fine traces can be caught and flow into next steps with ease. Non coders can thus build training sets and keep scarce samples from loss.
Catalog record
| CREATOR | 1 st Oliver Krumpek, 2 nd Oliver Heimann, 3 rd Prof. Jörg Krüger |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-05026
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Physical Annotation for Automated Optical Inspection: A Concept for In-Situ, Pointer-Based Trainingdata Generation 1 st Oliver Krumpek, 2 nd Oliver Heimann, 3 rd Prof. Jörg Krüger Abstract This paper introduces a novel physical annotation system designed to generate training data for automated optical inspection. The system uses pointer-based in-situ interaction to transfer the valuable expertise of trained inspection personnel directly into a machine learning (ML) training pipeline. Unlike conventional screen-based annotation methods, our system captures physical trajectories and contours directly on the object, providing a more intuitive and efficient way to label data. The core technology uses calibrated, tracked pointers to accurately record user input and transform these spatial interactions into standardised annotation formats that are compatible with open-source annotation software. Additionally, a simple projector-based interface projects visual guidance onto the object to assist users during the annotation process, ensuring greater accuracy and consistency. The proposed concept bridges the gap between human expertise and automated data generation, enabling non-IT experts to contribute to the ML training pipeline and preventing the loss of valuable training samples. Preliminary evaluation results confirm the feasibility of capturing detailed annotation trajectories and demonstrate that integration with CVAT streamlines the workflow for subsequent ML tasks. This paper details the system architecture, calibration procedures and interface design, and discusses its potential contribution to future ML data generation for automated optical inspection. Technical University Berlin Pascalstr. 8-9, 10587 Berlin, Germany I Introduction Fig. 1 : Pointer based labeling of a visible defect on the surface of a mechanical part in a manual inspection process using the bounding box feature. Fig. 2 : Illustration of the system’s components, along with the work- and dataflow involved in generating training data. Recent advances in AI models have enabled a multitude of novel optical inspection tasks in logistics and manufacturing. When trained properly, AI-driven systems have the ability to facilitate manual processes or, depending on requirements and complexity, fully automate inspection tasks. However, the performance of these AI-driven systems is highly dependent on the quality and accuracy of the data on which they are trained. Collecting such data in real-world scenarios with human inspectors in the loop remains a challenge. In many projects, the availability of manually labelled training data is a bottleneck for implementing robust automated inspection systems. In industrial quality inspection, the number of parts with actual defects can be relatively small compared to the total number of objects inspected. This is especially true in safety-related quality control, where a 100% detection rate is required for objects with high production standards. In addition, the process of digitizing and annotating defects is usually neither straightforward nor well integrated. It often requires the use of external digitisation devices and supplementary software. Furthermore, inspection processes are often well established and have been in place for many years. Changing them typically requires financial investments and a high level of acceptance at the management level. This aligns with current research, which identifies data availability and insufficient IT skills as the main challenges in implementing AI in manufacturing [ 9 ] . To address these challenges, we propose a physical, in-situ annotation system that captures the user’s expert knowledge directly within the inspection process. By leveraging intuitive pointer-based interactions, our approach enables inspectors to annotate classes and segments directly on the object surface. The framing idea and goal of the development is an inspection system based on the continuous and mutual exchange of expert knowledge between the human inspector and the AI expert system. In this way, the system aims to improve the quality and robustness of inspection tasks. It also increases acceptance of AI in the process by making the process of data generation more transparent and integrated into existing workflows. This paper details the system architecture, calibration procedures, and interface design, underscoring how our approach bridges the gap between expert insight and effective ML training data acquisition in manufacturing environments. II Contributions We present a novel in-situ approach to training data generation, specifically designed to address real-world problems in manual and assisted optical inspection tasks. Our approach allows data generation by non-IT experts through intuitive pointer-based interaction — simply by pointing at the target object, without the need for screen-based annotation tools. The system is designed to save time and avoid missing valuable and often rare training samples by integrating the annotation process directly into the inspection workflow. III Related Work Human-in-the-loop annotation methods, which are the focus of this article, emphasise semi-automatic and directly supervised approaches. While incorporating human expertise increases the time and effort required, it ultimately improves the quality and accuracy of the training data. These methods rely on specialised screen-based interfaces such as CVAT [ 6 ] , LabelImg [ 15 ] , Supervisely [ 11 ] and the VGG Image Annotator [ 8 ] , all of which support a wide range of annotation techniques for 2D, 3D and multimodal data. In addition, advances in segmentation technology, such as the Segment Anything Model (SAM) [ 12 ] , enable intuitive point-and-click interactions that rapidly select, highlight and refine regions of interest. In parallel, the annotation process has evolved into a significant business in its own right. The global data annotation market has experienced remarkable growth, with industry leaders such as Scale AI [ 3 ] and Appen [ 10 ] driving large-scale annotation operations. Annotation services are even expanding into low-income regions, with companies such as CloudFactory [ 5 ] taking advantage of lower labor costs in emerging markets. Moreover, the effective use of advanced annotation tools typically requires a certain level of IT expertise [ 2 ] , and the annotation process is often separate from the actual inspection activities, involving different workflows and personnel. Undoubtedly, the generation of training data represents a significant investment of both time and money, as highlighted by recent industry analyses [ 7 ] . IV Concept Formulation The hardware of our proposed system consists mainly of a 5MP industrial camera, an optical tracking system including a tracked pointer device and a conventional projector to provide visual feedback during the annotation process (Fig. 5 ). Both camera systems are mounted on a rigid aluminium support structure positioned 100 cm above the table surface. We use a 12 mm camera lens to cover a working area of 50 cm x 40 cm on the table surface. The projector and optical tracking system are mounted so that they are completely within the working area or projection cone of the camera. While the projection component can be considered an optional feature aimed at user convenience, early system tests and trials revealed that operators highly valued the visual feedback. This led to a more confident and intuitive annotation workflow. To provide core functionality, we developed a software consisting of a simple front-end and a back-end that handles processing logic such as trajectory approximation and system communication. In principle, generating text annotations is straightforward - one can choose a simple line-by-line format (e.g. normalised bounding box coordinates as used in YOLO), or opt for structured formats such as XML or JSON that embed detailed object attributes and spatial information. However, by integrating the CVAT API, we leverage its rich export options, editing capabilities and data consistency benefits. This