Falcon : Functional Assembly and Language for Compositional Reasoning in X-ray
Yonathan Michael †, Mohamad Alansari † ∗, Natnael Takele †, Andreas Henschel †‡, Naoufel Werghi †‡ · 2026
Read Falcon : Functional Assembly and Language for Compositional Reasoning in X-ray by Yonathan Michael †, Mohamad Alansari † ∗, Natnael Takele †, Andreas Henschel †‡, Naoufel Werghi †‡ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25701.txt.
Introduction
A bomb in a suitcase rarely travels whole: its battery, detonator, and charge sit apart, each harmless alone. Yonathan Michael, Mohamad Alansari, Natnael Takele, Andreas Henschel, and Naoufel Werghi build Falcon in 2026 to reason about such compositional threats in X-ray baggage scans. Instead of labelling single objects, Falcon turns region features from segmentation into a structured safety state: which components are present, which pairs could work together, and how risky the whole scene looks. That state feeds a language model as an explicit middle layer.
A companion benchmark, Falcon-X, grades models on dense grounding plus component completeness and risk judgement in cluttered scans. Existing multimodal models adapted to appearances but stumbled on the relational reasoning. Falcon improved functional grounding and gave more coherent threat assessments. The authors frame compositional safety reasoning as its own test track for multimodal systems, apart from ordinary object detection.
Catalog record
| CREATOR | Yonathan Michael †, Mohamad Alansari † ∗, Natnael Takele †, Andreas Henschel †‡, Naoufel Werghi †‡ |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.25701
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Falcon : Functional Assembly and Language for Compositional Reasoning in X-ray Yonathan Michael †, Mohamad Alansari † ∗, Natnael Takele †, Andreas Henschel †‡, Naoufel Werghi †‡ Abstract Conventional vision-language models are largely object-centric, focusing on detecting and describing individual entities. In safety-critical X-ray baggage screening, however, threat often emerges not from a single object but from the functional compatibility of spatially dispersed components, such as batteries, detonators, and explosive charges. We formalize this setting as compositional threat reasoning , where risk is modeled as a relational property of grounded regions rather than an independent detection outcome. We introduce Falcon , a multimodal framework that abstracts segmentation-aware region features into a structured safety state capturing component presence, pairwise functional compatibility, and scene-level risk. This structured representation is injected into the language model as an explicit intermediate interface, encouraging relationally consistent and safety-aware reasoning. To evaluate this problem, we present Falcon-X , a benchmark that unifies dense grounding with structured supervision over component completeness and risk inference in cluttered X-ray imagery. Experiments show that while existing multimodal models adapt to appearance, they struggle with compositional safety reasoning. Falcon improves functional grounding and produces more coherent threat assessments, establishing compositional safety reasoning as a distinct evaluation paradigm for multimodal systems. Figure 1 : Falcon enables segmentation-aware functional threat reasoning in X-ray imagery. It performs instance grounding, component presence recognition, referring functional grounding and more, while generating grounded and domain-aware natural-language explanations. Falcon reasons over spatially distant components of IED under heavy clutter, supporting structured threat analysis beyond object detection. 1 Introduction X-ray baggage screening is a safety-critical task characterized by heavy clutter, material transparency, and severe object overlap [ 33 ] . Recent deep learning approaches have significantly advanced prohibited-item detection [ 13 , 30 , 2 ] on large-scale X-ray benchmarks [ 22 , 36 , 31 , 38 , 21 ] , and multimodal models have enabled captioning and VQA in this domain [ 32 ] . However, existing systems remain fundamentally object-centric and threat is inferred from the presence of individual categories. In operational settings, however, risk is often compositional . Improvised explosive devices (IEDs) may be transported in dismantled form, with batteries, detonators, and explosive charges spatially separated within a bag. Individually benign, these components become hazardous only through functional compatibility . The central question is therefore not object recognition, but relational safety inference: Can spatially separated components under severe superposition collectively constitute a functional threat? Addressing this setting requires structured reasoning over multiple grounded regions and their interactions. Current X-ray benchmarks evaluate object-level detection, while multimodal large language models (MLLMs) [ 12 , 16 , 11 , 19 , 27 , 25 , 3 , 37 ] rely on implicit attention without explicit supervision over inter-component compatibility. Consequently, relationally consistent safety inference remains largely unexplored in cluttered X-ray imagery. We formalize compositional threat reasoning as a structured multimodal prediction problem in which safety is modeled as a relational property of grounded components. We introduce Falcon , a segmentation-aware multimodal framework that maps region-level features to an explicit safety state comprising component presence, pairwise functional compatibility, and calibrated scene-level risk. This structured state is injected into the language model via a Structured Safety Adapter (SSA) , enforcing relational consistency between perception and reasoning (Fig. 1 ). Table 1 : Comparison of Falcon-X with existing X-ray security benchmarks. DC denotes explicit modeling of dismantled components. Falcon-X is the only dataset that jointly supports dense grounding, multimodal understanding, and structured functional threat reasoning. Dataset # Samples DC Annotations Task Suite Avail. Labels Bbox Mask Caption RefSeg RefPan Pan VQA RL FG GG SA GDXray (JNDE’15) [ 17 ] 8,150 ✗ ✓ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ SIXray (CVPR’19) [ 22 ] 8,929 ✗ ✓ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ OPIXray (ACMMM’20) [ 34 ] 46,642 ✗ ✓ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ HiXray (ICCV’21) [ 31 ] 45,364 ✗ ✓ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ CLCxray (ITIFS’22) [ 38 ] 9,565 ✗ ✓ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ PIXray (ITMM’22) [ 18 ] 5,046 ✗ ✓ ✓ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ EDS (CVPR’22) [ 29 ] 14,219 ✗ ✓ ✓ ✗ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ LPIXray (CIPAE’23) [ 7 ] 60,950 ✗ ✓ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ PIDray (IJCV’23) [ 36 ] 47, 677 ✗ ✓ ✓ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ DvXray (ITIFS’24) [ 17 ] 5,496 ✗ ✓ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ STCray (CVPR’25) [ 32 ] 46,642 ✗ ✓ ✓ ✓ ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ Falcon-X 6,911 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Figure 2 : Qualitative comparison of structured threat reasoning. Existing models, including X-ray domain VLMs, do not model functional relation of components. RED: indicates hallucinations, BLUE: indicates weak semantic relevance, and GREEN: indicates correct description. To evaluate this setting, we introduce Falcon-X , a benchmark of \sim 7,000 real X-ray scans with instance-level annotations for key IED components. Beyond object detection, Falcon-X provides structured supervision for component completeness and calibrated risk, enabling evaluation of relational grounding and compositional safety reasoning under severe superposition. We benchmark Falcon-X across general-purpose and domain-adapted VLMs (Fig. 2 ). Our contributions are threefold: • We formalize compositional threat reasoning in X-ray imagery, modeling safety as a relational property of spatially grounded components. • We introduce Falcon-X, the first benchmark coupling dense grounding with structured supervision for functional completeness and safety-consistent risk inference. • We propose Falcon, a structured multimodal framework that injects an explicit safety state into region-level MLLMs to improve relationally coherent threat assessment. 2 Related Work General MLLMs: Multimodal large language models (MLLMs) extend language models to vision by coupling visual encoders with language backbones. Early methods [ 26 , 12 ] demonstrate the effectiveness of large-scale image-text pretraining for transferrable multimodal representations. Instruction tuned extensions [ 16 , 39 ] build conversational interfaces on top of frozen pretrained vision encoders to answer complex queries about whole images. To enable explicit target localization, recent works [ 25 , 3 , 19 ] incorporate region-level representations by encoding bounding boxes as tokens and [ 37 ] achieves that by extracting region features via pooling. [ 11 , 27 ] further extend language models to generate pixel-level masks for fine-grained localization. Despite these advances, such MLLMs have been evaluated mainly on everyday images and visually separable scenarios, and they do not target safety-critical tasks to explicitly reason over functional relationships or to detect disassembled threat components under heavy occlusion. X-ray baggage analysis: X-ray security images pose unique challenges due to material transparency, object overlap and heavy clutter. Large-scale benchmarks [ 21 , 36 , 34 , 22 , 31 ] have enabled progress in detecting prohibited items under the above conditions. These datasets, predominantly follow