Learning to See Inside Opaque Liquid Containers using Speckle Vibrometry
Matan Kichler, Shai Bagon, Mark Sheinin · 2025
Read Learning to See Inside Opaque Liquid Containers using Speckle Vibrometry by Matan Kichler, Shai Bagon, Mark Sheinin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-20757.txt.
Introduction
A sealed soda can hides its secret well: no camera can tell whether it is full or fizzing empty. Matan Kichler, Shai Bagon, and Mark Sheinin teach computers to read the answer from tiny surface vibrations instead. Their 2025 paper shines laser speckle on containers and films how the speckle shivers when sound touches the scene.
The team recorded vibration maps for many everyday containers, then trained a transformer model to name the container and its hidden fill level from the shiver alone. The model ignored whether the sound was staged or ambient, and it handled unseen cans from known families. The authors present the method as the first remote, hands-off way to check many sealed containers at once, with uses from stockrooms to recycling.
Catalog record
| CREATOR | Matan Kichler, Shai Bagon, Mark Sheinin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-20757
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning to See Inside Opaque Liquid Containers using Speckle Vibrometry Matan Kichler, Shai Bagon, Mark Sheinin Abstract Computer vision seeks to infer a wide range of information about objects and events. However, vision systems based on conventional imaging are limited to extracting information only from the visible surfaces of scene objects. For instance, a vision system can detect and identify a Coke can in the scene, but it cannot determine whether the can is full or empty. In this paper, we aim to expand the scope of computer vision to include the novel task of inferring the hidden liquid levels of opaque containers by sensing the tiny vibrations on their surfaces. Our method provides a first-of-a-kind way to inspect the fill level of multiple sealed containers remotely, at once, without needing physical manipulation and manual weighing. First, we propose a novel speckle-based vibration sensing system for simultaneously capturing scene vibrations on a 2D grid of points. We use our system to efficiently and remotely capture a dataset of vibration responses for a variety of everyday liquid containers. Then, we develop a transformer-based approach for analyzing the captured vibrations and classifying the container type and its hidden liquid level at the time of measurement. Our architecture is invariant to the vibration source, yielding correct liquid level estimates for controlled and ambient scene sound sources. Moreover, our model generalizes to unseen container instances within known classes ( e.g. , training on five Coke cans of a six-pack, testing on a sixth) and fluid levels. We demonstrate our method by recovering liquid levels from various everyday containers. 1 Introduction Figure 1 : Learning to ‘see’ the fill level of opaque containers. We develop a novel system to capture the surface vibrations of multiple liquid containers at multiple surface points. We vibrate the containers using a nearby speaker and train a novel Vibration Transformer to infer the container class and the hidden liquid level, expressed as a percentage of total capacity, from the measured multi-point surface vibrations. Since its inception in the 1960s, computer vision has sought to enable machines to infer a wide range of information about scene objects and events. In industrial settings, this capability enables the automation of various inspection and monitoring tasks involving manufactured goods, mechanical parts, or stored items across a range of environments. Today, much like in its early days, most computer vision systems rely on sensing using conventional cameras designed to replicate the human eye. However, using conventional cameras limits the scope of retrievable scene information. For instance, an image or video can be used to detect and identify a scene object, but the object’s internal properties, like its content or material composition, are mostly indeterminable from its captured surface appearance. In this paper, we focus on revealing one commonplace hidden object property: the fill level of opaque liquid containers. Our work joins a rich body of previous vision research focused on retrieving hidden object properties by leveraging unconventional imaging. Clever hyperspectral scene probing and polarization cues were used to classify [ 42 , 27 , 43 , 44 ] and segment [ 28 , 50 ] scene object materials. Material properties were also probed by thermal imaging [ 41 , 10 , 12 , 33 , 37 ] and structured light methods [ 30 ] . While effective, these methods can only probe object properties that are optically accessible at the object’s surface. To probe deeper, one must use a signal that permeates the object’s interior but can be optically sensed at the object’s surface. One such signal is object vibrations. In a series of seminal works, Bounman, Davis et al. , captured the minute vibrations of simple objects like rods and fabrics using high-speed cameras to extract properties like object density and Young’s modulus [ 13 , 6 ] . Later, Feng et al. , extended the vibrometry-based method to infer volumetric stiffness and density [ 19 , 35 ] . However, capturing high-frequency, low-amplitude vibrations on general object surfaces using high-speed cameras is challenging due to bandwidth and optical magnification constraints. To address these challenges, Sheinin et al. , recently demonstrated modal analysis using a dual-shutter camera prototype that leverages laser speckle to capture vibrations at high speeds for colinear scene points [ 46 ] . Later, Zhang et al. , used the same system to recover the anisotropy of different-material planar objects [ 58 ] . The visual vibration works described above focused on recovering low-level object properties like motion spectra, material stiffness, or density. In this paper, we seek to infer a higher-level object semantic property: the amount of liquid it presently holds. Specifically, similar to Davis et al. [ 13 ] , we excite the scene using a nearby speaker and measure the resulting object vibrations using a novel speckle-based imaging system. Our system is inspired by prior speckle-based vibration works [ 46 , 57 , 5 , 56 , 55 , 7 ] . However, unlike previous works, it can capture a 2D grid of scene points simultaneously, enabling vibration measurement of multiple objects at once, with each object probed at multiple points on its surface (see Figs. 1 and 3 ). Notably, several prior works tackled the liquid level recovery task by either recording the sound of liquid pouring [ 4 , 54 ] , or recording the sound resulting from a physical knock on the container [ 21 ] . Conversely, our approach eliminates the need for any physical interaction with the container or reliance on nearby microphones, enabling passive, remote inference of liquid content using only visual measurements. Inferring the hidden liquid level from a container’s vibrations is a challenging task. The vibrational response of an object having a simple shape and material composition can be modeled using a small set of material parameters ( e.g. , mass and stiffness). As shown in prior works, these parameters can be recovered by observing the object’s resonant frequencies and mode shapes [ 13 , 19 ] . While some liquid containers, such as certain wine glasses, exhibit a simple relationship between the fluid level and the resulting resonant frequencies, most everyday containers feature complex geometries and heterogeneous materials, resulting in a non-trivial relationship between the liquid level and the resulting vibrations. The challenge becomes substantial when inferring fluid levels of unseen containers from the same class, as slight manufacturing variations can shift resonant frequencies even among identical-looking items ( e.g. , cans in a six-pack). To address this challenge, we develop a learning-based approach introducing a novel physics-inspired ‘Vibration Transformer’ to classify the fluid level of various everyday liquid containers. The Vibration Transformer receives a spectral decomposition of the recorded two-axis, multi-point surface vibrations. It is, thus, invariant to the speaker excitation (content and duration). 1 1 1 As long as the excitation signal is sufficiently broadband. Our approach introduces a novel way to remotely assess the fill levels of multiple sealed containers at once without requiring any physical handling or manual weighing. We believe our method could facilitate the inspection of large warehouses storing consumer liquid products ( e.g. , soda cans, shampoos), heavy containers impractical to weigh, and containers holding hazardous, toxic, flammable, or radioactive liquids (