LSC-ADL: An Activity of Daily Living (ADL)-Annotated Lifelog Dataset Generated via Semi-Automatic Clustering
Minh-Quan Ho-Le [email protected], Duy-Khang Ho [email protected], Van-Tu Ninh [email protected], Cathal Gurrin [email protected], Minh-Triet Tran [email protected] · 2025
Read LSC-ADL: An Activity of Daily Living (ADL)-Annotated Lifelog Dataset Generated via Semi-Automatic Clustering by Minh-Quan Ho-Le [email protected], Duy-Khang Ho [email protected], Van-Tu Ninh [email protected], Cathal Gurrin [email protected], Minh-Triet Tran [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-02060.txt.
Introduction
Minh-Quan Ho-Le and co-authors add the missing layer to lifelog research: what the wearer was actually doing. Wearable camera archives are searchable by image content, but activity-level labels, which carry the temporal story of a day, were absent from the standard LSC data set. Their LSC-ADL release annotates that archive with activities of daily living.
The annotations were built semi-automatically: a density-based clustering algorithm grouped similar moments within each activity class, and humans verified the clusters. Action recognition woven into retrieval makes the results explainable in terms of routines, not just frames. The authors release the annotations openly, aiming at lifelog retrieval, activity recognition, and egocentric vision research that needs days, not snapshots.
Catalog record
| CREATOR | Minh-Quan Ho-Le [email protected], Duy-Khang Ho [email protected], Van-Tu Ninh [email protected], Cathal Gurrin [email protected], Minh-Triet Tran [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-02060
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LSC-ADL: An Activity of Daily Living (ADL)-Annotated Lifelog Dataset Generated via Semi-Automatic Clustering Minh-Quan Ho-Le [email protected], Duy-Khang Ho [email protected], Van-Tu Ninh [email protected], Cathal Gurrin [email protected], Minh-Triet Tran [email protected] Abstract Lifelogging involves continuously capturing personal data through wearable cameras, providing an egocentric view of daily activities. Lifelog retrieval aims to search and retrieve relevant moments from this data, yet existing methods largely overlook activity-level annotations, which capture temporal relationships and enrich semantic understanding. In this work, we introduce LSC-ADL, an ADL-annotated lifelog dataset derived from the LSC dataset, incorporating Activities of Daily Living (ADLs) as a structured semantic layer. We generate accurate ADL annotations to enhance retrieval explainability using a semi-automatic approach featuring the HDBSCAN algorithm for intra-class clustering and human-in-the-loop verification. By integrating action recognition into lifelog retrieval, LSC-ADL bridges a critical gap in existing research, offering a more context-aware representation of daily life. We believe this dataset will advance research in lifelog retrieval, activity recognition, and egocentric vision, ultimately improving the accuracy and interpretability of retrieved content. The ADL annotations can be downloaded at https://bit.ly/lsc-adl-annotations . Keywords Egocentric Action Recognition \cdot Activity of Daily Living \cdot Image Retrieval 1 Introduction Lifelogging refers to collecting and recording data about one’s everyday activities. Lifelog retrieval is searching personal lifelog data based on specific queries, aiming to provide users with relevant moments from their daily lives. In the context of this work, lifelog data primarily consists of images captured continuously by wearable cameras, offering a first-person, egocentric view of the user’s life. To improve retrieval performance, various techniques, such as query expansion, feature extraction, and similarity learning, have been explored to enhance accuracy and explainability. Among these, activity-level information—particularly Activities of Daily Living (ADLs), which refers to daily routine activities that people perform—remains an underutilized yet valuable source. Unlike static visual features, activity-level cues capture temporal relationships between frames, offering a more semantically rich understanding of the lifelogger’s behaviors. This is especially important for lifelog data because ADL labels can offer helpful insights into an individual’s lifestyle and behavior patterns. Since these patterns are repetitive, recognizing them can help build a more personalized and context-aware representation of daily life. Therefore, this presents a key opportunity for research. In this work, we present a new dataset called LSC-ADL, which is derived from a subset of the original Lifelog Search Challenge’s (LSC) dataset Gurrin et al. 2022 , enriched with additional ADL annotations for each image. These annotations were generated using a clustering-based labeling approach with human-in-the-loop verification. We believe that the newly available dataset would encourage lifelog researchers to validate our hypothesis that this metadata could enhance the lifelog retrieval systems’ performance and increase the retrieval results’ explainability. 2 Related work 2.1 Action recognition In recent years, action recognition has seen significant advancements, particularly with integrating deep learning techniques. Wang et al. 2023a has employed the concept of motion-augmented long-short contrastive learning, which enhances few-shot action recognition by embedding motion dynamics and long-range temporal context. Additionally, Zhou et al. 2023 has applied an object-aware attention module that enriches the feature representation with object information and improves overall action recognition accuracy. Wang et al. 2023b has showcased the effectiveness of multimodal learning by adapting language-image models, highlighting the potential of textual information in understanding actions. The ongoing development of intelligent human action recognition techniques continues to address challenges such as variability, complexity, and imperfect data, paving the way for more robust and efficient systems Kumar and Kumar 2024 . 2.2 Action recognition on egocentric lifelog data Egocentric lifelog data, collected from a first-person perspective through wearable devices and smartphones, provides a rich source of multimodal information for human activity recognition. Researchers have developed hybrid fusion frameworks that combine late and intermediate fusion mechanisms to enhance recognition performance Oh and Kim 2024 . Additionally, Nie et al. 2024 has utilised many machine learning techniques, such as Decision Tree, K-Nearest Neighbors, Logistic Regression, etc. to recognize activity patterns. On the other hand, Nguyen et al. 2018 has explored the utility of hand poses and object interactions to improve recognition accuracy. These methods aim to improve the precision and relevance of activity recognition, further enhancing the retrieval applications. 3 Methodology 3.1 Original dataset The original dataset Gurrin et al. 2022 is a multimodal lifelogging dataset collected by a single lifelogger every day over 18 months and is publicly available for research. It consists of: (1) a core image dataset containing wearable camera images captured with a narrative clip device at 1024 × 768 resolution, fully redacted to remove faces, text, and sensitive scenes for privacy compliance; (2) metadata files providing timestamps, geographic coordinates, and semantic location descriptions; and (3) visual concepts extracted from the non-redacted version, offering object and scene information. The total number of images in the dataset is over 700,000, making it a valuable resource for lifelog research, especially lifelong retrieval. It has been used in 4 consecutive editions of the challenge from 2022 to 2025. To construct the LSC-ADL dataset, we selected approximately 75% of the original images. Each image is annotated with a single activity label, informed by both the associated metadata (such as timestamps, GPS coordinates, and semantic locations) and the extracted visual features. While all available information contributes to the annotation process, the visual features play a pivotal role in determining the most representative activity for each image. 3.2 Label selection Figure 1: Label Selection Pipeline – The process begins with a random sampling of 20 days x 15 temporally ordered images from the LSC dataset. These images are processed by the LLaMA-3.2-11b-vision model to generate captions, which are concatenated into higher-level descriptions. Simultaneously, raw ADL labels are extracted from other ADL-related works. Both the generated descriptions and raw labels are fed into a GPT model for new label generation and categorization before further manual processing into final ADL classes. We developed a systematic label selection pipeline that combines insights from existing ADL datasets with a semi-automated selection process based on MLLM models to construct a well-structured set of activity labels for our dataset. This pipeline ensures that the final label set is both comprehensive and well-adapted to the LSC dataset. Firstly, raw activity labels are collected from four previous works Nie et al. 2024 , Nguyen et al. 2018 , Cartas et al. 2017 , Wang and Smeaton 2013 . These papers suggest a diverse range of daily activities, covering various aspects of human behavior. Compiling the labels from these sources creates an initial pool of candidate activities that serves as a foundation for our selection process. Since different datasets define and categorize ADLs in different ways accordingly to the individuals’ lifestyles, the collected labels required further processing to ensure consistency and relevance to LSC dataset. Next, to make the action labels more aligned with the LSC dataset’s lifestyle, 20 days are randomly selected. We extracted 15 images chronologically for each selected day to capture a representative sequence of activities throughout the day. Subsequently, we utilize Llama-3.2-11b-Vision AI@Meta 2024 to produce egocentric captions that emphasize the actions performed by the user in each frame. The generated captions were then concatenated into a single descriptive paragraph summarizing that day’s activities. With these daily descriptions in hand, a GPT-based approach is employed