lifeXplore at the Lifelog Search Challenge 2020 Conference: Proceedings of the Third Annual Workshop on the Lifelog Search Challenge; June 9, 2020; Dublin, Ireland Proceedings of the Third Annual Workshop on the Lifelog Search Challenge (LSC ’20), June 9, 2020, Dublin, Ireland Price: 15.00 DOI: 10.1

Andreas Leibetseder, Klaus Schoeffmann · 2025

Read lifeXplore at the Lifelog Search Challenge 2020 Conference: Proceedings of the Third Annual Workshop on the Lifelog Search Challenge; June 9, 2020; Dublin, Ireland Proceedings of the Third Annual Workshop on the Lifelog Search Challenge (LSC ’20), June 9, 2020, Dublin, Ireland Price: 15.00 DOI: 10.1 by Andreas Leibetseder, Klaus Schoeffmann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-21397.txt.

Introduction

Andreas Leibetseder and Klaus Schoeffmann present their lifelog search system as entered in the 2020 Lifelog Search Challenge, in a paper cataloged in 2025. Lifelogs are vast personal archives of photos and video from wearable cameras. The annual challenge races teams on finding a described moment inside such an archive, fast.

Their system, lifeXplore, mixes map-style browsing of visual features, concept search, filtering, and hand-drawn sketch queries. For this edition they added a deep object detector, text reading from images, and uniform sampling as an alternative to shot-based splitting. The paper reports the upgraded system’s design and challenge run. Tools like this point toward personal archives that answer questions instead of merely storing the past.

Catalog record

CREATORAndreas Leibetseder, Klaus Schoeffmann
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-21397

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

lifeXplore at the Lifelog Search Challenge 2020 Conference: Proceedings of the Third Annual Workshop on the Lifelog Search Challenge; June 9, 2020; Dublin, Ireland Proceedings of the Third Annual Workshop on the Lifelog Search Challenge (LSC ’20), June 9, 2020, Dublin, Ireland Price: 15.00 DOI: 10.1145/3379172.3391721 ISBN: 978-1-4503-7136-0/20/06 CCS: Information systems Multimedia and multimodal retrieval CCS: Information systems Search interfaces CCS: Human-centered computing Interactive systems and tools Andreas Leibetseder, Klaus Schoeffmann Abstract . Since its first iteration in 2018, the Lifelog Search Challenge (LSC) – an interactive competition for retrieving lifelogging moments – is co-located at the annual ACM International Conference on Multimedia Retrieval (ICMR) and has drawn international attention. With the goal of making an ever growing public lifelogging dataset searchable, several teams develop systems for quickly solving time-limited queries during the challenge. Having participated in both previous LSC iterations, i.e. LSC2018 and LSC2019, we present our lifeXplore system – a video exploration and retrieval tool combining feature map browsing, concept search and filtering as well as hand-drawn sketching. The system is improved by including additional deep concept YOLO9000, optical character recognition (OCR) as well as adding uniform sampling as an alternative to the system’s traditional underlying shot segmentation. 1. Introduction With the quantified self movement ( Lupton 2016 ) , self-tracking practices have gained widespread popularity. Followers of the movement hold the desire to gain and exhibit knowledge about themselves through numbers, which predominantly is linked to sportive activities. Nevertheless, there is ambition to go one step further and integrate collecting personal data into most aspects of one’s daily life. So called lifeloggers ( Gurrin et al. 2014 ) record and maintain a digital diary of their lives, at various levels of detail -- be it by automatically and periodically taking pictures of their every step, recording their heart rate throughout the day or even manually logging their calorie intake. Of course much more common activities certainly can also be attributed to the lifelogging movement. For instance, when uploading pictures of precious moments to social networks, often not only time and date of corresponding photos are included but even other attributes like their geographic location, rendering them very informative activity logs. Therefore, given that most content uploaded to platforms such as Facebook 1 1 1 https://www.facebook.com are very likely to somehow be related to persons maintaining their own profiles, they at least in a basic sense can indeed be considered lifeloggers. Although lifelogging can entail much personal gain such as the potential of recalling and reliving personal highlights, maintaining a life log can be costly: apart from certain lifestyle adjustments, technological advances as well as long-term storage have to be considered for the diverse and multimodal data collected. Finally, when considering the eventually overwhelming amount of data gathered over time, its maintenance and organization becomes harder and harder: how can a person revisit past memories without potentially knowing corresponding dates, places and other potentially important details? Albeit extensive lifelogs can certainly explored aimlessly, they can not easily be searched to find specific events in a way that complements human recollection typically fading over time. Therefore, it becomes apparent that managing big lifelogging archives requires smart interactive technologies not only for retrieval but also exploration, which possibly can trigger a person to remember more details. Figure 1. lifeXplore’s feature maps: hierarchical 2D shot-keyframe grids organized according to various criteria. The annual Lifelog Search Challenge ( Gurrin et al. 2020 ; Gurrin et al. 2019a ) (LSC), co-located at the ACM International Conference on Multimedia Retrieval (ICMR), addresses aforementioned issues and encourages international teams to develop systems for lifelog exploration and retrieval. It features a large public lifelogging dataset that needs to be made manageable by the participating teams. By adequately preprocessing and integrating the data into their individual systems prior to the competition, the teams later compete against each other in quickly retrieving specific lifelogging events, i.e. solving time-limited textual tasks announced at the conference during a live competition – the fastest team finding the most events wins. Similar to the annual Video Browser Showdown ( Lokoč et al. 2019 ; Lokoc et al. 2018 ; Schoeffmann 2014 ) (VBS), co-located at the International Conference on MultiMedia Modeling (MMM), not only the systems’ creators compete as expert users but, in order to encourage the development of easy-to-use systems, uninvolved persons such as challenge spectators are as well asked to participate as novice users, each handling one of the provided systems after being introduced to it briefly. In contrast to LSC2018 and LSC2019 employing an NTCIR-13 lifelogging test collection comprising roughly a month of collected data ( Gurrin et al. 2019b ) , LSC2020’s dataset ( Gurrin et al. 2020 ) is more than four times as big and contains several month-long periods of consecutive lifelogs recorded within the years of 2015-2018. Similar to the previous dataset, the one for LSC2020 consists of daily logs comprising photos taken periodically (roughly every 40 seconds) from a first-person perspective with automatically extracted deep visual concepts as well as time-stamped data from other sensors, e.g. geographic location, speed or heart rate. As already employed for previous LSC iterations, for LSC2020 we as well propose our evolving lifeXplore system ( Leibetseder et al. 2019 ; Münzer et al. 2018 ) – see Figure 1 depicting its main interface. It is based on diveXplore ( Leibetseder et al. 2020 ; Schoeffmann et al. 2019 ; Primus et al. 2018 ; Leibetseder et al. 2018 ; Schoeffmann et al. 2017a ) – a system iteratively developed for the most recent VBS challenges (VBS2017-VBS2020). Improvements to the current system are summarized as follows: YOLO9000:: We add YOLO9000 to our collection of deep concepts, cf. Section 4.1 . OCR:: We add optical character recognition (OCR) as a new filter, cf. Section 4.2 . Uniform Sampling:: Additionally to our traditional shot segmentation, we add uniform sampling as an alternative way to segment videos, cf. Section 4.3 . The remainder of this paper is structured as follows: we describe lifeXplore briefly in Section 2 , present example strategies of how to solve a past LSC task from an expert as well as novice perspective in Section 3 and outline major changes and improvements for LSC2020 in Section 4 . Section 5 finally concludes this work together with giving an outlook on future work. 2. The lifeXplore system The web-technologes-based lifeXplore system is derived from the interactive video exploration and retrieval system diveXplore, hence, it is build for working with pre-analyzed video clips. As the LSC dataset exclusively contains images, for preprocessing they need to be transcoded into videos: one video is created for a day’s worth of sequential image data at a frame-rate of 5 fps – this ensures an adequate playback rate for the in-system videoplayer and makes video navigation more precise than with faster frame-rates. This results in 114 video clips, which subsequently are organized into coherent scenes according to a custom shot detection algorithm ( Primus et al. 2016 ) based on optical flow. For every shot detected, a keyframe is chosen that the system uses as a representation for its entire content on multiple views. For instance, Figure 1 shows the system’s main interface – a

Read the complete text