StandUp4AI: A New Multilingual Dataset for Humor Detection in Stand-up Comedy Videos
Valentin BarriereUniversidad de Chile – DCCSantiago, Chile [email protected] Nahuel GomezUniversidad de Chile – DIESantiago, Chile [email protected] Leo HemamouWithout AffiliationParis, France [email protected] Sofia CallejasINRIA ChileSantiago, Chile [email protected], Brian R · 2025
Read StandUp4AI: A New Multilingual Dataset for Humor Detection in Stand-up Comedy Videos by Valentin BarriereUniversidad de Chile – DCCSantiago, Chile [email protected] Nahuel GomezUniversidad de Chile – DIESantiago, Chile [email protected] Leo HemamouWithout AffiliationParis, France [email protected] Sofia CallejasINRIA ChileSantiago, Chile [email protected], Brian R (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18903.txt.
Introduction
Valentin Barriere and co-authors release StandUp4AI in a 2025 paper, a large data set for finding humor in stand-up comedy videos. The set covers seven languages, from English and French to Hungarian and Czech.
The set holds more than three hundred hours of shows, the largest of its kind so far. Audience laughter is marked by machine for all clips, and the test part is marked by hand. Humor is tagged word by word, not as a yes-or-no clip label, to keep full joke context. The team also shows a trick that uses speech-to-text errors to boost laughter spotting. Model builders get a rich, shared test bed.
Catalog record
| CREATOR | Valentin BarriereUniversidad de Chile – DCCSantiago, Chile [email protected] Nahuel GomezUniversidad de Chile – DIESantiago, Chile [email protected] Leo HemamouWithout AffiliationParis, France [email protected] Sofia CallejasINRIA ChileSantiago, Chile [email protected], Brian R |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18903
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
StandUp4AI: A New Multilingual Dataset for Humor Detection in Stand-up Comedy Videos Valentin BarriereUniversidad de Chile – DCCSantiago, Chile [email protected] Nahuel GomezUniversidad de Chile – DIESantiago, Chile [email protected] Leo HemamouWithout AffiliationParis, France [email protected] Sofia CallejasINRIA ChileSantiago, Chile [email protected], Brian RavenetUniversité Paris Saclay -- LISNOrsay, France [email protected] Abstract Aiming towards improving current computational models of humor detection, we propose a new multimodal dataset of stand-up comedies, in seven languages: English, French, Spanish, Italian, Portuguese, Hungarian and Czech. Our dataset of more than 330 hours is automatically annotated in laughter (from the audience), and the subpart left for model validation is manually annotated. Contrary to contemporary approaches, we do not frame the task of humor detection as a binary sequence classification, but as word-level sequence labeling, in order to take into account all the context of the sequence and to capture the continuous joke tagging mechanism typically occurring in natural conversations. As par with unimodal baselines results, we propose a method for e propose a method to enhance the automatic laughter detection based on Audio Speech Recognition errors. Our code and data are available online: https://tinyurl.com/EMNLPHumourStandUpPublic 1 Introduction and Related Works Figure 1: Overview of humor detection modeled as a sequence labeling task, and the method relying on complementary errors from the ASR outputs. Omine et al. 2024 model detected no laughter. Video available here Humor detection remains a challenging tasks for computer systems Kalloniatis and Adamidis 2024 ; Hyun et al. 2024 . Yet, such mechanisms could be a massive improvement, in particular for conversational interactive systems such as chatbots and socially interactive agents. These kind of systems, which are designed to simulate a natural human-like conversation and its structure Ludusan and Schuppler 2022 , often struggle to identify or handle humorous attempts from the user, leading to inefficient and frustrating experiences Zargham et al. 2023 . While different theories of humor exist, most of them have in common the idea that humor emerges when the current situation surprisingly deviates from our expectations Warren and McGraw 2016 . Generally, a joke or funny story is based on the following sequence : the setup of a joke introduces some expectations on how a story usually ends and the punchline reveals the reality and the unexpected (and funny) twist of the story Martin and Ford 2018 . Sometimes, additional funny comments called tags can be added around and after the punchline to maintain the momentum of the laughter. Conversational humor often stems from unexpected deviations in content, behavior, or context, with timing and intensity being critical yet unpredictable triggers Wyer and Collins 1992 . Despite theoretical models, comedians rely on live testing to refine timing, phrasing, and delivery for audience engagement Raskin 1979 , as responses depend on cultural and contextual factors. This highlights the complexity of modeling humor computationally, necessitating diverse datasets to capture its multifaceted dynamics. Stand-up comedy, due to its nature aiming at recreating the spontaneity of everyday conversational humor, is a great context for studying these mechanisms and structures with computers. Many previous works investigating computational techniques to process humor relied on corpus of people speaking in less natural and more conventional and standardized ways. For instance, in Purandare and Litman 2006 ; Bertero and Fung 2016 ; Patro et al. 2021 ; Liu et al. 2024 , the authors relied on acted data from sitcoms. The UR-FUNNY and Ted Laughter Hasan et al. 2019 ; Chen and Lee 2017 datasets are composed of TED talks, which contain less outbursts of laughter and poorer language diversity than stand-up comedy. Most of the work participating in The MuSE challenges for the automatic estimation of humor are relying on public interviews Amiriparian et al. 2023 ; Amiriparian et al. 2024 , using the Passau-Spontaneous Football Coach Humour dataset Christ et al. 2022 . Additionally, while some of the previous work explored other languages Chauhan et al. 2021 , most of them are investigating english humorous content only. Another early work on stand-up humor is the one of Turano and Strapparava 2022 , which analyzes 90 scripts of 68 comedians, in English only. The closest work from ours would be the one described in Kuznetsova and Strapparava 2024 , which proposed a 40 hours dataset in Russian and English. Most humor detection models in videos treat humor as a sequence classification task, identifying punchlines only at the end of a sequence Choube and Soleymani 2020 ; Hasan et al. 2019 ; Kuznetsova and Strapparava 2024 ; Liu et al. 2024 . However, multiple laughs can occur within a single sentence, sometimes consecutively. To address this, we reframe the task as sequence labeling, enabling continuous prediction of audience laughter throughout the joke, rather than relying on end-only classification. In this article, we are presenting multiple contributions towards the development of humor detection models. First, we collected and annotated a dataset of stand-up comedy performance in different languages extracted from online videos. This dataset is the largest and most linguistically diverse multilingual dataset of live comedy performances. It has the ambition to be a reference dataset for any type of humor modeling tasks. Second, we propose a original methodology for the task of humor detection by using a sequence labeling approach we adapted to automatically predict laughter during a performance. Third, this led us to came up with new techniques for handling errors in automatic transcription and automatic laughter detection, validated on a manually laughter annotated test set. Fourth, we present first results of sequence labeling models built on our dataset and applied to predict laughter to be used as baselines by the community. Youtube Channels Language Videos Hours Words Laughter Comedy Central English 263 51.2 442,904 25,772 Comedy Central UK 319 18.6 174,369 13,486 Comedy Central Latam Spanish 971 59.2 499,329 21,708 Comedy Central España 404 18.0 150,256 5,215 Comedy Central Italia Italian 567 55.0 433,417 12,248 Comedy Central Magyarország Hungarian 73 11.4 78,002 6,875 Paramount Network CZ Czech 123 11.5 75,806 6,129 Montreux Comedy French 652 86.0 814,727 26,789 Comedy Central Brasil Portuguese 245 23.4 218,592 9,972 Total MLing 3,617 334.2 2,887,402 128,194 Table 1: The collection of videos retained for the dataset. Laughter is the number of words labeled as laughter. 2 Dataset The StandUp4AI dataset is composed of 3,617 standup videos in 7 languages. It contains the associated transcriptions and audience laughters that have been automatically refined, of comedians during Stand-up comedy performances in various languages. To build the dataset, we first collected a specific set of videos of Stand-up comedy from the internet, we then performed automatic transcription on these videos and we finally fixed some errors in the outputs by developing improved transcription and automatic laughter annotation techniques (overview in Figure 1 ). 2.1 Video Recollection In total, we gathered 334 hours of video in 7 morphologically diverse languages 1 1 1 latin, germanic, slavic, and uralic which is around 3M words and 130k laughter labels. Table 1 illustrates the quantity of videos collected per channel and per language. On each channels, we excluded videos from the Youtube Shorts section and videos where more than one comedian appeared. 2.2 Automatic laughter detection The next step was to run a task of automatic laughter detection on the videos. The