AffectEval : A Modular and Customizable Affective Computing Framework
Emily Zhou, Khushboo Khatri, Yixue Zhao, Bhaskar Krishnamachari · 2025
Read AffectEval : A Modular and Customizable Affective Computing Framework by Emily Zhou, Khushboo Khatri, Yixue Zhao, Bhaskar Krishnamachari (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-21184.txt.
Introduction
Emily Zhou, Khushboo Khatri, Yixue Zhao, and Bhaskar Krishnamachari introduce AffectEval, a software framework for affective computing. That field builds systems that sense and respond to human emotion. It serves uses in education, child development, and health, but each new project tends to rebuild the same pipeline by hand.
AffectEval offers modular, customizable parts so teams can assemble emotion recognition pipelines with less repeated work. The authors tested it by recreating earlier published experiments. In those tests, the framework cut programming effort by up to ninety percent, measured in lines of code. They argue such tooling helps results travel across domains instead of staying one-off builds.
Catalog record
| CREATOR | Emily Zhou, Khushboo Khatri, Yixue Zhao, Bhaskar Krishnamachari |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-21184
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AffectEval : A Modular and Customizable Affective Computing Framework Emily Zhou, Khushboo Khatri, Yixue Zhao, Bhaskar Krishnamachari Abstract . The field of affective computing focuses on recognizing, interpreting, and responding to human emotions, and has broad applications across education, child development, and human health and wellness. However, developing affective computing pipelines remains labor-intensive due to the lack of software frameworks that support multimodal, multi-domain emotion recognition applications. This often results in redundant effort when building pipelines for different applications. While recent frameworks attempt to address these challenges, they remain limited in reducing manual effort and ensuring cross-domain generalizability. We introduce AffectEval , a modular and customizable framework to facilitate the development of affective computing pipelines while reducing the manual effort and duplicate work involved in developing such pipelines. We validate AffectEval by replicating prior affective computing experiments, and we demonstrate that our framework reduces programming effort by up to 90%, as measured by the reduction in raw lines of code. 1. Introduction Affective computing is an interdisciplinary field of research for recognizing, interpreting, and responding to human affect with computational methodologies. While stimuli may vary, many affective computing pipelines share the same setup. Oliveira et al. (2023) outlines the key steps of signal acquisition, signal preprocessing, feature extraction and prediction as the key steps of the affective computing pipeline, which is illustrated in Figure 1 . As a result, several software libraries and frameworks have been developed to support the creation of affective computing pipelines ( Mertes et al., 2024 ; Carreiras et al., 15 ; Shandhi et al., 2022 ; Oliveira et al., 2023 ; Makowski et al., 2021 ) . However, due to the broad applications of affective computing—which encompasses mental health (MH) disorders ( Zucco et al., 2017 ; Assabumrungrat et al., 2022 ; Greene et al., 2016 ) , personalized education ( Bota et al., 2019 ) , and aging-related diseases such as Alzheimer’s ( Smith et al., 2021 ) —these software frameworks are not comprehensive and have additional limitations that create high manual costs and prevent them from being reusable. A comprehensive framework for affective computing should provide all the components necessary to implement a wide range of use cases across domains, signals, and methodologies. To guide the development of such a framework, we analyzed existing affective computing tools. Programming languages such as Python ( Van Rossum and Drake Jr, 1995 ) offer libraries such as scikit-learn ( Pedregosa et al., 2011 ) , pyHRV ( Gomes, 18 ) , and NeuroKit2 ( Makowski et al., 2021 ) that support parts of the affective computing pipeline. This includes signal preprocessing, feature extraction, and classification. Recent work has also introduced frameworks aimed at being more flexible ( Oliveira et al., 2023 ) and user-friendly ( Mertes et al., 2024 ) to facilitate the development of affective computing pipelines. However, we identify the following main limitations: (1) Researchers typically need to create pipelines from scratch using various existing libraries and frameworks. This often requires extensive software experience, is time-consuming, and results in duplicated work across applications. (2) Generally, affective computing pipelines are signal- and domain-specific; i.e. , they only support a limited set of signals and applications. As a result, it is difficult to reuse code across pipelines or make them generalizable across datasets. (3) There is no standardized interface to compare the performance of individual components across different affective computing techniques. To the best of our knowledge, there is no comprehensive framework for the development of end-to-end affective computing systems. To address the aforementioned limitations, such a framework should be modular , customizable , and flexible to support multimodal signals, i.e. , from multiple types of sensors and data streams, and multi-domain applications, such as general affect, stress, and depression detection. A framework consisting of individual modules, each responsible for a different component of the affective computing pipeline, provides an easily reusable structure. Users should also be able to customize modules as needed to create new pipelines or adapt existing ones for new applications. They can also compare the performance of different components in a fair playground by modifying their behaviors, such as using different signal preprocessing methods, feature extraction techniques, and classification models. We present AffectEval : a modular and customizable affective computing framework that generalizes to more signals and applications than existing frameworks. Our framework is designed under the object-oriented paradigm such that each component is responsible for a separate part of the affective computing pipeline. We program each component with default functionalities that can also be overwritten with custom methods, allowing researchers to easily set up basic end-to-end pipelines or create specialized applications as needed. We primarily focus on the use of time-series physiological signals for affective computing due to their extensive usage in emotion recognition and connection to mental states as suggested by the Somatic Marker Hypothesis ( Damasio et al., 1991 ) , but AffectEval can be easily extended to support other types of signals. We also establish a standard format for affective computing datasets in order to mitigate the amount of manual setup work required, which we detail in Section 3.1 . Additionally, we demonstrate the functionality of AffectEval by reproducing the experiments of Schmidt et al. ( Schmidt et al., 2018 ) and Zhou et al. ( Zhou et al., 2023 ) using the Anxiety Phases Dataset (APD) ( Senaratne et al., 2021 ) and Wearable Stress and Affect Detection dataset (WESAD) ( Schmidt et al., 2018 ) . These datasets were chosen because they are well-established multimodal datasets for stress and affect detection. Both authors conducted different types of affect recognition tasks: Schmidt et al. performed binary and three-class affect classification on WESAD, while Zhou et al. performed binary stress detection on both APD and WESAD. These datasets and experiments are therefore suitable for demonstrating the ease-of-use and reusability of AffectEval , as well as its ability to create signal- and domain-agnostic pipelines. Our AffectEval -based pipeline’s model performance largely matches or exceeds the performance achieved in previous work ( Zhou et al., 2023 ; Schmidt et al., 2018 ) , while using up to 90% fewer lines of code. We reuse the pipeline structure across our experiments and incorporate different preprocessing techniques, feature extraction methods, and models using AffectEval ’s modular design. In summary, this paper makes the following contributions: • We conduct an extensive literature review on existing affective frameworks to motivate our design of AffectEval . • We develop AffectEval , a comprehensive framework with pre-implemented behavior to facilitate affective computing research. We open-source the framework and artifacts to foster future research in this domain. • We migrate existing work to AffectEval , providing a starting point for researchers to compare affective computing works under a fair playground. • We reproduce existing work, quantifying the reduction of manual work AffectEval achieves while meeting or exceeding their model performance metrics. Section 2 provides an overview of existing methodologies, software libraries, and pipelines for affective computing. Section 3 describes the architecture of AffectEval and outlines the steps necessary to implement an affective computing pipeline using our framework. Section 4 describes our implementation of AffectEval to replicate Schmidt et al. and Zhou et al’s experiments. Section 5 discusses