A Multimodal Framework for Explainable Evaluation of Soft Skills in Educational Environments

Jared D.T. Guerrero-Sosa, Francisco P. Romero, Víctor Hugo Menéndez-Domínguez, Jesus Serrano-Guerrero, Andres Montoro-Montarroso, Jose A. Olivas · 2025

Read A Multimodal Framework for Explainable Evaluation of Soft Skills in Educational Environments by Jared D.T. Guerrero-Sosa, Francisco P. Romero, Víctor Hugo Menéndez-Domínguez, Jesus Serrano-Guerrero, Andres Montoro-Montarroso, Jose A. Olivas (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01794.txt.

Introduction

Judging soft skills like communication and creativity is hard to do fairly, since they show in fleeting gestures and expressions. Guerrero-Sosa and five co-authors present a framework that evaluates them from several channels at once. A fuzzy logic model breaks complex behavior into graded linguistic pieces, handling the vagueness that crisp scores hide. The tool watches signals such as facial expressions and gestures in undergraduate students.

In experiments, the framework merged its inputs into steady, meaningful assessments of decision-making, communication, and creativity. Combining modalities improved the quality of the scores over single-channel judging. Just as important, the fuzzy design keeps the reasoning visible, so teachers and students can see how a score arose. The authors aim the system at higher education settings where soft skills matter but unbiased, explainable grading is scarce.

Catalog record

CREATORJared D.T. Guerrero-Sosa, Francisco P. Romero, Víctor Hugo Menéndez-Domínguez, Jesus Serrano-Guerrero, Andres Montoro-Montarroso, Jose A. Olivas
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-01794

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Multimodal Framework for Explainable Evaluation of Soft Skills in Educational Environments Jared D.T. Guerrero-Sosa, Francisco P. Romero, Víctor Hugo Menéndez-Domínguez, Jesus Serrano-Guerrero, Andres Montoro-Montarroso, Jose A. Olivas Abstract In the rapidly evolving educational landscape, the unbiased assessment of soft skills is a significant challenge, particularly in higher education. This paper presents a fuzzy logic approach that employs a Granular Linguistic Model of Phenomena integrated with multimodal analysis to evaluate soft skills in undergraduate students. By leveraging computational perceptions, this approach enables a structured breakdown of complex soft skill expressions, capturing nuanced behaviours with high granularity and addressing their inherent uncertainties, thereby enhancing interpretability and reliability. Experiments were conducted with undergraduate students using a developed tool that assesses soft skills such as decision-making, communication, and creativity. This tool identifies and quantifies subtle aspects of human interaction, such as facial expressions and gesture recognition. The findings reveal that the framework effectively consolidates multiple data inputs to produce meaningful and consistent assessments of soft skills, showing that integrating multiple modalities into the evaluation process significantly improves the quality of soft skills scores, making the assessment work transparent and understandable to educational stakeholders. K eywords Multimodal Analysis \cdot Soft Skills Evaluation \cdot Fuzzy Logic \cdot Linguistic Perception Model 1 Introduction In recent years, the significance of soft skills, including decision-making, creativity, and communication, has been increasingly acknowledged in educational contexts. These skills are essential for personal and professional development [ 1 ] . Nevertheless, the accurate assessment of these skills remains a considerable challenge due to their intrinsic subjectivity, which is closely linked to human emotions and behaviour [ 2 ] . Given their complexity, soft skills require advanced assessment methods that can capture a range of expressive behaviours and adapt to individual differences among students. The evaluation of soft skills can be classified according to whether the assessment is conducted offline or online [ 3 ] . Offline methods do not utilise digital technology, including public speaking, face-to-face interactions and group discussions. In contrast, online methods employ digital platforms to assess skills such as communication, collaboration and creativity in virtual environments. Some examples of online methods are video resumes and asynchronous video interviews (AVIs). A video resume is a digital tool that allows people to present themselves through a combination of verbal and non-verbal cues [ 4 ] . In contrast, asynchronous video interviews entail more structured, prompt-driven responses, frequently employed in assessments or interviews [ 3 ] . These are scalable and can be conducted without the immediate presence of an interviewer. Although both methods are flexible, they are not without limitations in terms of depth. This is because they tend to focus on observable behaviours, which can obscure more subtle expressions of soft skills. Traditional assessment techniques often fail to capture the subtle nuances of soft skills, especially in dynamic educational environments where students are exposed to a wide range of technological tools [ 5 ] , some based solely on machine learning or convolutional neural networks, resulting in limited assessments [ 6 ] . These methods may be inadequate for the assessment of complex behavioural traits, which may lead to inconsistencies in the reliability and meaningfulness of soft skill evaluations. Nowadays, the development of artificial intelligence (AI) technologies, coupled with advanced techniques such as fuzzy logic and deep learning, has enabled the interpretation of subtle human behaviours, including facial expressions [ 7 ] , emotions [ 8 ] and tone of voice [ 9 ] . These multimodal assessments integrate AI-driven analysis with linguistic perception models and can capture and quantify emotional and behavioural expressions, providing deeper insights into students’ soft skills development [ 10 ] . The utilisation of multimodal approaches facilitates the examination of a multitude of data types, collectively imparting a more intricate understanding of student interactions and engagement. Such comprehensive evaluations are necessary for establishing an educational environment that prepares students for the evolving demands of the workplace [ 11 ] . This paper introduces an approach which employs a Granular Linguistic Model of Phenomena (GLMP) integrated with multimodal analysis to assess soft skills in undergraduate students. The study applies the model to address the inherent uncertainties in emotional and behavioural expressions, thereby enhancing the interpretability and reliability of the assessments. This approach permits a systematic assessment of soft skills, whereby intricate behavioural indicators are analysed at various levels to yield meaningful insights. With this aim, this work addresses the following research questions (RQ): • RQ1: How can soft skills be objectively evaluated using multimodal analysis, specifically in educational environments? • RQ2: What are linguistic models and artificial intelligence roles in interpreting non-verbal and verbal cues in soft skills assessment? • RQ3: Can applying deep learning and fuzzy logic enhance the precision and interpretability of assessments in multimodal soft skills evaluation systems? The main contributions of this article are as follows: • Definition of a comprehensive framework for evaluating soft skills using video analysis, natural language processing, and emotion recognition techniques. • Development of a multimodal assessment tool that utilises machine learning to analyse video, audio, and text data to assess various soft skills like decision-making, communication, and creativity. • Advancement in the explainability of AI models used in education, providing more precise explanations of how decisions are made to assess soft skills. This research article is organised as follows: Section 2 introduces the principal concepts and challenges associated with the evaluation of soft skills, covering multimodal analysis methods in the context of education and examines the GLMP. The methodology for multimodal soft skills analysis, including the assessment tool design and use of fuzzy logic and deep learning, is detailed in Section 3 . Section 4 focuses on evaluating decision-making, creativity, and communication skills among undergraduate students. Section 5 explores the implications of multimodal evaluation in education and the role of explainable AI in improving transparency. Finally, Section 6 summarises the study’s contributions, limitations, and suggestions for future research. 2 Theoretical Framework and Related Work This section presents the fundamental concepts and challenges associated with the evaluation of soft skills. It also provides an overview of multimodal analysis approaches in educational environments and an examination of the Granular Linguistic Model of Phenomena for capturing complex behavioural assessments. 2.1 Challenges in Soft Skills Evaluation The integration of soft skills into the technical undergraduate curriculum presents a number of challenges. Educators frequently encounter obstacles, including students’ attitudes, limited time, and large class sizes, which complicate the teaching of soft skills in these programmes [ 12 ] . Notwithstanding these challenges, the development of soft skills is crucial for enhancing employability and performance in engineering and IT professions [ 13 , 11 , 14 ] . In response to these challenges, some institutions have redesigned capstone courses to include team projects focused on soft skills training and assessment [ 13 ] . Others have introduced methodologies that incorporate soft skills without detracting from core technical content [ 15 ] . Nevertheless, a considerable number of students do not fully recognise the relevance of soft skills to their future careers [ 16 ] . Research conducted in Egypt has identified significant deficiencies in students’ understanding and proficiency in these skills, along with the necessity for universities to assume a

Read the complete text