Dynamic Agentic AI Expert Profiler System Architecture for Multidomain Intelligence Modeling

Aisvarya Adeseye, Jouni Isoaho, Seppo Virtanen, Mohammad Tahir · 2026

Read Dynamic Agentic AI Expert Profiler System Architecture for Multidomain Intelligence Modeling by Aisvarya Adeseye, Jouni Isoaho, Seppo Virtanen, Mohammad Tahir (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05345.txt.

Introduction

Adeseye, Isoaho, Virtanen, and Tahir build a profiler that rates how expert a speaker sounds from spoken answers. It sorts replies into Novice, Basic, Advanced, or Expert, so machines can match their tone to the person in front of them. Its layered design runs on a small open language model and scores each answer as it arrives. Tested on 82 recorded transcripts and 402 live interviews, its ratings matched speakers' own ratings in 83 to 97 percent of cases, with gaps traced to self-rating bias and unclear answers.

Catalog record

CREATORAisvarya Adeseye, Jouni Isoaho, Seppo Virtanen, Mohammad Tahir
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27447
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.05345

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dynamic Agentic AI Expert Profiler System Architecture for Multidomain Intelligence Modeling Aisvarya Adeseye, Jouni Isoaho, Seppo Virtanen, Mohammad Tahir Abstract In today’s artificial intelligence (AI) driven world, modern systems communicate with people with diverse backgrounds and skillsets. Hence, for this human-machine interaction to be more meaningful, it is important for machines to be aware of the context and user expertise. Therefore, this study proposes an agentic AI profiler that uses four levels: Novice, Basic, Advanced, and Expert to classify natural language responses. The system was built with a modular layered architecture and LLaMA v3.1 (8B) with components for text preprocessing, scoring, aggregation, and classification. The system evaluation was done in 2 phases: static and dynamic. The static phase analyzed pre-recorded transcripts from 82 participants. Similarly, the dynamic phase evaluated 402 live interviews conducted with an agentic AI interviewer. In both phases, participants’ self-ratings were compared with the profiler’s rating. However, during the dynamic phase, participants’ expertise was evaluated after every response, not at the end of the interview as in the static phase. Across domains, 83%–97% of profiler evaluations matched participants’ self-evaluations. The remaining differences came from participants overrating or underrating themselves, unclear responses, and occasional LLM misinterpretations of nuanced expertise. I Introduction Modern AI systems, especially Large Language Models (LLMs), can understand simple human language. They can also derive meanings from human responses and provide answers that fit the situation [ 1 , 2 ] . Because of this, machines can now communicate in a way that feels more natural and human. This helps people learn, share ideas, and work better in many areas, such as education, healthcare, and business [ 3 ] . The ability of LLMs to adapt to a particular language, tone or intent is a major strength [ 4 , 5 ] . However, for effective communication, AI systems must understand the user’s expertise level to adapt communication style [ 6 ] , so that it provides meaningful support that builds user trust [ 7 ] . Therefore, this study aims to develop and validate a profiling system that is capable of dynamically and transparently analyzing human expertise with a local LLM. When natural language capabilities of LLMs are combined with expertise level profiling, it enables LLM-driven systems to exchange thoughts at a more personalized level [ 8 , 9 , 10 , 11 ] . Thus, this research is guided by the key question: ”How can we design an AI-based system architecture that accurately evaluates and classifies human expertise from different domains?” The main contribution of the study is the design and validation of an expertise profiler system architecture. This is a framework that integrates linguistic scoring, reasoning analysis, and contextual evaluation in one unified process. It introduces a new approach to test expertise in two modes: static and offline mode, and dynamic and real-time mode. Hence, adaptive profiling helps reduce bias and improve accuracy caused by the way people perceive themselves. Additionally, it provides an explainable and interpretable structure, enhancing the transparency and fairness of AI-driven evaluation systems. II Related Works Borna et al. [ 12 ] built a Machine Learning (ML) model that processes medical data to find experts. It has layers for data cleaning, extracting features, and classifying experts. It was mainly used to match doctors with tasks. The raising of privacy issues because of patient data is a limitation. The work of Bukowski et al. [ 13 ] focused on using text mining to build profiles of emerging experts. The system scans publications and patients to find keywords and topics. It comprises data collection, text analysis, and a profile creation module. Limitations include not being able to capture experts with hidden skills and little publication. Also, Balog and Rijke [ 14 ] built a probabilistic model with similarity measures to match expert profiles inside organizations. The system architecture includes indexing, retrieval, and a ranking component. Drawbacks included the need for a large data set and structured company data. Similarly, Fazel-Zarand and Fox [ 15 ] built a time-aware recommender system and temporary models to track skillset change over a period of time for staff recruitment and career planning in companies. When skillsets change quickly, it fails to cope, which is a major limitation. Finally, Fu and Akbar [ 16 ] used a co-authorship graph network to group and cluster experts. It is used for research community mapping and leadership identification. Some limitation includes overdependence on complete data and difficulty in mapping overlapping communities. The literature indicates that there are no expert profilers that analyze human textual conversations to determine expertise level, which is the focus of this study. III System Architecture of Expert Profiler The Expert Profiler is an agentic AI system that uses written textual conversations like interview transcripts or live discussions to determine the expertise level of a person. The system is divided into several sequential but modular layers with their own responsibilities. Thus, it is scalable, transparent, and flexible, making it adaptable in various areas such as education, health, and research. The following section explains the architecture in Figure 1 . Fig. 1: Expert Profiler System Architecture: A Layered Framework for Expertise Classification from Textual Responses III-A Input Layer This is the first layer; it takes inputs in two ways, thus making it versatile: • Transcript Collector: It collects pre-recorded large interview transcripts for batch processing. • Response Stream: It collects live responses for real-time analysis during interview discussions. III-B Preprocessing Layer Raw conversational data contains noises that include inconsistencies and irrelevant patterns. Consequently, this layer eliminates such noises using three key processes: 1. Text Normalization: This focuses on eliminating formatting differences. For example, treating “AI,” “A.I.,” and “artificial intelligence” as the same thing. This eliminates confusion and improves consistency. 2. Segmentation: This process involves breaking long transcripts into shorter sentences and phrases to analyze them as separate chunks and not the entire transcript at once. This improves textual understanding. 3. Domain Lexicon Mapper: This includes defining domain-specific vocabulary that helps in improving the system’s transcript understanding. For example, words with more than one meaning, such as ”token” and ”fine-tuning” can be explicitly stated to the system as technical AI terms. . This ensures that the analysis is unbiased as a result of linguistic irregularities, enhancing interpretability, which is important for experts profiling in different domains. III-C Feature Scoring Engine This is the core of the system. It contains five parallel modules that cover different expertise indicators. • Terminology Scorer: This is a measure of how domain-specific vocabulary is applied accurately by the participants. For example, in cybersecurity, “phishing”, “encryption”, and “firewall” are accurate domain terminology. • Depth Analyzer: This measures how well the ideas are linked together and not just surface definitions, for example, an explanation of how encryption works and why it is important, shows deeper understanding. • Application Evaluator: This is a measure of the application of knowledge to real-life situations. For example, explaining how employee data could be protected indicates applied understanding. • Rigor Assessor: Involves the application of logical reasoning to check the structure, clarity, and evidence presented in the responses. • Uncertainty Handler: This focuses on finding phrases that indicates uncertainty like “ I think ” or “ maybe ” as

Read the complete text