Automatic identification of diagnosis from hospital discharge letters via weakly supervised Natural Language Processing Authors: Vittorio Torri, Elisa Barbieri, Anna Cantarutti, Carlo Giaquinto, Francesca Ieva Published: 2026 ABSTRACT Identifying patient diagnoses from hospital discharge letters is essential for large-scale cohort selection and epidemiological research, but traditional supervised approaches require extensive manual annotation, which is often impractical for large textual datasets. We present a weakly supervised Natural Language Processing (NLP) pipeline for classifying Italian discharge letters without document-level manual annotation. The method extracts diagnosis-related sentences, generates semantic embeddings using a transformer model further pre-trained on Italian medical documents, and applies a two-level clustering procedure to derive weak labels that are then used to train a document-level classifier. The approach was evaluated in a case study on bronchiolitis using 33,176 discharge letters of children admitted to 44 emergency rooms or hospitals in the Veneto Region, Italy, between 2017 and 2020. The best weakly supervised model achieved an AUROC of 77.68% (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pm 4.30\%$$\end{document}), an AUPRC of 73.13% (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pm 4.93\%$$\end{document}), and an F1-score of 78.14% (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pm 4.89\%$$\end{document}) against manually annotated data. Performance surpassed unsupervised baselines and approached fully supervised models, while reducing the need for manual annotation by more than 1,500 hours for a dataset of this size. Similar model rankings were observed in a secondary validation on a smaller bronchitis dataset (3,188 discharge letters, 2020-2025), where the best weakly supervised model achieved an AUPRC of 76.72% (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pm 5.02\%$$\end{document}). These results suggest the potential of weakly supervised NLP methods for scalable disease identification from clinical discharge letters. Introduction Electronic Health Records (EHR) contain a vast and continuously growing amount of patient information, but a significant portion of this data remains unstructured in formats like clinical notes, reports, discharge letters and referrals. Despite containing valuable information, this unstructured data is often underutilized in the downstream analysis due to their complexity and heterogeneity1,2. While Natural Language Processing (NLP) has made significant advancements in extracting and classifying information from text3,4, applying these techniques to clinical narratives remains challenging. Medical documents are characterized by a highly specialized lexicon, with ambiguous abbreviations and frequent misspellings, a lack of standard templates across hospitals and a limited amount of manually labelled data5–7. Additionally, most NLP tools are optimized for high-resource languages such as English, limiting their transferability to languages like Italian, where datasets and domain-specific linguistic resources are scarce8. Among the various types of clinical documents, hospital discharge letters play a particularly important role: they summarize the patient’s condition, diagnoses, and treatments, representing a key source of diagnostic information for both clinical follow-up and population-based studies. Although international standards such as the ICD-CM9 exist for encoding diagnoses, physicians often prefer to provide textual descriptions of their diagnoses because they are faster to write, more expressive, and often the only option available in document templates10,11. Even when ICD-CM codes are included, previous research has shown that they can be incomplete or inaccurate12–14. Automatic methods capable of identifying diagnoses directly from unstructured discharge letters are therefore crucial for cohort selection and epidemiological research15, reducing the need for manual review, which is labor-intensive and impractical at scale. In Italy, every hospital or emergency room (ER) admission generates a discharge letter that should contain at least one diagnosis describing the reason for admission or discharge. However, these letters vary widely in format and terminology, often mixing narrative and structured components. Developing robust NLP systems for such data is challenging: traditional approaches rely on supervised classification models that require a substantial amount of manually annotated data16, an effort that is costly and time-consuming for clinicians. This highlights a significant gap in the field: the need for methods that can effectively exploit large, unannotated datasets to extract clinically meaningful information without the burden of manual labelling. Previous studies applying NLP to Italian clinical texts have not addressed discharge letters. Lanera et al.17,18 worked on supervised models to identify varicella cases from notes of family pediatricians, while Sciannameo et al.19 used a BERT-based model to predict rehospitalizations for elderly patients from medication prescriptions and hospitalization records. Hammami et al.20 classified pathology reports for cancer morphology using a rule-based approach, and Viani et al.21,22 extracted entities from cardiology reports through ontologies and recurrent neural networks. Most international work on discharge letters has been conducted in English using resources such as the MIMIC database23,24 (see25–28 for some examples), but research has also been conducted in other European languages with limited resources, such as Bulgarian29,30, Dutch31,32, Czech33 and Swedish34. These studies, however, all relied on large annotated datasets, making them difficult to reproduce in settings where manual labelling is infeasible. Recent work has explored weakly supervised approaches to mitigate the need for large annotated clinical corpora. Many of these methods rely on task-specific heuristics, such as disease-related keywords, regular expressions, or the transfer of labels from structured data like diagnostic codes or medication records35–40. In parallel, the EHR phenotyping literature has proposed frameworks that combine surrogate signals extracted from structured and textual data to infer patient-level disease status, including approaches such as PheNorm41, sureLDA42, Anchor43, Polar44 and XPRESS45. These methods typically aggregate information across multiple encounters and rely on structured EHR variables (e.g., diagnosis codes, medications, laboratory tests or concept counts extracted from notes) to construct surrogate features for phenotyping. While effective in large longitudinal EHR datasets, such assumptions are not always satisfied in practice. In particular, structured clinical variables may be incomplete or unavailable, especially in emergency department admissions that do not lead to hospitalization, where discharge summaries often constitute the primary source of diagnostic information. In these settings, the task becomes one of document-level diagnosis identification directly from free-text clinical narratives, which remains comparatively less explored. More recently, large language models (LLMs) have also been used to support weak supervision pipelines46; however, in clinical settings, effective deployment often still relies on the integration of domain-specific knowledge47 or the use of proprietary models48. To address these limitations, we propose a weakly supervised NLP pipeline for automatically identifying patient diagnoses directly from Italian hospital discharge letters, operating solely on free-text clinical documents without requiring structured EHR variables or patient-level aggregation. Our approach eliminates the dependency on document-level manual annotations and offers a more generalizable solution than previous rule-based methods. The pipeline first identifies diagnosis-related sentences from a subset of documents and represents them through contextual embeddings generated by a transformer model, previously pre-trained with public data in Italian related to the medical domain. These embeddings are then grouped through a two-level clustering procedure, after which the resulting clusters are semantically summarized and mapped to diseases of interest to produce weak labels. The weakly labelled data are finally used to train a transformer-based classifier capable of recognizing those diseases directly from discharge letters. This design enables large-scale, disease-agnostic classification of clinical narratives while avoiding the costs of large-scale document-level manual labelling. We evaluate the proposed framework through a real-world case study focused on bronchiolitis, one of the leading causes of hospitalizations (20 %), emergency department visits (18 %) and office visits (15 %) among young children and a major driver of seasonal pediatric respiratory epidemics49. The dataset includes 33,176 discharge letters of 15,196 pediatric patients admitted to 44 emergency rooms or hospitals across the Veneto Region (Italy) between January 2017 and December 2020, collected through the Pedianet network50. To the best of our knowledge, there are no prior studies addressing the identification of bronchiolitis directly from medical text, whereas existing works have relied on structured data for diagnosis prediction51–53. To assess the robustness and generalizability of the proposed pipeline, we additionally conduct a secondary validation study on bronchitis using another smaller dataset from the same Pedianet network. This dataset includes 3,188 discharge letters collected between January 2020 and May 2025, involving 2,364 children living in Veneto Region, accessing 31 different hospitals. Although substantially smaller than the bronchiolitis dataset, this additional cohort provides an independent validation setting to assess whether the proposed weakly supervised framework can generalize to a different respiratory condition and data distribution. In our evaluation, we test the weakly supervised pipeline against manually labelled data, considering different classification backbones and comparing its performance with both fully unsupervised and fully supervised approaches. The proposed method achieves competitive performance, outperforming other unsupervised techniques and showing only a limited gap compared with supervised classifiers. Its performance is robust to weak label selection, and the analysis of the intermediate steps indicates a strong potential for generalization across diseases. Methods Fig. 1(a) Diagram of the full pipeline for weakly supervised classification of discharge letters. Step 0 (TPT) consists in additional pre-training of a transformer-based model on a dataset of medical documents in Italian, including documents automatically translated into Italian. Step 1 (SAL) is the core of the pipeline, consisting in the automatic extraction of diagnosis strings from the letters, their clustering (exploiting the model from TPT) and the mapping of the clusters to the disease labels. This procedure leads to a weakly labelled subset of data that is used in Step 2 (WLC) to train a weakly supervised classification model. (b) Detailed diagram of the clustering step for diagnosis strings, corresponding to Step 1.2 of the full pipeline. The extracted diagnoses are embedded with a transformer-based model, and, after a dimensionality reduction step, clustered. Clusters are summarized with relevant keywords and the clustering is repeated on these clusters’ representations. (a) Diagram of the full pipeline for weakly supervised classification of discharge letters. Step 0 (TPT) consists in additional pre-training of a transformer-based model on a dataset of medical documents in Italian, including documents automatically translated into Italian. Step 1 (SAL) is the core of the pipeline, consisting in the automatic extraction of diagnosis strings from the letters, their clustering (exploiting the model from TPT) and the mapping of the clusters to the disease labels. This procedure leads to a weakly labelled subset of data that is used in Step 2 (WLC) to train a weakly supervised classification model. (b) Detailed diagram of the clustering step for diagnosis strings, corresponding to Step 1.2 of the full pipeline. The extracted diagnoses are embedded with a transformer-based model, and, after a dimensionality reduction step, clustered. Clusters are summarized with relevant keywords and the clustering is repeated on these clusters’ representations. In this section, we describe in detail the pipeline we propose for the weakly supervised diagnosis identification. The pipeline, depicted in Fig. 1, can be divided into three main steps: 0TPT - Pre-Training of a Transformer-based model for Italian medical documents.1SAL - Semi-automatic labelling of the dataset.2WLC - Classification of discharge letters using the weakly labelled dataset.The next sections describe in detail the three steps. A summary of the main notation used throughout these sections is reported in Supplementary Material B. 0TPT - Pre-Training of a Transformer-based model for Italian medical documents. TPT - Pre-Training of a Transformer-based model for Italian medical documents. 1SAL - Semi-automatic labelling of the dataset. SAL - Semi-automatic labelling of the dataset. 2WLC - Classification of discharge letters using the weakly labelled dataset. WLC - Classification of discharge letters using the weakly labelled dataset. Pre-training of a Transformer-based model for Italian medical documents (TPT) Statistical and machine learning models for textual data require a mathematical representation of text. Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathscr {X}$$\end{document} denote the space of textual documents. A text encoder can be formalized as a mapping \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{\theta }: \mathscr {X} \rightarrow \mathbb {R}^d$$\end{document}, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_\theta$$\end{document} is a parametric function with parameters θ and d is the embedding dimension. Classical vocabulary-based representations (e.g., bag-of-words or TF–IDF) produce sparse vectors in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathbb {R}^{|{V}|}$$\end{document}, where V denotes the corpus vocabulary. While effective in certain settings, these representations do not explicitly encode semantic similarity between distinct lexical forms. Neural network-based embeddings address this limitation. Static word embeddings, such as Word2Vec54, define a fixed mapping \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$w \mapsto v_w \in \mathbb {R}^d$$\end{document} for each token w, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$v_w$$\end{document} is a dense, continuous vector, used for every occurrence of w, and d ≪ |V|. In contrast, contextual embeddings compute representations that depend on the full input sequence. Transformer-based models55, including BERT56 and RoBERTa57, define contextual encoders where the embedding of each token depends also on the surrounding tokens, capturing different meanings that can depend on the context. Several transformer models have been specialized for the biomedical domain in English, such as BioBERT58 and ClinicalBERT59. For Italian, however, most available transformer models are trained primarily on general-domain corpora60, with limited exposure to biomedical or clinical terminology. To improve domain adaptation, we perform additional pre-training on the European Clinical Case Corpus (E3C)61, a multilingual medical dataset containing documents from journal articles, specialty-school admission tests, patient information leaflets, and medical-science thesis abstracts (a summary of its composition is provided in Supplementary Material A.3). Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathscr {C} = \bigcup _{\ell \in \mathscr {L}} \mathscr {C}_\ell$$\end{document} denote the full E3C corpus, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathscr {L}$$\end{document} is the set of languages represented in the dataset and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathscr {C}_\ell$$\end{document} denotes the subset of documents in language ℓ. Let ℓ = it denote Italian. Since the corpus contains documents in multiple languages, we translate all non-Italian subsets into Italian using the Google Translate API. Automatic translation may introduce lexical or syntactic noise. However, the translated documents are used only for masked language modeling during domain-adaptive pre-training. As a result, occasional translation errors are unlikely to systematically bias the downstream task, while the enlarged corpus helps expose the model to a broader range of medical terminology. Formally, for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell \ne it$$\end{document}, we define \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_\ell : \mathscr {C}_\ell \rightarrow \mathscr {C}_{it}$$\end{document} (Step 0.1 in Fig. 1a). The resulting Italian medical corpus used for domain-adaptive pre-training is1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \mathscr {C}^{it} = \mathscr {C}_{it} \;\cup \; \bigcup _{\ell \in \mathscr {L} \setminus \{it\}} T_\ell (\mathscr {C}_\ell ). \end{aligned}$$\end{document}Starting from a pre-trained transformer encoder \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{\theta _0}$$\end{document}, we perform additional masked language modeling (MLM) pre-training on \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathscr {C}^{it}$$\end{document}. The objective function is2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \mathscr {L}_{\text {MLM}}(\theta ) = - \mathbb {E}_{(c,t)} \log P_\theta (w_t \mid c_{\setminus t}), \end{aligned}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$w_t$$\end{document} denotes a masked token in document c, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$c_{\setminus t}$$\end{document} denotes the surrounding context. The optimized parameters are obtained as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{\theta } = \arg \min _{\theta } \mathscr {L}_{\text {MLM}}(\theta )$$\end{document} (Step 0.2 in Fig. 1a), resulting in the domain-adapted encoder \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{\hat{\theta }}: \mathscr {X} \rightarrow \mathbb {R}^{d}$$\end{document} where d = 768 for the BERT-base encoders that we use. This encoder is subsequently used both in the semi-automatic labelling (SAL, Step 1) and in the weakly supervised classification (WLC, Step 2), which are described in the following sections. Semi-automatic labelling pipeline (SAL) Developing a weakly supervised classification pipeline for discharge letters requires a procedure to derive weak labels for at least a subset of data. Weak labels are labels that can be automatically associated to the data but that might contain some noise. This section presents a methodology to construct these weak labels by extracting diagnosis strings from discharge letters and clustering them. This allows the creation of groups of diagnoses related to the same disease or symptoms that can be exploited as labels. This step corresponds to Step 1 (SAL) of Fig. 1a. Our approach combines regular expressions for extracting diagnosis strings and a customized clustering pipeline based on a transformer model. In the following sections, we describe in detail these sub-steps. Extraction of diagnosis strings The first step (1.1 in Fig. 1a) consists of extracting diagnosis strings from discharge letters. Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathscr {D} = \{x_i\}_{i=1}^N$$\end{document} denote the set of discharge letters, where each \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$x_i \in \mathscr {X}$$\end{document} is a textual document. Italian discharge letters do not follow a standardized template, as each hospital has its own, and even different departments within the same hospital may personalize it further. Nevertheless, we have observed that these templates often include a designated section for diagnoses, where physicians report the conditions related to the current hospitalization. This section may contain one or multiple diagnosis entries (e.g., primary and secondary diagnoses referring to the present admission), which typically contain the information required to determine the document label. We formalize the extraction step as a deterministic function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$g : \mathscr {X} \rightarrow \mathscr {S} \cup \{\emptyset \},$$\end{document} where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathscr {S}$$\end{document} denotes the space of diagnosis strings. For each document \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$x_i$$\end{document}, the output is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_i = g(x_i),$$\end{document} where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_i \in \mathscr {S}$$\end{document} if a diagnosis section is detected and \emptyset otherwise. Three main challenges arise in handling diagnosis sections: Not all discharge summaries include this indication, as it may be absent or left empty.Different templates use varying terminologies and position this indication differently in the text.Diagnosis strings are typically written as free text, with the same disease described by different strings and levels of detailThe first point is the primary reason why our pipeline focuses on classifying the text of a discharge letter rather than relying solely on the extracted diagnosis strings. To address the second point, the function g is implemented using a predefined set of regular expressions that identify and extract the diagnosis section across heterogeneous templates. Regular expressions provide a flexible and reproducible pattern-matching mechanism for detecting structured fields with limited syntactic variation. The full set of patterns is reported in Supplementary Material D. Not all discharge summaries include this indication, as it may be absent or left empty. Not all discharge summaries include this indication, as it may be absent or left empty. Different templates use varying terminologies and position this indication differently in the text. Different templates use varying terminologies and position this indication differently in the text. Diagnosis strings are typically written as free text, with the same disease described by different strings and levels of detail Diagnosis strings are typically written as free text, with the same disease described by different strings and levels of detail After this step, for the majority of the letters we obtain a free-text diagnosis string describing the conditions associated with the current hospitalization. This extraction step focuses on diagnoses explicitly reported in the structured diagnosis section and does not aim to capture conditions mentioned exclusively in other narrative sections (e.g., past medical history). These strings are not directly used as labels due to their heterogeneity (point 3 above); instead, they are clustered, as detailed in the following section. Diagnosis clustering Once the diagnosis strings are extracted, we cluster them to identify semantically coherent groups corresponding to diseases or symptoms . This corresponds to Step 1.2 in panel a of Fig. 1 and is expanded in panel b of the same figure. Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${I} = \{ i \in \{1,\dots ,N\}: s_i \ne \emptyset \}$$\end{document} denote the subset of documents for which a diagnosis string was successfully extracted. In this clustering step, we consider only the set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{ s_i \}_{i \in {I}}$$\end{document}, ignoring documents with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_i = \emptyset$$\end{document}. Each diagnosis string undergoes minor normalization, including removal of punctuation, dates, and time expressions, through a deterministic pre-processing function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_s: \mathscr {S} \rightarrow \mathscr {S}$$\end{document}. The normalized string is denoted \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\bar{s}_i = p_s(s_i)$$\end{document} (Step 1.2.1). Each normalized diagnosis string is embedded using the domain-adapted transformer encoder \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{\hat{\theta }}$$\end{document} defined in the section “Pre-training of a Transformer-based model for Italian medical documents (TPT)” . The resulting embedding is3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} z_i = f_{\hat{\theta }}(\bar{s}_i)~\in ~\mathbb {R}^{d},~\quad i~\in ~{I}. \end{aligned}$$\end{document}Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Z = \{ z_i \}_{i \in {I}}$$\end{document} denote the set of all diagnosis embeddings (Step 1.2.2). Since high-dimensional embeddings are not directly suitable for density-based clustering, we apply Principal Component Analysis (PCA) to Z (Step 1.2.3). Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W \in \mathbb {R}^{d \times k}$$\end{document} denote the projection matrix obtained by fitting PCA on Z, where in our case studies d=768, k=100, retaining more than 80% of the variance in the embedding space. The projected embeddings are \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tilde{z}_i = W^\top z_i \in \mathbb {R}^{k}.$$\end{document} After this step, a clustering algorithm can be applied to the resulting embedding vectors (Step 1.2.4). We selected HDBSCAN62 because it does not require specifying the number of clusters a priori, allows leaving part of the data unclustered, and can identify clusters with heterogeneous shapes and densities. These properties are well suited to transformer-based text embeddings, which often lie on complex, nonlinear manifolds and may form clusters with varying densities, irregular geometries, and overlapping boundaries. Several commonly used clustering algorithms, such as K-Means, Gaussian Mixture Models, and standard hierarchical clustering, are less suitable for this type of data63,64, as further supported by a comparative analysis reported in Supplementary Material E. We apply HDBSCAN to the set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{ \tilde{z}_i \}_{i \in {I}}$$\end{document}. The clustering defines an assignment function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$h : \mathbb {R}^{k} \rightarrow \{1,\dots ,K\} \cup \{-1\},$$\end{document} where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$h(\tilde{z}_i) = k$$\end{document} indicates membership in cluster \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_k$$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$h(\tilde{z}_i) = -1$$\end{document} denotes noise points left unclustered. HDBSCAN is parameterized by a minimum cluster size and a minimum number of samples, selected via grid search as detailed in Supplementary Material E. The first-level clustering may produce multiple compact clusters corresponding to the same disease category. To merge semantically related clusters, we construct a keyword-based representation for each cluster. For each cluster \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_k$$\end{document}, we define a set of representative keywords \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Keywords}(C_k)$$\end{document} obtained by applying frequency-based filtering to the tokens appearing in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_k$$\end{document} after tokenization and stopword removal. These keywords provide a compact textual summary of the cluster. The complete keyword extraction procedure is described in Supplementary Material E. Each cluster is then represented by embedding its keyword summary using the same encoder \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{\hat{\theta }}$$\end{document}, yielding a cluster-level embedding \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$u_k \in \mathbb {R}^{d}$$\end{document}. A second-level clustering procedure is applied to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{u_k\}$$\end{document} to merge clusters that are semantically close (Step 1.2.6). Mapping clusters to diseases Step 1.3 consists of mapping clusters to weak labels that will be used to train the classification model. Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{C_k\}_{k=1}^K$$\end{document} denote the clusters obtained in Step 1.2 and let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Keywords}(C_k)$$\end{document} be the keyword set associated with cluster \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_k$$\end{document}. For each disease of interest, we specify one or more definitions. Each definition d consists of:a set of positive keywords \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_d = \{p_1, \dots , p_r\}$$\end{document} that must be present,optionally, a set of negative keywords \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_d = \{n_1, \dots , n_s\}$$\end{document} that must be absent.A cluster \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_k$$\end{document} is selected as positive for the disease if there exists at least one definition d such that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_d \subseteq \textrm{Keywords}(C_k) \quad \text {and} \quad N_d \cap \textrm{Keywords}(C_k) = \emptyset .$$\end{document} a set of positive keywords \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_d = \{p_1, \dots , p_r\}$$\end{document} that must be present, a set of positive keywords \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_d = \{p_1, \dots , p_r\}$$\end{document} that must be present, optionally, a set of negative keywords \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_d = \{n_1, \dots , n_s\}$$\end{document} that must be absent. optionally, a set of negative keywords \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_d = \{n_1, \dots , n_s\}$$\end{document} that must be absent. Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$L = \{ k \in \{1,\dots ,K\} : C_k \text { satisfies at least one } d \}$$\end{document} denote the set of clusters selected for the disease, h(i) the cluster assignment of diagnosis string \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_i$$\end{document} (Step 1.2), and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I_c = \{ i \in I : h(i) \ne -1 \}$$\end{document} the subset of documents whose diagnosis strings were assigned to a cluster (i.e., excluding noise points identified by HDBSCAN). The weak label is defined as4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \tilde{y}_i = {\left\{ \begin{array}{ll} 1 & \text {if } h(i) \in L, \\ 0 & \text {if } h(i) \notin L, \end{array}\right. } \quad i \in I_c . \end{aligned}$$\end{document}Documents with h(i) = -1 (i.e., diagnosis strings left unclustered) or i ∉ I (no diagnosis string extracted) do not receive a weak label and are excluded from the weakly labelled training subset. Thus, among the documents in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I_c$$\end{document}, discharge letters whose extracted diagnosis strings belong to selected clusters are assigned a positive weak label, while the remaining clustered documents receive a negative weak label. This procedure is fully deterministic once the disease definitions are specified and does not depend on gold annotations. The only domain input required is the specification of disease-related keywords, which is substantially less demanding than document-level manual annotation. In practice, keyword definitions are specified in collaboration with clinicians, with attention to the terminology typically used in discharge letters for the disease of interest. This step does not require document-level review and is performed independently of gold labels. While the coverage of weak labels depends on the completeness of the keyword definitions, the cluster-level matching strategy reduces sensitivity to minor lexical variations. A keyword-based selection at the cluster level is more robust than applying keywords directly to the raw diagnosis strings extracted in Step 1.1. Individual diagnosis strings are often heterogeneous, contain variable phrasing, and may include negations or references to past medical history. Direct keyword matching at the string level would therefore introduce a higher rate of false positives and false negatives. By contrast, clustering first aggregates semantically similar diagnoses, allowing keyword matching to operate on stabilized and semantically coherent cluster representations. Similarly, manually reviewing all extracted diagnosis strings would be impractical due to their large number. Classification of discharge letters using the weakly labelled dataset (WLC) The classification pipeline that exploits the weakly labelled data produced by Step 1 corresponds to Step 2 (WLC) in Fig. 1.a. Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tilde{\mathscr {D}} = \{(x_i, \tilde{y}_i): i \in {I_c}\}$$\end{document} denote the weakly labelled dataset constructed in Step 1.3. Only documents with an extracted diagnosis string are included in this training set. Each document undergoes a deterministic cleaning function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p: \mathscr {X} \rightarrow \mathscr {X}$$\end{document} that removes headers and footers containing non-informative administrative content (see Supplementary Material C for details). The processed document is denoted \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\bar{x}_i = p(x_i)$$\end{document} (Step 2.1). Since RoBERTa-based models are limited to 512 input tokens, we define a truncation operator \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau _{512}(\cdot )$$\end{document} that retains the first 512 tokens of the processed document. The final input to the classifier is therefore \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$x_i^{*} = \tau _{512}(\bar{x}_i).$$\end{document} In our datasets, approximately 90% of discharge letters contain fewer than 512 tokens after preprocessing, so truncation rarely affects the input. For datasets with a larger proportion of long documents, alternative strategies could be considered, such as splitting the document into segments of at most 512 tokens and aggregating segment-level probabilities, or using architectures designed for longer contexts, as discussed in studies on long-document classification with BERT-based models65. The classification model is based on the domain-adapted transformer encoder \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_{\hat{\theta }}$$\end{document} defined in the section “Pre-training of a Transformer-based model for Italian medical documents (TPT)”. A linear classification layer with parameters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$w \in \mathbb {R}^{d}$$\end{document} and bias \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$b \in \mathbb {R}$$\end{document} is added on top of the encoder. For an input document \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$x_i^{*}$$\end{document}, the predicted probability of the disease is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{p}_i = \sigma \!\left( w^\top f_{\hat{\theta }}(x_i^{*}) + b \right) ,$$\end{document} where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma (z) = 1/(1+e^{-z})$$\end{document} is the logistic function. Model parameters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\phi = (\hat{\theta }, w, b)$$\end{document} are optimized by minimizing the binary cross-entropy loss over the weakly labelled dataset:5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \mathscr {L}_{\text {WLC}}(\phi ) = - \sum _{i \in {I_c}} \left[ \tilde{y}_i \log \hat{p}_i + (1-\tilde{y}_i) \log (1-\hat{p}_i) \right] . \end{aligned}$$\end{document} Experimental outline The study uses anonymized discharge letters obtained from the Pedianet network50. Inclusion in the Pedianet database is voluntary and parents/legal guardians must provide informed consent for their children’s anonymized data to be used for research purposes. The study was performed in accordance with the Declaration of Helsinki. Ethical approval of the study and access to the database were granted by the Internal Scientific Committee of So.Se.Te. Srl, the legal owner of Pedianet. The main evaluation is conducted at the final classification stage of the pipeline, where predictions are compared against manually annotated gold labels \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y_i^{\text {gold}}$$\end{document}. In addition, intermediate components of the pipeline–diagnosis extraction (Step 1.1) and clustering (Steps 1.2–1.3)–are evaluated on a randomly selected subset of 1,000 diagnosis strings for each dataset, manually annotated by domain experts. Each string was labelled as correct, partially correct, or wrong with respect to (i) completeness of the extracted diagnosis string and (ii) correctness of its assignment to first- and second-level clusters. At the classification stage, we compare our pipeline with:Two unsupervised rule-based baselines:Rule-based full-text: keyword matching applied to the entire discharge letter;Rule-based diagnosis: keyword matching applied only to extracted diagnosis strings. The keywords correspond to the disease definitions introduced in Step 1.3.An XPRESS-style weak supervision baseline (WS-XPRESS-UmBERTo-TPT). Following the XPRESS paradigm45, we generate weak labels directly applying the keyword definitions defined in Step 1.3 (positive and negative sets) to the discharge letters and train the UmBERTo-TPT model used in our pipeline on these weak labels.A zero-shot large language model (LLM) for Italian (LLaMAntino-3-ANITA-8B-Inst-DPO-ITA)66, prompted to classify each discharge letter as positive or negative for bronchiolitis.To assess the contribution of the transformer backbone and domain-adaptive pre-training (TPT), we replace the classifier in Step 2.2 with: UmBERTo without TPT (WS-UmBERTo)60;XLM-RoBERTa-base67, with (WS-XLM-RoBERTa-TPT) and without (WS-XLM-RoBERTa) TPT;RemBERT68, with (WS-RemBERT-TPT) and without (WS-RemBERT-BPT) TPT;An LSTM-based recurrent neural network using Italian Word2Vec embeddings (WS-LSTM)54.The three transformer-based models with TPT are also evaluated in a fully supervised setting (FS-UmBERTo-TPT, FS-XLM-RoBERTa-TPT, FS-RemBERT-TPT) trained directly on gold labels, providing an upper bound on performance under full supervision. Two unsupervised rule-based baselines:Rule-based full-text: keyword matching applied to the entire discharge letter;Rule-based diagnosis: keyword matching applied only to extracted diagnosis strings. The keywords correspond to the disease definitions introduced in Step 1.3. Two unsupervised rule-based baselines:Rule-based full-text: keyword matching applied to the entire discharge letter;Rule-based diagnosis: keyword matching applied only to extracted diagnosis strings. The keywords correspond to the disease definitions introduced in Step 1.3. Rule-based full-text: keyword matching applied to the entire discharge letter; Rule-based full-text: keyword matching applied to the entire discharge letter; Rule-based diagnosis: keyword matching applied only to extracted diagnosis strings. Rule-based diagnosis: keyword matching applied only to extracted diagnosis strings. An XPRESS-style weak supervision baseline (WS-XPRESS-UmBERTo-TPT). Following the XPRESS paradigm45, we generate weak labels directly applying the keyword definitions defined in Step 1.3 (positive and negative sets) to the discharge letters and train the UmBERTo-TPT model used in our pipeline on these weak labels. An XPRESS-style weak supervision baseline (WS-XPRESS-UmBERTo-TPT). Following the XPRESS paradigm45, we generate weak labels directly applying the keyword definitions defined in Step 1.3 (positive and negative sets) to the discharge letters and train the UmBERTo-TPT model used in our pipeline on these weak labels. A zero-shot large language model (LLM) for Italian (LLaMAntino-3-ANITA-8B-Inst-DPO-ITA)66, prompted to classify each discharge letter as positive or negative for bronchiolitis. A zero-shot large language model (LLM) for Italian (LLaMAntino-3-ANITA-8B-Inst-DPO-ITA)66, prompted to classify each discharge letter as positive or negative for bronchiolitis. UmBERTo without TPT (WS-UmBERTo)60; UmBERTo without TPT (WS-UmBERTo)60; XLM-RoBERTa-base67, with (WS-XLM-RoBERTa-TPT) and without (WS-XLM-RoBERTa) TPT; XLM-RoBERTa-base67, with (WS-XLM-RoBERTa-TPT) and without (WS-XLM-RoBERTa) TPT; RemBERT68, with (WS-RemBERT-TPT) and without (WS-RemBERT-BPT) TPT; RemBERT68, with (WS-RemBERT-TPT) and without (WS-RemBERT-BPT) TPT; An LSTM-based recurrent neural network using Italian Word2Vec embeddings (WS-LSTM)54. An LSTM-based recurrent neural network using Italian Word2Vec embeddings (WS-LSTM)54. Since weak labels are derived from clustered diagnosis strings, classification models may over-rely on diagnosis sections. To assess this potential bias, we evaluate models under two conditions: Using the full discharge letter;Removing the diagnosis strings used to construct weak labels (when present).To better understand the effect of weak supervision, we perform additional analyses. First, we assess the robustness of the results with respect to the construction of weak labels by removing individual clusters from the set of positive weak labels and re-evaluating the models. Second, we analyze the impact of label noise by replacing varying proportions of gold labels with weak labels and measuring the resulting performance degradation. Finally, we evaluate classification performance against the weak labels themselves, allowing us to quantify the discrepancy between weak and gold supervision (in Supplementary Material F). Using the full discharge letter; Using the full discharge letter; Removing the diagnosis strings used to construct weak labels (when present). Removing the diagnosis strings used to construct weak labels (when present). Classification performance is evaluated using precision (P), recall (R), F1-score (F1) for the positive class, area under the ROC curve (AUROC), and area under the precision–recall curve (AUPRC). For classifiers producing a continuous score, the classification threshold is selected within training folds using Youden’s index69, and all evaluation metrics are computed on held-out test folds reflecting the original class prevalence. All metrics are computed using stratified cross-validation. For the bronchiolitis dataset, we use 10-fold cross-validation. For the smaller bronchitis validation dataset, which contains 97 positive cases, we use 5-fold cross-validation to ensure a sufficient number of positive examples in each test fold. For each metric, we report the mean and standard deviation across folds. To assess statistical significance of performance differences, we perform paired permutation (sign-flip) tests on fold-wise differences between our method and each other model. All tests are two-sided. To control for multiple comparisons across model variants, p-values are adjusted using the Holm–Bonferroni procedure within each metric. Differences are considered statistically significant when the Holm-adjusted p-value is below 0.05. AUROC and AUPRC are not reported for rule-based classifiers, as they do not produce probabilistic outputs. Fully supervised models are not evaluated on weak labels. Since data originate from multiple hospitals and Local Health Units (LHUs), additional experiments assessing inter-site variability are reported in Supplementary Material F. Training details Additional pre-training of the transformer-based models on the E3C corpus was performed using the masked language modeling objective described in the section “Pre-training of a Transformer-based model for Italian medical documents (TPT)”. The models were optimized using a learning rate of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2 \cdot 10^{-5}$$\end{document} and weight decay of 0.01. A validation set consisting of 20% of the corpus was used for early stopping. The LSTM baseline consists of a frozen Word2Vec embedding layer followed by a 32-unit LSTM layer, a dropout layer, a 16-unit fully connected layer, an additional dropout layer, and the final output layer. The model is trained using the Adam optimizer with a learning rate scheduler starting from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$10^{-2}$$\end{document}, decay rate 0.7, and decay step size of 100. Early stopping is applied based on validation performance. The transformer-based classifiers are trained using the AdamW optimizer with a linear learning rate scheduler starting from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$10^{-3}$$\end{document} and weight decay of 0.01. These hyperparameters were selected on training folds of the bronchiolitis dataset using grid search. The same configuration was applied unchanged to the bronchitis dataset. Training proceeds for a number of epochs determined by early stopping (6 epochs for UmBERTo models, 4 for XLM-RoBERTa models, and 3 for RemBERT models). To stabilize optimization, a gradual unfreezing strategy is applied. Initially, only the final classification layer is trained. After two epochs, the last transformer layer is unfrozen, and after four epochs (if reached), the last but one transformer layer is also unfrozen. To mitigate class imbalance during training, negative examples are undersampled within each training fold, limiting their number to at most twice that of the positive class. No undersampling is applied to test folds. Additional results supporting this choice are reported in Supplementary Material F. All experiments were implemented in Python. Training was performed on a single NVIDIA Tesla V100 GPU with 32 GB of VRAM. Results Datasets The dataset for the main case study comprises 33,176 discharge letters related to 15,196 children who were admitted to emergency rooms and/or hospitalized in the Veneto Region between January 2017 and December 2020, with a family pediatrician adhering to the Pedianet network50. The dataset covers 44 different hospitals in the region, which can be grouped into 9 Local Health Units (LHUs) and 2 university hospitals. Each discharge letter includes the complete text of the letter, along with the date of the letter and an anonymized identifier of the patient. A manual review of discharge letters was conducted to extract relevant diagnoses of bronchiolitis, resulting in 335 positive cases within this dataset. This review was performed by a data manager with a clinical background, with the support of clinicians for uncertain cases. These labels are the gold standard to validate our pipeline, and we do not make direct use of them in the pipeline training (except for the fully supervised experiments). The median length of these letters is 4,515 characters (1st–3rd quartiles 3,240–5,806). As an additional validation setting, we analyze a smaller dataset of 3,188 discharge letters related to 2,364 children who were admitted to emergency rooms and/or hospitalized in the Veneto Region between January 2020 and May 2025, with a family pediatrician adhering to the Pedianet network. This dataset covers 31 hospitals and 11 LHUs. Manual review, following the same approach of the bronchiolitis dataset, identified 97 cases of bronchitis, which serve as gold labels for evaluation. The median length of these letters is 5,387 characters (1st-3rd quartiles 2,601-6,759). Supplementary Material A.1 reports additional plots with data distributions among hospitals and LHUs for the two datasets. An example of a fictitious letter is reported in Supplementary Material A.2. A secondary annotator independently reviewed a stratified sample of 30 positive and 30 negative discharge letters for each dataset, resulting in perfect inter-annotator agreement for bronchiolitis (κ =1.00) and substantial agreement for bronchitis (κ =0.73), with discrepancies primarily involving less certain cases. Diagnosis extraction Our diagnosis extraction module (Step 1.1) successfully extracted a diagnosis sentence from 89% of the discharge letters in the bronchiolitis dataset. We extracted 15,038 unique diagnosis sentences, of which 11,279 occurred only once, confirming the need for clustering them in wider groups that can be more easily related to known diseases and symptoms. ICD-9-CM codes are present only in 737 diagnosis strings (4.9%), covering 3,436 discharge letters (10.4%). The most common strings are the simplest, while more articulate ones have single occurrences. The median length of the extracted diagnosis strings is 49 characters, with first and third quartiles of 17 and 156 characters. Among the 335 bronchiolitis-labelled letters, we extracted 138 unique diagnosis strings from 306 letters. The same extraction procedure was applied to the bronchitis validation dataset. Diagnosis strings were successfully extracted from 90% of the discharge letters, yielding 1,376 unique diagnosis strings, of which 1,092 occurred only once. Considering the 97 bronchitis-labelled letters, diagnosis strings were extracted from all of them, producing 88 unique diagnosis expressions. The manual evaluation results of this step on a random sample of 1,000 strings for each dataset are summarized in Fig. 2a. The number of extracted strings that are not diagnosis strings or that are incomplete is very limited (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\le 2\%$$\end{document}).Fig. 2Evaluation of the intermediate steps of the pipeline on the bronchiolitis and bronchitis datasets. a. Evaluation of diagnosis string extraction on 1,000 manually annotated diagnosis strings per dataset, reporting the percentage of extracted strings classified as correct, partially correct, or wrong. b. Evaluation of diagnosis clustering after first-level and second-level clustering on the same set of annotated diagnosis strings. c. Validation of weak labels against gold labels in terms of precision, recall, and F1-score, reported separately for labels obtained from first-level and second-level clustering. Evaluation of the intermediate steps of the pipeline on the bronchiolitis and bronchitis datasets. a. Evaluation of diagnosis string extraction on 1,000 manually annotated diagnosis strings per dataset, reporting the percentage of extracted strings classified as correct, partially correct, or wrong. b. Evaluation of diagnosis clustering after first-level and second-level clustering on the same set of annotated diagnosis strings. c. Validation of weak labels against gold labels in terms of precision, recall, and F1-score, reported separately for labels obtained from first-level and second-level clustering. Diagnosis clustering In the bronchiolitis dataset, the 15,038 extracted diagnosis strings were grouped into 590 clusters. Cluster sizes varied significantly, from a large cluster of 270 strings to several smaller clusters containing 5 strings. The subsequent second-level clustering reduced the number of clusters to 135. In the bronchitis dataset clusters were 77 and 37, respectively. A summary of the main clusters is reported in Supplementary Material E. The quality of the clustering assignments was evaluated on a random sample of 1,000 diagnosis strings per dataset. Figure 2b reports the proportion of assignments that were manually classified as correct, partially correct, or wrong at both clustering levels. Most of the partially correct cluster assignments are due to minor variations in disease specification, e.g., fracture of right hand assigned to a cluster for fracture of left hand, severe influenza assigned to a cluster for mild influenza. A smaller number of cases correspond to diagnoses mentioning multiple symptoms or diseases that only partially match the cluster definition, such as vomiting and fever assigned to a cluster describing fever alone.Table 1Keyword-based representation of the first 15 first-level clusters related to respiratory infections in each dataset, with the % of cases in each cluster, the % to the total number of cases in the dataset and the indication of the clusters that would be selected as weak labels at this stage.Cluster #Cluster keywords [ITA]Cluster keywords [ENG]Within clustercases %% of total casescoveredPositiveweak labelBronchiolitis1Bronchiolite lieveMild bronchiolitis85.196.87y2Acuta bronchiolite iniziale lieveAcute mild initial bronchiolitis85.1444.48y3Acuta bronchiolite corso insufficienza respiratoriaAcute bronchiolitis undergoing respiratory failure74.0511.94y4Acuto broncospasmo corso febbre pazienteAcute bronchospasm ongoing fever patient70.0015.02y5Difficolta’ alimentazione bronchiolite Feeding difficulties bronchiolitis50.002.09y6Acuta infezione insufficienza respiratoria transitoria virosiAcute respiratory infection transient virosis12.702.39n7Broncospasmo infezione respiratoriaBronchospasm respiratory infection6.670.30n8Broncospasmo disidratazione lieveBronchospasm mild dehydration1.170.60n9Flogosi alte vie respiratorie febbreHigh airways phlogosis fever0.000.00n10Broncospasmo infezione vie respiratorieBronchospasm airways infection0.000.00n11Acuto broncospasmoAcute bronchospasm0.000.00n12Broncospasmo flogosi vieBronchospasm airways phlogosis0.000.00n13Broncospasmo polmoniteBronchospasm pneumonia0.000.00n14Broncospasmo corsoOngoing bronchospasm0.000.00n15Broncopolmonite sinistraLeft bronchopneumonia0.000.00nBronchitis1Bronchite acutaAcute bronchitis90.4839.18y2Broncospasmo acuto parainfettivoAcute parainfectious bronchospasm22.735.15n3Broncospasmo acutoAcute bronchospasm16.675.15n4BronchiteBronchitis84.6222.68y5Nausea vomito febbreNausea vomit fever5.001.03n6Insufficienza respiratoria infezione basseRespiratory insufficiency lower infection30.006.19y7Polmonite basale destra riniteRight basal pneumonia rhinitis10.003.09n8BroncospasmoBronchospasm9.092.06n9Polmonite sinistra insufficienza respiratoriaLeft pneumonia respiratory insufficiency9.091.03n10Otite media acutaAcute otitis media0.000.00n11Infezione basse alte vie respiratorieRespiratory infection lower upper33.334.12n12Laringite ipoglotticaHypoglottic laryngitis0.000.00n13Bronchiolite lieveMild bronchiolitis4.761.03n14InfezioneInfection3.571.03n15Febbre flogosi alte vieFever upper tract phlogosis3.232.06n Table 1Keyword-based representation of the first 15 first-level clusters related to respiratory infections in each dataset, with the % of cases in each cluster, the % to the total number of cases in the dataset and the indication of the clusters that would be selected as weak labels at this stage.Cluster #Cluster keywords [ITA]Cluster keywords [ENG]Within clustercases %% of total casescoveredPositiveweak labelBronchiolitis1Bronchiolite lieveMild bronchiolitis85.196.87y2Acuta bronchiolite iniziale lieveAcute mild initial bronchiolitis85.1444.48y3Acuta bronchiolite corso insufficienza respiratoriaAcute bronchiolitis undergoing respiratory failure74.0511.94y4Acuto broncospasmo corso febbre pazienteAcute bronchospasm ongoing fever patient70.0015.02y5Difficolta’ alimentazione bronchiolite Feeding difficulties bronchiolitis50.002.09y6Acuta infezione insufficienza respiratoria transitoria virosiAcute respiratory infection transient virosis12.702.39n7Broncospasmo infezione respiratoriaBronchospasm respiratory infection6.670.30n8Broncospasmo disidratazione lieveBronchospasm mild dehydration1.170.60n9Flogosi alte vie respiratorie febbreHigh airways phlogosis fever0.000.00n10Broncospasmo infezione vie respiratorieBronchospasm airways infection0.000.00n11Acuto broncospasmoAcute bronchospasm0.000.00n12Broncospasmo flogosi vieBronchospasm airways phlogosis0.000.00n13Broncospasmo polmoniteBronchospasm pneumonia0.000.00n14Broncospasmo corsoOngoing bronchospasm0.000.00n15Broncopolmonite sinistraLeft bronchopneumonia0.000.00nBronchitis1Bronchite acutaAcute bronchitis90.4839.18y2Broncospasmo acuto parainfettivoAcute parainfectious bronchospasm22.735.15n3Broncospasmo acutoAcute bronchospasm16.675.15n4BronchiteBronchitis84.6222.68y5Nausea vomito febbreNausea vomit fever5.001.03n6Insufficienza respiratoria infezione basseRespiratory insufficiency lower infection30.006.19y7Polmonite basale destra riniteRight basal pneumonia rhinitis10.003.09n8BroncospasmoBronchospasm9.092.06n9Polmonite sinistra insufficienza respiratoriaLeft pneumonia respiratory insufficiency9.091.03n10Otite media acutaAcute otitis media0.000.00n11Infezione basse alte vie respiratorieRespiratory infection lower upper33.334.12n12Laringite ipoglotticaHypoglottic laryngitis0.000.00n13Bronchiolite lieveMild bronchiolitis4.761.03n14InfezioneInfection3.571.03n15Febbre flogosi alte vieFever upper tract phlogosis3.232.06n Keyword-based representation of the first 15 first-level clusters related to respiratory infections in each dataset, with the % of cases in each cluster, the % to the total number of cases in the dataset and the indication of the clusters that would be selected as weak labels at this stage. Table 1 reports the main first-level clusters related to respiratory infections for both datasets, including the number of bronchiolitis or bronchitis cases covered by each cluster. Given space constraints, we report here only the 15 largest clusters mentioning bronchiolitis, bronchitis, bronchospasm, pneumonia, respiratory infections, otitis, or fever, since these are the clusters most closely related to the respiratory diseases considered in our case studies. For the bronchiolitis dataset, four clusters (#1, #2, #3, #5) contain the keyword “bronchiolitis”. Three of them include a large majority of bronchiolitis cases, whereas the fourth cluster (#5) has only half of its diagnoses labelled as bronchiolitis. These four clusters together cover 65.38% of all bronchiolitis cases. Additionally, 13 clusters include the keyword “bronchospasm”, a symptom frequently associated with bronchiolitis. According to physicians who annotated the data, bronchospasm is associated with bronchiolitis only if accompanied by fever. Only one of these 13 clusters (#4) has both “bronchospasm” and “fever” keywords, and indeed, it contains a high proportion (70%) of bronchiolitis cases, accounting for an additional 15% of the total bronchiolitis cases. Thus, these five clusters are those that, without using the gold labels, would be selected to define the weak labels for the training set, according to the keyword-based definitions {positive={bronchiolitis},negative={} } and {positive={bronchospasm, fever},negative= {}}. Among the respiratory infection clusters, three additional clusters contain bronchiolitis cases (#6, #7, #8), although in a small number. For the bronchitis dataset, three clusters are selected at the first level (#1, #4, #6), according to the definitions {positive={bronchitis},negative={} } and {positive={lower, tract, respiratory, infection},negative={upper}}. These cover 68.05% of bronchitis cases. Table 2 reports the second-level clusters obtained from first-level clusters reported in Table 1. In particular, several clusters related to bronchiolitis, bronchospasm, bronchitis, and respiratory infections are aggregated into broader second-level clusters. In the bronchiolitis case study the selection of weak labels for training remains unchanged, as second-level clusters #1, #2, #3, and #4 cover exactly the clusters that would have been selected at the first level. Consequently, a downstream comparison between first- and second-level clustering would yield identical weak labels and identical classification results in this specific setting. In the bronchitis dataset, the second-level clustering merges two clusters that are partially related to lower tract respiratory infections, adding them to the set of positive weak labels.Table 2Keyword-based representation of the second-level clusters that are obtained as merge of first-level clusters reported in Table 1, with the % of cases in each cluster, the % to the total number of cases in the dataset and the indication of those selected as positive weak labels.1st lvlcluster2nd level cluster keywords [ITA]2nd level cluster keywords [ENG]1st lvlcluster1st level cluster keywords [ITA]1st level cluster keywords [ENG]Withinclusterbronch. %% of totalbronch.coveredPositiveweaklabelBronchiolitis1Bronchiolite lieveMild bronchiolitis1Acuta bronchiolite iniziale lieveAcute mild initial bronchiolitis85.1651.34y2Bronchiolite lieveMild bronchiolitis2Broncospasmo infezione respiratoriaBronchospasm respiratory infection7Broncospasmo infezione vie respiratorieBronchospasm infection airways2.900.30n10Broncospasmo infezione respiratoriaBronchospasm respiratory infection3BroncospasmoBronchospasm11Acuto broncospasmoAcute bronchospasm0.000.00n12Broncospasmo flogosi vieBronchospasm airways flogosis13Broncospasmo polmoniteBronchospasm pneumonia14Broncospasmo corsoOngoing bronchospasm22Broncospasmo otiteBronchospasm otitis23Broncospasmo parainfettivoBronchospasm parainfective24Broncospasmo virosiBronchospasm virosis17Broncospasmo episodioBronchospasm episode4Infezione vie respiratorieAirways infection18Infezione alte vie respiratorieHigh airways infection0.000.00n19Flogosi infezione alte vie respiratorieFlogosis high airways infection27Altre infezioni acute vie respiratorieOther acute respiratory infection airways26Crisi epilessia infezione paziente polmonare respiratoriaSeizure epilepsy respiratory infection lung patient16Febbricola febbrile infezione respiratoria virosiFebrile fever respiratory infection virosisBronchitis1BronchiteBronchitis1Bronchite acuteAcute bronchitis88.2461.86y4BronchiteBronchitis2BroncospasmoBronchospasm2Broncospasmo acuto parainfettivoAcute parainfectious bronchospasm16.2212.37n3Broncospasmo acutoAcute bronchospasm8BroncospasmoBronchospasm3infezione basse vie respiratorieLower tract respiratory infection6Insufficienza respiratoria infezione basseRespiratory insufficiency lower infection31.2510.31y11Infezione alte basse vie respiratorieRespiratory infection lower upper tract Table 2Keyword-based representation of the second-level clusters that are obtained as merge of first-level clusters reported in Table 1, with the % of cases in each cluster, the % to the total number of cases in the dataset and the indication of those selected as positive weak labels.1st lvlcluster2nd level cluster keywords [ITA]2nd level cluster keywords [ENG]1st lvlcluster1st level cluster keywords [ITA]1st level cluster keywords [ENG]Withinclusterbronch. %% of totalbronch.coveredPositiveweaklabelBronchiolitis1Bronchiolite lieveMild bronchiolitis1Acuta bronchiolite iniziale lieveAcute mild initial bronchiolitis85.1651.34y2Bronchiolite lieveMild bronchiolitis2Broncospasmo infezione respiratoriaBronchospasm respiratory infection7Broncospasmo infezione vie respiratorieBronchospasm infection airways2.900.30n10Broncospasmo infezione respiratoriaBronchospasm respiratory infection3BroncospasmoBronchospasm11Acuto broncospasmoAcute bronchospasm0.000.00n12Broncospasmo flogosi vieBronchospasm airways flogosis13Broncospasmo polmoniteBronchospasm pneumonia14Broncospasmo corsoOngoing bronchospasm22Broncospasmo otiteBronchospasm otitis23Broncospasmo parainfettivoBronchospasm parainfective24Broncospasmo virosiBronchospasm virosis17Broncospasmo episodioBronchospasm episode4Infezione vie respiratorieAirways infection18Infezione alte vie respiratorieHigh airways infection0.000.00n19Flogosi infezione alte vie respiratorieFlogosis high airways infection27Altre infezioni acute vie respiratorieOther acute respiratory infection airways26Crisi epilessia infezione paziente polmonare respiratoriaSeizure epilepsy respiratory infection lung patient16Febbricola febbrile infezione respiratoria virosiFebrile fever respiratory infection virosisBronchitis1BronchiteBronchitis1Bronchite acuteAcute bronchitis88.2461.86y4BronchiteBronchitis2BroncospasmoBronchospasm2Broncospasmo acuto parainfettivoAcute parainfectious bronchospasm16.2212.37n3Broncospasmo acutoAcute bronchospasm8BroncospasmoBronchospasm3infezione basse vie respiratorieLower tract respiratory infection6Insufficienza respiratoria infezione basseRespiratory insufficiency lower infection31.2510.31y11Infezione alte basse vie respiratorieRespiratory infection lower upper tract Keyword-based representation of the second-level clusters that are obtained as merge of first-level clusters reported in Table 1, with the % of cases in each cluster, the % to the total number of cases in the dataset and the indication of those selected as positive weak labels. Second-level clustering substantially reduces the number of clusters, to 20–50% of the original clusters in our datasets. In the specific case studies, Fig. 2c shows no effect on bronchiolitis weak labels and an increase in recall, at the cost of a decrease in precision, for bronchitis. Beyond these specific case studies, Fig. 2b provides an independent evaluation of clustering quality on a randomly selected sample of 1,000 diagnosis strings. In this broader assessment, second-level clustering increases the proportion of correct assignments and reduces partially correct or incorrect mappings in both datasets. This indicates a systematic refinement of cluster coherence, supporting the role of second-level clustering in improving weak label quality and interpretability, although this does not necessarily translate into substantial gains in downstream classification performance. The selected clusters cover 80% of the original bronchiolitis cases and 72% of the bronchitis cases, as illustrated in Fig. 2c, which compares weak labels derived from clustering with the manually annotated gold labels. To better understand the false negatives, we reviewed 20 discharge letters labelled as bronchiolitis in the gold standard but not covered by our weak labels. We found 10 cases where the diagnosis refers to other symptoms or diseases (e.g., otitis, dyspnea, fever) while bronchiolitis is mentioned elsewhere in the letter; 4 cases where bronchiolitis appears in the diagnosis alongside other diseases, leading to assignment to a cluster related to the other condition; 3 cases of respiratory issues without explicit mention of bronchiolitis, respiratory syncytial virus, or bronchospasm with fever; 2 cases with no diagnosis string present; and 1 case referring to bronchiolitis in a family member rather than the patient. These results suggest that most false negatives arise from the absence of explicit bronchiolitis indications in the diagnosis strings, rather than from clustering errors. To assess the effect of the TPT step on clustering results, Supplementary Material E reports the clusters obtained using the original UmBERTo model in our clustering pipeline, on the bronchiolitis dataset. The number of clusters related to respiratory infections is higher without TPT (34 vs 27), and their keywords appear more heterogeneous. Selecting weak labels by applying the same criterion we used on our clusters, we would cover only 47% of bronchiolitis cases with the clusters from the original UmBERTo model. Moreover, these clusters are more heterogeneous since they introduce more false negatives in the weak labels (precision decreases to 72 %). Additional results on the effect of clustering algorithms and hyperparameters are reported in Supplementary Material E. Disease classification Table 3Classification results on gold labels reported as mean (standard deviation) across cross-validation folds. Bold indicates best overall performance per metric, per dataset. Underline indicates best performance among non-fully-supervised models. Statistical significance was assessed using two-sided paired permutation tests across folds, comparing each model to WS-UmBERTo-TPT. P-values were adjusted using the Holm correction within each metric (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}p_{adj} < 0.05$$\end{document},\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}p_{adj} < 0.01$$\end{document}).LearningClassifierBronchiolitisBronchitisP [%]R [%]F1 [%]AUROC [%]AUPRC [%]P [%]R [%]F1 [%]AUROC [%]AUPRC [%]With diagnosis stringWeakly supervisedWS-UmBERTo-TPT 78.34 (6.77) 77.57 (6.89) 78.14 (4.89) 77.68 (4.30) 73.13 (4.93) 73.06 (5.95) 82.11 (5.82) 78.02 (5.25) 82.30 (5.10) 76.72 (5.02)WS-XLM-RoBERTa-TPT69.54\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.45)71.33(5.12)70.85\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.52)69.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.89)62.69\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.67)71.57(5.94)75.26\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.46)73.37\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.16)76.41\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.32)71.01(5.45)WS-RemBERT-TPT67.41\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(7.01)70.11(6.82)69.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.24)68.00\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.03)60.51\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.47)69.23(5.28)74.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.12)71.64\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.87)76.15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.17)68.39\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.56)WS-UmBERTo68.29(6.53)75.99(6.41)72.15(4.93)71.21(4.21)63.29(5.66)73.27(5.73)76.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.19)74.75(5.45)80.31(5.36)75.29(5.98)WS-XLM-RoBERTa67.49\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.28)68.67\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.09)68.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.32)67.65\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.65)61.11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.23)70.14(6.14)73.96\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.65)72.00\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.49)78.84(5.91)72.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.11)WS-RemBERT65.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.81)63.82\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.84)64.96\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.13)65.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.92)59.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.16)68.41\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.81)73.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.47)70.73\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.66)76.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.06)69.20\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.41)WS-LSTM57.39\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(8.31)60.86\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(11.52)59.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(9.69)63.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(7.36)49.94\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.13)60.78\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.25)63.92\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.66)62.31\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.28)70.12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.24)60.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.54)WS-XPRESS-UmBERTo-TPT16.57\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(2.21)60.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(9.18)29.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(9.04)54.18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(10.24)19.91\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(1.74)38.46\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.96)51.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.81)44.05\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.80)56.89\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.77)42.84\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.45)UnsupervisedRule-based full-text65.57\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.56)68.76\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.00)67.22\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.19)--15.79\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.51)32.88\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.67)21.33\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.81)--Rule-based diagnosis74.61(4.23)67.42\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.15)70.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.41)--21.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.15)20.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.68)21.13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.87)--LLaMAntino-378.26(5.72)74.64\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(7.48)75.12(4.29)71.67\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.48)67.92\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.47)65.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.51)72.61\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.89)68.71\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.67)75.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.68)68.13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.74)SupervisedFS-UmBERTo-TPT 80.30 (6.24) 81.52 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.59) 80.83 (4.12) 83.45 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.02) 77.57 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (2.02) 74.49 (4.45) 82.47 (4.68) 78.28 (4.51) 85.93 (4.47) 79.48 (3.52)FS-XLM-RoBERTa-TPT75.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.21)78.31(4.36)76.54(4.14)78.54(3.42)72.23(2.70)74.29(5.55)80.41(5.01)77.23(4.74)83.82(5.54)76.10(4.61)FS-RemBERT-TPT73.24(6.58)75.28(5.93)74.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.68)76.39(4.31)68.33(5.76)72.66(5.52)76.70(5.54)74.62(6.12)83.14(5.73)75.26(5.61)Without diagnosis stringWeakly supervisedWS-UmBERTo-TPT 74.25 (6.54) 73.52 (6.12) 73.89 (5.64) 74.27 (4.83) 69.27 (4.36) 72.09 (5.86) 79.90 (5.81) 75.79 (5.80) 79.26 (5.74) 74.71 (5.18)WS-XLM-RoBERTa-TPT66.83\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.69)67.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.88)67.19\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.57)67.25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.65)59.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.52)69.39(5.86)70.10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.51)69.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.84)75.13(5.64)69.33\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.54)WS-RemBERT-TPT64.53\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.58)63.66\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.21)64.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.84)65.47\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.97)56.75\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.43)69.31(5.56)72.16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.97)70.71\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.63)74.26\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.38)69.91\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.61)WS-UmBERTo67.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.38)76.40(6.25)71.74(5.47)69.98(4.59)63.27(2.66)72.00(6.01)74.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.67)73.10(5.73)73.45\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.39)69.09(5.84)WS-XLM-RoBERTa63.98\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.72)65.04\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.63)64.69\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.98)64.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.12)54.30\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.31)68.26(5.64)69.97\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.11)69.10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.88)70.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.42)65.39\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.15)WS-RemBERT62.25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.58)60.48\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.96)61.65\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.24)62.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.87)55.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.71)68.10(5.26)70.16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.69)69.11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.54)73.14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.91)69.34\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.82)WS-LSTM50.14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(7.14)63.97\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(10.12)56.34\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(8.65)59.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.51)43.30\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(4.74)58.45\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.51)61.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.25)60.11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.31)66.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.37)60.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.36)WS-XPRESS-UmBERTo-TPT16.03\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.52)64.50\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(12.62)27.36\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(8.48)52.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(10.12)17.89\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(2.33)35.47\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.54)50.51\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.24)41.67\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.94)52.28\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.64)45.43\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.02)UnsupervisedRule-based full-text62.33\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.78)60.05\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(8.42)61.15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.83)--14.09\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.61)30.00\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.89)19.18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.76)--LLaMAntino-370.42\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.26)72.98(5.84)71.16(3.98)68.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.55)62.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.22)60.12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.01)67.25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.15)63.49\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.23)68.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.02)59.31\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(2.74)SupervisedFS-UmBERTo-TPT 78.25 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.20)76.21(4.87) 77.15 (4.01) 78.51 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.84) 75.44 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.55) 72.99 (5.36)79.59(5.96) 76.36 (5.84) 82.63 (5.64) 76.82 (6.37)FS-XLM-RoBERTa-TPT72.54(4.26)73.65(4.16)72.96(3.19)74.08(3.34)70.61(3.39)73.33(5.51)79.38(6.11)76.24(5.43)81.15(5.30)74.34(5.38)FS-RemBERT-TPT71.49(5.46)70.87(4.12)71.21(3.95)72.68(3.87)64.06(4.59)71.57(5.63)75.26(5.84)73.37(5.69)78.26(5.88)71.28(5.67) Table 3Classification results on gold labels reported as mean (standard deviation) across cross-validation folds. Bold indicates best overall performance per metric, per dataset. Underline indicates best performance among non-fully-supervised models. Statistical significance was assessed using two-sided paired permutation tests across folds, comparing each model to WS-UmBERTo-TPT. P-values were adjusted using the Holm correction within each metric (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}p_{adj} < 0.05$$\end{document},\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}p_{adj} < 0.01$$\end{document}).LearningClassifierBronchiolitisBronchitisP [%]R [%]F1 [%]AUROC [%]AUPRC [%]P [%]R [%]F1 [%]AUROC [%]AUPRC [%]With diagnosis stringWeakly supervisedWS-UmBERTo-TPT 78.34 (6.77) 77.57 (6.89) 78.14 (4.89) 77.68 (4.30) 73.13 (4.93) 73.06 (5.95) 82.11 (5.82) 78.02 (5.25) 82.30 (5.10) 76.72 (5.02)WS-XLM-RoBERTa-TPT69.54\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.45)71.33(5.12)70.85\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.52)69.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.89)62.69\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.67)71.57(5.94)75.26\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.46)73.37\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.16)76.41\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.32)71.01(5.45)WS-RemBERT-TPT67.41\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(7.01)70.11(6.82)69.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.24)68.00\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.03)60.51\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.47)69.23(5.28)74.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.12)71.64\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.87)76.15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.17)68.39\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.56)WS-UmBERTo68.29(6.53)75.99(6.41)72.15(4.93)71.21(4.21)63.29(5.66)73.27(5.73)76.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.19)74.75(5.45)80.31(5.36)75.29(5.98)WS-XLM-RoBERTa67.49\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.28)68.67\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.09)68.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.32)67.65\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.65)61.11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.23)70.14(6.14)73.96\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.65)72.00\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.49)78.84(5.91)72.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.11)WS-RemBERT65.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.81)63.82\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.84)64.96\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.13)65.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.92)59.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.16)68.41\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.81)73.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.47)70.73\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.66)76.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.06)69.20\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.41)WS-LSTM57.39\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(8.31)60.86\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(11.52)59.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(9.69)63.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(7.36)49.94\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.13)60.78\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.25)63.92\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.66)62.31\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.28)70.12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.24)60.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.54)WS-XPRESS-UmBERTo-TPT16.57\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(2.21)60.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(9.18)29.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(9.04)54.18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(10.24)19.91\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(1.74)38.46\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.96)51.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.81)44.05\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.80)56.89\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.77)42.84\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.45)UnsupervisedRule-based full-text65.57\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.56)68.76\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.00)67.22\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.19)--15.79\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.51)32.88\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.67)21.33\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.81)--Rule-based diagnosis74.61(4.23)67.42\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.15)70.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.41)--21.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.15)20.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.68)21.13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.87)--LLaMAntino-378.26(5.72)74.64\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(7.48)75.12(4.29)71.67\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.48)67.92\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.47)65.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.51)72.61\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.89)68.71\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.67)75.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.68)68.13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.74)SupervisedFS-UmBERTo-TPT 80.30 (6.24) 81.52 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.59) 80.83 (4.12) 83.45 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.02) 77.57 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (2.02) 74.49 (4.45) 82.47 (4.68) 78.28 (4.51) 85.93 (4.47) 79.48 (3.52)FS-XLM-RoBERTa-TPT75.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.21)78.31(4.36)76.54(4.14)78.54(3.42)72.23(2.70)74.29(5.55)80.41(5.01)77.23(4.74)83.82(5.54)76.10(4.61)FS-RemBERT-TPT73.24(6.58)75.28(5.93)74.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.68)76.39(4.31)68.33(5.76)72.66(5.52)76.70(5.54)74.62(6.12)83.14(5.73)75.26(5.61)Without diagnosis stringWeakly supervisedWS-UmBERTo-TPT 74.25 (6.54) 73.52 (6.12) 73.89 (5.64) 74.27 (4.83) 69.27 (4.36) 72.09 (5.86) 79.90 (5.81) 75.79 (5.80) 79.26 (5.74) 74.71 (5.18)WS-XLM-RoBERTa-TPT66.83\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.69)67.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.88)67.19\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.57)67.25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.65)59.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.52)69.39(5.86)70.10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.51)69.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.84)75.13(5.64)69.33\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.54)WS-RemBERT-TPT64.53\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.58)63.66\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.21)64.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.84)65.47\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.97)56.75\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.43)69.31(5.56)72.16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.97)70.71\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.63)74.26\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.38)69.91\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.61)WS-UmBERTo67.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.38)76.40(6.25)71.74(5.47)69.98(4.59)63.27(2.66)72.00(6.01)74.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.67)73.10(5.73)73.45\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.39)69.09(5.84)WS-XLM-RoBERTa63.98\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.72)65.04\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.63)64.69\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.98)64.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.12)54.30\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(3.31)68.26(5.64)69.97\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.11)69.10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.88)70.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.42)65.39\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.15)WS-RemBERT62.25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(6.58)60.48\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.96)61.65\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.24)62.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.87)55.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.71)68.10(5.26)70.16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.69)69.11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.54)73.14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.91)69.34\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.82)WS-LSTM50.14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(7.14)63.97\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(10.12)56.34\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(8.65)59.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.51)43.30\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(4.74)58.45\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.51)61.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.25)60.11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.31)66.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.37)60.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.36)WS-XPRESS-UmBERTo-TPT16.03\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.52)64.50\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(12.62)27.36\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(8.48)52.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(10.12)17.89\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(2.33)35.47\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.54)50.51\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.24)41.67\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.94)52.28\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.64)45.43\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.02)UnsupervisedRule-based full-text62.33\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.78)60.05\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(8.42)61.15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(6.83)--14.09\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.61)30.00\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.89)19.18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document}(5.76)--LLaMAntino-370.42\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.26)72.98(5.84)71.16(3.98)68.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.55)62.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(4.22)60.12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.01)67.25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.15)63.49\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.23)68.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(5.02)59.31\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document}(2.74)SupervisedFS-UmBERTo-TPT 78.25 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.20)76.21(4.87) 77.15 (4.01) 78.51 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.84) 75.44 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.55) 72.99 (5.36)79.59(5.96) 76.36 (5.84) 82.63 (5.64) 76.82 (6.37)FS-XLM-RoBERTa-TPT72.54(4.26)73.65(4.16)72.96(3.19)74.08(3.34)70.61(3.39)73.33(5.51)79.38(6.11)76.24(5.43)81.15(5.30)74.34(5.38)FS-RemBERT-TPT71.49(5.46)70.87(4.12)71.21(3.95)72.68(3.87)64.06(4.59)71.57(5.63)75.26(5.84)73.37(5.69)78.26(5.88)71.28(5.67) Classification results on gold labels reported as mean (standard deviation) across cross-validation folds. Bold indicates best overall performance per metric, per dataset. Underline indicates best performance among non-fully-supervised models. Statistical significance was assessed using two-sided paired permutation tests across folds, comparing each model to WS-UmBERTo-TPT. P-values were adjusted using the Holm correction within each metric (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}p_{adj} < 0.05$$\end{document},\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}p_{adj} < 0.01$$\end{document}). 78.34 (6.77) 77.57 (6.89) 78.14 (4.89) 77.68 (4.30) 73.13 (4.93) 73.06 (5.95) 82.11 (5.82) 78.02 (5.25) 82.30 (5.10) 76.72 (5.02) 69.54\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.45) 71.33 (5.12) 70.85\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.52) 69.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.89) 62.69\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.67) 71.57 (5.94) 75.26\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.46) 73.37\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.16) 76.41\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.32) 71.01 (5.45) 67.41\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (7.01) 70.11 (6.82) 69.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.24) 68.00\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.03) 60.51\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.47) 69.23 (5.28) 74.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.12) 71.64\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.87) 76.15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.17) 68.39\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.56) 68.29 (6.53) 75.99 (6.41) 72.15 (4.93) 71.21 (4.21) 63.29 (5.66) 73.27 (5.73) 76.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.19) 74.75 (5.45) 80.31 (5.36) 75.29 (5.98) 67.49\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.28) 68.67\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.09) 68.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.32) 67.65\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.65) 61.11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.23) 70.14 (6.14) 73.96\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.65) 72.00\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.49) 78.84 (5.91) 72.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.11) 65.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.81) 63.82\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.84) 64.96\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.13) 65.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.92) 59.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.16) 68.41\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.81) 73.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.47) 70.73\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.66) 76.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.06) 69.20\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.41) 57.39\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (8.31) 60.86\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (11.52) 59.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (9.69) 63.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (7.36) 49.94\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.13) 60.78\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (6.25) 63.92\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.66) 62.31\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (6.28) 70.12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (6.24) 60.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.54) 16.57\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (2.21) 60.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (9.18) 29.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (9.04) 54.18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (10.24) 19.91\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (1.74) 38.46\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.96) 51.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.81) 44.05\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.80) 56.89\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.77) 42.84\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.45) 65.57\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.56) 68.76\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.00) 67.22\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.19) 15.79\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.51) 32.88\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.67) 21.33\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.81) 74.61 (4.23) 67.42\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.15) 70.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.41) 21.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (6.15) 20.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.68) 21.13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.87) 78.26 (5.72) 74.64\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (7.48) 75.12 (4.29) 71.67\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.48) 67.92\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.47) 65.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.51) 72.61\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.89) 68.71\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.67) 75.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.68) 68.13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.74) 80.30 (6.24) 81.52 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.59) 80.83 (4.12) 83.45 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.02) 77.57 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (2.02) 74.49 (4.45) 82.47 (4.68) 78.28 (4.51) 85.93 (4.47) 79.48 (3.52) 75.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.21) 78.31 (4.36) 76.54 (4.14) 78.54 (3.42) 72.23 (2.70) 74.29 (5.55) 80.41 (5.01) 77.23 (4.74) 83.82 (5.54) 76.10 (4.61) 73.24 (6.58) 75.28 (5.93) 74.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.68) 76.39 (4.31) 68.33 (5.76) 72.66 (5.52) 76.70 (5.54) 74.62 (6.12) 83.14 (5.73) 75.26 (5.61) 74.25 (6.54) 73.52 (6.12) 73.89 (5.64) 74.27 (4.83) 69.27 (4.36) 72.09 (5.86) 79.90 (5.81) 75.79 (5.80) 79.26 (5.74) 74.71 (5.18) 66.83\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.69) 67.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.88) 67.19\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.57) 67.25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.65) 59.29\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.52) 69.39 (5.86) 70.10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.51) 69.74\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.84) 75.13 (5.64) 69.33\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.54) 64.53\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.58) 63.66\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.21) 64.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.84) 65.47\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.97) 56.75\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.43) 69.31 (5.56) 72.16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.97) 70.71\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.63) 74.26\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.38) 69.91\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.61) 67.55\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.38) 76.40 (6.25) 71.74 (5.47) 69.98 (4.59) 63.27 (2.66) 72.00 (6.01) 74.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.67) 73.10 (5.73) 73.45\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.39) 69.09 (5.84) 63.98\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.72) 65.04\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.63) 64.69\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.98) 64.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.12) 54.30\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.31) 68.26 (5.64) 69.97\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.11) 69.10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.88) 70.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.42) 65.39\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.15) 62.25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (6.58) 60.48\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (6.96) 61.65\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.24) 62.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.87) 55.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.71) 68.10 (5.26) 70.16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.69) 69.11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.54) 73.14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.91) 69.34\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.82) 50.14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (7.14) 63.97\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (10.12) 56.34\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (8.65) 59.02\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (6.51) 43.30\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (4.74) 58.45\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.51) 61.87\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.25) 60.11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.31) 66.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.37) 60.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.36) 16.03\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (6.52) 64.50\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (12.62) 27.36\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (8.48) 52.23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (10.12) 17.89\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (2.33) 35.47\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.54) 50.51\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (6.24) 41.67\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.94) 52.28\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.64) 45.43\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (6.02) 62.33\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (6.78) 60.05\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (8.42) 61.15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (6.83) 14.09\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.61) 30.00\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.89) 19.18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{**}$$\end{document} (5.76) 70.42\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.26) 72.98 (5.84) 71.16 (3.98) 68.52\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.55) 62.24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (4.22) 60.12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.01) 67.25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.15) 63.49\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.23) 68.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.02) 59.31\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (2.74) 78.25 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (5.20) 76.21 (4.87) 77.15 (4.01) 78.51 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.84) 75.44 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{*}$$\end{document} (3.55) 72.99 (5.36) 79.59 (5.96) 76.36 (5.84) 82.63 (5.64) 76.82 (6.37) 72.54 (4.26) 73.65 (4.16) 72.96 (3.19) 74.08 (3.34) 70.61 (3.39) 73.33 (5.51) 79.38 (6.11) 76.24 (5.43) 81.15 (5.30) 74.34 (5.38) 71.49 (5.46) 70.87 (4.12) 71.21 (3.95) 72.68 (3.87) 64.06 (4.59) 71.57 (5.63) 75.26 (5.84) 73.37 (5.69) 78.26 (5.88) 71.28 (5.67) Fig. 3Precision-Recall curves on the bronchiolitis (a, b) and bronchitis (c, d) gold labels with (a, c) and without (b, d) diagnosis strings. Shaded areas represent 95% C.I. Precision-Recall curves on the bronchiolitis (a, b) and bronchitis (c, d) gold labels with (a, c) and without (b, d) diagnosis strings. Shaded areas represent 95% C.I. Table 3 reports the cross-validation results for the classification task, including precision, recall, F1-score, AUROC, and AUPRC. The table compares our weakly supervised pipeline with different transformer-based classifiers, fully unsupervised rule-based and zero-shot LLM approaches, and fully supervised versions of the transformer-based classifiers. Results are shown for both datasets and for two experimental settings: keeping the diagnosis string in the discharge letter or removing it. On the bronchiolitis dataset, our weakly supervised pipeline outperforms fully unsupervised approaches and maintains only a limited gap with the fully supervised models. The TPT step consistently improves performance across models: the differences are limited, but there are no cases in which TPT hurts performance. Among the transformer-based classifiers, UmBERTo generally achieves the best results, outperforming RemBERT and XLM-RoBERTa across most metrics. The LSTM classifier shows a clear performance gap with respect to the transformer-based models. Rule-based full-text, which verifies the presence of keywords in the entire letter, shows lower precision and recall than the majority of the weakly supervised models, emphasizing the need for a more sophisticated approach. The results further decline when the diagnosis string is excluded. Rule-based diagnosis, using only diagnosis strings, yields better precision than the rule applied to the complete text, though recall remains similar. The zero-shot approach with the Italian LLaMAntino-3 LLM does not outperform our best weakly supervised results but shows competitive performance. The model trained using weak labels derived from the XPRESS framework performs substantially worse than all other approaches in this setting. This is expected, since this implementation of the XPRESS framework applies the same keyword rules defined for our cluster selection across the entire clinical text, generating a larger number of noisy weak labels compared with the more targeted extraction of diagnosis strings. Results on the bronchitis validation dataset show a broadly similar ranking of models, suggesting consistent trends with the main case study. Despite the substantially smaller dataset size, transformer-based classifiers again outperform rule-based and zero-shot approaches, and UmBERTo-based models achieve the best overall performance among the weakly supervised methods. These findings, however, should be interpreted with caution given the limited number of positive bronchitis cases. We also repeated the bronchitis experiments without applying the second-level clustering step when generating weak labels. This led to a small decrease in performance (AUPRC reduction of 1-2%), indicating that the impact of second-level clustering on downstream performance is limited in this setting. Detailed results are reported in Supplementary Material F. Precision–recall curves for the different models are reported in Fig. 3, providing a complementary view of model ranking performance across recall levels.Fig. 4AUPRC with varying percentages of weak labels on the bronchiolitis (a) and bronchitis (b) datasets. Mean AUPRC (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pm 95\%$$\end{document} CI) on gold labels, with and without diagnosis strings, as different proportions of gold labels are replaced by weak labels. Results are averaged over 5 repetitions of cross-validation. The y-axis is truncated to improve visibility of the trend. AUPRC with varying percentages of weak labels on the bronchiolitis (a) and bronchitis (b) datasets. Mean AUPRC (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pm 95\%$$\end{document} CI) on gold labels, with and without diagnosis strings, as different proportions of gold labels are replaced by weak labels. Results are averaged over 5 repetitions of cross-validation. The y-axis is truncated to improve visibility of the trend. In Fig. 4, we observe the decrease in performance when replacing increasing percentages of gold labels with weak labels. This scenario is representative of many real-world settings where only a fraction of the data can be manually annotated. The results show a generally smooth and monotonic decrease in AUPRC as the proportion of weak labels increases, with similar trends observed for both bronchiolitis and bronchitis. In Table 4, we assess the sensitivity of our best weakly supervised model (WS-UmBERTo-TPT) to the selection of clusters used to generate weak labels. In particular, we remove one first-level cluster at a time from the set of positive clusters and evaluate the resulting classification performance. For the bronchiolitis dataset, excluding clusters #5 or #1 has little effect on performance, which is reasonable considering their smaller size. Removing clusters #3 or #4 leads to a moderate decrease in performance. In particular, cluster #4 corresponds to the second bronchiolitis definition (“bronchospasm” and “fever”); excluding this cluster is therefore equivalent to omitting that definition in the keyword specification step. Only excluding cluster #2 produces a substantial drop in performance and an increase in variance, which is expected since it accounts for almost half of the bronchiolitis cases. A similar trend is observed for the bronchitis dataset. Removing cluster #11 or #6 produces limited changes in performance, whereas excluding clusters #1 or #4 leads to a larger decrease in classification performance. Overall, these results confirm the robustness of the model with respect to moderate changes in the selection of clusters used to generate weak labels and, more generally, to variations in the disease definitions used in the weak label construction step. Additional analyses evaluating model performance when training and testing on weak labels, together with further stratified results across hospitals, LHUs, and pediatric vs non-pediatric emergency departments, are reported in Supplementary Material F. Some hospitals are more represented in the dataset, leading to superior performance, but also hospitals with a similar number of records show considerable performance variability, suggesting that reporting practices in discharge letters might influence disease detection.Table 4Classification results after removing one first-level cluster for weak labels at a time, on the best weakly supervised classification models. Cross validation results with std. dev. in parenthesis.Clusters for weak labelsP [%]R [%]F1 [%]AUROC [%]AUPRC [%]Bronchiolitis#1. #2. #3. #4.70.75 (6.48)81.27 (6.99)75.65 (4.29)75.49 (5.57)67.95 (4.32)#1. #2. #4. #5.68.42 (8.70)75.09 (8.58)71.60 (7.26)72.67 (6.87)61.44 (4.75)#2. #3. #4. #5.68.54 (6.30)82.35 (4.94)74.81 (3.82)75.83 (5.22)66.07 (3.64)#1. #3. #4. #5.57.29 (12.97)59.03 (15.94)58.15 (12.45)68.90 (10.04)47.79 (3.39)#1. #2. #3. #5.67.86 (6.36)73.42 (8.64)70.53 (5.33)70.22 (3.10)62.81 (3.19)Bronchitis#1. #4. #6.73.41 (5.84)79.12 (5.71)76.26 (5.19)79.66 (5.22)75.03 (5.87)#1. #4. #11.75.67 (5.96)77.23 (6.05)76.64 (5.85)77.22 (5.48)73.78 (6.09)#1. #6. #11.66.52 (6.41)70.12 (6.84)68.35 (6.35)70.52 (6.07)61.91 (5.68)#4. #6. #11.62.15 (6.96)60.23 (6.84)61.15 (6.40)61.22 (5.45)56.68 (6.25) Table 4Classification results after removing one first-level cluster for weak labels at a time, on the best weakly supervised classification models. Cross validation results with std. dev. in parenthesis.Clusters for weak labelsP [%]R [%]F1 [%]AUROC [%]AUPRC [%]Bronchiolitis#1. #2. #3. #4.70.75 (6.48)81.27 (6.99)75.65 (4.29)75.49 (5.57)67.95 (4.32)#1. #2. #4. #5.68.42 (8.70)75.09 (8.58)71.60 (7.26)72.67 (6.87)61.44 (4.75)#2. #3. #4. #5.68.54 (6.30)82.35 (4.94)74.81 (3.82)75.83 (5.22)66.07 (3.64)#1. #3. #4. #5.57.29 (12.97)59.03 (15.94)58.15 (12.45)68.90 (10.04)47.79 (3.39)#1. #2. #3. #5.67.86 (6.36)73.42 (8.64)70.53 (5.33)70.22 (3.10)62.81 (3.19)Bronchitis#1. #4. #6.73.41 (5.84)79.12 (5.71)76.26 (5.19)79.66 (5.22)75.03 (5.87)#1. #4. #11.75.67 (5.96)77.23 (6.05)76.64 (5.85)77.22 (5.48)73.78 (6.09)#1. #6. #11.66.52 (6.41)70.12 (6.84)68.35 (6.35)70.52 (6.07)61.91 (5.68)#4. #6. #11.62.15 (6.96)60.23 (6.84)61.15 (6.40)61.22 (5.45)56.68 (6.25) Classification results after removing one first-level cluster for weak labels at a time, on the best weakly supervised classification models. Cross validation results with std. dev. in parenthesis. Discussion This study introduces a novel pipeline demonstrating how weakly supervised learning can be effectively applied to clinical NLP. To our knowledge, this is the first work addressing diagnosis identification from Italian discharge letters. Although we primarily validated it using a case study on bronchiolitis, we also conducted a secondary validation on a smaller dataset for bronchitis. We observed a good mapping between diagnosis strings and their clusters’ descriptions in the manually evaluated subset of strings, and indeed a good match between weak and gold labels, confirming that the SAL step can effectively identify relevant cases. Despite the inevitable noise introduced by weak labels, its magnitude remains limited, allowing the model to perform well when evaluated against gold labels. Broadly similar trends are observed in both datasets; however, results on the bronchitis dataset should be interpreted with caution given the limited number of positive cases. Additionally, the differences between settings where we retained the diagnosis strings and those where we removed them are limited. This indicates that the model is effectively learning from the entire text of the discharge letters, making it particularly advantageous when the automatic extraction of diagnosis strings fails. Our manual exploration of some weak labels further revealed that diagnosis strings do not always mention the disease responsible for hospitalization. In some cases, they only refer to symptoms, and in cases of multiple diseases, not all may be reported. These findings highlight the necessity for a classification model based on the complete text of discharge letters, as the one we developed. In contrast, rule-based approaches underperformed because diseases like bronchiolitis and bronchitis are not always explicitly mentioned or may appear in negated or hypothetical contexts. While more sophisticated rule-based systems could address some of these issues, such approaches lack the scalability and generalizability of our pipeline. Conversely, our pipeline can be adapted to other diseases, making it a potentially versatile tool for clinical NLP. A similar issue arises when weak labels are generated using the XPRESS framework applied across the entire clinical text, which produces substantially noisier labels compared with the diagnosis-string-based weak labelling used in our pipeline. The zero-shot LLM performed better than rule-based and some weakly supervised settings, but it did not surpass the best weakly supervised configuration. Despite the reasoning capacities of modern LLMs, their training data rarely include documents and tasks of this type, limiting performance. Domain pretraining (TPT) improved results across all models. The best performance was obtained with UmBERTo, trained entirely on Italian data, confirming the advantage of language-specific representations even over larger multilingual models. Compared with fully supervised classifiers, our weakly supervised pipeline shows a reduction in performance of 4–6% AUPRC. While this difference is statistically significant, its practical relevance depends on the intended application. In the context of cohort selection from large collections of discharge letters, the goal is to identify groups of potentially relevant patients at scale, rather than to achieve perfect classification at the individual level. In this case, such a reduction may be considered acceptable given the substantial decrease in annotation effort, which would otherwise require more than 1,500 hours of expert time in our case study. This highlights the practical trade-off between performance and scalability offered by weakly supervised approaches. The experiments in which we progressively replaced gold labels with weak labels (Fig. 4) further support this trade-off in a realistic scenario where only a small portion of the data can be manually annotated. Even in this mixed-label setting, performance degraded smoothly and remained close to the fully supervised baseline as long as a modest fraction of gold labels was available, indicating strong resilience to label noise. The pipeline has also shown robustness to the cluster selection used to generate weak labels across both case studies. Performance degradation was proportional to the reduction in weak label coverage, indicating stable behavior under moderate variations in disease definitions. Although second-level clustering provided no or only modest improvements in downstream performance in our case studies, it improved cluster coherence and weak label correctness in the independent evaluation of clustering quality (Fig. 2b), thereby contributing to more interpretable cluster structures. It is important to clarify that, while the pipeline does not require document-level manual annotation, it relies on disease-specific keyword definitions provided as minimal domain input. This step requires collaboration with clinicians to reflect the terminology commonly used in discharge letters, but it is substantially less demanding than full annotation of individual documents. As shown in the robustness analysis, moderate variations in cluster selection lead to proportionally limited changes in performance, suggesting that the approach is not overly sensitive to minor imperfections in keyword specification. While our results are promising, several limitations should be acknowledged. First, the gold labels for both datasets were assigned primarily by a single annotator, introducing potential uncertainty in the reference standard. We performed an additional inter-annotator agreement analysis on a stratified subset of discharge letters, but this evaluation was limited in size and does not fully replace a systematic double-annotation process on the complete datasets. Although the task consisted of binary disease identification, which is less complex than fine-grained clinical annotation, some degree of subjectivity cannot be excluded. Second, a limitation concerns the additional validation on the bronchitis dataset. Due to the relatively small number of positive cases (97 in total), each test fold in the 5-fold cross-validation contains approximately 19 positive examples. This results in limited statistical power to detect differences between models and increases the variability of performance estimates. Consequently, results on the bronchitis dataset should be primarily viewed as indicative of generalization trends rather than as a definitive comparative evaluation. Third, the multi-centric nature of our dataset, though a strength in its diversity, also presents an imbalance in the number of records from different hospitals and LHUs, which are also all located within a single Italian region. This is not only due to the geographical distribution of the population, but might also reflect some biases in the distribution of the family pediatricians adhering to the Pedianet network. Testing the pipeline on more balanced datasets and externally validating it on data from other regions would strengthen its generalizability and allow a more precise assessment of performance differences among hospitals. Finally, the evaluation of intermediate steps relied on the manual review of a relatively small subset of diagnoses strings. Extending this annotation to a larger subset and assessing entire clusters would require substantial effort but could enable a more detailed analysis of the performance of each component in the pipeline. Future research can extend this work along multiple directions. Although we evaluated the pipeline on two different disease-specific datasets, further validation on additional clinical conditions and datasets is required to fully assess its generalizability. Future studies should also include complete double annotation to formally assess inter-annotator agreement and better quantify the reliability of reference labels. Incorporating hospital or LHU identifiers in hierarchical classification models could further improve performance by accounting for site-specific documentation practices. Moreover, enhancing the interpretability of model predictions will be crucial for widespread clinical adoption. Identifying which portions of each discharge letter most influence the classifier’s decision would help clinicians assess reliability and build trust. Techniques from explainable artificial intelligence could be applied to reveal the reasoning process behind model outputs and possibly recognize potential biases present in the training data. These future developments would further strengthen both the methodological robustness and clinical utility of weakly supervised NLP systems for healthcare text, supporting scalable and trustworthy solutions for real-world applications. Conclusions We presented a weakly supervised pipeline for disease identification from clinical discharge letters, designed to minimize the need for manual annotation while maintaining competitive classification performance. The proposed approach combines automatic extraction of diagnosis strings, clustering-based weak label generation, and transformer-based text classification. In a case study on bronchiolitis using Italian discharge letters, the pipeline outperformed unsupervised rule-based approaches and achieved performance close to fully supervised models. Similar model rankings were observed in a secondary validation on a smaller bronchitis dataset. The results also highlight the importance of targeted weak label generation: simpler strategies, such as in the XPRESS-based baseline, produced substantially noisier labels and markedly worse performance. Experiments with mixed gold and weak labels further showed that the proposed approach is resilient to label noise and can maintain strong performance even when only a limited portion of the dataset is manually annotated. In addition, the classification models were able to effectively learn from the entire discharge letter text, reducing dependence on perfectly extracted diagnosis strings. By substantially reducing the annotation burden while maintaining robust performance, this approach offers a practical and scalable framework for applying NLP to large collections of clinical documents. Supplementary Information Supplementary Information. Supplementary Information.