Skin-SOAP: A Weakly Supervised Framework for Generating Structured SOAP Notes
Sadia Kamal, Tim Oates, Joy Wan · 2025
Read Skin-SOAP: A Weakly Supervised Framework for Generating Structured SOAP Notes by Sadia Kamal, Tim Oates, Joy Wan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05019.txt.
Introduction
Sadia Kamal, Tim Oates, and Joy Wan present Skin-SOAP, a weakly supervised framework that generates structured clinical notes for skin cases, published in 2025. Skin carcinoma is the most common cancer globally and costs billions each year in care. Clinicians document visits in SOAP notes covering subjective and objective findings, assessment, and plan, but writing them by hand is laborious and feeds burnout.
Their multimodal system builds such notes from limited inputs: lesion images plus sparse clinical text. Weak supervision reduces the need for large hand-labeled data sets. Performance matched strong general models such as GPT-4o on clinical relevance measures, the authors report. They also introduce two new metrics that judge how well a note aligns with expert medical concepts and with the input features.
Catalog record
| CREATOR | Sadia Kamal, Tim Oates, Joy Wan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-05019
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Skin-SOAP: A Weakly Supervised Framework for Generating Structured SOAP Notes Sadia Kamal, Tim Oates, Joy Wan Abstract Skin carcinoma is the most prevalent form of cancer globally, accounting for over $8 billion in annual healthcare expenditures. Early diagnosis, accurate and timely treatment are critical to improving patient survival rates. In clinical settings, physicians document patient visits using detailed SOAP (Subjective, Objective, Assessment, and Plan) notes. However, manually generating these notes is labor-intensive and contributes to clinician burnout. In this work, we propose skin-SOAP , a weakly supervised multimodal framework to generate clinically structured SOAP notes from limited inputs, including lesion images and sparse clinical text. Our approach reduces reliance on manual annotations, enabling scalable, clinically grounded documentation while alleviating clinician burden and reducing the need for large annotated data. Our method achieves performance comparable to GPT-4o, Claude, and DeepSeek Janus Pro across key clinical relevance metrics. To evaluate this clinical relevance, we introduce two novel metrics MedConceptEval and Clinical Coherence Score (CCS) which assess semantic alignment with expert medical concepts and input features, respectively. 1 Introduction Skin cancer remains one of the most common and deadliest cancers in the United States, with approximately 9,500 new cases diagnosed daily Rogers et al. 2015 , highlighting the crucial role of clinical documentation as the foundation for effective communication, accurate diagnosis, and informed treatment planning. Structured formats like SOAP (Subjective, Objective, Assessment, Plan) notes are widely adopted in the United States to ensure consistency in recording patient encounters and minimizing communication errors among healthcare professionals Schloss and Konam 2020 . However, generating these notes is labor-intensive and time-consuming, which reduces direct patient interaction time and significantly contributes to physician burnout Li et al. 2024 Biswas and Talukdar 2024 . Automating the generation of SOAP notes presents a promising solution to reduce administrative burden, improve documentation consistency, and allow clinicians to focus more on patient centred care. Recent advances in large language models (LLMs) have enabled impressive progress in medical natural language processing tasks, including clinical summarization, question answering Singhal et al. 2023 , lab report interpretation He et al. 2024 , and deidentification of sensitive information Yang et al. 2023 . These models can produce coherent and fluent clinical narratives, making them useful tools for medical documentation. However, general purpose LLMs often lack the domain-specific reasoning required for clinical settings, struggle to understand subtle medical context, and are generally limited to text based inputs. Their performance is further constrained in tasks such as structured note generation, especially when applied to domains like dermatology. Figure 1: Overview of the proposed skin-SOAP framework, consisting of data generation, fine-tuning, and inference phases. Existing methods for automated SOAP note generation, such as K-SOAP Li et al. 2024 , rely heavily on extensive doctor-patient dialogues and large-scale annotated datasets resources that are particularly limited in dermatology and skin lesion documentation Wei et al. 2024 . Moreover, capturing both the visual features of skin conditions and the underlying clinical reasoning in a structured format remains a major challenge. To address these limitations, we propose skin-SOAP, a novel weakly supervised multimodal framework that generates structured SOAP notes from limited inputs including lesion images and sparse clinical text. Unlike prior approaches focused solely on text-based SOAP generation or dermatologic diagnosis, our method uniquely integrates retrieval-augmented clinical knowledge, weak supervision, and multimodal synthesis to enable domain-aligned documentation without requiring large-scale annotations. We also introduce two novel metrics to evaluate clinical and semantic quality: MedConceptEval and Clinical Coherence Score (CCS), which go beyond traditional NLP metrics by assessing alignment with defined concepts and feature consistency. By leveraging domain-guided retrieval and pseudo-labeling, our framework produces clinically relevant SOAP notes with minimal supervision, offering a scalable solution for dermatology and broader healthcare applications. Our main contributions are: • We developed skin-SOAP, a weakly supervised multimodal framework for structured SOAP note generation from lesion images and limited clinical text. • We introduced two novel evaluation metrics: MedConceptEval and Clinical Coherence Score (CCS) to evaluate semantic alignment with clinical concepts and input consistency. • We performed statistical analysis using two-way ANOVA to quantify the effects of SOAP sections and lesion types on semantic similarity scores. • We conducted qualitative evaluation using an LLM-as-a-Judge framework (Flow-Judge-v0.1) to assess structure, readability, completeness, and clinical relevance. 2 Related Work Artificial intelligence has been widely applied in the medical domain for tasks such as medical image captioning Cheng et al. 2025 , clinical text summarization Van Veen et al. 2023 , discharge note generation Jung et al. 2024 , and medical question answering Yan et al. 2024 . Recent efforts have focused on generating clinical notes, especially SOAP notes, from Electronic Health Records (EHRs) and doctor-patient conversations. In Li et al. 2024 , large language models (LLMs) are fine-tuned to generate K-SOAP notes, while Chen and Hirschberg 2024 evaluated generative models across general and SOAP-specific formats. Similar work has explored domains like pediatric rehabilitation Amenyo et al. 2025 and fine-tuning LLMs for efficient SOAP note generation Leong et al. 2024 . Earlier approaches to SOAP note generation, such as Schloss and Konam 2020 ; Singh et al. 2023 , used sequence-to-sequence models and transformer architectures trained on large dialogue datasets. Although general-purpose LLMs have improved clinical NLP tasks Singhal et al. 2023 ; Yang et al. 2023 , they often lack domain-specific reasoning and structured multimodal output capabilities. While methods like Ramprasad et al. 2023 have attempted to improve faithfulness in text-based settings, these methods heavily rely on large annotated dialogue corpora, which are difficult to obtain due to privacy concerns Chen and Hirschberg 2024 , and they remain text-centric without leveraging multimodal information, which is critical to fields like dermatology. SkinGPT-4 Zhou et al. 2024 recently demonstrated that integrating clinical images and text using a multimodal LLM improves dermatological diagnostic reasoning. However, it is primarily designed for diagnostic prediction and lacks structured documentation capabilities. Specifically, it does not generate clinically formatted SOAP notes or support section-level reasoning required for clinical documentation. In contrast, our work focuses on end-to-end generation of structured SOAP notes from limited multimodal inputs, introducing weak supervision and domain-guided retrieval to ensure both clinical reliability and scalability. 3 Methodology We proposed a three-phase weakly supervised multimodal framework for generating clinically structured SOAP notes from limited dermatologic inputs. As illustrated in Fig. 1 , our method consists of : (1) a data generation phase to synthesize weakly supervised SOAP notes using generative captioning and retrieval-augmented knowledge integration, (2) a fine-tuning phase to adapt a vision-language model using the synthesized notes, and (3) an inference phase to generate high-quality structured SOAP notes from new patient data. 3.1 Dataset We use the PAD-UFES-20 dataset Pacheco et al. 2020 , which consists of 2,298 dermoscopic images along with structured metadata for 1,641 skin lesions collected from 1,373 patients. The lesions are classified into six types: Basal Cell Carcinoma (BCC), Melanoma (MEL), Squamous Cell Carcinoma (SCC, including Bowen’s disease), and three non-cancerous conditions: Actinic Keratosis (ACK), Seborrheic Keratosis (SEK), and Nevus (NEV). Approximately 58% of the samples are confirmed through biopsy, while the remaining cases are clinically diagnosed based on expert consensus. Each lesion is linked to a CSV file containing 26 structured clinical features, which include lesion characteristics (such as size