Cross-cultural adaptation of the Four Habits Coding Scheme into Danish: a study of inter- and intra-rater reliability in face-to-face and video consultations Authors: Christian Emil Brinck, Ulrik Bak Kirk, Henrik Schou Pedersen, Maria Louise Køpfli, Linda Huibers, Morten Bondo Christensen, Alexandre Bellier, Edward Krupat, Anette Fischer Pedersen Published: 2026 ABSTRACT AbstractBackgroundEffective physician–patient communication is essential for high-quality care in general practice and can be systematically assessed using structured instruments such as the Four Habits Coding Scheme (4HCS). Although the 4HCS has been validated in English, no Danish version exists, limiting research, evaluation, and training opportunities in Denmark. This study aimed to translate and culturally adapt the 4HCS into Danish and evaluate its inter- and intra-rater reliability in both face-to-face and video consultations.MethodsFollowing internationally accepted guidelines, the original 4HCS was translated into Danish using forward–backward translation with minor cultural adaptations. The instrument contains 23 items rated on a 1–5 scale, grouped into Four Habits with a total possible score range of 23–115. Ten general practitioners recorded 150 consultations (78 face-to-face, 72 video consultations), of which 138 were eligible for assessment. Three trained physician raters independently evaluated the consultations using the Danish 4HCS. Inter- and intra-rater reliability were calculated using intra-class correlation coefficients (ICC).ResultsThe mean overall 4HCS total score was 79.80 (SD 14.30), with minimal differences between FTF and VC consultations. Inter-rater reliability for the total score was good (ICC = 0.75), with similar reliability for FTF (ICC = 0.74) and VCs (ICC = 0.77). Intra-rater reliability was good across all habits and modalities (ICC range = 0.67–0.75). Item-level reliability showed some variability (ICC range = 0.30–0.81).ConclusionsThe Danish 4HCS demonstrated good inter- and intra-rater reliability for assessing physician–patient communication in both FTF and VC consultations. It shows promise as a reliable tool for evaluating communication in Danish general practice across consultation modalities. Future studies may focus on enhanced rater calibration to further improve scoring consistency.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12909-026-09351-z.Keywords: Communication, Reliability, Physician–patient communication, Four Habits Coding Scheme, Video consultations Introduction Physicians’ communication skills are fundamental to high-quality clinical practice. Effective communication fosters trust and supports shared decision-making, which in turn has been shown to increase patient satisfaction, enhance treatment adherence, improve health outcomes, and reduce the likelihood of complaints [1–5]. While teaching physician–patient communication presents its own challenges, assessing it is inherently challenging, as it involves complex, context-dependent interactions that are not easily captured or measured objectively. To address the latter, a range of structured assessment instruments have been developed [6]. Among these, the Four Habits Coding Scheme (4HCS) is a widely used standardized observational instrument designed to assess physicians’ communication skills based on recorded consultations [7]. It is based on the Four Habits Model, an educational framework that conceptualizes the patient encounter as four key communication Habits: ‘Invest in the Beginning’, ‘Elicit the Patient’s Perspective’, ‘Demonstrate Empathy’, and ‘Invest in the End’ [8]. In the 4HCS, the Four Habits Model is operationalized as 23 observable items. Each item is rated on a 5-point Likert scale by trained external observers who use behaviorally anchored scale points to define different levels of performance [7]. Since its introduction, the 4HCS has been translated and culturally adapted into several languages, including Norwegian, German, Brazilian Portuguese, and French, facilitating its use across a range of healthcare and educational environments, from hospital wards and general practice to pharmacy consultations and medical education [7, 9–16]. While many instruments show good internal consistency, relatively few achieve satisfactory inter-rater reliability. This is a particular challenge in communication research, where assessment involves interpretation of nuanced and context-dependent behaviors. Without clearly defined rating criteria, such complexity can lead to substantial variability in scoring. The 4HCS appears to address this challenge effectively. A recent systematic review by Bellier et al. (2023) concluded that the 4HCS appears to be the most reliable instrument for assessing physician communication in face-to-face (FTF) consultations [6]. This high level of reliability may partly reflect its structured format and the use of exemplifying anchors for each item, which support more consistent scoring across raters. Despite its widespread international use and strong measurement properties, no Danish version of the 4HCS currently exists. The context in which physician–patient communication takes place is evolving. Video consultations (VCs) are becoming both widespread and recently mandatory in Danish general practice. Hence there is a growing need to measure the quality of physicians’ communication in this digital context [17–19]. As previous psychometric evaluations of the 4HCS have primarily been conducted in FTF consultations, it remains unclear whether its reliability is maintained in VCs. Until now, research on VCs in general practice has primarily employed qualitative approaches, exploring how clinicians and patients experience and adapt to digital consultations [20–22]. An objective and validated instrument for assessing communication quality in VCs is therefore still needed. Therefore, this study aimed to translate and culturally adapt the 4HCS into Danish and assess reliability measures using recordings of both FTF and VCs from Danish general practice. By addressing both linguistic adaptation and empirical testing in real-world settings, this study adds to the growing evidence base supporting the 4HCS as a feasible and effective instrument for evaluating clinical communication across modalities and healthcare settings. Materials and methods Study design This study employed a three-step approach. First, the 4HCS was translated and cross-culturally adapted into Danish. Second, the Danish version of the 4HCS was applied to evaluate recordings of both FTF and VCs conducted in Danish general practice. Third, inter- and intra-rater reliability measures were assessed. Translation and cross-cultural adaptation The translation process adhered to internationally recognized guidelines for cross-cultural adaptation of research instruments [23, 24]. The aim was to achieve a Danish version that retained the conceptual equivalence of the original US instrument by Krupat et al., remained culturally relevant in a Danish context, and was easily understandable to its intended users. The translation process consisted of three main steps: Forward Translation: Three native Danish speakers independently translated the original 4HCS; a psychologist and associate professor with expertise in rating scales; a physician and PhD student with research experience in clinical communication, and a non-medical academic linguistic expert. Discrepancies were discussed within the translation team until consensus was reached. Forward Translation: Three native Danish speakers independently translated the original 4HCS; a psychologist and associate professor with expertise in rating scales; a physician and PhD student with research experience in clinical communication, and a non-medical academic linguistic expert. Discrepancies were discussed within the translation team until consensus was reached. Back Translation: This consensus version was back-translated by a native English speaker translator fluent in Danish, who was blinded to the original version but informed of the instrument’s purpose. Back Translation: This consensus version was back-translated by a native English speaker translator fluent in Danish, who was blinded to the original version but informed of the instrument’s purpose. Review: The back-translated version was reviewed for semantic and operational equivalence by a researcher experienced in the French adaptation of the 4HCS. A non-medical academic linguistic expert and two trained raters reviewed and pretested the back-translated version leading to minor linguistic adjustments. Review: The back-translated version was reviewed for semantic and operational equivalence by a researcher experienced in the French adaptation of the 4HCS. A non-medical academic linguistic expert and two trained raters reviewed and pretested the back-translated version leading to minor linguistic adjustments. 4HCS scoring Each of the 23 items in the 4HCS is scored on a 5-point Likert scale, where higher values indicated better performance. To support consistent scoring, behaviorally anchored examples are provided for points 1, 3, and 5. As an example for item five (Elicit patient perspectives), a score of 5 is given if ‘The clinician encourages the patient to elaborate on his or her concerns (e.g. using various continuation markers like aha, tell me more and please go on)’. A score of 3 is given if ‘The clinician does not interrupt the patient, but also does not display great interest in learning more (listens but does not encourage elaboration or further discussion)’. A score of 1 is given if ‘The clinician interrupts or cuts off the patient, when he or she attempts to elaborate (i.e. is obviously not particularly interested).’ Scores of 2 and 4 are used when the observed communication behavior falls between two defined anchor points [7]. Study sample and data collection Due to practical and economic considerations, we aimed to recruit ten general practitioners. (GPs) through a recruitment email, supplemented by personal invitations to colleagues with prior collaboration experience. Each GP was instructed to record 5–10 FTF and VCs during standard working hours. As shown in Fig. 1, the recording process differed for FTF and VCs. FTF were recorded using a standard tablet filming both physician and patient via a GDPR-compliant app, with recordings automatically stored on a secure cloud server in Denmark [25]. For VCs, GPs used a unique consultation ID and a third-party screen recording tool in a standard web browser, while patients participated via the MinLæge (‘MyDoctor’) app on their smartphones [26, 27]. Recordings from both modalities were subsequently transferred manually to a local server at the Research Unit for General Practice in Aarhus, Denmark, and simultaneously deleted from the cloud servers. Overview of the recording process of face-to-face and video consultations. Abbreviations: FTF = face-to-face consultations; VCs = video consultations GPs received approx. 30-min instruction in hardware and software use from CB and received financial compensation: 134€ for set-up and instructions, and 27€ per recorded consultation. Eligible patients were > 17 years old, fluent in Danish, unaccompanied, and not cognitively impaired. Prior to recording, preliminary verbal consent was obtained. The consultation began with the GP reading a standardized information statement, after which the patient’s verbal consent was recorded on video. Consent was recorded as part of the consultation; therefore, the reported duration includes the consent procedure. Rater training and rating process Ratings were performed by three physicians: an MD and PhD student, a GP with a PhD, and an internal medicine trainee. All had experience in rating scales in research and in teaching physician–patient communication to medical students. The three raters attended two 4-h training sessions led by a psychologist, associate professor and senior researcher with expertise in rating scales. Training consisted of watching and independently coding five test-recordings using the 4HCS, followed by group discussions to calibrate ratings and ensure a shared understanding of the scoring criteria. After completing training, raters were given access to all recordings and started rating. All consultations were assessed in the same randomized order by all raters. Initial ratings (Sample 1) were used to assess inter-rater reliability. A subset of consultations was re-rated after a minimum interval of two months to assess intra-rater reliability (Sample 2; see Fig. 2). The 4HCS rating scores were captured using an electronic survey software, with mandatory completion of each item, ensuring no missing values [28]. Exclusion of recorded face-to-face consultations and video consultations by ten general practitioners. Abbreviations: FTF = face-to-face consultations; VCs = video consultations Sample size As the 4HCS is intended for use with multiple raters in observational research contexts, our primary interest was the reliability of the average ratings rather than single-rater estimates. To determine an appropriate sample size, we conducted a simulation partly based on the estimates proposed by Bellier et al. [9]. We assumed an individual interrater reliability coefficient of 0.60 and a mean 4HCS total score of 76.44 (SD = 12.4). We decided on a conservative estimate of 5.5 for the between-rater standard deviation and kept the number of raters fixed at three. By deciding on a type 1 error rate of 5%, our simulation showed that a sample of 70 consultations would yield only 30% power to detect an individual interrater reliability exceeding 0.50. However, under the same assumptions, we would achieve 81.6% power to detect an average interrater reliability above 0.50. An interrater reliability of 0.50 is generally considered to represent fair agreement [29]. We therefore aimed to record at least 70 consultations for both consultation types. To assess intra-rater reliability while ensuring balanced representation of both consultation types, each rater reassessed 40 consultations (20 FTF and 20 VCs). Although this sample is smaller than that used in some previous studies (e.g., Bellier et al., 2020, who included 117 videos), our design ensured that repeated evaluations reflected both types of consultation while remaining feasible within the logistical constraints of the study [9, 30]. Statistical analysis For each recording, we calculated the 4HCS overall sum score, the subscale scores for the Four Habits, and scores for individual items. We reported the frequencies and percentages of responses at each anchor point (1 = floor effect, 3 = midpoint, and 5 = ceiling effect). We defined floor and ceiling effects as present if more than 15% of item scores were at the lowest or highest possible rating, respectively [31]. To assess inter- and intra-rater reliability, we calculated intra-class correlation coefficients (ICCs) using a two-way random-effects model with absolute agreement. This assumes that both raters and subjects are randomly sampled and allows generalization beyond the specific raters and subjects included. ICCs were calculated and reported for both single and average measures. However, as noted above, our main emphasis was placed on the average ICCs, as the 4HCS is primarily intended for research and structured assessment contexts in which final scores are based on the mean of multiple raters to enhance reliability [32]. ICCs were calculated for all 23 individual items, the Four Habits subscale scores, and the total sum score. For both inter- and intra-rater reliability, ICCs were interpreted as follows: < 0.40 poor, 0.40–0.59 fair, 0.60–0.74 good, and 0.75–1.00 excellent reliability. These thresholds were applied in line with established guidelines for reliability research and interpreted in relation to the intended use of the 4HCS as an observational research instrument [29]. All ICC-estimates were reported together with a 95% confidence interval. Intra-rater reliability was further assessed using Bland–Altman plots to examine potential systematic bias (e.g., rater drift) and limits of agreement between initial and repeated ratings [33]. All analyses were performed in Stata, version 18.0 (StataCorp LLC, College Station, TX, USA). Ethics The study (ID: 2292) was registered in the Record of Processing Activities at the Research Unit for General Practice, Aarhus, in accordance with the General Data Protection Regulation (GDPR). In line with Danish law, this type of observational study did not require assessment by an ethics committee, nor written patient consent [34]. However, to uphold ethical standards and participant autonomy, verbal informed consent was recorded from all patients whose consultations were included [35]. Results Translation The original English version of the 4HCS was translated into Danish with following cross-cultural adaptation principles, with only minor linguistic adjustments and no changes the conceptual content. This version was used in all subsequent rating procedures. The full Danish version is available as Supplementary Material (S1). General practitioners and recordings In April 2024, fourteen GPs expressed interest in participating. Due to practical constraints, including scheduling conflicts, four were not enrolled in the study. The remaining ten GPs (four males) received training in the use of the recording equipment. Of these, eight completed the requested number of consultations (five to ten each of FTF and VCs). One GP recorded eleven FTF consultations instead of the planned maximum of ten, and one GP withdrew after recording a single VC. As shown in Fig. 2, a total of 150 consultations were recorded: 78 FTF and 72 VCs. Eligible consultations involved patients aged 18 years or older, fluent in Danish, unaccompanied, and without cognitive impairment. Twelve recordings were excluded due to technical failures or not meeting the inclusion criteria. This resulted in 138 consultations eligible for rating: 73 FTF and 65 VCs. In the VC group, 42 patients were female and 23 were male, with a mean age of 40.3 years (range 18–68). In the FTF group, 40 patients were female and 33 were male, with a mean age of 52.5 years (range 18–88). The mean duration of the 73 FTF consultations was 12 min 09 s (range: 03 min 41 s—29 min 03 s). The mean duration of the 65 VCs was 10 min 09 s (range: 02 min 58 s to 30 min and 04 s). For both groups, the durations include the time spent recording consent. Ratings Between January and April 2025, the three trained raters each evaluated a total of 178 consultations with the 4HCS: The included 138 consultations (73 FTF and 65 VC) were rated for inter-rater reliability (Sample 1), and an additional 40 consultations (20 FTF and 20 VCs) were re-rated for intra-rater reliability (Sample 2) after a minimum interval of two months. As shown in Table 1, the mean 4HCS total score across all 138 recorded consultations was 79.80 (SD = 14.30; range: 41–109). For FTF consultations, the mean score was 79.68 (SD = 13.69, range: 43–109), while VCs had a mean of 79.94 (SD = 15.00, range: 41–109). Floor effects were observed for four items (6, 8, 12, and 22), while ceiling effects were present for all items except four (3, 6, 12, and 22). Across the three raters, mean total scores were 85.40 (SD = 12.56; range: 47–109) for Rater 1, 79.48 (SD 14.14; range 41–107) for Rater 2 and 74.53 (SD = 14.14; range: 43–108) for Rater 3. Table 1Descriptive statistics for the Danish Four Habit Coding Scheme, in total and stratified for consultation typeFTF, n = 73VCs, n = 65All consultations, n = 138Mean (SD)RangeMean (SD)RangeMean (SD)RangeFloor effect, n (%)Midpoint, n (%)Ceiling effect, n (%)Habit 1: Invest in the beginning19.71(3.50)9–2819.71(3.57)9–3019.71(3.53)9–30------ 1. Show familiarity3.40(1.32)3.72(1.13)3.55(1.25)31(7.5)130(31.4)130(31.4) 2. Greet warmly3.83(0.87)3.66(0.83)3.75(0.86)6(1.5)133(32.1)79(19.1) 3. Engage in small talk3.16(0.86)2.99(0.88)3.08(0.87)23(5.6)249(60.0)27(6.5) 4. Question style3.49(1.09)3.50(1.11)3.49(1.10)19(4.6)166(40.1)97(23.4) 5. Expansion of concerns3.83(1.08)3.75(1.11)3.79(1.10)16(3.9)96(23.2)130(31.4) 6. Elicit full agenda2.00(1.15)2.09(1.12)2.04(1.13)186(44.9)100(24.2)12(2.9)Habit 2: Elicit the patient’s perspective9.58(2.91)3–1510.05(2.99)3–159.80(2.95)3–15------ 7. Patient’s understanding3.48(1.11)3.64(1.08)3.56(1.10)27(6.5)140(33.8)91(22.0) 8. Goals for visit2.82(1.30)3.22(1.35)3.01(1.33)74(17.9)128(39.0)75(18.1) 9. Impact on life3.28(1.36)3.19(1.32)3.24(1.34)67(16.2)135(32.6)95(23.0)Habit 3: Demonstrate empathy13.47(3.71)4–2013.57(3.94)6–2013.52(3.82)4–20------ 10. Encourage emotional expression3.43(1.17)3.44(1.26)3.43(1.21)37(8.9)146(35.3)101(24.4) 11. Accept feelings3.14(1.29)3.19(1.30)3.16(1.29)59(14.3)152(36.7)87(21.0) 12. Identify feelings2.78(1.22)2.86(1.28)2.81(1.25)82(19.8)154(37.2)51(12.3) 13. Show good nonverbal behaviour4.13(0.99)4.08(0.94)4.11(0.97)4(1.0)72(17.4)180(43.5)Habit 4: Invest in the end36.91(6.51)18–5036.62(7.21)19–5036.77(6.85)18–50------ 14. Use patient’s frame of reference4.11(1.00)3.99(0.98)4.06(0.99)9(2.2)85(20.5)171(41.3) 15. Allow time to absorb3.54(1.02)3.48(1.07)3.51(1.04)13(3.1)182(44.0)95(23.0) 16. Give clear explanation4.28(0.83)4.30(0.86)4.29(0.84)0(0)62(15.0)211(51.0) 17. Offer rationale for tests4.18(0.88)4.12(0.93)4.15(0.90)5(1.2)93(22.5)183(44.2) 18. Test for comprehension3.33(1.13)3.43(1.29)3.38(1.21)46(11.1)139(33.6)83(2.1) 19. Involve in decision3.89(1.04)3.85(1.16)3.87(1.10)19(4.6)84(20.3)140(33.8) 20. Explore plan acceptability3.73(1.02)3.77(1.12)3.75(1.07)15(3.6)147(35.5)130(31.4) 21. Explore barriers3.08(1.21)3.23(1.20)3.15(1.21)48(11.6)152(36.7)68(16.4) 22. Encourage questions2.72(1.27)2.49(1.31)2.61(1.29)118(28.5)125(30.2)36(8.7) 23. Plan for follow-up4.04(1.24)3.96(1.26)4.00(1.24)31(7.5)76(18.4)209(5.5)Total79.68(13.69)43–10979.94(15.01)41–10979.80(14.31)41–109------Abbreviations: SD Standard deviation, FTF Face-to-face consultations, VCs Video consultations. Floor effect, midpoint and ceiling effect: Number and percentage of ‘anchor’ scores 1, 3 and 5, respectively. Each recording was independently rated by three different raters. Max possible total score is 115 Descriptive statistics for the Danish Four Habit Coding Scheme, in total and stratified for consultation type Abbreviations: SD Standard deviation, FTF Face-to-face consultations, VCs Video consultations. Floor effect, midpoint and ceiling effect: Number and percentage of ‘anchor’ scores 1, 3 and 5, respectively. Each recording was independently rated by three different raters. Max possible total score is 115 Inter-rater reliability Average inter-rater reliability was excellent for the total 4HCS score across all consultations (ICC = 0.75) and similarly strong when analyzed separately for FTF (ICC = 0.74) and VCs (ICC = 0.77) (see Table 2). Table 2Intra-class correlation coefficient estimates for both inter-rater reliability (stratified for consultation type) and intra-rater reliability of the Danish 4HCSAll consultations, n = 138FTF n = 73VCs, n = 65All consultations, n = 40Average inter-rater ICC (CI95%)Average intra-rater ICC (CI95%)Habit 1. Invest in the beginning0.69(0.58–0.77.58.77)0.71(0.57–0.81.57.81)0.66(0.49–0.78.49.78)0.73(0.56—0.83) 1. Show familiarity0.76(0.68–0.82.68.82)0.81(0.72–0.88.72.88)0.66(0.48–0.78.48.78)0.80(0.72—0.86) 2. Greet warmly0.60(0.46–0.70.46.70)0.64(0.48–0.77.48.77)0.52(0.27–0.69.27.69)0.66(0.49—0.77) 3. Engage in small talk0.42(0.23–0.56.23.56)0.50(0.26–0.67.26.67)0.32(0.02–0.55.02.55)0.54(0.34—0.68) 4. Question style0.57(0.40–0.69.40.69)0.56(0.35–0.71.35.71)0.57(0.34–0.73.34.73)0.74(0.62—0.82) 5. Expansion of concerns0.62(0.46–0.73.46.73)0.57(0.35–0.71.35.71)0.67(0.48–0.80.48.80)0.78(0.67—0.85) 6. Elicit full agenda0.30(0.09–0.47.09.47)0.31(0.03–0.53.03.53)0.30(−0.01−0.53)0.56(0.37—0.70)Habit 2. Elicit the patient’s perspective0.61(0.35–0.75.35.75)0.52(0.16–0.72.16.72)0.69(0.49–0.81.49.81)0.67(0.07—0.85) 7. Patient’s understanding0.53(0.37–0.65.37.65)0.48(0.25–0.66.25.66)0.58(0.38–0.73.38.73)0.61(0.16—0.79) 8. Goals for visit0.62(0.48–0.73.48.73)0.52(0.28–0.68.28.68)0.70(0.55–0.81.55.81)0.58(0.12—0.77) 9. Impact on life0.58(0.24–0.75.24.75)0.56(0.15–0.76.15.76)0.61(0.30–0.78.30.78)0.76(0.57—0.85)Habit 3. Demonstrate empathy0.63(0.48–0.73.48.73)0.61(0.41–0.74.41.74)0.65(0.45–0.78.45.78)0.74(0.30—0.87) 10. Encourage emotional expression0.61(0.34–0.76.34.76)0.59(0.30–0.75.30.75)0.64(0.33–0.79.33.79)0.66(0.23—0.82) 11. Accept feelings0.63(0.29–0.79.29.79)0.62(0.25–0.79.25.79)0.65(0.30–0.81.30.81)0.74(0.55—0.84) 12. Identify feelings0.68(0.55–0.77.55.77)0.74(0.61–0.83.61.83)0.60(0.37–0.75.37.75)0.70(0.47—0.81) 13. Show good nonverbal behaviour0.71(0.44–0.84.44.84)0.73(0.43–0.86.43.86)0.70(0.41–0.84.41.84)0.68(0.46—0.80)Habit 4. Invest in the end0.41(0.23–0.56.23.56)0.33(0.02–0.55.02.55)0.49(0.24–0.67.24.67)0.75(0.44—0.87) 14. Use patient’s frame of reference0.61(0.47–0.71.47.71)0.49(0.25–0.66.25.66)0.71(0.55–0.82.55.82)0.54(0.23—0.71) 15. Allow time to absorb0.35(0.14–0.52.14.52)0.44(0.18–0.63.18.63)0.25(−0.06−0.50)0.54(0.21—0.72) 16. Give clear explanation0.56(0.41–0.67.41.67)0.54(0.33–0.70.33.70)0.58(0.37–0.73.37.73)0.62(0.45—0.73) 17. Offer rationale for tests0.46(0.19–0.64.19.64)0.40(0.12–0.60.12.60)0.51(0.16–0.71.16.71)0.62(0.44—0.74) 18. Test for comprehension0.63(0.49–0.73.49.73)0.51(0.29–0.68.29.68)0.71(0.53–0.82.53.82)0.67(0.33—0.82) 19. Involve in decision0.62(0.50–0.72.50.72)0.66(0.50–0.77.50.77)0.59(0.39–0.74.39.74)0.62(0.39—0.76) 20. Explore plan acceptability0.52(0.37–0.65.37.65)0.48(0.24–0.65.24.65)0.57(0.36–0.72.36.72)0.71(0.52—0.81) 21. Explore barriers0.63(0.49–0.73.49.73)0.62(0.44–0.75.44.75)0.63(0.44–0.76.44.76)0.63(0.44—0.75) 22. Encourage questions0.81(0.75–0.86.75.86)0.78(0.67–0.85.67.85)0.85(0.77–0.91.77.91)0.79(0.70—0.86) 23. Plan for follow—up0.75(0.65–0.82.65.82)0.71(0.57–0.81.57.81)0.79(0.66–0.87.66.87)0.78(0.62—0.86)Total0.75(0.59–0.84.59.84)0.74(0.54–0.85.54.85)0.77(0.59–0.86.59.86)0.74(0.22—0.88)Abbreviations: ICC Intra-class correlation coefficient, CI Confidence interval, FTF Face-to-face consultations, VCs Video consultations Intra-class correlation coefficient estimates for both inter-rater reliability (stratified for consultation type) and intra-rater reliability of the Danish 4HCS Abbreviations: ICC Intra-class correlation coefficient, CI Confidence interval, FTF Face-to-face consultations, VCs Video consultations Scores for Habits 1–3 demonstrated good inter-rater reliability across all consultations (ICC range: 0.60–0.69), with fair to excellent estimates for both FTF and VCs (ICC range: 0.52–0.79). No consistent pattern emerged favoring either consultation type. In contrast, scores for Habit 4 (Invest in the End) showed poor inter-rater reliability for FTF (ICC = 0.33). At the item level, most ICC estimates ranged from fair to excellent (ICC range: 0.41–0.85). Using an ICC threshold of ≥ 0.60 (indicating at least good reliability), 14 of 23 items met this criterion across all consultations. Most of the remaining items demonstrated fair reliability, while two items, Item 6 (Elicit Full Agenda) and Item 15 (Allow time to absorb), demonstrated poor inter-rater reliability. Despite variability in ICC values between consultation types, no systematic trend was observed at the Habit or item level. Intra-rater reliability Average intra-rater reliability was good to excellent for the total score and all Four Habit domains across consultations (ICC range: 0.67–0.75). At the item level, all but four items (3, 6, 14, and 15) demonstrated fair to excellent reliability (ICC range: 0.54–0.80). To further assess intra-rater agreement, Bland–Altman plots were generated for each Habit (see Fig. 3). Across all Four Habits, re-ratings were generally higher than initial ratings, indicating possible rater drift over time. The broad spread of differences, especially for Habit 4, suggests notable variation at the individual rater level, reflecting both possible inconsistent measurement and lack of variance in scoring distribution in certain items. Intra-rater agreement: Bland–Altman plots for the Four Habits. Each Bland–Altman plot represents 120 dots (40 repeated ratings with three raters) of which some may overlap. Each dot represents the difference between first and second ratings by the same rater, plotted against their means. Solid lines indicate mean difference; dashed lines show 5th and 95th percentiles ICC estimates for both inter- and intra-rater reliability were consistently lower for individual measures compared to average measures. For example, the ICC for the total score increased from 0.50 for individual ratings to 0.75 when averaged across raters (full results for individual measures are presented in Appendix A). Discussion The 4HCS was translated and culturally adapted from English into Danish. We found nearly identical mean total scores across consultation types (FTF = 79.68; VCs = 79.94), but notable inter-rater variation of mean total scores ranging from 74.53 to 85.40. Average inter-rater reliability was excellent for the total 4HCS score. ICC estimates were 0.74 (CI = 0.54–0.85.54.85) for FTF consultations, 0.77 (CI = 0.59–0.86.59.86) for VCs, and 0.75 (CI = 0.59–0.84.59.84) across all consultations. At the Habit level, ICC estimates ranged from fair to good but were poor for Habit 4 in FTF consultations. At the item level, 14 of 23 items demonstrated at least good ICC estimates, while most of the remaining items were fair. Two items (6 and 15) showed poor ICC estimates. Intra-rater ICC estimates at the Habit level ranged from 0.67 to 0.75, with a total ICC of 0.74. These values fall within the upper range of good reliability and support the use of the 4HCS as an observational research instrument in general practice, enabling systematic assessment of physician–patient communication across raters and studies. Lower ICC values may reflect not only rater disagreement but also limited between-physician score variability, as ICC estimates depend on the variability between subjects relative to measurement error [30, 32]. When most physicians receive similar ratings, this reduces between-subject variance and may lead to lower ICC estimates despite reasonable agreement. The mean total score observed in our study (79.80) aligns closely with those reported by Bellier et al. (76.44) and Clayton et al. (83.1) but is notably higher than the scores reported by Krupat et al. (68.0) and Fossli Jensen et al. (60.10) [7, 9, 10, 36]. Three factors may explain this discrepancy. First, differences in rater training procedures and/or calibration criteria (i.e. continuously aligning scoring standards across raters) across studies could have led to variability in how the 4HCS scale was interpreted and applied. Second, rater characteristics may have played a role. As all raters in the present study were physicians, this could have introduced a ‘similar-to-me’ bias, wherein raters may be more lenient when evaluating clinicians whose communication style or professional context resembles their own [37, 38]. This may partly explain the relatively higher scores observed in our study as well as those reported by Bellier et al. and Clayton et al. Third, differences in study context may have influenced ratings. While Krupat et al. and Fossli Jensen et al. analyzed hospital-based consultations, Bellier et al. employed standardized actor scenarios, whereas Clayton et al. and the present study focused on general practice encounters [7, 10, 36]. These differences may affect how communication is expressed and evaluated, for example in general practice, where continuity of care and long-term relationships are more common. The inter-rater reliability for the total 4HCS score in our study (ICC = 0.75) is consistent with previous findings, which have reported average ICCs ranging from 0.69 to 0.82 [7, 9–11]. We used a two-way random effects model, which provides more conservative yet generalizable estimates by assuming that both raters and subjects are randomly sampled. In contrast, Scholl et al. used a two-way mixed effects model with fixed raters, which may yield higher but less generalizable estimates. However, their reported ICC (0.82) was only slightly higher than ours [11]. Our approach is therefore well suited to real-world applications involving different raters. The 4HCS demonstrated good reliability when applied to real-world general practice consultations, including both FTF and VCs. However, the use of real-world consultations in this study likely contributed to variability. While real-life data increases ecological validity, they also introduce greater variation in patient concerns, clinical complexity, and clinician communication styles. Although this variation may both support and challenge reliability, inter-rater agreement for the total 4HCS score was nevertheless excellent across both FTF and VCs (ICC = 0.74 and 0.77, respectively). Importantly, the similarity in ICC values between FTF and VCs suggests that the reliability of the 4HCS ratings appears comparable across consultation types, despite potential real-world differences in individual encounters. At the item level, ICCs varied considerably, with no consistent differences observed between FTF and VCs. This suggests that individual item ratings are influenced by rater’s interpretation rather than consultation modality. To our knowledge, this is the first study to apply the 4HCS to VCs in a general practice setting, supporting its applicability across both traditional and digital formats. Our findings indicate that the 4HCS can be used reliably to assess communication in VCs, despite potential differences in interaction style or technical setting. In addition, we successfully established the technical infrastructure required to capture and analyze a large volume of VCs during routine care. Our results suggest that in diverse clinical contexts, certain 4HCS items may lose discriminative utility or be applied inconsistently by raters. This was particularly evident for Item 3 (Engage in Small Talk) and 6 (Elicit Full Agenda) from Habit 1, both of which showed limited variance and reduced reliability in our study. Consistent with the general considerations outlined above, the lower ICC estimates for these items may reflect constrained variability in observed behaviors, for example due to limited contextual relevance or uniformly similar performance, as reflected in the observed floor effect and clustering around the midpoint for these items. Alternatively, they may reflect inconsistencies in how raters interpret and apply specific item criteria across different consultation types [39]. This may also reflect the relational context of general practice, where continuity of care and established doctor–patient relationships can influence how specific communication behaviors, such as small talk or agenda-setting, are expressed and evaluated. In this regard, observer-based communication assessment instruments face the inherent challenge of capturing and evaluating nuanced, relational, and context-dependent behaviors [40]. Even with a well-structured instrument like the 4HCS, subjective interpretation can vary considerably between raters. In our study, this was reflected in notable differences in total scores between raters, despite prior calibration efforts, and in low inter-rater reliability for specific items. These findings suggest that even trained raters may apply the instrument idiosyncratically, shaped by their individual perceptions of communicative subtleties [41]. This variability highlights the importance of employing multiple raters or consensus scoring in research contexts and suggests that the instrument is best suited for group-level analyses, while caution is warranted in high-stakes or summative assessments relying on single-rater evaluations [42]. In addition to inter-rater variation, we observed systematic changes over time, with higher mean Habit scores on re-ratings, suggesting a potential learning effect or scoring drift. This finding highlights the importance of not only initial rater training but also ongoing calibration, feedback, and reliability monitoring, as demonstrated in the study by Fossli Jensen et al. [10]. Without such measures, even well-constructed assessment instruments may yield less reliable scores at the individual level. While our study reflects real-world constraints with limited time and resources for extensive rater training, it is plausible that more comprehensive training combined with periodic feedback could have enhanced inter-rater consistency and minimized scoring drift over time. Practice implications The Danish version of the 4HCS demonstrates good to excellent reliability for assessing physician communication skills in general practice, across both FTF and VCs. The instrument appears well-suited for use in observational research settings, particularly when applied to group-level analyses, and may also have potential applications in clinical quality improvement and educational settings. However, ensuring robust rater training that targets lower-performing items may enhance scoring consistency. The observed drift in rater scoring over time suggests that temporal effects should be accounted for in future study designs and rating protocols. As the dimensionality of the Danish 4HCS was not formally examined, interpretation should primarily focus on reliability estimates. Future research is needed to determine whether the total score reflects a sufficiently coherent underlying construct and to formally validate the instrument's factor structure and construct validity within the Danish context, including examination of its relationship with patient-reported outcomes. Future studies could also explore whether inter-rater reliability varies across different clinician populations, such as medical students or early-career physicians, where greater between-subject variability may influence reliability estimates. Finally, with access to larger datasets, it would be relevant to explore the feasibility of a shortened version of the 4HCS that retains its core dimensions while preserving psychometric strength. Conclusion Eight hours of training in the use of the Danish version of the 4HCS yielded good average inter-rater reliability for the total score across both FTF consultations and VCs conducted in routine general practice. Most items demonstrated fair to excellent reliability. However, a few items showed lower reliability, likely due to limited score variability—stemming from restricted contextual relevance or uniformly high performance—as well as residual inconsistencies in how raters interpreted and evaluated nuanced communicative behaviors. Future training approaches may benefit from particular focus on lower-performing items to enhance scoring consistency. Notably, the reliability of the 4HCS was consistent across consultation types, despite potential differences in case type or complexity. These findings add to the growing body of evidence supporting the applicability of the 4HCS in varied clinical and technological contexts, underscoring its utility as a robust instrument for assessing physician communication skills in contemporary general practice. Supplementary Information Supplementary Material 1 Acknowledgements We are grateful to all participating general practitioners, and to Emil Vilstrup and Dennis Graversen for rating the consultations. We also thank Lone Niedziella for her valuable assistance with the translation process. Appendix A Intra-Class Correlation Coefficients (ICCs) for Inter- and Intra-Rater Reliability of the Danish Four Habits Coding Scheme (n = 138) Inter-rater ICC (n = 138) (CI 95%)Intra-rater ICC (n = 40) CI 95%IndividualAverageIndividualAverageHabit 1. Invest in the beginning0.420.32—0.520.690.58—0.770.580.38—0.710.730.56—0.83 1. Show familiarity0.520.42—0.610.760.68—0.820.670.56—0.760.800.72—0.86 2. Greet warmly0.330.22—0.440.600.46—0.700.490.33—0.620.660.49—0.77 3. Engage in small talk0.190.09—0.300.420.23—0.560.370.21—0.510.540.34—0.68 4. Question style0.300.18—0.420.570.40—0.690.580.45—0.690.740.62—0.82 5. Expansion of concerns0.350.22—0.470.620.46—0.730.640.50—0.740.780.67—0.85 6. Elicit full agenda0.130.03—0.230.300.09—0.470.390.23—0.540.560.37—0.70Habit 2. Elicit the patient’s perspective0.340.15—0.500.610.35—0.750.500.04—0.740.670.07—0.85 7. Patient’s understanding0.270.17—0.380.530.37—0.650.440.09—0.660.610.16—0.79 8. Goals for visit0.360.24—0.470.620.48—0.730.410.06—0.630.580.12—0.77 9. Impact on life0.320.09—0.500.580.24—0.750.610.40—0.750.760.57—0.85Habit 3. Demonstrate empathy0.450.21—0.630.710.44—0.840.580.18—0.770.740.30—0.87 10. Encourage emotional expression0.360.23—0.480.630.48—0.730.490.13—0.690.660.23—0.82 11. Accept feelings0.340.15—0.510.610.34—0.760.590.38—0.720.740.55—0.84 12. Identify feelings0.370.12—0.560.630.29—0.790.530.30—0.690.700.47—0.81 13. Show good nonverbal behaviour0.410.29—0.530.680.55—0.770.510.30—0.670.680.46—0.80Habit 4. Invest in the end0.500.32—0.640.750.65—0.820.600.28—0.770.750.44—0.87 14. Use patient’s frame of reference0.190.09—0.300.410.23—0.560.370.13—0.550.540.23—0.71 15. Allow time to absorb0.340.23—0.460.610.47—0.710.370.12—0.560.540.21—0.72 16. Give clear explanation0.150.05—0.260.350.14—0.520.450.29—0.580.620.45—0.73 17. Offer rationale for tests0.300.19—0.410.560.41—0.670.450.29—0.590.620.44—0.74 18. Test for comprehension0.220.07—0.370.460.19—0.640.510.20—0.690.670.33—0.82 19. Involve in decision0.360.24—0.470.630.49—0.730.450.24—0.610.620.39—0.76 20. Explore plan acceptability0.360.25—0.460.620.50—0.720.550.35—0.680.710.52—0.81 21. Explore barriers0.270.16—0.380.520.37—0.650.450.28—0.600.630.44—0.75 22. Encourage questions0.360.25—0.470.630.49—0.730.660.53—0.750.790.70—0.86 23. Plan for follow—up0.590.50—0.680.810.75—0.860.640.45—0.760.780.62—0.86Total0.500.32—0.640.750.59—0.840.590.12—0.790.740.22—0.88Abbreviations: CI Confidence interval, ICC Intraclass correlation coefficient. Inter-rater ICC is based on Sample 1, where each recording was rated once by each of three raters. Intra-rater ICC is based on Sample 2, where each recording was rated twice by each of the same three raters, with a minimum interval of two months between the two rating rounds (see Fig. 2) Abbreviations: CI Confidence interval, ICC Intraclass correlation coefficient. Inter-rater ICC is based on Sample 1, where each recording was rated once by each of three raters. Intra-rater ICC is based on Sample 2, where each recording was rated twice by each of the same three raters, with a minimum interval of two months between the two rating rounds (see Fig. 2) Authors’ contributions CEB, AFP, and UBK conceived and designed the study. CEB, AFP, and UBK developed the methodology. HSP and CEB performed validation of analyses, and CEB, AFP, UBK and HSP conducted formal data analysis and interpretation. CEB carried out data collection and management, with CEB and HSP contributing to data curation. The manuscript was drafted by CEB and critically revised by AFP, UBK, HSP, AB, EK, MLK, MBC, and LH. CEB prepared the visualizations. Supervision was provided by AFP, MBC, LH, HSP and UBK, and project administration was handled by CEB. Funding was obtained by CEB, AFP and UBK. Funding The study was funded by the Health Insurance 'danmark' Foundation [Grant no. 2022–0149 and 2020–0117] and by the Quality and Development Fund (KEU) of the Central Denmark Region [Grant no. 1–30-72–183-24]. The funding body did not influence any part of the scientific process. Data availability The datasets generated and analysed during the current study are not publicly available due to restrictions related to participant confidentiality and Danish data protection regulations, but may be available from the corresponding author upon reasonable request and pending ethical approval. Declarations Ethics approval and consent to participate The study (ID: 2292) was registered in the Record of Processing Activities at the Research Unit for General Practice, Aarhus, in accordance with General Data Protection Regulation. In line with Danish law (§14(2) of the Act on Research Ethics Review of Health Research Projects, Consolidation Act no. 1338 of 1 September 2020), observational studies not involving human biological material og interventions do not require approval from the regional or national ethics committees. Therefore, ethics approval was not required from The Danish National Committee on Health Research Ethics. Verbal informed consent was obtained from all participants. The study was conducted in accordance with the principles of the Declaration of Helsinki. Consent for publication Not applicable. Competing interests The authors declare no competing interests. Footnotes Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Supplementary Materials Supplementary Material 1 Data Availability Statement The datasets generated and analysed during the current study are not publicly available due to restrictions related to participant confidentiality and Danish data protection regulations, but may be available from the corresponding author upon reasonable request and pending ethical approval.