Nonparticipation in a Digital Health Intervention Study Among Older Adults: Uneven Involvement, Biased Outcomes, and the Effect of Weighting Authors: Arianna Poli, Ingemar Kåreholt, Susanne Kelfve, Katarina Berg, Andreas Motel-Klingebiel, Lewis A Lipsitz Published: 2024 ABSTRACT AbstractBackgroundThe involvement of older adults in research on digital health is uneven with respect to, for example, age, gender, health status, and digital skills. However, little is known regarding the effect of the uneven involvement of older adults in digital health research on researched outcomes. This study helps to fill this knowledge gap, identifies the effects of uneven involvement of older adults in digital health research on researched outcomes, and also develops a correction for this.MethodsData are retrieved from a digital health intervention for postoperative monitoring of people who underwent day surgery in Sweden. Based on field information on the recruitment process and researched outcomes for the intervention, this study (i) tested intervention effects by using 2 standard unweighted procedures in a sample of 281 individuals aged 50 years or older, and then (ii) used the information on participants, nonparticipants, and their respective probabilities to be involved in the intervention study to perform a weighting of the intervention effects for each step of selection and for the study group membership.ResultsThe intervention effects were found to be overestimated due to overrepresentation of groups that gained from receiving the intervention. No intervention effects were found after adjustment for participation bias.ConclusionsSelective participation of older adults in digital health research biases research outcomes and can lead to overestimation of intervention effects. Weighting allows researchers to correct and describe the effect of selective participation on researched outcomes.Keywords: Biased outcomes, Digital technologies, Exclusion, Inequalities, Participation in research Digitally based services are increasingly used in healthcare. It can be expected that access to and use of healthcare services will be progressively more dependent on the use of digital technologies by individuals (1,2). Digital health technologies enable, for example, tele-consultations, tele-monitoring, tele-rehabilitation, and electronic access to booking systems and medical records and can be used to supplement or replace traditional analog services (3). The amount of research and development regarding digital health technologies has been growing for several years, and a number of evaluations have been conducted to assess the utility of new digital health technologies for users. Older adults are one of the main user groups of healthcare services and, therefore, constitute a major target group for digital shifts in healthcare provision (2,4,5). However, digital participation is unequally distributed among older adults, and some groups of older adults can be especially disadvantaged by a digital transformation of healthcare provision (6–9). Previous studies have shown that nonuse and low use of digital technologies are found more often among groups of older adults with disadvantaged backgrounds or living circumstances, defined with respect to, for example, age, education, previous occupation, living settings, and health status (6,8,10–14). Groups of older adults with disadvantaged backgrounds tend to, for example, have more limited access to digital devices, have lower levels of digital skills, lack support for using and learning how to use digital technologies, and, as a result, use digital technologies less or less effectively than their counterparts do (5,7,15–18). In addition to these mechanisms, other factors have been identified as leading to digital exclusion and inequalities in old age in recent studies, including, for example, the uneven involvement of older adults in research on digital health (19–22). The participation of older adults in digital health research has been found to be selective. Some groups of older adults are systematically less likely to participate in digital health research compared with others (21,23,24). These groups are typically older and have lower socioeconomic status, poorer health status, and poorer digital skills than those who are involved in digital health research (19,20,22,25,26). Selective participation in research on digital health is created in the course of the recruitment process of study participants through 3 successive steps of selection: prescreening, recruitment, and self-selection. All of these selection steps have been analyzed in previous works (19,20,22). In these studies, it was found that each of these steps of selection results in the participation of some groups of older adults and in the nonparticipation of others and eventually defines a group of participants that can differ significantly from those who do not participate (19,22). The underrepresentation of some groups of older adults and the overrepresentation of others in studies evaluating digital health technologies are likely to influence the research results (21,23,27). In particular, the observed associations between intervention exposures and outcomes may represent an overestimation or an underestimation of the intervention effects within the overall population of interest due to an unbalanced representation of different groups of older adults in the sample (28–30). An overestimation of intervention effects would indicate that the intervention was less effective among the individuals who were less likely to participate compared with those who were more likely to participate. An underestimation of the effects would suggest that those who were less likely to participate gained on average more from the intervention than those who were more likely to participate. In both cases, the results can guide inadequate digital health strategies that overlook the needs of those groups, which are less likely to be involved in digital health research. It is therefore of crucial importance to understand and address the effect of selective participation of older adults in studies evaluating the effect of digital health technologies on researched outcomes. To our knowledge, no previous studies have addressed this issue. This paper contributes to filling this knowledge gap. Building on previous work, this study aims at identifying the effects of uneven involvement of older adults in digital health research on researched intervention outcomes, as well as developing corrections for this bias. This investigation is conducted in the context of an intervention study evaluating a mobile phone system for postoperative monitoring of individuals after undergoing day surgery, that is, “Mobile Phone in Recovery after Ambulatory Surgery” (MIRAS). We hypothesize that selective participation of older adults leads to an overestimation of the effects of the MIRAS intervention. Such overestimation would be due to the underrepresentation of those groups, which are less likely to participate in digital health research and tend to show lower benefits from the intervention. Adjusting for the underrepresentation of different groups within the targeted population of interest by weighting procedures can give more accurate estimations of the intervention effects among targeted individuals. Method The data for the present study was gathered between 2018 and 2020 in the context of the MIRAS intervention study. Beyond the intervention study, an analysis of the involvement of older adults in the evaluation of the mobile phone system was performed. The Intervention MIRAS was designed as a randomized controlled trial with 2 groups, an intervention group and a control group. Individuals who were eligible and accepted to participate in the intervention were randomized between the 2 groups. The control group received the traditional postoperative face-to-face monitoring at the day surgery department. The intervention group was provided with a data system installed on their own mobile phones, which performed postoperative monitoring. Individuals in this group were invited to answer short monitoring questions on their recovery progress once a day during the first 2 weeks after the day surgery and once a week in Weeks 3 and 4 after day surgery. Both the control and the intervention groups were assessed at baseline (ie, on the same day of the surgery) and after 28 days from the day of surgery (ie, on the day of the postsurgical follow-up visit) through self-administered paper questionnaires. The recruitment for the MIRAS intervention study was conducted at 2 Swedish public day surgery units. MIRAS’ recruitment targeted individuals who underwent day surgery in the fields of orthopedics, general surgery, including breast-cancer operations, and urology, during the MIRAS recruitment timeframe. Recruitment was conducted on the day of surgery by a research nurse. Participants to the MIRAS study who were randomized to the intervention group were registered into the monitoring system by the research nurse and received an SMS on their own mobile phones with a link for the system installation and information on its use and data protection (eg, GDPR-related information). This was done on the day of the surgery. By clicking on the link, installation of the monitoring system started, and the patient could receive the MIRAS’s monitoring questions as scheduled. The study criteria for being eligible to participate in MIRAS were the following: (i) having a mobile phone with an internet connection (no requirements for specific mobile operating systems); (ii) being able to speak and comprehend Swedish; (iii) not showing or reporting health problems that could interfere with participating in the intervention (eg, aphasia, dementia); (iv) being older than 18 years of age. MIRAS was granted ethical approval by the competent ethical board (dnrs: 2014/422–31 and 2016/474–31). Involvement of Older Adults in the Intervention To study the effect of the involvement of older adults in the MIRAS intervention study on the outcomes researched for MIRAS, we employed the multistep conceptual model for the study of selective participation described in a previous work (22). The conceptual model differentiates 3 steps of selection (ie, prescreening, recruitment, and self-selection) through which a group of study participants is selected within a certain targeted population. In prescreening, some individuals from the targeted population are excluded before being screened with respect to the study criteria, often with no documented reasons for such exclusion. This may happen, for example, due to the absence of the candidate at the time of the screening, the absence of the recruiter, or the assumed ineligibility of an individual by the recruiter. Those individuals who are not excluded in the prescreening step undergo an eligibility check during which they can be recruited or deemed as ineligible and, thus, excluded (ie, the recruitment step). Those individuals who are recruited are invited to participate in the intervention study and can either accept or decline the invitation (ie, the self-selection step). We investigated the effect that each of these selection steps had on the MIRAS results. Data The sample for the present study consists of a subsample of the MIRAS’s sample, which includes all individuals aged 50 years and older who underwent day surgery at the 2 surgery units involved in the MIRAS recruitment in 2018 (N = 717). The sample comprises both individuals who participated in the MIRAS intervention study and those who did not because of prescreening selection, ineligibility (ie, non-recruited individuals), or invitation refusal (ie, decliners). The mean age of the sample was 65 years (SD = 9.6). More than half of the individuals (n = 375) were 65 years old or older. About a third of the individuals were men (n = 256), and about two thirds were women (n = 461). Outcome Measures for the MIRAS Intervention Study The effects of the studied intervention were assessed on 4 main outcome measures: health-related quality-of-life, general health status, postsurgical recovery, and sense of security. Health-related quality-of-life was measured through the EuroQoL (EQ5D) instrument before and after the intervention (31). The instrument consists of 2 parts. The first part includes 5 items investigating aspects of quality-of-life related to mobility, self-care, daily activities, pain or discomfort, anxiety, or depression. Each item allows 3 possible answers: no problem, some problems, or severe problems. The answers to the 5 items were combined in a final score using the scoring STATA command EQ5D, which is based on estimates and modeling of the United Kingdom general population conventionally used as a default population (32,33). The outcome variable was constructed as the difference between the EQ5D score measured at post-intervention and the EQ5D score assessed before the intervention. General health status and postsurgical recovery were measured by the post-discharge Surgical Recovery instrument (PSR) (34,35). The PSR instrument includes a total of 12 items, divided into 2 sets and answered on a 10-point scale. The first set of items comprises 8 questions that cover aspects related to health status, daily activities, and fatigue and were administered before and after the intervention. The answers to these items were combined into 2 final scores, 1 pre-intervention and 1 post-intervention, corresponding to the sum score of the items divided by the maximum possible score and multiplied by 100. The scoring method was retrieved from a previous study (34). Based on the pre- and post-intervention scores, we constructed an outcome variable named “general health status,” which takes as its value the difference between the post-intervention score and the pre-intervention score. The “general health status” variable ranges between 10 (ie, very poor health status) and 100 (ie, very good health status). The second set of PSR items includes 4 questions that assess the overall level of postsurgical recovery (eg, work ability, recovery quality) and were administered only after the intervention. By combining the scores from the first set of 8 questions collected post-intervention with this second set of 4 items, we created the outcome variable called “postsurgical recovery,” expressing the health status and the level of postsurgical recovery after the MIRAS intervention. The variable is constructed as the sum score of the 12 items administered after the intervention divided by the maximum possible score and multiplied by 100. Postsurgical recovery ranges between 10 (ie, very poor recovery) and 100 (ie, very good recovery). The scoring was performed according to the procedures used in a previous work (34). Individual sense of security was measured by 3 items of the Sense of Security in Care – Patients’ Evaluation (SEC-P) instrument (36). The items were: “Do you feel confident that you can handle your daily life?,” “Do you feel you have control over your situation?,” and “Do you feel healthcare personnel have adequately informed you about what to expect in your care?.” Based on the scores from the 3 items, we constructed an outcome variable called “Sense of security” taking the mean of the 3 items as its values. For this calculation, we used all the individuals who answered at least 2 items out of 3. Weight Measures for Selective Participation To measure the effect of selective participation on the MIRAS’ intervention effects, we constructed 4 weight variables representing 4 conditions, namely the 3 steps of selection (ie, being screened, being eligible, being participant) as described in a previous work (22) and the randomization into the control or the intervention group. The weight variables are calculated as the inverse of the probability of being screened (Weight Variable 1), being eligible (Weight Variable 2), being a participant (Weight Variable 3), and being sorted into either the control or intervention group (Weight Variable 4), based on age and gender. Weights are centered at 1 on average (see Supplementary Table 1 presenting descriptive statistics for the 4 weights). The weight variables range from 0.63 to 3.56. Each weight variable is meant to compensate for unequal selection into one of the 4 conditions (ie, being screened, being eligible, being participant, randomization) by reducing or increasing the relevance of the observations in that specific condition depending on proportions of participation by age and gender. This is done by assigning each observation a weight that is inversely proportional to the probability of being in that condition. Analyses Descriptive statistics are computed to describe MIRAS participants and nonparticipants in terms of age and gender. Pearson’s Chi-square tests and one-way ANOVA are computed to test differences between participants and the 3 groups of nonparticipants, namely the non-screened, the ineligible, and the decliners, with respect to age and gender. Further descriptive statistics are computed to characterize study participants in the control and in the intervention groups by age and gender. Pearson’s Chi-square tests and Student’s t-tests are computed to test differences among participants with respect to age and gender. MIRAS’ intervention effects are tested by using 2 standard unweighted procedures and then adjusting for participation biases through a weighting procedure. For the first standard procedure, we ran 4 linear regression models to compare control and intervention groups on the 4 outcomes, namely EQ5D, “general health status,” “postsurgical recovery,” and “sense of security,” controlling for age and gender. In the second standard procedure, we ran the same 4 linear regression models and controlled for age, gender, and the propensity score of being a participant. To adjust for the effect of selective participation on the MIRAS’ intervention effects, we included the weight variables one by one in the 4 linear regression models by using the STATA command for probability weights, pweight. In this way, individuals in the control and in the intervention group were compared on the 4 outcomes (ie, EQ5D, “general health status,” “postsurgical recovery,” and “sense of security”), but attributing different weights to different individuals on the effects. We controlled for age and gender in all the models. Age was used as a categorical variable (ie, 50–59, 60–64, 65–69, and 70 years and older) to capture possible nonlinear relationships. Gender was treated as a binary variable. Finally, because the proportions of men and women participating and not participating in the MIRAS intervention study were different, separate analyses for men and women were made to study the effects of the intervention on the 2 gender groups. Two models were used. In the first model, we compared the control and the intervention groups on the 4 outcomes controlling for age. In the second model, we include a weight variable corresponding to the step of selection that had an effect on the results based on the first set of analyses. All analyses were performed using STATA 15 (37). Results Descriptive Results A total of 717 individuals were scheduled for day surgery in the 2 recruitment units during the MIRAS’ recruitment timeframe. Of these, 281 participated in the intervention study (39%), whereas 436 (61%) were nonparticipants (Table 1). Among nonparticipants, thirteen percent of the individuals did not participate due to non-screening (N = 95), ten percent of the individuals were not recruited due to ineligibility according to the study criteria (N = 68), and 38% of the individuals were nonparticipants through refusing the invitation to participate in the intervention study (N = 273). Regarding those individuals who were ineligible based on the study criteria, the most often registered reason for ineligibility was not having a mobile phone (N = 44) (including, e.g., not having a own mobile phone, having a malfunctioning or too old mobile phone, not having access to the internet), followed by not being able to speak Swedish (N = 16) and having health conditions that could interfere with taking part in the study (N = 8). No one was excluded because of the age criterion. Reasons for non-screening and refusal were not registered. Table 1.Main Characteristics of the Participants and Nonparticipants in the MIRAS Intervention studyTotal SampleNonparticipantsStudy Participants X 2/F p ValueNon-ScreenedIneligibleDeclinersGroup size (N (row %))717 (100)95 (13.2)68 (9.5)273 (38.1)281 (39.2)AgeMean (SD)65 (9.6)67 (9.6)73 (11.1)67 (9.3)62 (7.7)33.63 .000 †,* N (column %)Age groups50–59223 (31.1)23 (24.2)10 (14.7)65 (23.8)125 (44.5)79.35 .000 * 60–64119 (16.6)13 (13.7)5 (7.4)45 (16.5)56 (19.9)65–69118 (16.5)15 (15.8)7 (10.3)54 (19.8)42 (15.0)70+257 (35.8)44 (46.3)46 (67.7)109 (39.9)58 (20.6)GenderMen256 (35.7)49 (51.6)27 (39.7)99 (36.3)81 (28.8)16.73 .001 * Women461 (64.3)46 (48.4)41 (60.3)174 (63.7)200 (71.2)Men50–5990 (12.6)15 (15.8)3 (4.4)29 (10.6)43 (15.3)105.96 .000 * 60–6429 (4.0)1 (1.1)1 (1.5)13 (4.8)14 (4.5)65–6936 (5.0)7 (7.4)3 (4.4)15 (5.5)11 (3.9)70+101 (14.1)26 (27.4)20 (29.4)42 (15.4)13 (4.6)Women50–59133 (18.6)8 (8.4)7 (10.3)36 (13.2)82 (29.2)60–6490 (12.6)12 (12.6)4 (5.9)32 (11.7)42 (15.0)65–6982 (11.4)8 (8.4)4 (5.9)39 (14.3)31 (11.0)70+156 (21.8)18 (19.0)26 (38.2)67 (24.5)45 (16.0) *Bold values indicate p < .05. †One-way ANOVA was calculated. Main Characteristics of the Participants and Nonparticipants in the MIRAS Intervention study *Bold values indicate p < .05. †One-way ANOVA was calculated. The non-screening rate (ie, the percentage of non-screened individuals based on all individuals who were scheduled for one of the eligible day surgeries) was 13%, but the ineligibility rate (ie, the percentage of ineligible individuals based on all individuals who were screened for participating in MIRAS) was 11%, and the refusal rate (ie, the percentage of decliners based on all individuals who were invited to participate in MIRAS) was 49%. Participants were significantly younger (62 years old on average) than the individuals who were non-screened (67 years old on average), ineligible (73 years old on average), and decliners (67 years old on average) (p < .001). Individuals between the ages of 50 and 59 years were more often participants (45%) than nonparticipants (non-screened: 24%; ineligible: 15%; decliners: 24%) (p < .001). Individuals between the ages of 60 and 64 years were more often study participants (20%) than non-screened (14%), ineligible (7%) and decliners (17%). On the contrary, individuals who were 70 years old and older represented a larger share among the nonparticipants (in the non-screened group, 46%; in the ineligible group, 68%; in the declining group, 40%) than among the participants (21%) (p < .001). The participant group included significantly more women (71%) than the nonparticipant group (in the non-screened group, 48%; in the ineligible group, 60%; in the declining group, 64%) (p < .05). The 281 participants were randomly allocated to the intervention group (n = 139) and the control group (n = 142). Table 2 shows the main baseline characteristics of the MIRAS’ participants. Overall, individuals in the intervention and control groups did not differ statistically by age and gender. Table 2.Main Characteristics of the MIRAS Intervention ParticipantsParticipants X 2/t p ValueTotal (N = 281)Intervention Group (n = 139)Control Group (n = 142)AgeMean (SD)62 (7.7)62 (7.5)61 (8.0)−0.58.563* N (column %)Age groups 50–59125 (44.5)60 (43.2)65 (45.8)4.77.189 60–6456 (19.9)27 (19.4)29 (20.4) 65–6942 (15.0)27 (19.4)15 (10.6) 70+58 (20.6)25 (18.0)33 (23.2) missing000Gender Men81 (28.8)42 (30.2)39 (27.5)0.26.611 Women200 (71.2)97 (69.8)103 (72.5) missing000 Note: MIRAS = Mobile Phone in Recovery after Ambulatory Surgery. *Mean comparisons were executed through calculating Student’s t-test. Main Characteristics of the MIRAS Intervention Participants Note: MIRAS = Mobile Phone in Recovery after Ambulatory Surgery. *Mean comparisons were executed through calculating Student’s t-test. Unweighted and Weighted Intervention Effects Unweighted intervention effects Intervention effects are reported in Table 3 (descriptive results) and Table 4 (regression model results). Undergoing the intervention was associated with better “postsurgical recovery” (β = 6.50; p = .005), compared with being in the control group (Table 4). The intervention was associated with a higher SEC-P, significantly at the 10% level (β = 0.24; p = .054). No intervention effects were observed on the other 2 outcomes, namely EQ5D and “general health status.” The same intervention effects were found when controlling for the propensity score of being a participant. Table 3.Description of the Outcomes for the 2 Groups of ParticipantsParticipants (N = 281) t p Value†Intervention Group (n = 139)Control Group (n = 142)Health-related quality of life (EQ5D), pre–post change (mean [SD])−0.01 (0.19)0.03 (0.18)1.57.118General health status, pre–post change (mean [SD])7.49 (18.25)4.27 (16.30)−1.42.158Postsurgical recovery, at Day 28 (mean [SD])78.59 (17.53)71.68 (17.94)−3.00 .003 * Sense of security, at Day 28 (mean [SD])5.09 (0.89)4.86 (1.02)−1.85.065 *Bold values indicate p < .05. †All mean comparisons were executed through calculating by Student’s t-test. Description of the Outcomes for the 2 Groups of Participants *Bold values indicate p < .05. †All mean comparisons were executed through calculating by Student’s t-test. Table 4.Unweighted and Weighted Intervention EffectsHealth-Related Quality of Life, Pre–Post Change (EQ5D)General Health Status, Pre–Post ChangePostsurgical Recovery, at Day 28Sense of Security, at Day 28Coeff. p 95% CICoeff. p 95% CICoeff.p95% CICoeff.p95% CI Simple linear regression model, controlled by age, gender (standard procedure 1) † Study group Control (ref.) 0.00 0.00 0.00 0.00 Intervention−0.04.100−0.090.012.83.216−1.677.346.50 .005 * 1.9411.060.24.0540.000.49 Simple linear regression model, controlled by age, gender and propensity score of being participant (standard procedure 2) † Study group Control (ref.) 0.00 0.00 0.00 0.00 Intervention−0.04.102−0.090.012.84.216−1.677.356.50 .005 * 1.9511.060.24.0540.000.49 Weighting for selective participation Step 1. Linear regression model weighted by the inverse of the probability of being screened by age and gender (ie, prescreening selection) † Study group Control (ref.) 0.00 0.00 0.00 0.00 Intervention−0.04.126−0.080.012.62.265−2.007.246.20 .009 * 1.5810.810.23.068−0.020.48 Step 2. Linear regression model weighted by the inverse of the probability of being eligible to participate by age and gender (ie, recruitment) † Study group Control (ref.) 0.00 0.00 0.00 0.00 Intervention−0.03.168−0.080.012.40.319−2.347.155.92 .014 * 1.2110.630.23.079−0.030.48 Step 3. Linear regression model weighted by the inverse of the probability of being willing to participate by age and gender (ie, self-selection) † Study group Control (ref.) 0.00 0.00 0.00 0.00 Intervention−0.01.722−0.060.040.82.783−5.066.704.37.138−1.4110.160.17.284−0.140.49 Step 4. Linear regression model weighted by the inverse of the probability of being willing to participate by age and gender (ie, self-selection) and study group membership † Study group Control (ref.) 0.00 0.00 0.00 0.00 Intervention−0.01.728−0.060.041.07.73−5.017.154.37.143−1.4810.230.17.292−0.150.50 *Bold values indicate p < .05. †All models were estimated controlling for age and gender. Unweighted and Weighted Intervention Effects *Bold values indicate p < .05. †All models were estimated controlling for age and gender. Intervention effects weighted by the inverse of the probability of being included in the study The effects of the intervention were adjusted for selective participation by performing a weighting for the inverse of the probability of being a study participant. The effect of the 4 studied conditions (ie, the 3 steps of selection and the randomization into the intervention group or the control group) on the intervention effects was calculated one at a time to check for the specific effect of each condition on the intervention effects. In weighting Step 1 (based on the inverse of the probability of being screened), the intervention effects on “postsurgical recovery” for the study participants compared with the control group remained statistically significant but were slightly reduced (β = 6.20; p = .009). The intervention effects on “sense of security” stayed unchanged and significant at the 10% level (β = 0.23; p = .068). In weighting Step 2 (based on the inverse of the probability of being eligible), the effects of the intervention on “postsurgical recovery” in the intervention group were further reduced but remained significant (Step 1: β = 6.20; p = .009 vs Step 2: β = 5.92; p = .014). The effects of the intervention on the ”sense of security” among participants remained stable compared with what obtained at Step 1 of weighting and also remained significant at the 10% level (Step 1: β = 0.23; p = .068 vs β = 0.23; p = .079). By weighting for the inverse of the probability of being willing to participate (ie, weighting Step 3), no significant intervention effects were found. The intervention was no longer significantly associated either with better “postsurgical recovery” or with a higher “sense of security.” When weighting for both self-selection and study group membership (ie, weighting Step 4), intervention effects remained unchanged compared with Step 3. Unweighted and Weighted Intervention Effects by Gender Unweighted intervention effects by gender Among women, undergoing the intervention was significantly associated with better “postsurgical recovery” (β = 7.64; p = .005) and with a higher “sense of security” (β = 0.32; p = .025), compared with the control group (Table 5). Also, the intervention was associated with a negative change in EQ5D compared with the baseline assessment; the association was marginally significant (β = −0.05; p = .050). No statistically significant effects of the intervention were observed among men. Table 5.Unweighted and Weighted Intervention Effects for Women and MenHealth-Related Quality-of-Life, Pre–Post Change (EQ5D)General Health Status, Pre–Post ChangePostsurgical Recovery, at Day 28Sense of Security, at Day 28Coeff.p95% CICoeff.p95% CICoeff.p95% CICoeff.p95% CI WOMEN. Simple linear regression model weighted by age (standard procedure) † Study group Control (ref.) 0.00 0.00 0.00 0.00 Intervention−0.05.050−0.110.003.09 .23 −1.988.177.64 .005 * 2.3912.880.32 .025 * 0.040.60 WOMEN. Linear regression model weighted by the inverse of the probability of being willing to participate by age (ie, self-selection) † Study group Control (ref.) 0.00 0.00 0.00 0.00 Intervention−0.04.084−0.090.013.28.231−2.118.676.74 .011 * 1.5711.920.34 .018 * 0.060.62 MEN. Simple linear regression model weighted by age (standard procedure) * Study group Control (ref.) 0.00 0.00 0.00 0.00 Intervention−0.01.831−0.110.090.46.924−9.2210.143.16.519−6.5912.920.00.997−0.530.53 MEN. Linear regression model weighted by the inverse of the probability of being willing to participate by age (ie, self-selection) † Study group Control (ref.) 0.00 0.00 0.00 0.00 Intervention0.03.593−0.070.12−2.04.748−14.6710.591.31.831−10.9413.56−0.07.828−0.750.61 *Bold values indicate p < .05. †All models were controlled by age. Unweighted and Weighted Intervention Effects for Women and Men *Bold values indicate p < .05. †All models were controlled by age. Intervention effects for women and men weighted by the inverse of the probability of being willing to participate in the MIRAS intervention study Weighting for the inverse of the probability of being willing to participate in MIRAS by age resulted in an absence of significant effects of the intervention on EQ5D and in a decreased effect of the intervention on the PSR12, although the latter effect remained significant (standard procedure: β = 7.64; t = 2.87; p = .005 vs weighted model: β = 6.74; t = 2.57; p = .011). The effect on “sense of security” remained stable and significant (β = 0.32; p = .025 vs weighted model: β = 0.34; p = .018). Estimating the weighted models did not yield statistically significant effects among men on any of the 4 outcomes. Discussion In this study, we described and identified a way to correct for the effect of the uneven involvement of older adults in an intervention study evaluating the effects of using a mobile phone system for postoperative monitoring of individuals after day surgery, that is, the MIRAS intervention study. We demonstrated the applicability and the utility of a weighting procedure for correcting biased intervention outcomes and identifying how outcomes are impacted by selective participation of older adults in digital health research. Based on the multistep conceptual model of selective participation in research described in previous works (19,20,22), we were able to correct for over and underrepresentation of groups of individuals in the study sample and to explore which steps of selection in the recruitment process affect intervention outcomes and in which way. We have shown that weighting allows for correcting and more accurately assessing the effect of selective participation on intervention outcomes. Although the MIRAS intervention had a positive effect on the level of postsurgical recovery and individuals’ sense of security after the surgery, we demonstrate that these effects are overestimated due to selective participation in the MIRAS study. The observed association between the MIRAS intervention exposure and the level of postsurgical recovery and the individual sense of security outcomes is driven by an overrepresentation of those groups of individuals who gained from receiving the MIRAS intervention. Indeed, adjusting for such overrepresentation leads to an absence of overall intervention effects. This is in line with our hypothesis and with similar studies in other fields (28,29). We ascertained which step of selection contributed to the overestimation of intervention effects. In the MIRAS intervention study, it is especially the step of self-selection that decreases the representation of those groups with smaller or no benefits from the intervention and, in turn, overincreases the effect of their counterpart groups. In this way, the positive intervention effects among the overrepresented group become predominant in the results over the limited or null intervention effects among the underrepresented groups. We indeed show that men were less likely to accept to participate compared with women in the MIRAS intervention study, and that men also were more likely to not experience any effects of the intervention (22). On the contrary, the MIRAS intervention has significant positive effects among women, which is the overrepresented group, regarding postsurgical recovery and individuals’ sense of security. In the light of our results, we argue that weighting can be used for making research results better reflect the diversity of experiences of older adults with digital health technologies and, in this way, to more accurately inform digitalization strategies in healthcare. In a previous work, it was shown that underrepresented groups were more likely to be older, to be men, to have lower socioeconomic status, to have poorer health status, and to have lower digital competence (22). Overestimating the benefits of digital technologies among these groups of older adults can misguide digitalization strategies towards promoting innovations that may actually be disadvantageous for some groups of older adults. In this respect, selective participation in digital health research can represent an inequality mechanism through which older adults with disadvantaged backgrounds are, first, less likely to be involved in relevant research and then, next, more likely to be exposed to digitally based services that do not suit them. Our study has some limitations. First, limited information was available on nonparticipants, which restricted our ability to characterize the nonparticipants compared with participants. This also leads to the second limitation. We could base the weigh calculations only on age and gender. Further factors could have been relevant for predicting nonparticipation, but available information was too limited or inadequate for performing such analyses. Although information on surgery unit and type of surgery was available, this data was confounded with gender because 2 out of the 3 assessed types of surgery typically involved either women (ie, breast cancer surgery) or men (ie, urology surgery) and were each performed at a distinct surgery unit. Based on this fact, we decided to employ only gender in the analyses, which cover institutional differences also. Third, as a consequence, although weighting is a commonly applied procedure for accounting for nonresponse and selection bias (38), the weighted effects that are presented in this paper should be considered cautiously as an approximation of how results would look like if selection bias were removed. Fourth, we showed that self-selection was most important in contributing to the overestimation of the intervention effects. However, the reasons for declining to participate were not registered during recruitment and thus not amenable to analysis. Taking refusal reasons into account would have allowed a more detailed description of the mechanisms behind refusal to participate in MIRAS. Overall, this study nevertheless gives solid indications that selective participation prevents some groups of older adults from influencing researched outcomes in ways that guide innovations in healthcare service provision. Selective participation should be approached as a mechanism possibly contributing to unequal digital inclusion among older adults. Our findings show that digital technologies that are found to be effective among highly selected groups of older participants can be less effective or ineffective for other groups. As we show in this study and in accordance with previous works, the groups who are more often excluded from intervention studies can be those who derive the least benefit from these interventions and face higher risks of exclusion from related services (19,22). In times of rapid digital transformation in healthcare, this exclusivity from new digital-based services can exacerbate social and health inequalities in old age (4,22). It is therefore essential to improve inclusivity of intervention studies and to analyze systematically how the effects of innovative digital-based interventions are distributed among groups of older people. Digital health research should invest more efforts in making intervention studies more inclusive regarding groups of older adults who are more likely to be underrepresented. This includes taking into account that digital access, use, and skills are not equally distributed among older adults. As we showed in this paper, requiring digital access to participate in intervention studies leads to ineligibility and exclusion of some groups of individuals from the research and is likely to influence self-selection as well (due to poor digital skills). These problems could be addressed, for example, by providing devices and internet access to individuals who do not have digital access and would otherwise be unable to participate, offering support to those who decline the invitation to participate due to poor digital skills, or involving older people’s organizations that can voice the needs of underrepresented groups. Moreover, digital health research could focus part of its recruitment processes on groups of older adults, which are known to be underrepresented more often in intervention studies. Finally, research should systematically incorporate selective participation investigations, collecting both recruitment information and individual perspectives on (non)participation, to allow researchers having a greater knowledge of their target populations to implement needed adjustments for the possible effect of selective participation on intervention results. When selective participation occurs, weighting can be an appropriate procedure for making research results more meaningful and informative, strengthening the role of research in enabling the identification of actual suitable digital-based interventions and the definition of successful digital inclusion strategies for older adults. Supplementary Material Contributor Information Arianna Poli, Division of Ageing and Social Change, Department of Culture and Society, Linköping University, Norrköping, Sweden. Ingemar Kåreholt, Institute of Gerontology, Jönköping University, Jönköping, Sweden; Aging Research Center, Karolinska Institutet and Stockholm University, Stockholm, Sweden. Susanne Kelfve, Division of Ageing and Social Change, Department of Culture and Society, Linköping University, Norrköping, Sweden. Katarina Berg, Department of Health, Medicine and Caring Science, Linköping University, Linköping, Sweden. Andreas Motel-Klingebiel, Division of Ageing and Social Change, Department of Culture and Society, Linköping University, Norrköping, Sweden. Funding This work was supported by the Swedish Research Council for Health, Working Life and Welfare (FORTE) (reference number 2014-4100). It was conducted as part of “Supporting Self-care by Information and Communication Technology for Older People with Long-term Conditions” (ICT4SELFCARE). Conflicts of Interest None. Author Contributions A.P., S.K., and A.M.-K. conceived the study. A.P., S.K., and I.K. planned the statistical analyses, and A.P. analyzed the data and wrote the paper. K.B. designed and coordinated the data collection. All authors have given their final approval of the version to be published and have each made substantial contributions to revising the manuscript critically so as to strengthen intellectual content. References Associated Data Supplementary Materials