Abstract
Introduction
Within the value-based healthcare framework, outcome data can be used to inform patients about (treatment) options, and empower them to make shared decisions with their health care professional. To facilitate shared decision-making (SDM) supported by outcome data, a multicomponent intervention has been designed, including patient decision aids on the organisation of post-treatment surveillance (breast cancer); discharge location (stroke) and treatment modality (advanced kidney disease), and training on SDM for health care professionals. The SHared decision-making supported by OUTcome information (SHOUT) study will examine the effectiveness of the intervention and its implementation in clinical practice.
Methods and analysis
Multiple interrupted time series will be used to stepwise implement the intervention. Patients diagnosed with either breast cancer (N=630), stroke (N=630) or advanced kidney disease (N=473) will be included. Measurements will be performed at baseline, three (stroke), six and twelve (breast cancer and advanced kidney disease) months. Trends on outcomes will be measured over a period of 20 months. The primary outcome will be patients’ perceived level of involvement in decision-making. Secondary outcomes regarding effectiveness will include patient-reported SDM, decisional conflict, role in decision-making, knowledge, quality of life, preferred and chosen care, satisfaction with the intervention, healthcare utilisation and health outcomes. Outcomes regarding implementation will include the implementation rate and a questionnaire on the health care professionals’ perspective on the implementation process.
Ethics and dissemination
The Medical research Ethics Committees United in Nieuwegein, the Netherlands, has confirmed that the Medical Research Involving Human Subjects Act does not apply to this study. Bureau Onderzoek & Innovatie of Santeon, the Netherlands, approved this study. The results will contribute to insight in and knowledge on the use of outcome data for SDM, and can stimulate sustainable implementation of SDM.
Trial registration number
NL8374, NL8375 and NL8376.
Keywords: Stroke, Chronic renal failure, Breast surgery, MEDICAL EDUCATION & TRAINING
Strengths and limitations of this study.
Key stakeholders participated in the development of a multicomponent intervention designed to facilitate shared decision-making supported by personalised outcome information.
By using stepwise implementation in all participating hospitals, lessons learnt can be used to facilitate implementation in subsequent hospitals.
The proposed multiple interrupted time-series design allows the multicomponent intervention to be refined and evaluated over time.
The study design does not allow evaluation of each individual component of the multiple component intervention.
The expected effect size may not be clinically meaningful.
Introduction
Value-based healthcare (VBHC) is gaining momentum worldwide.1 2 The VBHC framework strives to maximise value for patients by achieving the best outcomes while controlling costs.3 Per patient group, clinical and patient-reported outcomes, costs and process data are measured and compared in a structured, standardised manner. These data are used to identify variation across the care cycle to collectively enhance the value of healthcare provision on patient group level.2 Besides the use of outcome data on group level, outcome data can also be used on the individual patient level, by integrating outcomes and value in patient communication. However, in clinical practice, the role of outcome data in patient communication is not common practice. On individual patient level, most importantly, outcome data can provide insight into benefits and harms of treatment options. Integrating outcome data in discussing treatment options between health care professionals and patients, is where VBHC and shared decision-making (SDM) entangle.4 5
SDM is the process in which patients and health care professionals make well-informed, collaborative choices by combining the best available evidence and patients’ values and preferences.6 7 So far, SDM has shown to lead to well-informed, preference-based patient decisions, and to improve patients’ relationship with their health care professional.6 8 9 Using outcome data can further strengthen the motivation of health care professionals to apply SDM and empower patients to make shared decisions with their health care professional. In this way, outcome data can accelerate the implementation of SDM and strengthen VBHC.4 5 10 11
To support SDM, outcome data should be presented to patients in a meaningful way. The four-step conversational SDM model can be used for this purpose.6 7 In each step, outcome data, both on patient individual and group level (aggregated), can be incorporated (see figure 1, based on refs. 6 10).
The individual outcome data can be used to introduce a care decision and to determine available options for the patient (step 1). Related benefits and harms of these options are explained in step 2. As these may differ between patients depending on clinical and personal characteristics, it is highly encouraged to display personalised outcomes (‘patients-like-me data’),10 or to use prediction models in which these characteristics can be entered to display personal estimated risks and to support personalised aftercare paths.12 Next (step 3), the health care professional and the patient discuss the patient’s preferences. This process of value clarification can be fostered by being informed on outcome data of previous patients. In step 4, the health care professional and the patient together integrate outcome data and preferences to make a shared decision.
Currently, outcome data are often not readily available to be used for SDM in clinical practice. To lower this threshold, we developed a multicomponent intervention for three patient groups with an oncological (breast cancer), cardiovascular (stroke) and chronic (advanced kidney disease; AKD) condition. It consists of condition-specific patient decision aids (PtDAs) with personalised outcome data, as well as training for health care professionals and an accompanying implementation strategy. So far, little is known about the impact of using outcome data for SDM.10 11
The aim of the SHared decision-making supported by OUTcome information (SHOUT) study is to assess the effectiveness of the intervention, facilitating SDM supported by personalised outcome data, and to evaluate its implementation in clinical practice. The SHOUT study will contribute to obtaining insight in, and knowledge on, the use of personalised outcome data for SDM, and can stimulate sustainable implementation of SDM in clinical practice.
Methods and analysis
We use multiple interrupted time series (mITS)13 to compare the intervention with standard care. We follow the Standard Protocol Items: Recommendations for Interventional Trials checklist (see online supplemental appendix A).14 15 mITS will allow for initial testing and refinement of the intervention. In participating hospitals, trends on outcomes will be evaluated through a continuous sequence of observations taken repeatedly at equal time intervals from November 2019 onwards (see figure 2). Trends in the preimplementation phase will be ‘interrupted’ at planned timepoints by the stepwise implementation of the intervention in each hospital. Direct effects (level change) will be examined, as well as gradual changes over time (slope change).
bmjopen-2021-055324supp001.pdf (81.5KB, pdf)
Study setting
Seven independent large Dutch teaching hospitals, which together form the Santeon hospital group, will participate in this study. The hospitals are geographically spread across the Netherlands and account for about 11% of the Dutch hospital care volume. By using VBHC principles, comparing outcome data, collaborating in multidisciplinary improvement teams, and by focusing on SDM supported by personalised outcome data as part of the Experiment Outcome Indicators, Santeon continuously aims to improve quality of care on patient group level.16 17 The next step is to use the collected, real-world outcome data to better inform individual patients and health care professionals. Up to now, aggregated outcome data have been gathered in international studies using homogenous samples and population averages. Real-world outcomes in larger, heterogeneous groups of patients provide complementary evidence.18
Study population
Patients diagnosed with either breast cancer, stroke or AKD, treated in Santeon hospitals, will be asked to participate in this study. These patient groups are sufficiently large and diverse to cover a relatively broad spectrum of hospital healthcare. In addition, both breast cancer and stroke are in the top-20 list of largest medical conditions in terms of national disease burden.19
Inclusion criteria
All participants must be aged 18 years or older, and able to understand the Dutch language in speech and writing. Inclusion criteria will be:
Patients facing the decision for the organisation of post-treatment surveillance after curative treatment for invasive non-metastasised breast cancer.
Hospitalised patients with a (ischaemic or haemorrhagic) stroke that have to decide on their discharge location and type of care after discharge from the hospital.
Patients with AKD (ie, CDK-KDIGO G4-G5A1-3) who have to make a treatment modality decision (haemodialysis, peritoneal dialysis, kidney-transplantation or conservative care management).
Exclusion criteria
Patients with severe cognitive impairment or physical inability to complete a questionnaire will be excluded. Exclusion criteria per patient group are displayed in table 1.
Table 1.
Breast cancer | Stoke | Advanced kidney disease |
|
|
|
Intervention
A multicomponent intervention was developed including PtDAs and a training for health care professionals. Because the implementation of SDM is not only a matter of introducing PtDAs, nor that it is achieved by providing personalised outcome data, we designed an implementation strategy focusing on awareness, willingness and behaviour of both health care professionals and patients.
Interactive PtDAs containing personalised outcome data
A PtDA was developed for each patient group in an iterative process of five cocreation sessions with a multidisciplinary team consisting of patients, patient representatives and health care professionals. A literature review and needs assessment studies among patients and health care professionals served as input.20 Development was guided by the International Patient Decision Aid Standards Collaboration framework,21 and in line with the Dutch guidelines for developing PtDAs.22 Content was critically revised by the teams in an iterative process and rewritten to B1 language level (Common European Framework of Reference for Languages). Usability testing consisted of going through the PtDA, combined with think-aloud sessions with patients, an online survey (stroke) and/or interviews by telephone (breast cancer, stroke, and AKD) among health care professionals. Detailed results of the developmental process of the PtDAs will be/have been published.23 24
Each PtDA is composed of three components which contain personalised (patient-reported and clinical) outcome data, both on individual as well as aggregated level. Personalised data are entered into the PtDA by both health care professionals and patients. From the transition phase onwards (figure 2), the health care professional will introduce the PtDA to patients by means of a paper or digital consultation sheet (component 1). Health care professionals provide personalised clinical data (eg, for patients with stroke: type of stroke, National Institutes of Health Stroke Scale score) when introducing the PtDA. Next, patients will receive a personal login code to access the online interactive PtDA at home or during hospital admission (component 2). Each PtDA contains evidence-based information about the options and pros and cons. Information is tailored to relevant options for the patient and presented without favouring any particular outcome. Patients enter patient-reported data, by means of patient-reported outcome measures (PROMs), into the PtDA during use (eg, for patients with AKD: physical condition, treatment goals). The PtDAs actively encourage patients to weigh their options. Once patients have completed the PtDA, a summary sheet will automatically be created, containing an overview of patient-reported personalised data and patient’s preferences and considerations, which can be used as a base for final decision-making in a consultation with their healthcare professional (component 3).
Breast cancer PtDA
The breast cancer PtDA focusses on the organisation of post-treatment surveillance after receiving curative treatment for invasive non-metastasised breast cancer. The PtDA includes patients’ personal risks for locoregional recurrences estimated using the INFLUENCE-nomogram,12 a validated prediction model with which the 5-year risk for locoregional recurrences can be estimated, and a PROM questionnaire on fear of cancer recurrence (sections of the PtDAs were translated for publication; see online supplemental appendix B).
bmjopen-2021-055324supp002.pdf (1.6MB, pdf)
Stroke PtDA
The stroke PtDA focusses on discharge location and type of care after discharge from the hospital. The PtDA includes an interactive ‘patients-like-me’ model on the discharge location of comparable patients based on historical Santeon data (N>5000) and a PROMs questionnaire on physical and mental well-being (see online supplemental appendix B).
AKD PtDA
The AKD PtDA focusses on the treatment modality decision in AKD. The PtDA contains an interactive ‘patients-like-me’ model on median survival and mean hospitalisation rates per treatment modality based on Santeon and national data and a PROMs questionnaire on, for example, the physical condition (see online supplemental appendix B).
Training of health care professionals
Health care professionals will be asked to complete an e-learning on applying (personalised) outcome data to support SDM. Consequently, they will be asked to participate in a group training of one daypart. The e-learning is focused on providing theoretical background and practical tips and tricks on applying outcome information in the four steps of SDM in clinical consultations (including text, videos and self-assessment tests). Completion of the e-learning takes approximately one hour. The group training includes theoretical background information on SDM, reflection on audio-taped consultations (provided by participating health care professionals as part of the data collection for the study), cases introduced by participants, and practising SDM conversational skills with an actor. By offering the e-learning before the group training sessions, we reduce the time spent on theoretical background in the training, leaving more time to practice on SDM conversational skills. Upon completion of the group training, follow-up will be offered after one day (by offering a plasticised card or poster containing short written instructions on SDM, and by presenting a publication on using outcome data to support SDM), after one month (by offering tips, tricks, a testimonial by a colleague health care professional and an instruction clip on SDM) and after two months (by offering the possibility to receive individualised feedback by sending an audio-taped consultation to the trainer).
Implementation strategy for the multicomponent intervention
The implementation strategy is based on prior successful implementation strategies for PtDAs25 and a web-based self-management application using PROMs to monitor quality of life and focuses on awareness, willingness and behaviour of both health care professionals and patients.26 Core elements are listed in box 1.
Box 1. Implementation strategy.
Inform and create support for using the patient decision aid (PtDA) by deciding on the key moment for introducing a PtDA for these three patient groups, developing the PtDA by means of a participatory design approach, including both health care professionals and patient advocates, and by customising the PtDA for each individual hospital (ie, by applying the individual hospital logo).
Document the current care path in each hospital to find the best way to incorporate the PtDA. Involving both the timing of the PtDA and the health care professionals who will present it.
Remove organisational barriers that represent obstacles to the process of implementing the PtDA, such as reorganisations, or the simultaneous implementation of different innovations, by asking hospitals when it is most convenient for them to proceed with the implementation.
Informing and involving all (health care) professionals in the care path by means of an information meeting, and by offering the possibility to make use of an e-learning for these professionals also on applying outcome data in shared decision-making (SDM).
Instruction on how to introduce the PtDA to eligible patients by means of a kick-off meeting organised in the hospitals shortly before the start of the implementation of the PtDA.
Offering support in the workplace, that is, by providing plasticised cards containing short written instructions in line with the SDM four-step conversation model, and by stimulating implementation for example, by distributing promotional posters and informative video’s for patients on SDM with personalised outcome data. Support and technical assistance for both health care professionals and patients will be centralised and available through a helpdesk.
Closely monitoring of progress and stimulating implementation by local ambassador and informed by a dashboard containing usage data of the PtDA.
Offering the training and the PtDA free of charge during the study period.
Study design and procedures
The intervention will be stepwise implemented in the hospitals over a period of 20 months (see figure 2). In each hospital, there will be 6–12 months in which standard care will be thoroughly assessed (preimplementation phase), followed by a transition phase of 2 months in which health care professionals will be trained and the PtDA will be introduced. Finally, there will be 6–12 months in which the intervention will be assessed (postimplementation phase). Data collection is ongoing. The moment by which hospitals switch from standard care to use of the intervention will not be randomised. To promote that PtDAs will become successfully implemented into routine clinical settings, we will ask involved health care professionals when it will be most convenient for them to proceed with implementation. Internal validity will be increased, as each hospital will act as its own historical control group and the hospitals will not switch at the same time.
Patients will be asked by their health care professional to participate in this study: (1) patients with breast cancer will be informed and asked to participate during the follow-up consultation on the occasion of their first post-treatment surveillance with imaging about one year after surgery, (2) patients with stroke will be asked during admission to the hospital and (3) patients with AKD will be asked when a decision has to be made about kidney replacement therapy or conservative care. When interested, patients will receive a patient information letter about the study. They will be asked for written informed consent. In the postimplementation phase, patients who decline participation in the SHOUT-study will still be offered the SDM supported by outcome information as the standard form of care.
Data collection and methods
To assess the effectiveness of the multicomponent intervention, first, a baseline questionnaire (T0) will be sent to patients, via e-mail, post or will be handed out to patients with stroke during admission at the hospital. Subsequently, patients will receive a follow-up questionnaire after three months (T1) for patients with stroke, and after six (T1) and twelve (T2) months for patients with breast cancer or AKD. The time it takes to complete the questionnaires differs per measurement moment. The T0 questionnaire takes about 30–45 min to complete and the T1 and T2 questionnaires take 15–20 min. The timing of follow-up questionnaires differs between the three conditions due to the course and nature of and the care pathways for the three conditions. Furthermore, some outcome measures are disease-specific and will therefore only be assessed in the patient groups for which they are suitable.
Second, the consultations, in which the options are being discussed, will be audio-taped to assess patients’ involvement in the decision-making process from observers’ viewpoint. Also, the length of the consultations will be determined. Third, to assess the extent to which the intervention leads to changes in the utilisation and outcomes of healthcare, information will be retrieved from patients’ electronic health records.
To evaluate the implementation, first, the estimated total number of eligible patients and the total number of patients who received the PtDA will be determined. Second, participating health care professionals will receive a questionnaire six months after start of the post implementation phase, to assess their perspective on the implementation process.
Participant timeline
The participant timeline is displayed in figure 3.
Outcomes
Effectiveness
Primary outcome measure
The primary outcome to assess effectiveness will be patients’ perceived level of involvement in decision-making, measured with the 9-item SDM Questionnaire (SDM-Q-9).27 28 Each item describes a different step in the SDM process and will be scored by patients on a 6-point Likert scale. The sum of the item scores will range from 0 to 45, with higher scores indicating a greater level of perceived involvement in SDM.
Secondary outcome measures
Secondary outcomes will be: (1) patient-reported SDM, measured with the CollaboRATE; (2) decisional conflict, measured with the Decisional Conflict Scale (DCS); (3) decision regret for patients with stroke and AKD, measured with the Decision Regret Scale (DRS); (4) preferred and perceived role in decision-making, measured with the Control Preference Scale (CPS); (5) patients’ knowledge regarding their disease and treatment options, measured with patient group-specific items; (6) quality of life, measured with the 12-item Short Form Health Survey (SF-12) for patients with breast cancer and AKD, and measured with the Patient Reported Outcomes Measurement Information System Global Health (PROMIS-10), five-dimension EuroQol five-levels questionnaire (EQ-5D-5L), and EuroQol Visual Analogue Scale (EQ-VAS) for patients with stroke; (7) preferred and chosen care (and the role of the consultation and outcome data therein), measured with patient group-specific items; (8) satisfaction with the intervention, measured with the Preparation for Decision Making scale (Prep-DM) and study-specific questions; (9) perceived risk and fear of recurrence for patients with breast cancer, measured with the Cancer Worry Scale (CWS), two subscales of the Revised Illness Perception Questionnaire for breast cancer survivors (IPQ-BCS) and patient group-specific questions and (10) participation/functioning and caregivers’ strain for patients with stroke, measured with the modified Ranking Scale (mRS), Utrecht Scale for Evaluation of Rehabilitation-Participation (USER-P) and the Caregiver Strain Index (CSI) (see table 2, also for references).
Table 2.
Measure | Description | Scoring range | Preimplementation phase | Postimplementation phase | ||||
Baseline | T1 | T2 | Baseline | T1 | T2 | |||
All patient groups: | ||||||||
Shared decision-making | ||||||||
|
9-item, 6-point scale measures patients’ perceived level of involvement in decision-making. | Range 0–45, higher scores indicate a greater level of perceived involvement in decision-making. | X | X | ||||
|
3-item, 10-point scale measures patient-reported SDM. | Range 0–100, higher scores indicate a higher patient-reported SDM. | X | X | ||||
Decisional conflict | ||||||||
|
16-item, 5-point scale measures personal perceptions of (a) uncertainty in choosing options, b) modifiable factors contributing to uncertainty and c) effective decision-making. | Range 0–100, higher scores indicate greater decisional conflict. | X | X | ||||
Decision regret | ||||||||
Stoke and advanced kidney disease:
|
5-item, 5-point scale measures distress or remorse after a healthcare decision. | Range 0–100, higher scores indicate greater regret. | X | X | X | X | ||
(Preferred) role in decision-making | ||||||||
|
1-item with 5 response options to assess the patient’s preferred or perceived degree of control when decisions about treatment are being made. | X | X | |||||
Patients’ knowledge regarding their disease and treatment options (patient group-specific items) | ||||||||
Breast cancer: Stroke: Advanced kidney disease: |
10 items with 3 response options. 7 items with 3–7 response options. 7 items with 3–5 response options. |
X X X |
X X X |
|||||
Quality of life | ||||||||
Breast cancer and advanced kidney disease:
|
12-items with 2–6 response options on quality of life. | Mental and physical component score based on the US population scoring system, higher scores indicate greater quality of life. | X | X | X | X | X | X |
Stroke: | 10 items with 5–11 response options on quality of life. 5 items, 5-point scale measures patients’ health-related quality of life. VAS measures patients’ health-related quality of life. |
Physical and mental health summary scores based on the US population scoring system, higher scores indicate greater quality of life. Range −0.446–1 based on the Dutch population tariff, higher scores indicate greater health-related quality of life. Range 0–100, higher scores indicate greater health-related quality of life. |
X X X |
X X X |
||||
Preferred and chosen care (and the role of the consultation and outcome data therein) (patient group-specific items) | ||||||||
Breast cancer: Stroke: Advanced kidney disease |
48 items with 3–10 response options/open-ended. 6 items with 3–8 response options/open-ended. 9 items with 2–9 response options/open-ended. |
X X X |
X X X |
|||||
Satisfaction with the intervention | ||||||||
|
10-item, 5-point scale measures patients’ perception of how useful the PtDA is in preparing them to communicate with their health care professional during consultations, and for making a healthcare decision. | Range 0–100, higher scores indicate higher perceived level of preparation for decision-making. | X | |||||
|
24 items with 2–8 response options/open-ended to assess (1) the way in which the PtDA has been presented to the patient, (2) the experience of the patient with using the PtDA, and (3) the extent to which the PtDA is of value to the patient. | X | ||||||
Breast cancer: | ||||||||
Perceived risk and fear of recurrence | ||||||||
|
6-item, 4-point scale measures concerns about cancer recurrence and the impact of these concerns on daily functioning. | Range 6–24, higher scores indicate greater worrying. | X | X | X | X | X | X |
|
2×4 item, 5-point scale measures (a) patients’ cure beliefs and (b) personal control over the risk for recurrences. | X | X | X | X | X | X | |
9 items with 4–6 response options to assess patients’ feelings about imaging during follow-up and worry about cancer recurrence, and to assess patients’ perceived (absolute and comparative) risk of recurrence. | X | X | X | X | X | X | ||
Stroke: | ||||||||
Participation/functioning | ||||||||
|
5-item, 2-point scale measures the degree of dependence of patients with stroke. | Range 0–5, higher scores indicate greater dependence. | X | X | ||||
|
11-item, 5-point scale measures experienced restrictions on 11 domains of participation. | X | X | |||||
Caregivers’ strain | ||||||||
|
13-item, 2-point scale measures strain related to care provision. | Range 0–13, ≥7 indicates a higher level of strain. | X | X |
CPS, Control Preference Scale; CRHWS, Cancer-Related Health Worries Scale; CSI, Caregiver Strain Index; CWS, Cancer Worry Scale; DCS, Decisional Conflict Scale; DRS, Decision Regret Scale; EQ-5D-5L, five-dimension EuroQol five-levels questionnaire; FCR7, seven-item Fears of Cancer Recurrence; FoP-Q, Fear of Progression Questionnaire; IPQ-BCS, modified version of the Revised Illness Perception Questionnaire for breast cancer survivors; mRS, modified Ranking Scale; Prep-DM, Preparation for Decision Making scale; PROMIS, Patient Reported Outcomes Measurement Information System; PtDA, patient decision aid; SDM-Q-9, 9-item Shared Decision Making Questionnaire; SF-12, 12-item Short Form Health Survey; USER-P, Utrecht Scale for Evaluation of Rehabilitation-Participation; VAS, Visual Analogue Scale.
Observer-reported SDM
We will combine the SDM-measurement tools, with a more objective score of SDM, as this score may differ from the patients’ subjective interpretation.29 The Observing Patient Involvement in decision-making scale (OPTION-5)30 will be used to analyse the audiorecordings of encounters from clinical settings. All audiorecordings will be double coded by two raters who have been trained on rating the OPTION-5. In case of disagreement, a third rater will be consulted. The OPTION-5 includes five core SDM steps, to which a sixth is added to assess the role of personalised outcome data (‘the health care professional informs the patient on outcomes of different treatment options’). The item scores will be summed and rescaled to a 0–100 scale, with higher scores indicating greater SDM.
Healthcare utilisation and outcomes
Patients’ healthcare utilisation and clinical outcomes will be extracted from their electronic health records. For patients with breast cancer, the number of hospital visits, the number of mammograms and other imaging during follow-up, and mortality will be extracted. For patients with stroke, the length of stay, the number of readmissions to the hospital, and the number of (treatment-related) complications during admission will be extracted. For patients with AKD, the number of visits to outpatient clinics, hospital admissions and hospitalisation days, and the rate of major treatment-related complications will be extracted.
Moderators
Socio-demographic characteristics
In the baseline questionnaire, patients’ sex, birth year, marital status, occupation and education level will be asked.
Clinical characteristics
Relevant medical characteristics will be extracted from the baseline questionnaire and the electronic health records. For patients with breast cancer, tumour and treatment characteristics will be extracted. For patients with stroke, aetiology of stroke, and whether or not the patient has been treated with reperfusion therapy will be extracted. For patients with AKD, kidney function, aetiology and duration of kidney failure, whether these patients have had other treatment modalities for kidney failure in the past, comorbidity and definite treatment modality will be extracted.
Health literacy
Patients’ health literacy will be assessed in the baseline questionnaire by the Set of Brief Screening Questions (SBSQ).31 The mean score on the three items will be calculated, with higher scores reflecting higher health literacy skills.
Implementation
Implementation rate
The implementation rate will be calculated as the proportion of patients who received the PtDA compared with the estimated total number of eligible patients during the period of 6–12 months in which the PtDA will be handed out.
Health care professionals’ view on the implementation process and use of the PtDA
Determinants of implementing an innovation
Health care professionals will fill out a questionnaire based on the Measurement Instrument for Determinants of Innovations (MIDI).32 The MIDI assesses barriers and facilitators of implementation at the level of the innovation, the user (health care professionals) and the organisation (hospital).
Physicians’ willingness to incorporate SDM
Health care professionals will also fill out a questionnaire based on items from the incorpoRATE, a brief and broadly applicable measure of physicians’ willingness to incorporate SDM into practice.33
Sample size
The sample size was estimated using Statistical Analysis System (SAS) with the SDM-Q-9 as primary outcome measure with the statistical significance level set at alpha=0.05 (two sided). Since there is no agreement on what constitutes a clinically meaningful difference on the SDM-Q-9, we estimated the size of the expected effect on previous studies using the SDM-Q-9. The size of the expected effect of the intervention on the SDM-Q-9 was set to be small to moderate (Cohen’s d=0.3–0.4) as relatively high scores on the SDM-Q-9 are common in the Netherlands.34 The mITS with seven clusters (ie, hospitals) had 18 measurement periods (excluding the transition phase, see figure 2). For patients with breast cancer and stroke, a non-large Intraclass Correlation Coefficient (0.05) was assumed. The correlation between monthly measurements was expected to be high (0.7–0.9) throughout a period of 18 months, although correlations between months farther apart could be lower than for months closer by. A correlation structure where the correlation decreases exponentially with the distance between months (autoregressive correlation structure) turned out too conservative and a correlation structure where the correlation between months is the same regardless of the distance between them (compound symmetry correlation structure) was too optimistic and not realistic for this purpose. Therefore, power calculations were primarily based on assuming that the correlation between months decreases from 0.9, for subsequent months, to 0.7, for months that are the farthest apart (ie, the first and last month). To be precise, the correlation decreases linearly on the log scale from log(0.9) to log(0.7) (linear exponent autoregressive correlation structure).35 Five patients per hospital per month was considered feasible, and with a 25% lost to follow-up, this results in a monthly inclusion rate of four patients. This yields more than 80% power and amounts to a study population of N=504–630.
For patients with AKD, an inclusion rate of four patients was deemed feasible within the hospitals. Assuming a 25% lost to follow-up, three patients per month would give at least 80% power for detecting a Cohen’s d=0.4 assuming a correlation between subsequent months of at least 0.8 and a correlation between the first and last month of at least 0.6. This amounts to a study population of N=378–473.
Statistical methods
An overview of the demographic and clinical characteristics will be provided using descriptive statistics. Continuous data will be expressed as a mean with the SD, or as the median (IQR) where appropriate. Categorical data will be expressed as frequencies (%) unless stated otherwise.
Separate ITS analyses will be performed to analyse the data per patient group per hospital. Segmented regression will be employed, with the period before and after the introduction of the intervention as segments. In each segment, linear regression will be fitted to the data, allowing each segment of the time series to exhibit different levels and trends. Correlation between repeated measurements in each time series will be accounted for by modelling the error structure. The effect of the intervention will be examined by comparing the slopes and intercepts in both the preimplementation and postimplementation phase using the following model:
where will represent the baseline level at , will be interpreted as the change in outcomes associated with a time unit increase (representing the underlying trend in the preimplementation phase), = 1 when the hospital is at the time in the intervention and I = 0 otherwise, will be the level change in the postimplementation phase and will indicate the slope change following the implementation phase (using the interaction between time since the intervention started and the indicator for being in the intervention: ). A change in will constitute an immediate effect, while a change in will imply an effect that was experienced over time (which also allows us to measure the sustainability of the impact). Moreover, segmented regression will enable us to control for other variables, that can cause a change in level or trend of the outcomes of interest.
Seasonal patterns and outliers will be identified by visualising the multiple time series. The percentage of drop-out and missings at each follow-up timepoint will be recorded. If necessary, either imputation techniques or sensitivity analyses will be used to assess their impact on the trial results.
To correct for multiple testing and the risk of type-1 errors a Bonferroni-Holm procedure will be applied across the set of primary and secondary endpoints.
To explore the average effect per patient group across all hospitals, a meta-analysis of the hospital-specific effects will be conducted. To examine the overall effect of the SHOUT study, also, meta-analysis across all patient groups and hospitals will be performed. Finally, implementation across all patient groups will be investigated by using several the same outcome measures at a similar points in time.
Patient and public involvement
Santeon supports that patients with ‘lived experiences’ become members of a research team. Since the very beginning (composing the grant application), we have engaged a core group of patients and patient representatives of the patient associations involved. We designed the multicomponent intervention in collaboration with patients and health care professionals (see the Methods and Analysis section). In addition, patient representatives were involved in the development of the study. Our collaboration with the patient associations will continue throughout the study. Study findings about the potential benefits of the multicomponent intervention will be disseminated by means of our project website.
Ethics and dissemination
The Medical research Ethics Committees United in Nieuwegein, the Netherlands, has confirmed that the Medical Research Involving Human Subjects Act (WMO) does not apply to this study (reference number W19.154). Bureau Onderzoek & Innovatie of Santeon, the Netherlands, approved this study (reference numbers METC 2019-075, 2019-076 and 2019-077).
The study will be conducted in accordance with local laws and regulations. Eligible patients will fully be informed about the study and asked to participate. They will receive a patient information letter and will be informed by telephone about the implications of participation. Patients will have sufficient opportunity to ask questions and to consider the implications before providing written informed consent. They will be allowed to withdraw from the study without giving a reason, at any time.
The SHOUT study is part of a larger Santeon programme on using outcome data for SDM (‘Experiment Uitkomstindicatoren’). It will contribute to the limited understanding of the impact of using (clinical and patient-reported) outcome data for SDM. We will share our findings through peer-reviewed journals, (inter)national conferences, workshops webinars, and newsletters and social media.
Supplementary Material
Acknowledgments
We thank all patients, patient representatives and health care professionals for their contribution to designing the multi-component intervention and execution the SHOUT study. The SHOUT study is part of a larger program on using outcome data for SDM (‘Experiment Uitkomstindicatoren Santeon’), which is part of the Outcome-based Health care program initiated by the Dutch Ministry of Health, Welfare and Sports. We would like to thank ZonMw for funding this project.
Footnotes
Collaborators: VBHC breast cancer group: YEA van Riet, Catharina Hospital, Eindhoven, the Netherlands; LJA Strobbe, Canisius Wilhelmina Hospital, Nijmegen, the Netherlands; AE Dassen, Medisch Spectrum Twente, Enschede, the Netherlands; AFT Olieman, Martini Hospital, Groningen, the Netherlands; HHG Witjes, OLVG, Amsterdam, the Netherlands; R Koelemij, St. Antonius Hospital, Utrecht, the Netherlands; A Doeksen, St. Antonius Hospital, Utrecht, the Netherlands; CME Contant, Maasstad Hospital, Rotterdam, the Netherlands.VBHC stroke group: PJAM Brouwers, Medisch Spectrum Twente, Enschede, the Netherlands; GW van Dijk, Canisius Wilhelmina Hospital, Nijmegen, the Netherlands; RAR Gons, Catharina Hospital, Eindhoven, the Netherlands; PHE Hilkens, St. Anontius Hospital, Nieuwegein, the Netherlands; R Saxena, Maasstad Ziekenhuis, Rotterdam, the Netherlands; SM van Schaik, OLVG, Amsterdam, the Netherlands; ES Schut, Martini Hospital, Groningen, the Netherlands. VBHC chronic kidney disease group: WJW Bos, St. Antonius Hospital, Nieuwegein, the Netherlands; MA van den Dorpel, Maasstad Hospital, Rotterdam, the Netherlands; CEH Siegert, OLVG, Amsterdam, the Netherlands; CJAM Konings, Catharina Hospital Eindhoven, the Netherlands; CG ter Meulen, Canisius Wilhelmina Hospital, Nijmegen, the Netherlands; WMT Janssen, Martini Hospital, Groningen, the Netherlands; JN Brinkman, Medisch Spectrum Twente, the Netherlands.
Contributors: JWA, NE, JCMP, SS, CHCD, LJAS, YEAvR, RMAvdD, WJWB, SMvS and CFvU-K developed the multicomponent intervention. MQNH, ST, PBvdN, PJvdW and CFvU-K contributed to the design of the study. JWA, NE and JCMP are conducting this study in fulfillment of a PhD and will be responsible for data collection. JWA, NE, JCMP and ST will be responsible for data analysis. All authors will be responsible for interpretating the data. The present manuscript was drafted by MQNH and CFvU-K. JWA, NE, JCMP, ST, SS, CHCD, LJAS, YEAvR, RMAvdD, WJWB, PBvdN, RMvdB-V, SMvS, MMG and PJvdW critically revised this manuscript. All authors read and approved the final manuscript.
Funding: This research project is funded by ZonMw as part of the ‘Experiment Uitkomstindicatoren Santeon’.
Disclaimer: This funding had no involvement in collection, management, analysis, and interpretation of the data; writing this manuscript or the decision to submit the article for publication.
Competing interests: None declared.
Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review: Not commissioned; externally peer reviewed.
Availability of data and materials: Data will be collected and recorded in CASTOR EDC, a cloud-based electronic data capture platform. This platform is fully compliant with GCP, 21 CFR part 11, GDPR, HIPAA, ISO27001 and ISO 9001. All data will be coded and password protected. Study participants will be assigned a participant identification number (PIN). A digital, password protected identifying list relating medical information of participants to their PIN numbers will be kept on a secured server in the Santeon hospitals. All data and study documents will deleted and discarded after 15 years. The datasets used and / or analysed during the SHOUT study are available from JWA (breast cancer), NE (AKD) and JCMP (stroke) on reasonable request. The (intellectual) property rights with regard to the generated data will reside at Santeon, Utrecht, The Netherlands. Interested parties can request a non-exclusive license for research and educational purposes. The non-exclusive license may be requested only after the completion of the theses to be written reserving the generated data.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
Contributor Information
the Santeon VBHC breast cancer, stroke and chronic kidney disease group:
YEA van Riet, LJA Strobbe, AE Dassen, AFT Olieman, HHG Witjes, R Koelemij, A Doeksen, CME Contant, PJAM Brouwers, GW van Dijk, RAR Gons, PHE Hilkens, R Saxena, SM van Schaik, ES Schut, WJW Bos, MA van den Dorpel, CEH Siegert, CJAM Konings, CG ter Meulen, WMT Janssen, and JN Brinkman
Ethics statements
Patient consent for publication
Not applicable.
References
- 1. Kampstra NA, Zipfel N, van der Nat PB, et al. Health outcomes measurement and organizational readiness support quality improvement: a systematic review. BMC Health Serv Res 2018;18:1005. 10.1186/s12913-018-3828-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Porter ME, Larsson S, Lee TH. Standardizing patient outcomes measurement. N Engl J Med 2016;374:504–6. 10.1056/NEJMp1511701 [DOI] [PubMed] [Google Scholar]
- 3. Porter ME. What is value in health care? N Engl J Med 2010;363:2477–81. 10.1056/NEJMp1011024 [DOI] [PubMed] [Google Scholar]
- 4. Steinmann G, van de Bovenkamp H, de Bont A, et al. Redefining value: a discourse analysis on value-based health care. BMC Health Serv Res 2020;20:862. 10.1186/s12913-020-05614-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. van der Nat PB. The new strategic agenda for value transformation. Health Serv Manage Res 2022;35:1–5. 10.1177/09514848211011739 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Stiggelbout AM, Pieterse AH, De Haes JCJM, JCJM deH. Shared decision making: concepts, evidence, and practice. Patient Educ Couns 2015;98:1172–9. 10.1016/j.pec.2015.06.022 [DOI] [PubMed] [Google Scholar]
- 7. Elwyn G, Frosch D, Thomson R, et al. Shared decision making: a model for clinical practice. J Gen Intern Med 2012;27:1361–7. 10.1007/s11606-012-2077-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Stacey D, Légaré F, Lewis K. Decision AIDS for people facing health treatment or screening decisions. Cochrane Database Sys Rev 2017;4:CD001431. 10.1002/14651858.CD001431.pub5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Elwyn G, Frosch DL, Kobrin S. Implementing shared decision-making: consider all the consequences. Implement Sci 2016;11:114. 10.1186/s13012-016-0480-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Damman OC, Jani A, de Jong BA, et al. The use of PROMs and shared decision-making in medical encounters with patients: an opportunity to deliver value-based health care to patients. J Eval Clin Pract 2020;26:524–40. 10.1111/jep.13321 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. La Grouw Y, Kee K, Ybema S, et al. Samen Beslissen in de praktijk: praktijkonderzoek i.o.v. Het Zorginstituut Nederland. Equalis / vu, 2019. Available: https://www.zorginstituutnederland.nl/publicaties/rapport/2019/06/14/rapportage-samen-beslissen-in-de-praktijk
- 12. Witteveen A, Vliegen IMH, Sonke GS, et al. Personalisation of breast cancer follow-up: a time-dependent prognostic nomogram for the estimation of annual risk of locoregional recurrence in early breast cancer patients. Breast Cancer Res Treat 2015;152:627–36. 10.1007/s10549-015-3490-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Biglan A, Ary D, Wagenaar AC. The value of interrupted time-series experiments for community intervention research. Prev Sci 2000;1:31–49. 10.1023/a:1010024016308 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Chan A-W, Tetzlaff JM, Altman DG, et al. Spirit 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med 2013;158:200–7. 10.7326/0003-4819-158-3-201302050-00583 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Chan A-W, Tetzlaff JM, Gøtzsche PC, et al. Spirit 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ 2013;346:e7586. 10.1136/bmj.e7586 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. EIT Health . Implementing value-based health care in Europe: Handbook for pioneers (director: GREGORY Katz), 2020.
- 17. Santeon , 2021. Available: www.santeon.nl
- 18. Templeton AJ, Booth CM, Tannock IF. Informing patients about expected outcomes: the efficacy-effectiveness gap. J Clin Oncol 2020;38:1651–5. 10.1200/JCO.19.02035 [DOI] [PubMed] [Google Scholar]
- 19. RIVM . Ranglijst aandoeningen op basis van ziektelast (in DALY’s), 2018. Available: https://www.volksgezondheidenzorg.info/ranglijst/ranglijst-aandoeningen-op-basis-van-ziektelast-dalys
- 20. Ankersmid JW, Drossaert CHC, van Riet YEA, et al. Needs and preferences of breast cancer survivors regarding outcome-based shared decision-making about personalised post-treatment surveillance. J Cancer Surviv 2022. 10.1007/s11764-022-01178-z. [Epub ahead of print: 04 Feb 2022]. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Joseph-Williams N, Newcombe R, Politi M, et al. Toward minimum standards for certifying patient decision AIDS: a modified Delphi consensus process. Med Decis Making 2014;34:699–710. 10.1177/0272989X13501721 [DOI] [PubMed] [Google Scholar]
- 22. Patiëntenfederatie Nederland, Nederlands Huisartsen Genootschap (NHG), Federatie Medisch Specialisten (FMS), Verpleegkundigen & Verzorgenden Nederland (V&VN) . Hoe maak IK Een keuzehulp bij Een richtlijn? Den Haag, The Netherlands, 2018. [Google Scholar]
- 23. Prick JCM, van Schaik SM, Deijle IA, et al. Development of a patient decision aid for discharge planning of hospitalized patients with stroke. BMC Neurol 2022;22:245. 10.1186/s12883-022-02679-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Engels N, van der Nat PB, Ankersmid JW, et al. Development of an online patient decision aid for kidney failure treatment modality decisions. BMC Nephrol 2022;23:236. 10.1186/s12882-022-02853-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. van Tol-Geerdink JJ, van Oort IM, Somford DM, et al. Implementation of a decision aid for localized prostate cancer in routine care: a successful implementation strategy. Health Informatics J 2020;26:1194–207. 10.1177/1460458219873528 [DOI] [PubMed] [Google Scholar]
- 26. Matthijs de Wit L, van Uden-Kraan CF, Lissenberg-Witte BI, et al. Adoption and implementation of a web-based self-management application "Oncokompas" in routine cancer care: a national pilot study. Support Care Cancer 2019;27:2911–20. 10.1007/s00520-018-4591-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Rodenburg-Vandenbussche S, Pieterse AH, Kroonenberg PM, et al. Dutch translation and psychometric testing of the 9-Item shared decision making questionnaire (SDM-Q-9) and shared decision making questionnaire-physician version (SDM-Q-Doc) in primary and secondary care. PLoS One 2015;10:e0132158. 10.1371/journal.pone.0132158 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Kriston L, Scholl I, Hölzel L, et al. The 9-item shared decision making questionnaire (SDM-Q-9). development and psychometric properties in a primary care sample. Patient Educ Couns 2010;80:94–9. 10.1016/j.pec.2009.09.034 [DOI] [PubMed] [Google Scholar]
- 29. Ubbink DT, van Asbeck EV, Aarts JWM, et al. Comparison of the collaborate and SDM-Q-9 questionnaires to appreciate the patient-reported level of shared decision-making. Patient Educ Couns 2022;105:2475–9. 10.1016/j.pec.2022.03.007 [DOI] [PubMed] [Google Scholar]
- 30. Elwyn G, Grande SW, Barr P. Observer OPTION5 manual. measuring shared decision making by assessing recordings or transcripts of encounters from clinical settings. Lebanon, USA: Dartmouth Institute for Health Policy and Clinical Practice; 2018. [Google Scholar]
- 31. Chew LD, Griffin JM, Partin MR, et al. Validation of screening questions for limited health literacy in a large va outpatient population. J Gen Intern Med 2008;23:561–6. 10.1007/s11606-008-0520-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Fleuren MAH, Paulussen TGWM, Van Dommelen P, et al. Towards a measurement instrument for determinants of innovations. Int J Qual Health Care 2014;26:501–10. 10.1093/intqhc/mzu060 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Berkowitz J, Martinez-Camblor P, Stevens G. The development of incorpoRATE: a measure of physicians’ willingness to incorporate shared decision making into practice. Patient Educ Couns 2021;S0738-3991:00140–3. 10.1016/j.pec.2021.02.040 [DOI] [PubMed] [Google Scholar]
- 34. Van der Hulst F, Meijer M, Holst L. Mate waarin arts en patiënt samen beslissen over een behandeling nauwelijks veranderd tussen 2016-2020. Gezamenlijke besluitvorming. Utrecht: Nivel, 2021. [Google Scholar]
- 35. Simpson SL, Edwards LJ, Muller KE, et al. A linear exponent AR(1) family of correlation structures. Stat Med 2010;29:1825–38. 10.1002/sim.3928 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Elwyn G, Barr PJ, Grande SW, et al. Developing collaborate: a fast and frugal patient-reported measure of shared decision making in clinical encounters. Patient Educ Couns 2013;93:102–7. 10.1016/j.pec.2013.05.009 [DOI] [PubMed] [Google Scholar]
- 37. O’Connor AM. Manual – decisional conflict scale. 1. OHRI, 1993: 1–16. [Google Scholar]
- 38. O’Connor AM. User manual – decision regret scale. Ottawa: Ottawa Hospital Research Institute, 1996. [Google Scholar]
- 39. Degner LF, Sloan JA, Venkatesh P. The control preferences scale. Can J Nurs Res 1997;29:21–43. [PubMed] [Google Scholar]
- 40. Ware J, Kosinski M, Keller SD. A 12-Item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996;34:220–33. 10.1097/00005650-199603000-00003 [DOI] [PubMed] [Google Scholar]
- 41. Ware JE, Kosinski MA, Keller SD. SF-12: How to score the SF-12 Physical and Mental Health Summary Scales. Boston Massachusetts: The Health Institute, New England Medical Center, 1995. [Google Scholar]
- 42. Hays RD, Bjorner JB, Revicki DA, et al. Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Qual Life Res 2009;18:873–80. 10.1007/s11136-009-9496-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Herdman M, Gudex C, Lloyd A, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011;20:1727–36. 10.1007/s11136-011-9903-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. M Versteegh M, M Vermeulen K, M A A Evers S, et al. Dutch tariff for the five-level version of EQ-5D. Value Health 2016;19:343–52. 10.1016/j.jval.2016.01.003 [DOI] [PubMed] [Google Scholar]
- 45. Graham ID, O’Connor AM. User manual – preparation for decision making scale. Ottawa: Ottawa Hospital Research Institute; 1995. [Google Scholar]
- 46. Custers JAE, Kwakkenbos L, van de Wal M, et al. Re-validation and screening capacity of the 6-item version of the cancer worry scale. Psychooncology 2018;27:2609–15. 10.1002/pon.4782 [DOI] [PubMed] [Google Scholar]
- 47. Moon Z, Moss-Morris R, Hunter MS, et al. Measuring illness representations in breast cancer survivors (BCS) prescribed tamoxifen: modification and validation of the revised illness perceptions questionnaire (IPQ-BCS). Psychol Health 2017;32:439–58. 10.1080/08870446.2016.1275629 [DOI] [PubMed] [Google Scholar]
- 48. Deimling GT, Bowman KF, Sterns S, et al. Cancer-Related health worries and psychological distress among older adult, long-term cancer survivors. Psychooncology 2006;15:306–20. 10.1002/pon.955 [DOI] [PubMed] [Google Scholar]
- 49. Humphris GM, Watson E, Sharpe M, et al. Unidimensional scales for fears of cancer recurrence and their psychometric properties: the FCR4 and FCR7. Health Qual Life Outcomes 2018;16:30. 10.1186/s12955-018-0850-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Herschbach P, Berg P, Dankert A, et al. Fear of progression in chronic diseases: psychometric properties of the fear of progression questionnaire. J Psychosom Res 2005;58:505–11. 10.1016/j.jpsychores.2005.02.007 [DOI] [PubMed] [Google Scholar]
- 51. Bruno A, Akinwuntan AE, Lin C, et al. Simplified modified rankin scale questionnaire. Stroke 2011;42:2276–9. 10.1161/STROKEAHA.111.613273 [DOI] [PubMed] [Google Scholar]
- 52. Post MWM, van der Zee CH, Hennink J, et al. Validity of the utrecht scale for evaluation of rehabilitation-participation. Disabil Rehabil 2012;34:478–85. 10.3109/09638288.2011.608148 [DOI] [PubMed] [Google Scholar]
- 53. Robinson BC. Validation of a caregiver strain index. J Gerontol 1983;38:344–8. 10.1093/geronj/38.3.344 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2021-055324supp001.pdf (81.5KB, pdf)
bmjopen-2021-055324supp002.pdf (1.6MB, pdf)