Are shared decision making studies well enough described to be replicated? Secondary analysis of a Cochrane systematic review

Abstract

Background

Interventions to change health professionals’ behaviour are often difficult to replicate. Incomplete reporting is a key reason and a source of waste in health research. We aimed to assess the reporting of shared decision making (SDM) interventions.

Methods

We extracted data from a 2017 Cochrane systematic review whose aim was to determine the effectiveness of interventions to increase the use of SDM by healthcare professionals. In a secondary analysis, we used the 12 items of the Template for Intervention Description and Replication (TIDieR) checklist to analyze quantitative data. We used a conceptual framework for implementation fidelity to analyze qualitative data, which added details to various TIDieR items (e.g. under “what materials?” we also reported on ease of access to materials). We used SAS 9.4 for all analyses.

Results

Of the 87 studies included in the 2017 Cochrane review, 83 were randomized trials, three were non-randomized trials, and one was a controlled before-and-after study. Items most completely reported were: “brief name” (87/87, 100%), “why” (rationale) (86/87, 99%), and “what” (procedures) (81/87, 93%). The least completely reported items (under 50%) were “materials” (29/87, 33%), “who” (23/87, 26%), and “when and how much” (18/87, 21%), as well as the conditional items: “tailoring” (8/87, 9%), “modifications” (3/87, 4%), and “how well (actual)” (i.e. delivered as planned?) (3/87, 3%). Interventions targeting patients were better reported than those targeting health professionals or both patients and health professionals, e.g. 84% of patient-targeted intervention studies reported “How”, (delivery modes), vs. 67% for those targeting health professionals and 32% for those targeting both. We also reported qualitative analyses for most items. Overall reporting of items for all interventions was 41.5%.

Conclusions

Reporting on all groups or components of SDM interventions was incomplete in most SDM studies published up to 2017. Our results provide guidance for authors on what elements need better reporting to improve the replicability of their SDM interventions.

Background

Shared decision making (SDM) is a patient-centered approach whereby clinicians and patients work together to make decisions based on the best available evidence and according to the patient’s values and preferences [1, 2]. SDM is particularly relevant when there are different preference-sensitive options to consider [3]. It has been shown that SDM has positive effects on patients’ health outcomes and their satisfaction with the decisions made [4, 5]. It improves the experience of both patients and health professionals [6, 7]. Indeed, SDM represents a set of essential actions (e.g. define/explain problem, present options, discuss pros/cons) [8] that must be achieved by patients and health professionals together [9]. It is possible to specify behaviors that each party must adopt for SDM to occur and then to intervene to increase adoption of these behaviors [10, 11].

Despite its benefits, SDM has not yet been widely adopted in clinical practice [12]. Many barriers to SDM have been identified in the literature [13, 14]. Some studies demonstrate missed opportunities for SDM in various contexts [15, 16]. For example, in one study oncologists rarely expressed that a treatment decision needed to be made in consultations concerning cancer care [15]. Another study in mental health contexts suggested that patients could participate more in goal setting [16].

Numerous interventions using behavioral change strategies have been developed to increase the adoption of SDM by health care professionals and the involvement of patients in their own care [13]. A Cochrane systematic review informed us on interventions for increasing the use of SDM by health professionals in a variety of contexts [17]. Many interventions targeting patients, healthcare professionals or both were multi-component interventions designed to change health professionals’ performance and behavior, such as the way they present information and interact with patients [17]. Authors concluded that the certainty of the evidence was too low to determine if the interventions were effective or not for increasing the use of SDM, and explained that this was mainly due to poor reporting of results [17].

The completeness of reporting of interventions in primary studies is one of the key challenges for assessing their quality [18]. Incomplete reporting limits our ability to learn through systematic reviews, meta-analyses, and other forms of research synthesis [19]. It also makes it difficult to replicate or scale up effective interventions or learn from ineffective ones [20]. Incomplete reporting of intervention components makes it hard to identify the active ingredients linked to improvement [21]. In sum, incomplete reporting is a major source of waste in health research [22].

Initiatives such as the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network aim to improve the reliability and value of published health research literature by promoting transparent and accurate reporting and wider use of robust reporting guidelines [23]. The template for intervention description and replication (TIDieR), a reporting guideline, provides a structured framework for reporting interventions [22].

Reporting quality was not covered by the parent Cochrane review. Therefore, in order to promote better replication and scaling up of effective SDM interventions, we aimed to describe the quality of reporting of SDM interventions included in the review.

Methods

We performed a secondary analysis of a 2017 Cochrane systematic review entitled “Interventions for increasing the use of shared decision making by healthcare professionals” [17]). The Cochrane review sought eligible studies in CENTRAL, MEDLINE, Embase, Health Technology Assessment Database, NHS Economic Evaluation Database, PubMed, CINAHL EBSCO, PsycINFO Ovid, two clinical trial registries and proceedings of relevant conferences. The 87 studies included in the 2017 Cochrane review reported on interventions targeting either patients, health professionals or both [17]. The present study, a secondary analysis of the Cochrane review, analyzed all 87 included articles. Details on information sources, search methods, study selection, data collection and analysis, assessment of risk of bias, data synthesis and summary of findings of the original systematic review are available in full in the Cochrane Library [17].

There are currently no reporting guidelines specifically for secondary analyses of systematic reviews. Thus we used the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) checklist [24] (S1 Checklist) and the Guidelines for reporting meta-epidemiological methodology research [25] to report this study.

Data collection

Theoretical underpinnings

This secondary analysis used the Template for Intervention Description and Replication (TIDieR) checklist [22] as a reference for extracting items reported in SDM interventions. The TIDieR checklist was inspired by the SPIRIT checklist (Standard Protocol Items: Recommendations for Interventional Trials) and the CONSORT statement (Consolidated Standards of Reporting Trials) created to address poor reporting in protocols and randomized control trials [19, 20, 22]. While CONSORT and SPIRIT cover some elements of reporting on interventions (e.g. outcomes, population), TIDieR provides a complementary list of items to offer guidance for more complete reporting with the ultimate purpose of improving the efficacy and replicability of interventions [22].

The TIDieR checklist contains 12 items, or questions: 1) “Brief name?” i.e. a name or a phrase that describes the intervention; 2) “Why?” i.e. any rationale, theory, or goal of the elements essential to the intervention; 3) “What (materials)?” i.e. any physical or informational materials used in the intervention, including those provided to participants or used in intervention delivery or in the training of intervention providers; 4) “What (procedures)?” i.e. procedures, activities, and/or processes used in the intervention, including any enabling or support activities; 5) “Who?” i.e. expertise, background and any specific training given for each category of intervention provider (e.g. psychologist, nursing assistant); 6) “How?” i.e. the modes of delivery (such as face-to-face, internet or telephone) of the intervention and whether it was provided individually or in a group; 7) “Where?” i.e. the type(s) of location(s) where the intervention occurred, including any necessary infrastructure or relevant features; 8) “When and how much?” i.e. the number of times the intervention was delivered and over what period of time, including the number of sessions, the schedule, and the duration, intensity or dose; 9) “Tailoring?” i.e. if the intervention was planned to be personalized, titrated or adapted, what, why, when, and how; 10) “Modifications?” i.e. if the intervention was modified during the course of the study, what changes were made (what, why, when, and how); 11) “How well (planned)?” i.e. if intervention adherence or fidelity was assessed, how and by whom, and if any strategies were used to maintain or improve fidelity; 12) “How well (actual)?” i.e if the intervention adherence or fidelity was assessed, the extent to which the intervention was delivered as planned [22]. Unlike items 1 to 8, items 9 to 12 are conditional, i.e. reporting is only required if the intervention was planned to be tailored (item 9), if it was modified during the course of the study (item 10), or if adherence or fidelity were assessed (items 11 and 12) (See Table 1) [22].

Table 1. TIDieR items and examples of data extracted.

TIDieR item number	TIDieR item name	Data extracted	Examples
1	Brief name	a) Name	MyAsthma, Fiks 2015
		b) Acronym	BRIDGES (Building Recovery of Individual Dreams and Goals). Pickett 2012
		c) Brief description of the intervention	Behavioral SDM intervention for inpatients with schizophrenia. Hamann 2017
2	Why	a) Goal	“The CCPP package for patients aims to change patient behavior, and through these changes, alter physician behavior.” Butow 2004
		b) Rationale	“Decision-making preferences of patients with cancer are not always met, and often oncologists do not elicit these. Oncologists’ perceptions may be inconsistent with patients’ stated preferences, for example, in elderly patients [19]. These difficulties are heightened when discussing a clinical trial [20–23]. These issues suggest that targeted training in SDM is warranted. Communication training in decision making regarding standard care has been shown to be effective in a randomised controlled trial in the family practice setting.” Bernhard 2011
		c) Theory of the intervention	“Developed in 1986 after an extensive literature review and needs assessment, it was built around a new model of clinician-patient communication, the “4E Model” (Engage, Empathize, Educate, and Enlist) (Keller and Carroll, 1994), which includes key clinician-patient communication competencies detailed in the Kalamazoo Consensus Statement (Makoul, 2001).” Haskard 2008
3	What (materials)	a) Materials	“Control physicians received a brochure on prostate cancer screening that was distributed by the Centers for Disease Control and Prevention, whereas intervention physicians were exposed to an interactive, 30-minute, Web-based curriculum that included interactive roulette wheels,16 illustrative video vignettes, and other content to illustrate the potential harms, benefits, and downstream consequences of receiving prostate cancer screening, as well as methods of enhancing shared decision making.” Feng 2013
		b) Where they can be accessed (URL, appendix)	“The clinician can obtain an estimate of the patient’s 45 day pretest probability for acute coronary syndrome and download the decision aid corresponding to the appropriate level of risk at http://shareddecisions.mayoclinic.org/decision-aid-information/chest-pain-choice-decision-aid/. Write the patient’s name in the top left corner, and give the decision aid to the patient for subsequent review.” Hess 2016
4	What (procedures)	Procedure and/or Activities and/or Processes	“Patients randomly assigned to Group 1 (intervention) received three, two-hour trainings in active participation, patient empowerment, and communication. The 3 trainings occurred at approximately 2 weeks, 1–2 months, and 3–4 months after enrollment. The curriculum was developed in Namibia by local content experts and was framed by the social cognitive theory of self-efficacy [references]. The content was translated into the local Namibian languages of each participating region and site. Session 1: Learning to Speak to Providers begins to teach patients how to ask questions and explain their symptoms to doctors… All trainings were held on-site, at the ART facility, in a designated clinic space that was private and large enough for groups of five to six individuals. Six months after their enrollment date, participants in the control group (Group 2) were also offered training sessions as an ethically important intervention benefit.” Maclachlan 2016
5	Who provided	a) Intervention provider	“Two facilitators employed by the primary care trust delivered the training and also provided access to self management support activities and resources in the primary care trust.” Kennedy 2013
		b) Intervention provider’s expertise and/or background	“Nurse educators were trained to adopt a neutral stance regarding the performance of prostate cancer screening.” Myers 2011
6	How	Mode of delivery	“Physician training was delivered in small groups and office data collection depended upon the scheduling of research assistants.” Haskard 2008
7	Where	a) Setting	“The patient intervention and accompanying surveys were delivered to participants prior to regularly scheduled medical appointments in a private room in each practice.” Sheridan 2012
		b) Location	“The trial took place at 11 primary care and family medicine sites within the Mayo Clinic Health System and Olmsted Medical Center, all in southeast Minnesota.” Mullan 2009
		c) Infrastructure or relevant features	“Three of the four sites provided a computer for patient use at the office but the fourth required patient access to a computer at home or elsewhere.” Roter 2012
8	When and how much	a) When	“Clinicians in the intervention group were to use the decision aid during the consultation with their patients, while clinicians in the control arm did not have access to the decision aid (usual care).” LeBlanc 2015
		b) Frequency	“PCOMS therapists received 12 h of training during two days, with four weeks apart, with respectively eight and four hours of training.”Rise, 2012
		c) Duration	“The decision aid took between 11 and 34 minutes to complete, depending on which modules users chose to review.” Schroy 2016
9	Tailoring	If the intervention was planned to be personalised or adapted, and how.	“Building on previously developed evaluative guidelines we designed and piloted two different versions of a decision aid. Both versions included individualised risk and benefit presentation and a section to support shared decision-making.” Thomson 2007
10	Modifications	If the intervention was modified during the course of the study, and how.	“One version used explicit value elicitation employing the standard gamble method and a Markov decision analysis (“explicit tool”), the other included only the risk/benefit presentation (“implicit tool”). Early in the trial, the observational study showed that participants in the explicit arm found the elicitation of utilities using the standard gamble to be difficult, so this arm was discontinued (see Murtagh et al.) [references].” Thomson 2007
11	Fidelity (Planned)–If assessed	a) How and by whom intervention fidelity was assessed	“We also assessed, by reviewing the video-recorded encounters, the fidelity with which the decision aid was delivered and used as intended during these encounters using the osteoporosis fidelity checklist” Leblanc 2015b
		b) Strategies were used to maintain or improve fidelity	-
12	Fidelity (Actual)–If assessed	Extent to which the intervention was delivered as planned	“We also found that the fidelity with which the decision aid items were covered was high in the Decision Aid arm [67%, 95%CI (63, 78)]” Leblanc 2015b

To inform our qualitative observations for the latter two items, we referred to the notion of implementation fidelity as proposed by Carroll et al. [26]. Implementation fidelity refers to the degree to which an intervention or program is delivered as intended. This is important to gain a better understanding of how and why an intervention works, and the extent to which outcomes can be improved [26]. The measurement of implementation fidelity is the measurement of adherence, i.e. how far those responsible for delivering an intervention, or else its participants, actually adhere to the intervention as outlined by its designers. Adherence includes the subcategories of content, frequency, duration and coverage (also defined as “dose”) [26].

Procedure

Data extractors were given extraction instructions in an information session given by TTA, during which they pretested extraction methods on two studies. The extraction was performed by TTA, PR, ELA, AB, ATN, JVAF, MVRY and RA.

Pairs of extractors were chosen and each pair member extracted data from individually assigned studies. Pair members then met to compare and discuss extraction until consensus was reached. Any discrepancies in coding between pairs of coders were resolved by RA who was the designated conflict resolver.

Data extraction

For each study, we extracted intervention content according to the target group. In an Excel file we recorded the name of the first author, year, target group (patients, health professionals, or both) and each TIDieR item. To assess reporting of the 12 items we used a methodology developed by Tie P. Yamato and al. (2018) [27]. Each item was assessed on a 3-point scale with the following categories: not reported (NR), incompletely reported (IR), and adequately reported (R). For items classified R or IR, extractors quoted the author. For items containing sub-items (items 3, 5, 7, 8 and 11, see Table 1), all sub-items needed to be adequately reported to give the item the score “R”. If the item criterion was only achieved for one of the sub-items, the item as a whole was scored as “IR” (See S1 Table). Interventions targeting both health professionals and patients needed a unique reporting assessment so we merged their attributed categories. For instance, to declare Item 2-“Why?” adequately reported (“R”) in studies targeting both patients and health professionals, the rationale had to be adequately reported for both. The TIDieR items used for data collection and quotations representing items and sub-items from the intervention studies are presented in Table 1, which also lists the elements of each item extracted.

As not all items could be treated equally (the reporting of some items being conditional), we were not able to calculate an overall reporting score. Therefore, we present here the percentage of reporting of each item for each target group, and then of each item for the three groups as a whole. However, the results need to be interpreted carefully. “Not reported” for items 9 to 12 could represent a lack of description of the intervention (as for items 1 to 8) or that the intervention was not planned to be tailored, modified, or assessed.

The interventions were described using TIDieR by both researchers and service deliverers (clinicians, nurses etc.) to represent diverse voices [28].

Data analysis

We performed a descriptive analysis using frequency counts (number and percentage). For each target population of the interventions (patients, health professionals, and both) we recorded levels of reporting for each of the 12 items in the TIDieR checklist. We calculated the percentage of studies in which interventions were adequately reported (R), incompletely reported (IR) or not reported (NR) in general and with respect to the target population of the interventions. Analyses were performed using version 9.4 of SAS software. For items classified R or IR, we quoted the author and performed descriptive analyses of the extracted quotes.

Results

In the Cochrane systematic review on interventions to increase adoption of SDM by health professionals (version 2018), 87 intervention studies were included, of which 44 studies (51%) targeted only patients, 15 studies (17%) only health professionals, and 28 studies (32%) both health professionals and patients. Types of studies included were randomized trials (n = 83), non-randomized trials (n = 3) and a controlled before-after study (CBAs) (n = 1). They included intervention studies on SDM training programs as well as on SDM tools such as decision aids. See S2 Table for description of the 87 SDM interventions.

Adequacy of reporting per item

In general, of the 87 SDM interventions the percentage of items reported was as follows: Items with most complete reporting were: “brief name” (87/87, 100%), “why” (rationale) (86/87, 99%), and “what” (procedures) (81/87, 93%). Items with least complete reporting (under 50%) were “materials” (29/87, 33%), “who” (23/87, 26%), and “when and how much” (18/87, 21%); and the conditional items “tailoring” (8/87, 9%), “modifications” (3/87, 4%), “how well (actual)”, i.e. delivered as planned? (3/87, 3%). No intervention completely reported on “how well (planned—e.g. how was fidelity improved?) (Table 2). However, we could not always tell if studies had met the condition for these items to apply. Without eliminating the conditional items, the overall completeness of reporting would have been 41.5%. If items 9 to 12 are eliminated because of their conditionality, the overall completeness of reporting was 60.2%.

Table 2. Level of reporting of TIDieR items in SDM interventions.

Target group (n)	Patients (44)			Health professionals (15)			Both (28)			All (N = 87) (patients, health professionals and both)
Item number and meaning	R* n (%)	IR n (%)	NR n (%)	R n (%)	IR n (%)	NR n (%)	R n (%)	IR n (%)	NR n (%)	R n (%)	IR n (%)	NR n (%)
1-Brief name	44 (100)	-	-	15 (100)	-	-	28 (100)	-	-	87 (100)	-	-
2-Why	44 (100)	-	-	14 (93)	-	1 (7)	28 (100)	-	-	86 (99)	-	1 (1)
3-What (materials)	23 (52)	19 (43)	2 (5)	4 (27)	9 (60)	2 (13)	2 (7)	25 (89)	1 (4)	29 (33)	53 (61)	5 (6)
4-What (procedures)	44 (100)	-	-	14 (93)	1 (7)	-	23 (82)	5 (18)	-	81 (93)	6 (7)	-
5-Who	12 (27)	26 (59)	6 (14)	8 (53)	4 (27)	3 (20)	3 (11)	23 (82)	2 (7)	23 (26)	53 (61)	11 (13)
6-How	37 (84)	3 (7)	4 (9)	10 (67)	1 (7)	4 (26)	11 (39)	14 (50)	3 (11)	58 (66)	18 (21)	11 (13)
7-Where	38 (86)	4 (9)	2 (5)	-	15 (100)	-	-	28 (100)	-	38 (44)	47 (54)	2 (2)
8-When and how much	17 (39)	26 (59)	1 (2)	-	15 (100)	-	1 (4)	27 (96)	-	18 (21)	68 (78)	1 (1)
9-Tailoring	5 (11)	3 (7)	36 (82)	-	2 (13)	13 (87)	3 (11)	5 (18)	20 (71)	8 (9)	10 (12)	69 (79)
10-Modifications	3 (7)	-	41 (93)	-	-	15 (100)	-	-	28 (100)	3 (4)	-	84 (96)
11-How well (planned)	-	8 (18)	36 (82)	-	-	15 (100)	-	8 (29)	20 (71)	-	16 (18)	71 (82)
12-How well (actual)	3 (7)	25 (57)	16 (36)	-	13 (87)	2 (13)	-	21 (75)	7 (25)	3 (3)	59 (68)	25 (29)

* R = adequately reported; IR = incompletely reported; NR = not reported.

Adequacy of reporting per target population

Overall, when comparing reporting per item per intervention target (patients, health professionals or both), interventions targeting patients were better reported than the others. For example, 84% of patient-targeted intervention studies reported “How” (delivery modes), vs. 67% for health professionals, vs. 32% for both. Only “Who,” was better reported for interventions targeting health professionals (full details in Table 2).

Reporting levels over time

Studies were published from 1995 to 2017. In isolation, there was no clear trend in the level of reporting of each item by year of publication. But when reporting of items was compiled as a whole and compared by year, there was an upward trend in reporting levels as the years progressed (Fig 1). Since 2017 they have likely progressed further.

Fig 1. Level of reporting of TIDieR items by year of publication.

Qualitative reporting per item

Item 3-materials

Some authors provided a URL (link) or appendix to access the materials. Four out of the 23 links (17%) provided were no longer functional [29–32]. Among the functional links, seven (30%) led to a home page [33–39] from which it was not always clear where to go for the material. For 3 others, or about 16%, access to the material on the URL provided required a username and password [40–42].

Item 5-who provided

Of the 44 studies on patient-targeted interventions, 26 incompletely reported this item, e.g. did not report the expertise, background or any specific training given to the people who provided the intervention. Twenty of the 26 (77%) were interventions provided by clinicians (e.g. general practitioners, surgeons, nurses, endocrinologists) [39, 42–51], researchers [32, 52–56], clinician-researchers [57, 58] or both clinicians and researchers [36]. In three of the patient-targeted interventions where interventions were mailed to participants, authors did not provide any information about the person in charge of sending emails [31, 59, 60]. For the 28 studies on interventions targeting both health professionals and patients, 23 studies incompletely reported this item. Among these, nine studies [35, 61–68] provided full information for one target group (patients) but not for the other (health professionals). In ten of the 23 studies (43%) the interventions were provided by a member of the research team or project staff (e.g. study coordinator, lead investigator) [38, 39, 69–76]. This was also the case in two out of four studies that incompletely reported this item in the 15 studies on interventions targeting health professionals [77, 78].

Item 6-how

Eighteen studies overall incompletely reported this item. In six of these [52, 58, 66, 70, 77, 79], authors incompletely reported how one among the multiple intervention components was delivered. Twelve of the 15 studies on interventions targeting both patients and health professionals provided complete information for either the patients [35, 38, 61–64, 68, 71, 80, 81] or the health professionals [65, 82], but not both.

Item 7-where

In studies on interventions targeting health professionals (including studies targeting both health professionals and patients), most authors reported the clinical setting (e.g. primary care) but not the location where the intervention occurred (e.g. consulting room). As location was not completely reported, the infrastructure or relevant features of the location was missing. For interventions targeting patients, both setting and location were reported.

Item 8-when and how much

Compared to the “How much”, the “When” was rarely reported for interventions targeting health professionals, whether health professionals alone (13/15 studies, or 87%) or both health professionals and patients (21 studies out of 28 studies, or 75%), whereas in patient studies, the item was completely reported at 39% or incompletely reported at 59%.

Item 9-tailoring

Eighteen studies overall reported a form of tailoring [30, 36, 39, 50, 51, 55–57, 66, 70, 73, 75, 82–87]. Eight tailoring studies targeted patients [30, 36, 50, 51, 55–57, 85], two targeted healthcare professionals [83, 87] and eight targeted both [39, 66, 70, 73, 75, 82, 84, 86]. While ten studies incompletely reported tailoring [36, 39, 51, 55, 66, 73, 83, 84, 86, 87], eight studies reported the complete what, why, when and how of the tailoring [30, 50, 56, 57, 70, 75, 82, 85]. All studies reported what was tailored and how tailoring was performed. However, five interventions did not report why the need for tailoring [36, 51, 66, 83, 86] and six did not report or incompletely reported when tailoring occurred [36, 39, 55, 66, 83, 84, 86].

Item 10-modifications

Three studies reported on modifications, all of them on patient-targeted interventions [48, 85, 88]. Two specified the complete what, why, when, and how the intervention was modified during the course of the study [85] while the third did not specify when the intervention was modified [88].

Item 11-how well (planned)

Among the 16 studies that did report on this item, none reported it completely. Fifteen did not report by whom fidelity was assessed [33, 38, 39, 44, 46, 51, 68, 74, 76, 82, 86, 88–91]. Ten studies reported how they assessed fidelity to the content [38, 44, 51, 68, 76, 82, 86, 88, 90, 91], three reported how they assessed fidelity to coverage [33, 46, 89], two reported how they assessed both fidelity to content and to coverage [39, 74]. One study reported how and by whom fidelity to content was assessed, but did not report how and by whom other types of fidelity were assessed [57]. No study reported strategies used to maintain or improve fidelity. However, we do not know if the 71 studies who did not report on this item had performed an assessment at all.

Item 12-how well (actual)

Among the 62 studies that did report on this item, three studies reported on it completely [51, 57, 91]. These three studies reported fidelity to content and to coverage, while the other dimensions were not applicable [51, 57, 91]. Among the 59 studies that incompletely reported this item, 48 studies reported fidelity to coverage only [29, 30, 32, 33, 35, 36, 41–43, 45–47, 49, 50, 54, 55, 63, 64, 68–73, 75, 77–81, 83, 85, 87, 89, 92–106]. Four studies reported fidelity to content only [38, 44, 82, 86]. Four studies reported fidelity to content and to coverage [39, 74, 76, 90]. One study reported fidelity to content and to frequency [88]. One study reported fidelity to duration and to coverage [53]. However, we do not know if the 25 studies that did not report on this item had performed an assessment at all.

Discussion

In this secondary analysis of SDM intervention studies published up to 2017, we analyzed the quality of the reporting of SDM interventions using the TIDieR checklist. The three items most adequately reported were a brief name, why (rationale) and what (procedures). Other than the conditional items (9–12), the least reported items were what (materials), who, and when and how much. Interventions targeting patients were better reported than those targeting health professionals or targeting both. In multi-component studies, few authors reported on all components. Without taking into account that items 9 to 12 [item 9-Tailoring, item 10-Modifications, item 12-How well (actual), item 11-How well (planned)] were conditional, overall adequacy of reporting was 41.5%. With items 9 to 12 eliminated because of their conditionality, the overall adequacy of reporting was 60.2%. Also, we found that most links to relevant materials were not usable (e.g. no longer functional, password needed). These results lead us to make the following observations.

First, although authors made some effort to report on what materials were used (61%), this item was one of the least completely reported. For studies that did not report on how to access the material, it is possible that some of the materials are copyrighted and there may be a valid reason not to share them. We found that about 17% of URLs to the material were non-functional, and 16% of those that were functional required a username and password. This is consistent with the literature where the mortality rate of URLs found in five biomedical informatics journals was about 16.3% within the first year and reached 43.2% within five years [107]. Limited access to the materials prevents others from building on this work [108], and blocks replication and scaling up of effective strategies. Assessing the quality of the materials is also important for understanding why the intervention did or did not work as expected. Care must be taken when providing links to material that could change within a short time or that reside on restricted intranets or require user authentication [107]. An improvement in digital preservation is called for, as mentioned in the CONSORT-EHEALTH checklist [109], including defining strategies to allow permanent availability of URL references in scientific articles [107]. One strategy could be to reference URLs but include other means of locating the source if the URL expires [108]. Also, archival services (e.g. Internet Archive or Google’s cache) could be used to conserve information that has become unavailable elsewhere [107].

Second, our study shows that up to 2017, authors of most of the studies we analyzed had reported incompletely on their targeted groups or components of their multi-component SDM interventions. Reporting on interventions targeting health professionals was much less rigorous than on interventions targeting patients, perhaps because researchers are more used to following data collection protocols for patient participants than for health professional participants. Describing all the components of an intervention indicates to those wishing to replicate the intervention which elements are essential as opposed to optional or incidental [22]. However, while TIDieR captures many important aspects of SDM interventions, it may miss other more complex ones. For example, there is an apparent contradiction between fidelity (staying true to the original intervention) and tailoring it to fit new contexts. Although core components of an intervention may change over time, it may be helpful to distinguish between core components of an intervention and its modifiable components [28, 110, 111]. Indeed, complex interventions pose numerous problems for evaluators, including the practical and methodological difficulties that any successful evaluation must overcome [112]. Especially in a field where interventions are both multi-component and highly heterogenous, as shown in the Cochrane review, comparing the effect of interventions is difficult. However, if the separate components of multi-component interventions are well reported, it is more feasible. It is not possible for authors of incompletely reported multi-component intervention studies to explain the observed effect of the components of their interventions, or to compare the effect of these components, or indeed to compare them to other studies [17]. This highlights the importance of following guidance [112] or standards for reporting of all components such as the standards for reporting implementation studies (StaRI) [113].

Third, the percentage of items reported for the SDM interventions overall, using the TIDieR checklist, was less than 50%. This is congruent with other studies of the quality of reporting on behavior change interventions. A report on smoking cessation interventions found that published descriptions of behavioral support mentioned fewer than half of the behavior change techniques specified in the intervention manuals [114]. Indeed, according to a review of nearly 1,000 behaviour change outcome studies, interventions were described in detail in only 5% to 30% of experimental studies [115]. In the pharmacological field, a recent analysis found that only 11% of 262 trials of cancer chemotherapy provided complete details of the trial treatments [22]. Using the CONSORT checklist, a review over 100 herbal medicine trials (in the field of alternative medicine) found an average of 38% of the information was reported [116]. Under-reporting of interventions is thus common to several areas in the world of health research. This is not only a threat to fidelity and replicability of interventions, but also prevents the scaling up of interventions that could potentially benefit much larger populations. However, while the TIDieR checklist is in theory applicable to any clinical intervention study, some details relevant to SDM interventions in particular are not considered. In reporting on materials used with patients (e.g. decision aids or question prompts), few studies included details about how these tools were developed, e.g. if they were field tested or if they complied with the International Patient Decision Aid Standards (IPDAS) criteria [117, 118]. These details could improve the reporting of the quality of the materials. Our study highlights the need to reach a consensus on how to report this type of intervention.

Fourth, the least reported items were the optional TIDieR items. Tailoring, modifications during the course of the study and planned and actual fidelity of the intervention are rarely reported. The problem here was that we could not tell if tailoring, modification or fidelity assessments had really not taken place, or if the author omitted to present them. This was a limitation of the conceptual framework used, as it gave us a misleading picture of reporting completeness. The conditional items should be subdivided into two levels. For example, the modification question should begin “Did you modify the intervention?” followed by, at a second level, “If so, describe the modifications.” Also, all experts involved in the development of StaRI agreed that fidelity to the core components of the planned intervention and any modifications or adaptations during the course of the study should be reported first, then reported in the results section [113]. However, in the 87 studies we analyzed, modifications were reported in the methodology and in the discussion but not in the results. Placing this information in the results section would encourage reporting on the results of modifications and their impacts on the results of the study. This information is necessary to reconstruct the puzzle of the study and determine causal links between actions taken and results observed.

Strengths and limitations

The 2017 Cochrane review on SDM interventions provided us with a rich and unexplored source of data on a topic important for all implementation science researchers and publishers. Our use of TIDieR to verify the completeness of the published information on the SDM interventions and the shortcomings it highlighted underlined the importance of using reporting guidelines such as TIDieR to improve replicability and scalability.

Our study has some limitations. First, it was a secondary analysis and data were therefore not collected primarily for the purpose of this study. Second, since 2017 there has been an increase in studies on SDM in mental health. While a few were considered in this 2017 search, most were not. Thus the current recommendations do not represent/address all conditions and patient populations. Lastly, we did not contact authors of the included studies and thus our assessment is purely based on published information.

Conclusions

In this review, we analyzed the reporting of SDM interventions using the TIDieR checklist. The results revealed not only that many items were incompletely reported, but that few authors reported on all groups targeted or all components of their SDM interventions. These results will provide guidance for the community of SDM intervention developers on what elements need better reporting to improve the replicability of their interventions.

Supporting information

S1 Checklist. PRISMA 2009 checklist.

(DOC)

S1 Table. Rating of sub-items.

(PDF)

S2 Table. Description of shared decision making interventions.

(PDF)

S1 File. Database.

(XLSX)

Abbreviations

CONSORT

Consolidated Standards of Reporting Trials

CONSORT-EHEALTH

Consolidated Standards of Reporting Trials of Electronic and Mobile HEalth Applications and onLine TeleHealth

DA

Decision aid

IPDAS

International Patient Decision Aid Standards

SDM

Shared decision making

SPIRIT

Standard Protocol Items: Recommendations for Interventional Trials

StaRI

Standards for Reporting Implementation Studies

TIDieR

Template for Intervention Description and Replication

Data Availability

All relevant data are within the paper and its Supporting information files.

Funding Statement

Financial support for this study was provided entirely by the Tier 1 Canada Research Chair in Shared Decision Making and Knowledge Translation. FL holds the Chair. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.