Abstract
Introduction
Deprescribing (medication dose reduction or cessation) is an integral component of appropriate prescribing. The extent to which deprescribing recommendations are included in clinical practice guidelines is unclear. This scoping review aimed to identify guidelines that contain deprescribing recommendations, qualitatively explore the content and format of deprescribing recommendations and estimate the proportion of guidelines that contain deprescribing recommendations.
Methods
Bibliographic databases and Google were searched for guidelines published in English from January 2012 to November 2022. Guideline registries were searched from January 2017 to February 2023. Two reviewers independently screened records from databases and Google for guidelines containing one or more deprescribing recommendations. A 10% sample of the guideline registries was screened to identify eligible guidelines and estimate the proportion of guidelines containing a deprescribing recommendation. Guideline and recommendation characteristics were extracted and language features of deprescribing recommendations including content, form, complexity and readability were examined using a conventional content analysis and the SHeLL Health Literacy Editor tool.
Results
80 guidelines containing 316 deprescribing recommendations were included. Deprescribing recommendations had substantial variability in their format and terminology. Most guidelines contained recommendations regarding for who (75%, n=60), what (99%, n=89) and when or why (91%, n=73) to deprescribe, however, fewer guidelines (58%, n=46) contained detailed guidance on how to deprescribe. Approximately 29% of guidelines identified from the registries sample (n=14/49) contained one or more deprescribing recommendations.
Conclusions
Deprescribing recommendations are increasingly being incorporated into guidelines, however, many guidelines do not contain clear and actionable recommendations on how to deprescribe which may limit effective implementation in clinical practice. A co-designed template or best practice guide, containing information on aspects of deprescribing recommendations that are essential or preferred by end-users should be developed and employed.
Trial registration number
osf.io/fbex4.
Keywords: Clinical practice guidelines; Decision support, clinical; Evidence-based medicine; Medication safety; Clinical pharmacology
WHAT IS ALREADY KNOWN ON THIS TOPIC.
WHAT THIS STUDY ADDS
While deprescribing recommendations are increasingly being incorporated into guidelines, there is significant variability in their content and format. Few guidelines contain clear, comprehensive and actionable recommendations on how to deprescribe.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Guideline developers need to consider deprescribing within the scope of guidelines and optimise the content, format and language of recommendations to enable ease of interpretation and implementation.
Introduction
Potentially inappropriate medicines (PIMs) are medicines where the potential harm outweighs the expected benefit.1 2 In the UK, an estimated 10% of the 1.1 billion medicines dispensed each year are deemed potentially inappropriate,3 contributing to undesired outcomes including drug-drug interactions, adverse drug reactions, non-adherence, falls, hospitalisations and mortality.4 5 High rates of PIM use have been observed internationally, with one-third to a half of older adults taking one or more PIMs.6,8
Clinical practice guidelines (guidelines) are statements containing recommendations informed by a systematic review of the evidence, intended to mitigate variation in practice and optimise patient care.9 While guidelines play a vital role in promoting evidence-based care, they may not prompt consideration of how to optimally manage the complexities or competing demands of multimorbidity or frailty, nor take into account the potential impacts of drug-drug and drug-disease interactions.10 11 Well-intentioned healthcare professional adherence to prescribing recommendations contained in guidelines may therefore be a driver of PIM use.
Deprescribing, an integral component of appropriate prescribing,12 refers to the systematic process of dose reducing or discontinuing a medicine under the supervision of a healthcare professional.13 At a societal level, deprescribing plays an important role in reducing low-value care and overtreatment, as well as avoiding preventable medicine-related harm.14 15 At the patient level, benefits may include improved adherence, reduced medicine-related costs and burden, and resolution of adverse drug reactions.16 17 However, implementation of deprescribing in practice remains limited due to various patient, prescriber and organisation-level barriers.18 Specifically, limitations in healthcare professionals’ knowledge of when and how to safely and effectively deprescribe, and their perceived lack of self-efficacy to cease medications have been reported.19 Such findings suggest that additional guidance for healthcare professionals may increase deprescribing in practice.
Guidelines have the potential to support deprescribing, yet historically, they have included limited deprescribing recommendations.11 20 21 Over the last decade, medication-class-specific deprescribing guidelines have been developed,22,27 but their recommendations have not yet been integrated into treatment guidelines. Additionally, despite language being recognised as a core determinant of guideline implementability,28 29 there is limited evidence on whether the language of deprescribing recommendations is clear and actionable, facilitating safe and effective deprescribing in practice. This review aimed to identify guidelines containing deprescribing recommendations, qualitatively explore the content and format of deprescribing recommendations and estimate the proportion of guidelines that contain deprescribing recommendations.
Methods
A scoping review was conducted in accordance with the Joanna Briggs Institute methods30 and the Preferred Reporting Items for Systematic review and Meta-Analysis extension for Scoping Reviews statement,31 by researchers and healthcare professionals with expertise in deprescribing, guideline development and scoping review methodology. The review protocol is hosted on Open Science Framework.32
Operational definitions
Clinical practice guideline: A guidance document developed by a multidisciplinary team that includes recommendations (or equivalent based on different terminology used internationally) that can be applied by a healthcare professional to inform patient care.9 Recommendations must be informed by a systematic literature review (or termed an ‘extensive evidence review’ for National Institute for Health and Care Excellence guidelines). For the purpose of this review, clinical practice guidelines are either;
Treatment guidelines
A clinical practice guideline that primarily focuses on the treatment of a condition and contains prescribing recommendations, but may also contain deprescribing recommendations.
Deprescribing-focused guidelines
A clinical practice guideline that primarily focuses on deprescribing medications.
Recommendations
Actionable statements contained within a clinical practice guideline, intended to optimise patient care. Recommendations can be;
Evidence-based
Guidance informed by a systematic review of the evidence in addition to an assessment of the benefits/harms of alternative treatment options.9
Good practice statements
Guidance informed by indirect evidence outside the scope of a systematic search strategy that a guideline panel feels are important to include but are not appropriate for formal ratings of the quality of evidence.33 Also referred to as ‘best practice statements’ or ‘practice points’.34 35
Data sources and searches
Three search strategies were developed to identify eligible guidelines, executed in; (1) bibliographic databases, (2) Google and (3) guideline registries.
A systematic, bibliographic database search of MEDLINE via OVID, EMBASE via OVID and CINAHL Plus via EBSCOhost was conducted with results limited to the past 10 years (January 2012 to November 2022). Search terms related to ‘deprescribing’ and ‘guidelines’ were used (see online supplemental file 1 for full search strategy).
A restricted Google search was performed in February 2023 using incognito mode and the search term; ‘Guidelines (deprescribing or withdrawal or stop or discontinue or medication)’.
Four guideline registries were queried in February 2023; Guidelines International Network, Canadian Medical Association Infobase, MAGICapp and GuidelineCentral, to identify all guidelines published in English from January 2017 to February 2023. The purpose of the guideline registry search was to identify additional eligible guidelines for inclusion in the review and estimate the proportion of guidelines containing deprescribing recommendations. Restricting the registry search to the last 5 years (from the date of our initial search in November 2022) was both a pragmatic decision based on anticipation of a large number of guidelines that would be identified, and a mechanism to allow for an up-to-date estimate of the prevalence of deprescribing recommendations in current guidelines.
Additional eligible guidelines were identified through citation searching of articles identified in each of the aforementioned search strategies.
Study selection
Eligibility criteria
Guidelines were included if they met our operational definition of a clinical practice guideline (based on the Institute of Medicine’s criteria9), were published in English from 2012 onwards, and contained at least one deprescribing recommendation (ie, for who, when, when not, why or how to dose reduce, cease or substitute a medication with a medication from a different class). If multiple versions of a guideline were identified, only the most recent update was used.
Guidelines that focused on acute treatment (eg, emergency department or intensive care unit settings, medicines used short term for acute conditions) or related exclusively to surgery or interventional investigations were excluded. Guidelines that only contained recommendations for pharmacological substitution to a medication within the same drug class, or suggested dose adjustments based on physiological characteristics (eg, renal or hepatic function) and those relating to treatment of substance use disorders, vaccines, blood products or illicit substances were also excluded. The full eligibility criteria are presented in online supplemental file 2.
Screening
A piloting exercise for each search strategy was conducted prior to screening commencement.
All records identified through the bibliographic database search were imported into Covidence. Following de-duplication, two rounds of screening (title and abstract, then full text) were performed in duplicate by two independent reviewers.
For the restricted Google search, the first 200 retrieved results were extracted into an Excel spreadsheet. Two independent reviewers screened 50 results at a time, starting with the most relevant results at the top of the page. An a priori decision was made for screening to conclude when no relevant results were identified in the previous batch of 50 records.
For the registries search, all guidelines were exported into Excel, from which a 10% sample from each registry was randomly selected using the ‘Randomise Range’ function. This random 10% sample was chosen due to the large number of guidelines contained in the registries, and their limited search functionality. This sample provided a practical number of guidelines for screening and enabled an estimation of the prevalence of relevant guidelines containing at least one deprescribing recommendation. Two screening phases were performed by two independent reviewers. The first screening phase applied all eligibility criteria to the full-text guidelines, except for whether the guideline contained a deprescribing recommendation or not. The second screening phase determined whether the guidelines identified contained one or more deprescribing recommendations.
For all three search strategies, any potentially eligible guidelines identified through citation searching were screened in duplicate to determine eligibility. Disagreements regarding study inclusion were resolved through discussions between the reviewers and the first and senior author if required.
Data extraction
Data extraction was piloted and performed by two independent reviewers using a standardised data collection form (online supplemental file 3). Characteristics of the guideline (eg, region and year of publication) and recommendation characteristics (eg, strength and certainty of evidence, based on an approach such as Grading of Recommendations Assessment, Development and Evaluation (GRADE))36 were extracted. Each guideline was assigned a first-level Anatomical Therapeutic Classification code, developed by the WHO, based on the physiological system that the medication(s) in the guideline targeted. This classification was used to group guidelines and explore which medical specialties routinely include deprescribing recommendations in their guidelines. The strength and certainty of the evidence for each recommendation were extracted and converted to a standardised format using methods described by Klugar, et al.37
Data synthesis and analysis
Descriptive statistics were used to summarise the characteristics of the included guidelines and deprescribing recommendations. Frequencies and percentages were used for categorical characteristics and means with SD or medians and ranges were reported for numerical characteristics, dependent on whether the data was normally distributed.
A conventional content analysis38 was performed to examine the language features of each deprescribing recommendation. An inductive approach was employed whereby the text of each recommendation was reviewed word by word and open coding was performed to create categories and subcategories. These were iteratively refined following group discussions between the research team. Each deprescribing recommendation was coded by two independent reviewers, then codes were compared and an agreement was reached. Inductively derived categories were then mapped to a framework, informed by Bloom and Lahey’s model for defining language which recognises three separate but intersecting language concepts influencing meaning: content, form and use.39 As written, rather than spoken text was examined, inductively derived categories were mapped to the form and content components.
Using NVivo (V.13),40 matrix tables were produced to explore variability in characteristics of the deprescribing recommendation language, dependent on whether the guideline was deprescribing-focused or not. The complexity and readability of recommendations were calculated using the SheLL (Sydney Health Literacy Lab) Health Literacy Editor,41 providing a quantitative assessment of text complexity by assessing factors such as sentence length, syllables, uncommon words and acronyms. A higher value represents a greater level of complexity. A readability score was also generated, representing the grade of literacy expected for someone to be able to read/understand specific text (correlating with school grades). Complexity and readability scores were calculated to determine the ease at which a guideline end-user could comprehend the content of each deprescribing recommendation.
Results
Search results
2866 records were identified from the bibliographic database search, 200 from the Google search, and 2023 from the registry search (figure 1). Following citation searching, screening and duplicate removal, 80 guidelines containing one or more deprescribing recommendations were included.
Figure 1. Study selection - Preferred Reporting Items for Systematic review and Meta-Analysis 2020 flow diagram.
Guideline characteristics
The majority of the 80 included guidelines were developed in Europe (41%) and North America (40%), with a minority originating from Australia (6%), Asia (1%) and Africa (1%) (online supplemental table 1). A general upward trend in the number of guidelines containing deprescribing recommendations was observed over the examined timeframe (online supplemental figure 1). Most guidelines (90%) were treatment rather than deprescribing-focused (10%), and predominantly concerned diseases and/or medicines relating to the nervous system (38%), antineoplastic and immunomodulating agents (26%) and the alimentary tract and metabolism (13%). Less than half the guidelines (43%) contained a deprescribing recommendation for each medicine that had a prescribing recommendation. For example, a guideline on chronic obstructive pulmonary disease contained recommendations about initiating a range of medication classes including long-acting β2-agonists, long-acting muscarinic antagonists, inhaled corticosteroids and opioids for dyspnoea, yet only included a deprescribing recommendation for inhaled corticosteroids.42
Deprescribing recommendation characteristics
The 80 guidelines contained a total of 3569 evidence-based recommendations, of which 248 pertained to deprescribing (table 1). The median number of recommendations per guideline was 24.5 (range; 2–233) and the median number of evidence-based deprescribing recommendations per guideline was 2 (range; 0–14). On average, 7% (248/3569) of evidence-based recommendations identified in the guidelines pertained to deprescribing. An additional 68 deprescribing good practice statements were identified, resulting in a total of 316 deprescribing recommendations. Of these deprescribing recommendations, 9% (28/316) advised on when not to deprescribe (table 1).
Table 1. Deprescribing recommendation characteristics.
| Direction of deprescribing recommendations (n=316) | |
| For deprescribing* | 288 (91%) |
| Against deprescribing† | 28 (9%) |
| Readability score‡ of deprescribing recommendations (n=316) | |
| Mean (SD) | 17.3 (3.54) |
| Complexity score§ of deprescribing recommendations (n=316) | |
| Mean (SD) | 40.7 (10.85) |
| Evidence-based recommendation (EBR) or good practice statements (GPS) (n=316) | |
| EBR | 248 (78%) |
| GPS | 68 (22%) |
| Strength of EBRs (n=248) | |
| Strong¶ | 41 (17%) |
| Weak or conditional** | 77 (31%) |
| Consensus | 17 (7%) |
| Not specified | 113 (46%) |
| Certainty of evidence for EBRs (n=248) | |
| High | 5 (2%) |
| Moderate | 25 (10%) |
| Low | 32 (13%) |
| Very low | 43 (17%) |
| No evidence | 1 (0.4%) |
| Not specified | 142 (57%) |
Recommendations about for Who, What, When, How or Why to deprescribe were considered For deprescribing.
Recommendations advising on when not to deprescribe were considered Against deprescribing.
A lower grade reading score means the text is easier to read. Grade 88 or lower is appropriate for most audiences.
A lower complexity score means the text is easier to read. Higher complexity suggests a higher use of words that are uncommon in English.
Strong recommendations suggest that all or almost all persons would choose that intervention.43
Conditional/Wweak recommendations imply that there is likely to be an important variation in the decision that informed persons are likely to make.43
The 248 evidence-based deprescribing recommendations were classified as either strong43 (17%), or conditional/weak43 (31%). Of the remainder, 113 (46%) did not specify the strength, and 17 (7%) self-described the recommendation as consensus-based. The certainty of the evidence informing the evidence-based deprescribing recommendations was variable, ranging from high (2%) to very low (17%). Over half (57%) did not specify the evidence certainty.
The average language complexity score (41%) and reading grade of deprescribing recommendations (17) were high (table 1), with only one recommendation falling below the recommended reading grade for the public (grade 8).41 Most readability scores were above 16, which are deemed ‘very difficult to read’.44
Conventional content analysis
Each of the 316 deprescribing recommendations were individually coded to seven categories, with subcategories. These categories were mapped to two overarching language concepts; (1) content and (2) form (figure 2).39 The content concept comprised categories relating to the recommendation’s subject matter, specifically whether a recommendation indicated for who, what, when/why or how to deprescribe. The form concept related to the presentation of the recommendation and the terminology used.
Figure 2. Language framework of deprescribing recommendations.
Content
Most guidelines contained recommendations about who may be eligible for deprescribing (ie, identifying the population of interest), what to deprescribe (ie, the medicine/drug class) and when or why to deprescribe (table 2). When and why were grouped due to significant overlap between these concepts (eg, a recommendation suggesting medication cessation when a person was in disease remission encompassed both when and why to trial deprescribing). Although 91% of guidelines contained one or more recommendations about when or why to deprescribe, clarity was often lacking. For example, one recommendation stated ‘…should be minimised to less than 7.5 mg/day (prednisone equivalent) and, when possible, withdrawn’,45 alluding to when to deprescribe, yet not providing clear directives in terms of duration of use.
Table 2. Content characteristics of deprescribing recommendations identified through conventional content analysis.
| Content categories and subcategories | Number of guidelines, n (%), total N=80
|
Example quotes |
| Who | ||
| Recommendation specified the population of interest by defining their age, sex, disease state, medication use or specific, clinical characteristics. | 60 (75)
|
‘In adults aged 65 years and older…’‘In patients with Crohn’s disease…’‘For patients taking triple therapy…’ |
| What | ||
| Recommendation specified the medication or medication class to be/not be deprescribed and/or the indication for the medication. | 89 (99)
|
‘…antipsychotics for the treatment of insomnia’‘opioids for chronic non-cancer pain…’ |
| When/why | ||
| Recommendation provided a rationale and or/timing for deprescribing, including: | 73 (91)
|
|
| Adverse drug reaction | 35 (44) | ‘If tics are stimulant related, reduce the stimulant dose…’‘For patients who cannot tolerate a statin, we suggest a washout period, followed by…’ |
| Specific timeframe/duration | 34 (43) | ‘an attempt to taper and withdraw the drug should be made within 4 months.’‘After a person has been seizure-free for 2 years…’ |
| Remission/stable disease | 28 (35) | ‘Tapering and discontinuing glucocorticoids is strongly recommended after inactive disease has been attained.’ |
| Change in specific clinical parameter(s) | 25 (31) | ‘It is suggested to discontinue metformin for the management of type 2 diabetes mellitus in patients with 2 or more of the follow risk factors: age>80; gastrointestinal complaints during the last year; GFR<60 mL/min…’ |
| Treatment is no longer effective | 23 (29) | ‘Discontinue opioid therapy for lack of response…’‘If symptoms have not improved, stop LAMA+LABA+ICS…’ |
| Changes in individual health circumstances | 21 (26) | ‘Individualising glycaemic targets to goals of care…and deprescribing accordingly.’ |
| Pre-conception, pregnancy or breast feeding | 10 (13) | ‘If a woman becomes pregnant, discuss stopping the medication…’‘If a woman taking psychotropic drugs is planning a pregnancy…’ |
| Progressed stages of disease | 10 (13) | ‘…the individual has severe or end-stage dementia…’‘For persons with a severe opioid use disorder…’ |
| Patient preference/decision | 9 (11) | ‘a decision by a person with dementia and/or their family/carer to discontinue the medication…’‘…patients taking triple therapy who wish to discontinue a DMARD…’ |
| Other | 10 (13) | ‘Where rapid access to care is challenging…’‘…a person with dementia’s refusal or inability to take the medication…’ |
| How – brief process | ||
| Recommendation provided some information about how to deprescribe, but information was not comprehensive. | 54 (68)
|
‘We recommend gradual tapering of opioids…’‘Taper or terminate therapy.’ |
| How - detailed process | ||
| Recommendation provided detailed information on how to deprescribe, including: | 46 (58)
|
|
| Detailed dose reduction plan | 26 (33) | ‘Reduce to 75%, 50%, and 25% of the original dose on a biweekly basis before stopping.’ |
| Counselling patient/shared-decision making | 22 (28) | ‘…a discussion of the potential benefits and risks of bDMARD discontinuation after dose reduction may form part of a shared decision-making plan.’ |
| Developing a deprescribing plan | 6 (8) | ‘We suggest developing and implementing a deprescribing plan…’ |
| Preventing or managing withdrawal symptoms | 5 (6) | ‘For people who have been on long-term opioid therapy…the rate of reduction may need to be slower to prevent withdrawal’. |
| Monitoring or follow-up | 27 (34) | ‘…a trial discontinuation, with close periodic monitoring (eg, every 4 weeks)…’ |
| Recommending use of co-interventions | 21 (26) | ‘emphasising the use of non-pharmacological options…’‘Options include: adding a group exercise intervention…’ |
| Individualising patient care | 14 (18) | ‘tapering might need to be individualised depending on the starting dose, available dosage forms, and how tapering is tolerated.’ |
bDMARD, biological DMARD; DMARD, disease-modifying anti-rheumatic drugs; GFR, glomerular filtration rate; ICS, inhaled corticosteroid; LABA,long-acting beta-agonistLAMA,long-acting muscarinic antagonist
The how category was dichotomised into two subcategories to convey the difference in detail provided by recommendations. A brief process described recommendations that had some information about how to deprescribe (eg, ‘reduce the dose gradually’46), whereas detailed process recommendations provided additional guidance on dose reduction (eg, ‘Reduce to 75%, 50%, and 25% of the original dose on a biweekly basis before stopping’25) monitoring, counselling or the implementation of co-interventions to facilitate deprescribing. Although 68% of the 80 guidelines contained one or more recommendations about how to deprescribe, only 58% provided detailed guidance.
When comparing deprescribing-focused guidelines and treatment guidelines, deprescribing-focused guidelines consistently addressed each content category of Bloom and Lahey’s framework to a greater extent than their treatment counterparts (table 2).
Form
The syntax/structure of the 316 deprescribing recommendations was commonly a single sentence (54%), multiple sentences (22%) or as a stem-sentence with accompanying bullet points (19%) (online supplemental table 2). Significant variability in the terminology used to indicate deprescribing was identified. In total, 12 deprescribing synonym subcategories were defined, with the terms discontinue, reduce, stop, withdraw and taper being the most prevalent. The term deprescribe was not used in any treatment guidelines and was only identified in three deprescribing-focused guidelines. Multiple terms were used within a single guideline, and even within individual recommendations. Similarly, an array of synonyms for the term recommend were used, with varying levels of obligation. The vocabulary of should, recommend and can or may appear most frequently, with the term suggest only used 8% of the time (online supplemental table 2).
Proportion of guidelines from the registries search containing a deprescribing recommendation
The guideline registry search identified 2023 guidelines, of which a 10% sample (n=202) were screened. Of these, 49 guidelines were potentially eligible for inclusion (ie, fulfilled all eligibility criteria except the requirement to have a deprescribing recommendation). After the second screening phase, 14 guidelines were included in the review as they contained one or more deprescribing recommendations. Based on these findings, 29% (n=14/49) of relevant guidelines from the past 5 years contained at least one deprescribing recommendation.
Discussion
Integration of deprescribing recommendations into clinical practice guidelines has been proposed as a mechanism to minimise potentially inappropriate medicine use and reduce medicine-related harm.47 The large number of deprescribing recommendations identified in this review (n=316), as well as the upward trend in guidelines containing deprescribing recommendations over time, may reflect a growing awareness of the need to consider deprescribing. However, less than one-third of potentially eligible guidelines identified in the registries search contained a deprescribing recommendation and only 7% of evidence-based recommendations from the 80 included guidelines pertained to deprescribing. Additionally, less than half of these guidelines had a deprescribing recommendation for all medications with a prescribing recommendation. Such findings indicate a paucity of adequate deprescribing guidance and a need for developers to consider including deprescribing when defining the scope of guidelines and when formulating recommendations.
Internationally, guideline development handbooks and checklists stipulate that recommendations should specify core information, including the population of interest and a description of which options are appropriate in which situations.48,51 The conventional content analysis captured these factors via the who, what and when/why categories, finding that each were commonly addressed. In contrast, insufficient information on how to deprescribe was identified, despite this being recognised as a key enabler to achieving safe and effective deprescribing in practice.52 Lack of detailed guidance on how to deprescribe could be attributable to a range of factors, including the typical Population, Intervention, Comparison, Outcome format used to generate guideline questions,50 as well as limited high-certainty evidence to inform an optimal deprescribing approach.53 Given deprescribing studies often lack comprehensive information about interventions and relevant outcomes (eg, how long tapering took, when and what withdrawal symptoms occurred),54,57 guideline panels may be deterred from making specific how-focused recommendations. It is also possible that how is a concept highly relevant to deprescribing, but less pertinent for prescribing recommendations as medication initiation may be more straightforward or familiar to guideline end-users. Deprescribing-focused guidelines addressed the how category more consistently than their treatment counterparts. However, deprescribing guidelines currently exist for only a limited number of medication classes22,27 and expanding them to cover every medication class could lead to guideline fatigue and underuse.58 As such, routine integration of deprescribing recommendations into all clinical practice guidelines may enhance both the reach and adoption of deprescribing recommendations and support changing the culture of medicine so that deprescribing is seen as a regular part of care.47 Deprescribing recommendations can be developed using the same methods used for other recommendations.51 However, several challenges to including deprescribing recommendations in treatment guidelines have been suggested, including additional time, funds and expertise required for development,47 59 as well as a lack of incentives or requirements for developers to do so.3648,50 It could also be argued that deprescribing is not within the scope of all treatment guidelines (as they may focus only on the early stages of treatment), and there are potential benefits of standalone deprescribing guidelines, such as having the opportunity to provide detailed and comprehensive deprescribing information which may not be a priority within the broader scope of a treatment guideline.
While more comprehensive deprescribing recommendations may facilitate the implementation of deprescribing in practice, there remains a need for recommendations to be simple and digestible. Guideline language is integral to implementability60 and recommendations should be uncomplicated, clear and persuasive.29 The deprescribing recommendations analysed in this review were often complex and contained imprecise language, (eg, ‘reduce the dose gradually’).46 Such guidance does not provide sufficient detail to enable the development of a specific deprescribing plan, or mitigate variation in clinicians’ practice, despite this being a key objective of guidelines.61 Although clinician expertise and discretion can be used to bridge the gap between recommendations and practice, in the absence of clear deprescribing recommendations, there is concern that recommendations will be misapplied, leading to unintended harm. For example, in the USA, the 2016 National Centres for Disease Control Guideline for Prescribing Opioids for Chronic Pain recommended avoiding opioid doses above defined thresholds and suggested tapering and discontinuation of opioids if the benefits did not outweigh the harms.62 A lack of specific and actionable opioid deprescribing recommendations may have contributed to observed unintended consequences, including inappropriate tapering and termination of care.63 64 As such, it is imperative that deprescribing recommendations are thoughtfully worded to enable them to be applied as intended. Further, as language complexity is inversely proportional to recommendation adherence,65 66 strategies to reduce the high complexity and readability scores of recommendations may be required, even for healthcare professionals who are assumed to be able to understand information pitched at a higher reading grade than the general public.
Substantial variability in the terminology used to convey deprescribing and recommend was noted in this review, suggesting that there is no standardised approach for formatting deprescribing recommendations. Some variability in terminology is to be expected, with guideline panels likely using discipline-specific vocabulary (eg, step-down was frequently identified in rheumatology-related guidelines). However, terms with multiple meanings, such as withdraw can be unclear, referring to medication cessation but also adverse drug withdrawal reactions. The use of imprecise language runs counter to the Institute of Medicine’s charge for guidelines to use ‘unambiguous language and define terms precisely’.9 Similarly, the GRADE group advocates for guideline developers to make explicit connections between recommendation wording and the intended level of obligation to avoid subjectivity,67 promoting the term recommend or should for strong recommendations, and suggest or might for conditional recommendations.68 Despite GRADE being widely available and endorsed for use in guideline development, these terms were not used uniformly. Efforts are therefore required to explore contextual considerations across disciplines, countries and cultures that may influence preferred language, as well as understand how to incentivise developers to employ consistent approaches when crafting deprescribing recommendations, recognising that multiple terms may be required to convey nuances in meaning.
Strengths and limitations
To our knowledge, this is the largest review of deprescribing recommendations in clinical practice guidelines, exploring multiple disease states and drug classes. A rigorous search of the literature was conducted, supplemented by comprehensive citation searching. Screening, data extraction and content analysis were performed in duplicate by independent reviewers and each step was piloted to ensure process consistency. A limitation was only screening a 10% sample of the guideline registries, however, this was deemed practically necessary. The estimate of prevalence obtained from this sample only considered guidelines focused on long-term treatment, therefore the overall proportion of guidelines with deprescribing recommendations is likely lower. Additionally, as this was a scoping review, we have not conducted any statistical analyses to determine trends or associations between guideline characteristics. For the content analysis, only the deprescribing recommendations were extracted and analysed. It is possible that information about deprescribing was contained in other parts of the guideline, however, we focused on the recommendations alone. A further limitation was the exclusion of guidelines that were not published in the English language, potentially limiting the number and diversity of guidelines included for analysis. This decision was intentional as the review aimed to explore the language of recommendations, and translation may not appropriately convey the meaning of the original text. Similarly, the exclusion of guidelines pertaining to acute medical conditions may have resulted in the omission of potentially relevant recommendations about deprescribing short-term medications. However, we chose to focus this review on deprescribing of chronic medications, as this is the area where the term deprescribing is most often applied, and where healthcare professionals have called for additional guidance.19
Conclusion
While the majority of the 80 guidelines included in this scoping review presented one or more recommendations about for whom, what medicines and when or why to deprescribe, treatment guidelines did not routinely contain clear and actionable recommendations on how to deprescribe. Producing comprehensive deprescribing recommendations may be achieved by co-designing a template, reporting standard, or best practice guide, containing information on aspects of deprescribing recommendations that are essential, or preferred, to enable consistency of content (eg, details of how to taper) and format (terminology and presentation). Such guidance may help ensure that the intended meaning of deprescribing recommendations are adequately conveyed and that recommendations are actionable for end-users, to ultimately reduce inappropriate medicine use and improve patient care.
supplementary material
Acknowledgements
We would like to thank Matthew Garrett, Wenhao (Byron) Zhu and Mandeep Chahal for their contributions to the screening of records identified in the guideline registries and Google searches.
Footnotes
Funding: Dr Reeve is supported by a NHMRC Investigator Grant (1195460). Dr Langford is supported by a NHMRC Investigator Grant (2025289).
Provenance and peer review: Not commissioned; externally peer reviewed.
Patient consent for publication: Not applicable.
Ethics approval: Not applicable.
Contributor Information
Aili Veronica Langford, Email: aili.langford@monash.edu.
Imaan Warriach, Email: imaan.warriach.19@ucl.ac.uk.
Aisling M McEvoy, Email: ash.mcevoy@monash.edu.
Elisa Karaim, Email: ekar0007@student.monash.edu.
Shyleen Chand, Email: scha0105@student.monash.edu.
Justin P Turner, Email: justin.turner@monash.edu.
Wade Thompson, Email: wade.thompson@ubc.ca.
Barbara J Farrell, Email: bfarrell@bruyere.org.
Danielle Pollock, Email: danielle.pollock@adelaide.edu.au.
Frank Moriarty, Email: frankmoriarty@rcsi.ie.
Danijela Gnjidic, Email: danijela.gnjidic@sydney.edu.au.
Nagham J Ailabouni, Email: n.ailabouni@uq.edu.au.
Emily Reeve, Email: emily.reeve@monash.edu.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
References
- 1.Moriarty F, Hardy C, Bennett K, et al. Trends and interaction of Polypharmacy and potentially inappropriate prescribing in primary care over 15 years in Ireland: a repeated cross-sectional study. BMJ Open. 2015;5:e008656. doi: 10.1136/bmjopen-2015-008656. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Malakouti SK, Javan-Noughabi J, Yousefzadeh N, et al. A systematic review of potentially inappropriate medications use and related costs among the elderly. Value Health Reg Issues. 2021;25:172–9. doi: 10.1016/j.vhri.2021.05.003. [DOI] [PubMed] [Google Scholar]
- 3.Department of Health and social care Good for you, good for us, good for everybody. A plan to reduce overprescribing to make patient care better and safer, support the NHS, and reduce carbon emissions. 2021. [15-Dec-2022]. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1019475/good-for-you-good-for-us-good-for-everybody.pdf Available. Accessed.
- 4.Maher RL, Hanlon J, Hajjar ER. Clinical consequences of Polypharmacy in elderly. Expert Opin Drug Saf. 2014;13:57–65. doi: 10.1517/14740338.2013.827660. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Lavan AH, Gallagher P. Predicting risk of adverse drug reactions in older adults. Ther Adv Drug Saf. 2016;7:11–22. doi: 10.1177/2042098615615472. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Tian F, Chen Z, Zeng Y, et al. Prevalence of use of potentially inappropriate medications among older adults worldwide: A systematic review and meta-analysis. JAMA Netw Open. 2023;6:e2326910. doi: 10.1001/jamanetworkopen.2023.26910. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Charles L, Kumar E, Elghol E, et al. Potentially inappropriate medication use in the elderly. Can J Aging. 2022;41:176–83. doi: 10.1017/S0714980821000234. [DOI] [PubMed] [Google Scholar]
- 8.American Geriatrics society beers criteria® update expert panel. American Geriatrics society 2023 updated AGS beers criteria® for potentially inappropriate medication use in older adults. J Am Geriatr Soc. 2023;71:2052–81. doi: 10.1111/jgs.18372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Graham R, Mancher M, Miller Wolman D, editors. Clinical Practice Guidelines We Can Trust. Washington (DC): National Academies Press (US) Copyright 2011 by the National Academy of Sciences. All rights reserved. 2011. Institute of medicine Committee on standards for developing trustworthy clinical practice G. [PubMed] [Google Scholar]
- 10.Boyd CM, Darer J, Boult C, et al. Clinical practice guidelines and quality of care for older patients with multiple comorbid Diseasesimplications for pay for performance. JAMA. 2005;294:716–24. doi: 10.1001/jama.294.6.716. [DOI] [PubMed] [Google Scholar]
- 11.Okeowo D, Patterson A, Boyd C, et al. Clinical practice guidelines for older people with Multimorbidity and life-limiting illness: what are the implications for Deprescribing. Ther Adv Drug Saf. 2018;9:619–30. doi: 10.1177/2042098618795770. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.De Vries T, Henning RH, Hogerzeil HV, et al. Guide to good prescribing: a practical manual: world health organization. 1994.
- 13.Reeve E, Gnjidic D, Long J, et al. “A systematic review of the emerging de Fi Nition of 'Deprescribing' with network analysis: implications for future research and clinical practice”. Br J Clin Pharmacol. 2015;80:1254–68. doi: 10.1111/bcp.12732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Wu H, Kouladjian O’Donnell L, Fujita K, et al. Deprescribing in the older patient: A narrative review of challenges and solutions. IJGM. 2021;Volume 14:3793–807. doi: 10.2147/IJGM.S253177. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Page AT, Clifford RM, Potter K, et al. The feasibility and effect of Deprescribing in older adults on mortality and health: a systematic review and meta-analysis. Br J Clin Pharmacol. 2016;82:583–623. doi: 10.1111/bcp.12975. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Reeve E, Shakib S, Hendrix I, et al. The benefits and harms of Deprescribing. Med J Aust. 2014;201:386–9. doi: 10.5694/mja13.00200. [DOI] [PubMed] [Google Scholar]
- 17.Reeve E, Wiese MD. Benefits of Deprescribing on patients’ adherence to medications. Int J Clin Pharm. 2014;36:26–9. doi: 10.1007/s11096-013-9871-z. [DOI] [PubMed] [Google Scholar]
- 18.Sawan M, Reeve E, Turner J, et al. A systems approach to identifying the challenges of implementing Deprescribing in older adults across different health-care settings and countries: a narrative review. Expert Rev Clin Pharmacol. 2020;13:233–45. doi: 10.1080/17512433.2020.1730812. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Anderson K, Stowasser D, Freeman C, et al. Prescriber barriers and Enablers to minimising potentially inappropriate medications in adults: a systematic review and thematic synthesis. BMJ Open. 2014;4:e006544. doi: 10.1136/bmjopen-2014-006544. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Jansen J, McKinn S, Bonner C, et al. Systematic review of clinical practice guidelines recommendations about primary cardiovascular disease prevention for older adults. BMC Fam Pract. 2015;16:104. doi: 10.1186/s12875-015-0310-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.van der Ploeg MA, Floriani C, Achterberg WP, et al. Recommendations for (discontinuation of) Statin treatment in older adults: review of guidelines. J Am Geriatr Soc. 2020;68:417–25. doi: 10.1111/jgs.16219. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Farrell B, Black C, Thompson W, et al. Deprescribing Antihyperglycemic agents in older persons: evidence-based clinical practice guideline. Can Fam Physician . 2017;63:832–43. [PMC free article] [PubMed] [Google Scholar]
- 23.Pottie K, Thompson W, Davies S, et al. Deprescribing benzodiazepine receptor agonists: evidence-based clinical practice guideline. Can Fam Physician . 2018;64:339–51. [PMC free article] [PubMed] [Google Scholar]
- 24.Farrell B, Pottie K, Thompson W, et al. Deprescribing proton pump inhibitors: evidence-based clinical practice guideline. Can Fam Physician . 2017;63:354–64. [PMC free article] [PubMed] [Google Scholar]
- 25.Bjerre LM, Farrell B, Hogel M, et al. Deprescribing antipsychotics for behavioural and psychological symptoms of dementia and insomnia: evidence-based clinical practice guideline. Can Fam Physician . 2018;64:17–27. [PMC free article] [PubMed] [Google Scholar]
- 26.Reeve E, Farrell B, Thompson W, et al. Evidence-Based Clinical Practice Guideline for Deprescribing Cholinesterase Inhibitors and Memantine in People with Dementia. Sydney, Australia: Recommendations The University of Sydney; 2018. [Google Scholar]
- 27.Langford AV, Lin CC, Bero L, et al. Clinical practice guideline for Deprescribing opioid Analgesics: summary of recommendations. Med J Aust. 2023;219:80–9. doi: 10.5694/mja2.52002. [DOI] [PubMed] [Google Scholar]
- 28.Brouwers MC, Makarski J, Kastner M, et al. The guideline Implementability decision excellence model (GUIDE-M): a mixed methods approach to create an international resource to advance the practice guideline field. Implement Sci. 2015;10:36. doi: 10.1186/s13012-015-0225-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Kastner M, Bhattacharyya O, Hayden L, et al. Guideline uptake is influenced by six Implementability domains for creating and communicating guidelines: a realist review. J Clin Epidemiol. 2015;68:498–509. doi: 10.1016/j.jclinepi.2014.12.013. [DOI] [PubMed] [Google Scholar]
- 30.Khalil H, Peters M, Godfrey CM, et al. An evidence-based approach to Scoping reviews. Worldviews Evid Based Nurs. 2016;13:118–23. doi: 10.1111/wvn.12144. [DOI] [PubMed] [Google Scholar]
- 31.Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for Scoping reviews (PRISMA-SCR): checklist and explanation. Ann Intern Med. 2018;169:467–73. doi: 10.7326/M18-0850. [DOI] [PubMed] [Google Scholar]
- 32.Langford AV. Characteristics and language of Deprescribing recommendations in clinical practice guidelines: A Scoping review. Open Science Framework. 2023 doi: 10.17605/OSF.IO/FBEX4. [DOI] [Google Scholar]
- 33.Guyatt GH, Alonso-Coello P, Schünemann HJ, et al. Guideline panels should seldom make good practice statements: guidance from the GRADE working group. Journal of Clinical Epidemiology. 2016;80:3–7. doi: 10.1016/j.jclinepi.2016.07.006. [DOI] [PubMed] [Google Scholar]
- 34.National Health and Medical Research Council (NHMRC) 27. Procedures and requirements for meeting the 2011 NHMRC standard for clinical practice guidelines 2011. [7-Dec-2022]. https://www.nhmrc.gov.au/about-us/publications/meeting-2011-nhmrc-standard-clinical-practice-guidelines#block-views-block-file-attachments-content-block-1 Available. Accessed.
- 35.Dewidar O, Lotfi T, Langendam MW, et al. Good or best practice statements: proposal for the Operationalisation and implementation of GRADE guidance. BMJ Evid Based Med. 2023;28:189–96. doi: 10.1136/bmjebm-2022-111962. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Schünemann HBJ, Guyatt G, Oxman A. GRADE Handbook for grading quality of evidence and strength of recommendations.: the GRADE working group. BMJ. 2013 [Google Scholar]
- 37.Klugar M, Kantorová L, Pokorná A, et al. Visual transformation for guidelines presentation of the strength of recommendations and the certainty of evidence. J Clin Epidemiol. 2022;143:178–85. doi: 10.1016/j.jclinepi.2021.11.009. [DOI] [PubMed] [Google Scholar]
- 38.Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88. doi: 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]
- 39.Bloom L, Lahey M. Language development and language disorders. 1978
- 40.NVIVO [program], 2022
- 41.Sydney Health Literacy Lab Health literacy editor. [15-Dec-2022]. https://shell.techlab.works/ Available. Accessed.
- 42.Nici L, Mammen MJ, Charbek E, et al. Pharmacologic management of chronic obstructive pulmonary disease. an official American Thoracic society clinical practice guideline. Am J Respir Crit Care Med. 2020;201:e56–69. doi: 10.1164/rccm.202003-0625ST. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Atkins D, Best D, Briss PA, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004;328:1490. doi: 10.1136/bmj.328.7454.1490. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Ferguson C, Merga M, Winn S. Communications in the time of a pandemic: the Readability of documents for public consumption. Aust N Z J Public Health. 2021;45:116–21. doi: 10.1111/1753-6405.13066. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Fanouriakis A, Kostopoulou M, Alunno A, et al. Update of the EULAR recommendations for the management of systemic lupus erythematosus. Ann Rheum Dis. 2019;78:736–45. doi: 10.1136/annrheumdis-2019-215089. [DOI] [PubMed] [Google Scholar]
- 46.The National Institute for Health and Clinical Excellence (NICE) Bipolar Disorder: Assessment and Management (NICE Guideline, No.185) London: National Institute for Health and Care Excellence (NICE); 2020. [Google Scholar]
- 47.Moriarty F, Pottie K, Dolovich L, et al. Deprescribing recommendations: an essential consideration for clinical guideline developers. Res Social Adm Pharm. 2019;15:806–10. doi: 10.1016/j.sapharm.2018.08.014. [DOI] [PubMed] [Google Scholar]
- 48.Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010;182:E839–42. doi: 10.1503/cmaj.090449. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.The National Institute for Health and Clinical Excellence (NICE) The Guidelines Manual: Process and Methods. 2012. [Google Scholar]
- 50.World Health Organization (WHO) WHO Handbook for Guideline Development. World Health Organization (WHO); 2014. [Google Scholar]
- 51.Farrell B, Pottie K, Rojas-Fernandez CH, et al. Methodology for developing Deprescribing guidelines: using evidence and GRADE to guide recommendations for Deprescribing. PLoS One. 2016;11:e0161248. doi: 10.1371/journal.pone.0161248. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Doherty AJ, Boland P, Reed J, et al. Barriers and Facilitators to Deprescribing in primary care: a systematic review. BJGP Open. 2020;4:bjgpopen20X101096. doi: 10.3399/bjgpopen20X101096. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Ulley J, Harrop D, Ali A, et al. Deprescribing interventions and their impact on medication adherence in community-dwelling older adults with Polypharmacy: a systematic review. BMC Geriatr. 2019;19:15. doi: 10.1186/s12877-019-1031-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Reeve E, Moriarty F, Nahas R, et al. A narrative review of the safety concerns of Deprescribing in older adults and strategies to mitigate potential harms. Expert Opin Drug Saf. 2018;17:39–49. doi: 10.1080/14740338.2018.1397625. [DOI] [PubMed] [Google Scholar]
- 55.Langford AV, Schneider CR, Lin C-WC, et al. Patient-targeted interventions for opioid Deprescribing: an overview of systematic reviews. Basic Clin Pharmacol Toxicol. 2023;133:623–39. doi: 10.1111/bcpt.13844. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Reeve E, Jordan V, Thompson W, et al. Withdrawal of antihypertensive drugs in older people. Cochrane Database Syst Rev. 2020;6:CD012572. doi: 10.1002/14651858.CD012572.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Ross, MSc candidate SB, Wu PE, Atique, MD candidate A, et al. Adverse drug events in older adults: review of adjudication methods in Deprescribing studies. J American Geriatrics Society. 2020;68:1594–602. doi: 10.1111/jgs.16382. [DOI] [PubMed] [Google Scholar]
- 58.Neta G, Glasgow RE, Carpenter CR, et al. A framework for enhancing the value of research for dissemination and implementation. Am J Public Health. 2015;105:49–57. doi: 10.2105/AJPH.2014.302206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Reeve E, Thompson W, Farrell B. Deprescribing: A narrative review of the evidence and practical recommendations for recognizing opportunities and taking action. Eur J Intern Med. 2017;38:3–11. doi: 10.1016/j.ejim.2016.12.021. [DOI] [PubMed] [Google Scholar]
- 60.Gupta S, Tang R, Petricca K, et al. The guideline language and format instrument (GLAFI): development process and international needs assessment survey. Implement Sci. 2022;17:47. doi: 10.1186/s13012-022-01219-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Woolf SH, Grol R, Hutchinson A, et al. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318:527–30. doi: 10.1136/bmj.318.7182.527. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain--United States, 2016. JAMA. 2016;315:1624–45. doi: 10.1001/jama.2016.1464. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Agnoli A, Xing G, Tancredi DJ, et al. Association of dose tapering with overdose or mental health crisis among patients prescribed long-term opioids. JAMA. 2021;326:411–9. doi: 10.1001/jama.2021.11013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Fenton JJ, Magnan E, Tseregounis IE, et al. Long-term risk of overdose or mental health crisis after opioid dose tapering. JAMA Netw Open. 2022;5:e2216726. doi: 10.1001/jamanetworkopen.2022.16726. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Rashidian A, Eccles MP, Russell I. Falling on stony ground? A qualitative study of implementation of clinical guidelines' prescribing recommendations in primary care. Health Policy. 2008;85:148–61. doi: 10.1016/j.healthpol.2007.07.011. [DOI] [PubMed] [Google Scholar]
- 66.Gurses AP, Marsteller JA, Ozok AA, et al. Using an Interdisciplinary approach to identify factors that affect Clinicians' compliance with evidence-based guidelines. Crit Care Med. 2010;38:S282–91. doi: 10.1097/CCM.0b013e3181e69e02. [DOI] [PubMed] [Google Scholar]
- 67.Yang N, Yu Y, Zhang A, et al. Reporting, presentation and wording of recommendations in clinical practice guideline for gout: a systematic analysis. BMJ Open. 2019;9:e024315. doi: 10.1136/bmjopen-2018-024315. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Andrews JC, Schünemann HJ, Oxman AD, et al. GRADE guidelines: 15. going from evidence to recommendation-determinants of a recommendation’s direction and strength. J Clin Epidemiol. 2013;66:726–35. doi: 10.1016/j.jclinepi.2013.02.003. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
All data relevant to the study are included in the article or uploaded as supplementary information.


