Abstract
Background
The GRADE system is a transparent approach to assess the quality of evidence and strength of recommendations in health. It aids in converting evidence into guidelines, enhancing clarity and application in clinical practice. This study aims to identify scoping and systematic reviews using GRADE in top rehabilitation and neurology journals from 2020 to 2022.
Methods
The top 13 journals of rehabilitation and/or neurology area were selected by Scopus Sources. All systematic reviews and scoping reviews published between 2020 and 2022 were included. Studies that did not use the original GRADE system or failed to describe the GRADE details were excluded. The data of interest were extracted and analyzed by three researchers for an Excel table.
Results
There is no relation between the journals’ quality and the number of systematic reviews that used the GRADE system. The Neurorehabilitation and Neural Repair and Journal of Stroke and Cerebrovascular Diseases had the highest number of systematic reviews that used the GRADE system (16.7% and 4.8%, respectively). The most used downgrading criteria are both risk of bias and imprecision (100%).
Conclusions
The GRADE system is not widely used in the top 13 journals in the rehabilitation and neurology journals.
Keywords: GRADE, Certainty of evidence, Rehabilitation, Neurology, Systematic reviews
Background
The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) is a sensible and transparent approach to rank the quality of evidence and strengths of recommendations in the health area, mainly for converting this evidence into guidelines [1, 2] The level of evidence reflects the level of confidence in the information of the recommendation [3, 4]. Thus, it is one of the most influential systems that rates the quality of evidence in systematic reviews and other evidence synthesis [5]. One of the advantages of its use is the understanding of (1) quality of evidence and (2) strengths of recommendation. Also, observational studies can be “upgraded” if they reach certain criteria [6]. Therefore, the GRADE system can provide a clear quality of evidence and strength of recommendations, allowing for easy reading and application of information in clinical practice [2, 4, 7].
Systematic reviews are a study design that follows specific methodological criteria (to identify, select, synthesize, and assess the primary studies) to answer relevant questions. It summarizes all relevant information of a specific clinical question and, using an explicit methodology that minimizes bias, leads to high-quality, relevant, accessible, and up-to-date information that can also point to literature lacks and limitations [8–10]. The amount of scoping and systematic reviews in rehabilitation and neurology journals has been growing, and important questions have been answered by them, such as the effect of balance-training interventions on postural control in children and adolescents with cerebral palsy, the effectiveness of virtual reality in children and young adults with cerebral palsy, and the measurement properties of self-report physical activity assessment tools for patients with stroke, and so on [11–13]. Therefore, it is important to investigate the quality of the evidence in these studies, which has been increasing.
Due to the recognition of the importance of applying the GRADE system, previous studies from different areas (urology, gynecology and obstetrics, and nutrition) have carried out this investigation [14–16]. Less than 15% of the published systematic reviews used the GRADE system in the previous studies [14–16], which highlighted the need to encourage the application of GRADE and gave the reader a more critical look at the quality of the evidence. However, it is not known whether the systematic reviews published in the neurological area used an adequate system to determine the quality of the evidence and the strength of the recommendation, allowing for clear application in clinical practice. Although the quality of evidence and strengths of recommendation are essential to evidence-based practice use, there is no previous study that investigated if scoping and systematic reviews published in rehabilitation and neurology journals have used the GRADE system to evaluate the quality of evidence.
Therefore, the aim of the present study was to identify and describe all scoping and systematic reviews that have used the GRADE system to evaluate the quality of evidence and strengths of recommendations published between 2020 and 2022 in the top 13 rehabilitation and neurology journals, and (2) to summarize and present the GRADE-specific information.
Methods
The aim of the study is to analyze relevant systematic and scoping reviews published between 2020 and 2022 in the top 13 journals of neurological rehabilitation and/or neurology area that used the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system to evaluate the quality of evidence and the strength of recommendations. There was no language restriction to the journals. This study included articles that applied the original GRADE methodology and its implementation using the GRADE pro GDT tool, since both follow the same methodological principle for assessing the quality of evidence. Also, only studies that clearly applied the downgrading domains—such as risk of bias, inconsistency, imprecision, indirectness, and publication bias—and, when applicable to observational studies, the upgrading domains—such as large magnitude of effect, dose–response relationship, and confounding factors that would reduce the observed effect—were included.
The top 13 journals were selected in the Scopus Sources in November 2022, using the following filters: “neurology” subject and “journal” source type. Only the journals that had “rehabilitation” “neurology” and “neurology (clinical)” in the subject area were included. All scoping and systematic reviews that used the original version of the GRADE system, published in the top 13 rehabilitation and neurology journals with higher scores in the impact factor between 2020 and 2022, were included. Searching for full text was performed in the PubMed database and journal’s website between October 2022 and November 2022. Scoping reviews were included because they may also report assessments of evidence certainty using the GRADE approach; so, for the purposes of this study, they were analyzed together with systematic reviews.
Selection of documents
The inclusion criteria of the present study were scoping and systematic reviews in rehabilitation and neurology journals that were published between 2020 and 2022. Furthermore, only studies that used the original GRADE domains were included.
The first step was a pilot testing, in which all authors performed a data extraction simulation from an article to standardize the extraction process. Following this, three researchers (PHVC, ABV, JMV) independently screened and identified articles by title and abstract. Second, the researchers independently obtained and checked all potentially relevant articles for final inclusion, verifying the use of the GRADE system. Disagreements were resolved by discussion with a fourth researcher (JCP) to promote inter-rater reliability metrics. Third, we identified if the included studies used the original version of the GRADE system.
Data extraction
The reviewers extracted the data about the Cite Score and impact factor of each journal included, and the number of reviews published per year (2020, 2021, 2022). The three reviewers (PHVC, ABV, JMV), independently, extracted mainly data about the use of GRADE by including studies and put the information on an Excel table. The studies’ characteristics extracted were year of publication, journal published, number of primary studies included, main disease, number of participants, number of outcomes, the rating of the certainty per outcome, outcomes rated, risk of bias and scale used to evaluate risk of bias, inconsistency, indirectness, imprecision, publication bias, magnitude, residual confounding, effect size, meta-analysis conducted (yes or no), categories of certainty of evidence ratings (very low, low, moderate and high), summary of findings table reported (yes or no) [14].
GRADE has specific domains for each type of study included in the systematic reviews (randomized controlled trials or observational studies) [2, 17]. In the randomized controlled trials, the following domains are included: risk of bias which evaluates the compatibility with reality, imprecision which represents the 95% confidence interval of each outcome, inconsistency which indicates the heterogeneity of the study, indirectness which analyzes secondary outcomes, and publication bias [2, 17]. In the observational studies, the following domains are included: large effect which identifies whether there has been an exaggerated reduction in the incidence of an outcome, dose–response which evaluates the probability of the cause-effect occurring, and plausible confounding which assesses factors associated with the main outcomes [17, 18].
Results
Two hundred and forty-four systematic and scoping reviews were published in the selected journals between 2020 and 2022. The top 13 rehabilitation and neurology journals were IEEE Transactions on Neural Systems and Rehabilitation, Neurorehabilitation and Neural Repair, Journal of Neurologic Physical Therapy, Journal of Head Trauma Rehabilitation, Journal of Intellectual Disability, Spinal Cord, PM and R, Journal of Stroke and Cerebrovascular Diseases, Topics in Stroke Rehabilitation, NeuroRehabilitation, Research and Practice in Intellectual and Developmental Disabilities, Neurologie and Rehabilitation, and Motricité Cérébrale.
Only one hundred eighty-two systematic reviews were available to read since the rest were not freely available and the authors did not respond to requests for access to the studies by e-mail and other communication platforms. Only thirteen have used the GRADE system (Fig. 1). Furthermore, it is important to note that nine systematic reviews metanalysis with 10 studies, and one systematic review has metanalyses with only nine studies.
Fig. 1.
Flowchart of included studies
Table 1 describes the distribution of the included systematic reviews according to the published year, impact factor, and category. The journal with the lower impact factor was Neurologie and Rehabilitation, and Journal of Neurologic Physical Therapy presented the higher impact factor, with 1 and 3 systematic reviews published, respectively. Journal of Stroke and Cerebrovascular Diseases published a higher number of systematic reviews (n = 83). The Neurorehabilitation and Neural Repair and Journal of Stroke and Cerebrovascular Diseases had the highest number of systematic reviews that used the GRADE system (16.7% and 4.8%, respectively. Thus, there is no association between a high-impact factor in the journal and the use of the GRADE system.
Table 1.
The distribution of systematic reviews (n = 13) according to the published year, impact factor, and category
| Number of systematic reviews published,n | Number of systematic reviews rating the outcome specific certainty of evidence with GRADE,n(% of systematic reviews published) | Cite score SJR/impact factorSNIP | Journal category according to Scopus | |
|---|---|---|---|---|
| Total, n | ||||
| 2020 | 89 | – | – | |
| 2021 | 81 | – | – | |
| 2022 | 87 | – | – | |
| Journal | ||||
| IEEE transactions on neural systems and rehabilitation | 5 | 0 | 1.257/1.642 | Neurological and/or rehabilitation |
| Neurorehabilitation and neural repair | 24 | 4 | 1.188/1.635 | Neurological and/or rehabilitation |
| Journal of Neurologic Physical Therapy | 3 | 0 | 0.936/1.999 | Neurological and/or rehabilitation |
| Journal of Head Trauma Rehabilitation | 25 | 0 | 0.785/1.213 | Neurological and/or rehabilitation |
| Journal of Intellectual Disability | 15 | 0 | 0.841/1.411 | Neurological and/or rehabilitation |
| Spinal cord | 31 | 2 | 0.772/1.324 | Neurological and/or rehabilitation |
| PM and R | 28 | 1 | 0.581/1.042 | Neurological and/or rehabilitation |
| Journal of Stroke and Cerebrovascular Diseases | 83 | 4 | 0.698/0.901 | Neurological and/or rehabilitation |
| Topics in Stroke Rehabilitation | 17 | 1 | 0.626/1.252 | Neurological and/or rehabilitation |
| NeuroRehabilitation | 22 | 1 | 0.494/0.900 | Neurological and/or rehabilitation |
| Research and Practice in Intelectual and Developmental Disabilities | 2 | 0 | 0.380/0.621 | Neurological and/or rehabilitation |
| Neurologie and Rehabilitation | 1 | 0 | 0.112/0.137 | Neurological and/or rehabilitation |
| Motricite Cerebrale | 1 | 0 | 0.161/0.352 | Neurological and/or rehabilitation |
SJR Scientific Journal Ranking, SNIP source normalized impact per paper
*Journals are presented according to their Scopus Impact Factor
Table 2 describes the results of the systematic reviews that included the GRADE system (n = 13). Risk of bias (100%) and imprecision (100%) were the domains with the most downgrades. These domains were also the ones with the most downgrading per outcome, also present in 100% of the systematic reviews. None of the studies used an upgrade domain; even five systematic reviews included observational studies as primary studies, and this is an incorrect form to use GRADE [19–24].
Table 2.
Details on the certainty of evidence rating—downgrading and upgrading per outcome of the systematic reviews that used the GRADE system published by included journals (n = 13)
| Total (%) | |
|---|---|
| The rating of the certainty per outcome, n | |
| High, n (%) | 20 (12.5) |
| Moderate, n (%) | 71 (44.4) |
| Low, n (%) | 69 (43.1) |
| Very low, n (%) | 32 (20) |
| Total number of downgrading domains, n | |
| Risk of bias, n (%) | 13 (100) |
| Imprecision, n (%) | 13 (100) |
| Inconsistency, n (%) | 12 (92.3) |
| Indirectness, n (%) | 11 (84.6) |
| Publication bias, n (%) | 12 (92.3) |
| Upgrading, n | |
| Large effect, n (%) | 0 (0) |
| Dose–response, n (%) | 0 (0) |
| Plausible confounding, n (%) | 0 (0) |
| Frequency of the rating domains | |
| Mean frequency,n of downgrading domains, n/the rating of the certainty per outcome, n | |
| Risk of bias, n (% of outcomes downgraded) | 13 (100) |
| Imprecision, n (% of outcomes downgraded) | 13 (100) |
|
Indirectness, n (% of outcomes downgraded) Inconsistency, n (% of outcomes downgraded) |
10 (76.9) 12 (92.3) |
| Publication bias, n (% of outcomes downgraded) | 11 (84.6) |
| Mean frequency, n of upgrading, n/the rating of the certainty per outcome, n | |
| Large effect n (% of outcomes upgraded) | 0 (0) |
| Dose–response n (% of outcomes upgraded) | 0 (0) |
| Plausible confounding, n (% of outcomes upgraded) | 0 (0) |
Table 3 describes the systematic reviews that use some GRADE domains. The risk of bias (62.7%) was the downgrading domain that was most used, followed by the publication bias domain (50.2%). Risk of bias (47.3%) was also the most downgrading domain per outcome used. The upgrading domain that was most used was plausible confounding (11.2%). Finally, risk of bias (10%) and large effect (10%) were the most upgrading domains per outcome used.
Table 3.
Details on the certainty of evidence rating—downgrading and upgrading per outcome of the systematic reviews that use some GRADE domains
| Total (%) | |
|---|---|
| Total number of downgrading domains, n | |
| Risk of bias, n (%) | 106 (62.7) |
| Imprecision, n (%) | 23 (13.6) |
| Inconsistency, n (%) | 36 (21.3) |
| Indirectness, n (%) | 10 (5.9) |
| Publication bias, n (%) | 85 (50.2) |
| Upgrading, n | |
| Large effect, n (%) 18 (10.6%) | 18 (10.6) |
| Dose–response, n (%) 5 (2.9%) | 5 (2.9) |
| Plausible confounding, n (%) 19 (11.2%) | 19 (11.2) |
| Frequency of the rating domains | |
| Mean frequency, n of downgrading domains, n/the rating of the certainty per outcome, n | |
| Risk of bias, n (% of outcomes downgraded) | 80 (47.3) |
| Imprecision, n (% of outcomes downgraded) | 21 (12.4) |
| Inconsistency, n (% of outcomes downgraded) | 30 (17.7) |
| Indirectness, n (% of outcomes downgraded) | 9 (5.3) |
| Publication bias, n (% of outcomes downgraded) | 49 (29.9) |
| Mean frequency, n of upgrading, n/the rating of the certainty per outcome, n | |
| Large effect n (% of outcomes upgraded) | 17 (10) |
| Dose–response n (% of outcomes upgraded) | 4 (2.3) |
| Plausible confounding n (% of outcomes upgraded) | 17 (10.0) |
Seven studies reported which publication bias tool was used. Two studies used the Cochrane Collaboration Tool and two used the PEDro Scale [22, 25–27]. One study used the Jadad Scale, Risk-of-Bias (RoB), and Risk of Bias in Non-randomized Studies of Interventions (ROBINS-I) [23, 28]. The Agency for Healthcare Research and Quality assessment form was used in a cross-sectional/prevalence study [24]. One study used more than one tool to evaluate the publication bias [23].
Table 4 presents the characteristics of the systematic reviews that used the GRADE system to evaluate their outcomes. It is observed that Stroke is the most investigated health condition. Only four studies made the summary of findings table.
Table 4.
Characteristics of the systematic reviews that used GRADE system
| Systematic reviews (n = 13) | |
|---|---|
| Number of primary studies, median (range) | 32 (3 - 191) |
| Number of participants, median (range) | 90592 (55 - 1146252) |
| Summary of findings table, n | 4 |
| Meta-analysis conducted, n | 9 |
| Outcomes rated in a systematic reviews, median (range) | 2 (1 - 4) |
| Disease | |
| Stroke | 7 |
| Spinal cord injury | 3 |
| Parkinson disease | 1 |
| Multiple sclerosis | 1 |
| Chronic diseases | 1 |
Discussion
The present study evaluates the use of the GRADE system in systematic reviews published in the top 13 rehabilitation and/or neurological journals between 2020 and 2022. In general, few studies used the GRADE system to rank the quality of evidence and strength of recommendations. A small proportion (12.5%) of systematic reviews in rehabilitation and/or neurology journals demonstrates strong evidence of an effect of intervention. One explanation could be the lack of strong evidence to support the intervention, or the intervention has positive effects but did not use an adequate system to classify the levels of evidence [6]. The evaluation of the available evidence is important to present more reliable studies. Thus, it is notable that the domains of Risk of Bias and Imprecision reduce the quality and strength of the evidence. Another finding was that systematic reviews that used observational studies as primary studies applied just downgrading domains, and this is an incorrect use of the GRADE system.3
Interestingly, there was no association between the quality of the journal (impact factor) and the number of systematic reviews that used the GRADE system. Given the great importance of quality evidence for clinical practice, the instructions for authors at the journals could have the strong recommendation to use a valid and reliable system to evaluate the quality of evidence of systematic reviews, such as GRADE [17]. The Neurorehabilitation and Neural Repair Journal and Journal of Stroke and Cerebrovascular Diseases had the major number of systematic reviews that used the GRADE system, although it does not have this specific recommendation in the instructions for authors. Despite these journals having the major number of systematic reviews that used the GRADE system, the proportion between published systematic reviews and systematic reviews that used the GRADE system is low (16.7% and 4.8%). Therefore, even though systematic reviews summarize the evidence and present a reliable methodology, they may not present strong quality and strength for recommendation.
Systematic reviews with metanalysis are on top of the evidence pyramid [29]. Nonetheless, the top journals in the rehabilitation and/or neurologic area published few systematic reviews with metanalysis between 2020 and 2022. For example, Motricité Cérébrale Journal published only one systematic review without metanalysis in the three investigated years. Few studies have used the GRADE system to evaluate the quality of evidence, and few systematic reviews published may not support the clinical practice evidence based as necessary. Guidelines are used to support clinical practice, but, without high-quality systematic reviews, it turns difficult to develop new guidelines with the best available evidence. A recent study from Gianola et al. (2022) related that one third of systematic reviews assess the certainty of evidence, and from this percentage of study, just 69% of them applied the GRADE system [30]. In this sense, it was previously observed that over half of clinical practice guidelines of different areas used non-systematic methods to inform recommendations for clinical practice [31].
However, even though systematic reviews present great methodological reliability, they are not free of bias. For example, if the metanalyses included few studies, publication bias is observed [7, 18]. In the current study, one systematic review presented publication bias; nine systematic reviews present metanalysis with 10 studies, and one systematic review presents metanalyses with only nine studies [32]. It is recommended that systematic reviews with metanalysis with less than 10 studies include the funnel plot in order to investigate the presence of publication bias, even with its substantial limitations [7, 18]. However, the included paper did not present this analysis [32]. Another important bias observed in the included systematic reviews is the imprecision that happened when the metanalyses include large confidence intervals and small sample sizes [7, 18]. In the included papers, the mean of sample sizes included on metanalyses was 88,173. This finding could be explained by the fact of the disease’s characteristics since the recruitment and program retention of individuals with neurological diseases is difficult [33–35].
Only three previous studies analyzed the use of the GRADE system in systematic reviews in high-impact factor journals from different areas: urology, gynecology and obstetrics, and nutrition [14–16]. The study that analyzed the use of GRADE in systematic reviews in urology journals, from 2016 to 2020, found a total of 445 systematic reviews, but only 13.5% used GRADE [14]. The study that analyzed the use of GRADE in systematic reviews in gynecology and obstetrics journals, from 2016 to 2020, found a total of 952 systematic reviews, but only 10.0% used GRADE [15]. The study that analyzed the use of GRADE in systematic reviews in nutrition journals, from 2015 to 2019, found a total of 800 systematic reviews, but only 6.8% used GRADE [16]. The present study found 257 systematic reviews with 5.7% of them using the GRADE system, between 2020 and 2022. Rehabilitation and neurology journals had a lower percentage of articles using GRADE when compared to urology and nephrology. This may be explained by the number of years that the study included [14].
Although articles in the areas of neurology and rehabilitation show particularly low adherence to the use of GRADE, other specialties, such as urology and nutrition, also show insufficient GRADE application. This limitation can be explained by factors such as the authors’ lack of familiarity with the methodology, the absence of requirements from journals regarding the assessment of the certainty of the evidence, and the complexity involved in applying GRADE, especially in systematic reviews that include non-randomized studies. This fact is observed in the present study, since none of the included studies used an upgrade domain, even five systematic reviews included observational studies as primary studies. This represents a methodological inconsistency in the application of the GRADE approach, since in such cases the use of upgrading domains would be expected. Some hypotheses may help explain this trend. First, the lack of specific training among reviewers in applying GRADE criteria—particularly the upgrading domains such as large effect size, dose–response gradient, or plausible confounding—may lead to underutilization of these components. Second, poor or inconsistent reporting in the primary studies often limits the information necessary to justify upgrading, even when the data might warrant it. Additionally, discipline-specific challenges within rehabilitation and neurology, such as heterogeneous interventions, multifactorial outcomes, and reliance on complex clinical populations, may make it more difficult to apply the upgrading criteria rigorously or consistently. These factors underscore the need for improved methodological guidance and targeted training to support accurate and complete GRADE evaluations in observational research. Werner et al. (2021) point out that many researchers misinterpret the GRADE assessment criteria, raising the certainty of the evidence based on inappropriate factors, such as low risk of bias, narrow confidence intervals, very low p-values, and slight statistical heterogeneity. It is also important to emphasize that the underuse of the GRADE system can significantly affect the development of clinical guidelines, as it may lead to recommendations based on low-certainty evidence or lacking transparent justification. This compromises the reliability of guidelines and can result in inconsistent clinical decisions. Ultimately, the absence of rigorous evidence grading may negatively impact patient outcomes, as healthcare professionals might adopt interventions with uncertain effectiveness or safety, potentially reducing the overall quality of care.
Although the present study considered only journals that were included in the Scopus system, this is the first study that investigated the use of GRADE in high-impact factor rehabilitation and/or neurology journals. In addition, the present study provides a wide range of relevant information extracted from the systematic reviews, such as inconsistency, indirectness, imprecision, publication bias, and others. Presently, we provide information to make future studies better, raising awareness of the use of the GRADE system. However, the present study included only 3 years of analyses.
A recent study from Dixon et al. [16] showed that most guidelines from the USA use the GRADE system, although they do not use all GRADE domains [36]. Therefore, we recommend that the researchers use the GRADE system to evaluate the quality of evidence of their systematic reviews. Also, it is important to emphasize that researchers need to use the GRADE system correctly, considering all the domains. Considering the relevance of the GRADE system in enhancing the quality and transparency of evidence, journals should consider encouraging its inclusion in author guidelines. Furthermore, promoting training opportunities for researchers may help support the appropriate and consistent application of GRADE, ultimately contributing to more robust evidence-based clinical practice [37]. Following the purpose of the present study, similar studies need to be conducted in other areas because it is necessary to evaluate the quality of published systematic reviews and raise awareness among both researchers and professionals in other areas as well.
Conclusion
The GRADE system was used in a small percentage of the systematic reviews from the Top 13 journals of rehabilitation and neurology journals in the years 2020, 2021, and 2022. As the GRADE system verifies the certainty of evidence and the strength of recommendations, there is a necessity to enhance the use of GRADE in order to provide better knowledge for the professional’s clinical practice and to find gaps in the literature, leading the researchers to produce specific studies.
Acknowledgements
We gratefully acknowledge the financial support of Fundação do Amparo de Minas Gerais (FAPEMIG) and Pró-Reitoria de Pesquisa of Universidade Federal de Minas Gerais (PRPq/UFMG).
Authors’ contributions
ABV, JMV, PHVC, and JCP contributed to the conceptualization, formal analysis, research, and visualization stages of this scoping review. ABV, JMV, PHVC drafted the manuscript. AAS, CDCMF, and JCP validated the initial work. AAS, CDCMF, and JCP reviewed the manuscript. All authors read and approved the final manuscript.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Data availability
All relevant information is included in the main article.
Declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
All authors authorize the publication of this study. They have all been duly informed of the study’s objectives, its implications, and the publication process.
Competing interests
The authors declare that they have no competing interests.
Footnotes
This study was developed at Department of Physical Therapy, Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brazil.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Caplan AM, Caplan L. The GRADE method. Rheum Dis Clin North Am. 2022;48(3):589–99. 10.1016/j.rdc.2022.04.002. [DOI] [PubMed] [Google Scholar]
- 2.Kirmayr M, Quilodrán C, Valente B, Loezar C, Garegnani L, Franco JVA. The GRADE approach, Part 1: how to assess the certainty of the evidence. Medwave. 2021;21: e8109. 10.5867/medwave.2021.02.8109. [DOI] [PubMed] [Google Scholar]
- 3.Goldet G, Howick J. Understanding GRADE: an introduction. J Evid Based Med. 2013. 10.1111/jebm.12018. [DOI] [PubMed] [Google Scholar]
- 4. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.5 (updated August 2024). Cochrane; 2024. 10.1002/9781119536604
- 5.Perillat L, Mercuri M. Clinical recommendations: the role of mechanisms in the GRADE framework. Stud Hist Philos Sci. 2022;96:1–9. 10.1016/j.shpsa.2022.08.018. [DOI] [PubMed] [Google Scholar]
- 6.Xie CX, Machado GC. Clinimetrics: Grading of recommendations, assessment, development and evaluation (GRADE). J Physiother. 2021;67:66. 10.1016/j.jphys.2020.07.003. [DOI] [PubMed] [Google Scholar]
- 7.Herbert R, Jamtvedt G, Hagen KB, Elkins M. Practical Evidence-Based Physiotherapy. Third Edition. London: Elsevier; 2022.
- 8.Mancini MC, Cardoso JR, Sampaio RF, Costa LC, Cabral CM, Costa LO. Tutorial for writing systematic reviews for the Brazilian journal of physical therapy (BJPT). Braz J Phys Ther. 2014;18(6):471–80. 10.1590/bjpt-rbf.2014.0077. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA. (Ed.). Cochrane handbook for systematic reviews of interventions. John Wiley & Sons, 2019. 10.1002/14651858.ED000142 [DOI] [PMC free article] [PubMed]
- 10.Elkins MR, Herbert RD, Moseley AM, Sherrington C, Maher C. Rating the quality of trials in systematic reviews of physical therapy interventions. Cardiopulm Phys Ther J. 2010;21(3):20–6. [PMC free article] [PubMed] [Google Scholar]
- 11.Araújo PA, Starling JMP, Oliveira VC, Gontijo APB, Mancini MC. Combining balancetraining interventions with other active interventions may enhance effects on postural control in children and adolescents with cerebral palsy: a systematic review and meta-analysis. Braz J Phys Ther. 2020;24(4):295–305. 10.1016/j.bjpt.2019.04.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Fandim JV, Saragiotto BT, Porfírio GJM, Santana RF. Effectiveness of virtual reality in children and young adults with cerebral palsy: a systematic review of randomized controlled trial. Braz J Phys Ther. 2021;25(4):369–86. 10.1016/j.bjpt.2020.11.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Martins JC, Aguiar LT, Nadeau S, Scianni AA, Teixeira-Salmela LF, Faria CDCM. Measurement properties of self-report physical activity assessment tools for patients with stroke: a systematic review. Braz J Phys Ther. 2019;23(6):476–90. 10.1016/j.bjpt.2019.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Zhang S, Wu QJ, Liu SX. A methodologic survey on use of the GRADE approach in evidence syntheses published in high-impact factor urology and nephrology journals. BMC Med Res Methodol. 2022;22(1): 220. 10.1186/s12874-022-01701-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Yang HJ, Zhang DY, Hao YY, Xu HL, Li YZ, Zhang S, Li XY, Gong TT, Wu QJ. Grade use in evidence syntheses published in high-impact-factor gynecology and obstetrics journals: a methodological survey. J Clin Med. 2023;12(2): 446. 10.3390/jcm12020446. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Werner SS, Binder N, Toews I, Schünemann HJ, Meerpohl JJ, Schwingshackl L. Use of GRADE in evidence syntheses published in high-impact-factor nutrition journals: a methodological survey. J Clin Epidemiol. 2021;135:54–69. 10.1016/j.jclinepi.2021.02.010. [DOI] [PubMed] [Google Scholar]
- 17.Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ, GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924–6. 10.1136/bmj.39489.470347.AD. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, Alonso-Coello P, Djulbegovic B, Atkins D, Falck-Ytter Y, Williams JW Jr, Meerpohl J, Norris SL, Akl EA, Schünemann HJ. GRADE guidelines: 5. Rating the quality of evidence–publication bias. J Clin Epidemiol. 2011;64(12):1277–82. 10.1016/j.jclinepi.2011.01.011. [DOI] [PubMed] [Google Scholar]
- 19.Schünemann HJ, Cuello C, Akl EA, Mustafa RA, Meerpohl JJ, Thayer K, Morgan RL, Gartlehner G, Kunz R, Katikireddi SV, Sterne J, Higgins JP, Guyatt G, GRADE Working Group. Grade guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol. 2019;111:105–14. 10.1016/j.jclinepi.2018.01.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Zwanenburg PR, Verdijk RWA, Lapid O, Obdeijn MC, Gans SL, Boermeester MA. A systematic review and meta-analysis of sensate versus non-sensate flaps for the prevention of pressure ulcer recurrence among individuals with spinal cord disease. Spinal Cord. 2021;59(5):463–73. 10.1038/s41393-020-00590-5. [DOI] [PubMed] [Google Scholar]
- 21.Fagundes NCF, Couto RSD, Brandão APT, Lima LAO, Bittencourt LO, Souza-Rodrigues RD, Freire MAM, Maia LC, Lima RR. Association between tooth loss and stroke: a systematic review. J Stroke Cerebrovasc Dis. 2020;29(8): 104873. 10.1016/j.jstrokecerebrovasdis.2020.104873. [DOI] [PubMed] [Google Scholar]
- 22.Mitsutake T, Imura T, Tanaka R. The effects of vestibular rehabilitation on gait performance in patients with stroke: a systematic review of randomized controlled trials. J Stroke Cerebrovasc Dis. 2020;29(11): 105214. 10.1016/j.jstrokecerebrovasdis.2020.105214. [DOI] [PubMed] [Google Scholar]
- 23.Solomon RM, Dhakal R, Halpin SJ, Hariharan R, O’Connor RJ, Allsop M, Sivan M. Telerehabilitation for individuals with spinal cord injury in low-and middle-income countries: a systematic review of the literature. Spinal Cord. 2022;60(5):395–403. 10.1038/s41393-022-00797-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Chen X, Zhang H, Xiao G, Lv C. Prevalence of suicidal ideation among stroke survivors: a systematic review and meta-analysis. Top Stroke Rehabil. 2021;28(7):545–55. 10.1080/10749357.2020.1846933. [DOI] [PubMed] [Google Scholar]
- 25.Hadoush H, Alawneh A, Kassab M, Al-Wardat M, Al-Jarrah M. Effectiveness of non-pharmacological rehabilitation interventions in pain management in patients with multiple sclerosis: systematic review and meta-analysis. NeuroRehabilitation. 2022;50(4):347–65. 10.3233/NRE-210328. [DOI] [PubMed] [Google Scholar]
- 26.da Silva ESM, Ocamoto GN, Santos-Maia GLD, de Fátima Carreira Moreira Padovez R, Trevisan C, de Noronha MA, Pereira ND, Borstad A, Russo TL. The effect of priming on outcomes of task-oriented training for the upper extremity in chronic stroke: a systematic review and meta-analysis. Neurorehabil Neural Repair. 2020;34(6):479–504. 10.1177/1545968320912760. [DOI] [PubMed] [Google Scholar]
- 27.Cugusi L, Prosperini L, Mura G. Exergaming for quality of life in persons living with chronic diseases: a systematic review and meta-analysis. PM R. 2021;13(7):756–80. 10.1002/pmrj.12444. [DOI] [PubMed] [Google Scholar]
- 28.Monteiro KB, Cardoso MDS, Cabral VRDC, Santos AOBD, Silva PSD, Castro JBP, Vale RGS. Effects of motor imagery as a complementary resource on the rehabilitation of stroke patients: a meta-analysis of randomized trials. J Stroke Cerebrovasc Dis. 2021;30(8): 105876. 10.1016/j.jstrokecerebrovasdis.2021.105876. [DOI] [PubMed] [Google Scholar]
- 29.Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016;21(4):125–7. 10.1136/ebmed-2016-110401. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Gianola S, Bargeri S, Nembrini G, Varvello A, Lunny C, Castellini G. One-third of systematic reviews in rehabilitation applied the grading of recommendations assessment, development, and evaluation (GRADE) system to evaluate certainty of evidence: a meta-research study. Arch Phys Med Rehabil. 2023;104(3):410–7. 10.1016/j.apmr.2022.09.005. [DOI] [PubMed] [Google Scholar]
- 31.Lunny C, Ramasubbu C, Puil L, Liu T, Gerrish S, Salzwedel DM, Mintzes B, Wright JM. Over half of clinical practice guidelines use non-systematic methods to inform recommendations: a methods study. PLoS One. 2021;16(4): e0250356. 10.1371/journal.pone.0250356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Reis SB, Bernardo WM, Oshiro CA, Krebs HI, Conforto AB. Effects of robotic therapy associated with noninvasive brain stimulation on upper-limb rehabilitation after stroke: systematic review and meta-analysis of randomized clinical trials. Neurorehabil Neural Repair. 2021;35(3):256–66. 10.1177/1545968321989353. [DOI] [PubMed] [Google Scholar]
- 33.Silva TRD, Luvizutto GJ, Martins LG, Costa RDMD, Souza JT, Winckler FC, Sartor LCA, Modolo GP, Ferreira NC, Rodrigues JCS, Kanda RG, Fogaroli MO, Borges GF, Rizzatti GRS, Ribeiro PW, Pires DS, Favoretto DB, Aguiar LR, Bazan SGZ, Betting LEG, Antunes LCO, Nunes HRC, Pereira VM, Edwards TGS, Pontes-Neto O, Conforto AB, Bazan R. Barriers to patient recruitment in a poststroke neurorehabilitation multicenter trial in Brazil. Braz J Med Biol Res. 2023;56: e12326. 10.1590/1414-431X2023e12326. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Gomes EP, Michaelsen SM, Rodrigues LC, Farias NC, da Silva R. Pesquisas científicas com indivíduos pós Acidente Vascular Encefálico: dificuldades no recrutamento, alocação e aderência em dois diferentes protocolos de intervenção fisioterapêutica. Fisioter Pesq. 2015;22(1):34–40. 10.590/1809-2950/13111022012015. [DOI] [Google Scholar]
- 35.Tsao CW, Aday AW, Almarzooq ZI, Anderson CAM, Arora P, Avery CL, Baker-Smith CM, Beaton AZ, Boehme AK, Buxton AE, Commodore-Mensah Y, Elkind MSV, Evenson KR, Eze-Nliam C, Fugar S, Generoso G, Heard DG, Hiremath S, Ho JE, Kalani R, Kazi DS, Ko D, Levine DA, Liu J, Ma J, Magnani JW, Michos ED, Mussolino ME, Navaneethan SD, Parikh NI, Poudel R, Rezk-Hanna M, Roth GA, Shah NS, St-Onge MP, Thacker EL, Virani SS, Voeks JH, Wang NY, Wong ND, Wong SS, Yaffe K, Martin SS, American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics-2023 update: a report from the American Heart Association. Circulation. 2023;147(8):e93–621. 10.1161/CIR.0000000000001123. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Dixon C, Dixon PE, Sultan S, Mustafa R, Morgan RL, Murad MH, Falck-Ytter Y, Dahm P. Guideline developers in the United States were inconsistent in applying criteria for appropriate grading of recommendations, assessment, development and evaluation use. J Clin Epidemiol. 2020;124:193–9. 10.1016/j.jclinepi.2020.01.026. [DOI] [PubMed] [Google Scholar]
- 37.Stern C, Munn Z, Barker TH, Porritt K, Stone JC, Pap R, Khalil H, Aromataris E. Implementing grade in systematic reviews that adhere to JBI methodological conduct. JBI Evid Synth. 2024;22(3):351–8. 10.11124/JBIES-23-00543. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
All relevant information is included in the main article.