Levels of Evidence within Cardiovascular Medicine Research in Saudi Arabia: A Systematic Review

Abstract

International commitments to reduce the prevalence of non-communicable disease have led the Government of Saudi Arabia to invest more in research related to cardiovascular disease. However, the strength of evidence derived from these research activities remains unclear. The aim of our study was to examine the level of evidence within clinical Cardiovascular Medicine research in Saudi Arabia. We conducted a systematic review of published articles that included a population from Saudi Arabia. Electronic databases EMBASE and MEDLINE (Ovid) were searched up to 25^th of April 2021, supplemented by a second search in CENTRAL (Cochrane Central Register of Controlled Trials) and www.clinicaltrials.gov. In addition, the Snowball- and Pearl-growing methods of search were conducted for finding additional eligible articles. Finally, a search was conducted in PubMed database for all eligible articles published by Journal of the Saudi Heart Association from the first indexed issue up to April 2021. Level of evidence of reviewed articles was determined using the Oxford Level of Evidence 2 scale. We calculated the mean level of evidence over 5-year periods, and explored evidence for a time trend for number of published articles and LOE using linear regression. Of the 1113 records identified, 418 met the inclusion criteria for analysis. The articles were published between September 1986 and March 2021. More than half of the included articles were level IV studies (n = 242, 57.8%). Furthermore, we observed no trend over the years for increased mean of level of evidence (β = −0.07, 95% CI [−0.20, −0.06], p = 0.236). Overall, the level of evidence produced by the articles in clinical Cardiovascular Medicine in Saudi Arabia is very low. Prioritizing higher-quality research is critical to produce the clinical practices and policies necessary to reduce the burden of cardiovascular diseases in Saudi Arabia.

1. Introduction

Cardiovascular diseases (CVDs) are a growing public health concern, especially in Saudi Arabia. CVDs remain the leading cause of morbidity and mortality in the country, with their prevalence only projected to increase in the coming decades [1–4]. By 2035, the economic burden of CVDs in Saudi Arabia, including both direct and indirect costs, is projected to increase threefold to US$ 9.8 billion [3].

International commitments to reduce the prevalence of non-communicable disease have led the Government of Saudi Arabia to invest more in CVDs-related research; however, the quality of this research remains unclear [5–7]. In 2015, the Government of Saudi Arabia signed Agenda 2030 for Sustainable Development. This agenda includes Target 3.4, which aims to reduce by one-third premature mortality from non-communicable diseases, including CVDs, by 2030. High-quality research and evidence are needed to guide clinical practice and health policy to achieve this target [5,7].

The rise of CVDs within Saudi Arabia has led the government to invest significantly in CVDs-related research; however, the quality of this research remains unclear [6,7]. The “Levels of Evidence (LOE)” ranking system by the Oxford Centre for Evidence Based Medicine is one of the standard models used to determine the quality of evidence-based research [8–10]. The model grades studies as level I, II, III, IV, or V based on quantity of data, study question (diagnosis, prognosis, treatment, screening), as well as study benefits and harms [8]. To date, the literature has examined the LOE within Saudi publications across various clinical disciplines, including gastroenterology and oncology. To the best of our knowledge, no studies have examined the LOE within Cardiovascular Medicine [11–15]. Thus, our aim is to examine the LOE within clinical Cardiovascular Medical research in Saudi Arabia.

2. Materials and methods

We conducted a systematic review according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [16]. This study protocol was submitted to the international prospective register of systematic reviews (PROSPERO). Risk of bias within and across studies was not assessed. Ethical approval was not required as primary data have not been collected.

2.1. Electronic searches

Electronic databases EMBASE and MEDLINE were searched on the 25^th of April 2021, with restriction to human studies in the English language. There were no restrictions to publication year. We used the platform Ovid SP, and identified keywords and index terms from key to conduct a second search in CENTRAL (Cochrane Central Register of Controlled Trials) and in www.clinicaltrials.gov. The Snowball- and Pearl-growing methods of search were conducted for finding additional eligible articles. Finally, a search was conducted in PubMed database for all eligible articles published by Journal of the Saudi Heart Association from the first indexed issue up to April 2021. Search engines as Google Scholar and the library of London School of Economics and Political Science were used to find the full text of chosen articles for review.

Search strategies were revised by a team of librarians. The detailed search strategy and keywords/MeSH-terms are shown in Supplementary Material, Table S1.

2.2. Study selection and collection process

2.2.1. Eligibility criteria

The inclusion criteria of articles for review included: full, peer-reviewed clinical Cardiovascular Medicine articles; articles that had at least one author affiliated with a Saudi institution; and articles that included a population from Saudi Arabia. We excluded basic science or cadaver studies, nonhuman related studies, non-English articles, instructional course lectures, non-systematic reviews, editorial correspondences, Cochrane Reviews, clinical guidelines, and letters to the editor. Quality studies and economics evaluation studies were also excluded.

2.2.2. Selection of studies

Two review authors (S.A., A.S.) independently scanned the abstracts, titles, or both, of every article retrieved to determine which studies should be assessed further. They examined full texts of all potentially relevant articles. Discrepancies were resolved through consensus or assessed by a third review author (B.P). The final selection of studies for this systematic review used Rayyan QCRI in the process of blinding results for both review authors (S.A., A.S) [17].

2.3. Data extraction and management

Two review authors (S.A, A.S) extracted data independently. When necessary, a third review author (B.P.) resolved any disparities between the two initial data extraction. Input data were saved in a pre-specified spreadsheet of a Microsoft® 365 Excel document. The extracted data elements consisted of: Publication details: last name of first author, publication year, journal’s name; study design of the analysis (not of the data collection) for each publication regardless of that stated by the authors. LOE was allocated to each study according to the Oxford Level of Evidence 2 scale (level I as the highest-level score, level V as the lowest) [8].

2.4. Data analysis

We summarized data using descriptive statistics (number of included articles, proportion each study design type, proportion of each LOE). Stratified by 5-year time periods, we calculated the mean LOE, and explored, using linear regression, evidence for a time trend for number of published articles and LOE. The Stata 16.1 SE (StataCorp., College Station, TX, USA) statistical package was used for statistical analysis.

3. Results

The selection process of the records is shown in Fig. 1. We retrieved a total of 1113 records. Six hundred and eighty-nine records were identified from the data base search. Additionally, 632 records were identified through other sources, of which 582 were from the Journal of Saudi Heart Association. Four hundred and eighteen articles met the inclusion criteria and were included in our review (Found in Supplementary Material, Table S2). The articles were published between September 1986 and March 2021. The most common type of study design was cross-sectional studies (29.1%), followed by case reports (27%), and retrospective studies (21%) (Table 1). We identified four systematic reviews, but none were restricted to randomized controlled trials or included a meta-analysis. Of the eight identified randomized clinical trials, two adopted an open label design.

Fig. 1.

Study selection process.

Table 1.

Classification of included articles according to study design.

Type of study design	Frequency (%)
Systematic reviews	4 (0.96)
Randomized clinical trials	8 (1.91)
Prospective studies	71 (16.99)
Retrospective studies	88 (21.05)
Cross-sectional studies	12 (29.43)
Case series	11 (2.63)
Case reports	113 (27.03)
Total	418 (100)

As for LOE, more than half of the included articles were level IV studies (n = 242, 57.8%). Level I studies made up only 1.9% of the included articles (n = 8). As shown in Fig. 2, the trend over the years for number of published articles increased from only 8 articles in 1986–1990 to 163 articles in 2016–2021 (β = 26.46, 95% CI [14.23, 38.69], p = 0.003). However, we observed no trend over the years for increased mean of LOE (β = −0.07, 95% CI [−0.20, −0.06], p = 0.236) (Table 2).

Fig. 2.

Number published articles in clinical Cardiovascular Medicine in Saudi Arabia with their proportion of level of evidence over the years (Stratified by 5-Year time periods).

Table 2.

Levels of evidence of clinical Cardiovascular Medicine research in Saudi Arabia (Stratified by 5-Year time periods).

Period	Level of evidence frequency (%)				Mean
	Level I	Level II	Level III	Level IV
1986–1990	0 (0.0)	0 (0.0)	0 (0.0)	8 (100.0)	4
1991–1995	0 (0.0)	2 (25.0)	2 (25.0)	4 (50.0)	3.3
1996–2000	0 (0.0)	1 (5.9)	4 (23.5)	12 (70.6)	3.6
2001–2005	1 (2.5)	11 (27.5)	8 (20.0)	20 (50.0)	3.2
2006–2010	0 (0.0)	6 (10.9)	10 (18.2)	39 (70.9)	3.6
2011–2015	3 (2.4)	20 (15.7)	24 (18.9)	80 (63.0)	3.4
2016–2021a	4 (2.5)	31 (19.0)	49 (30.1)	79 (48.5)	3.2
Total	8 (1.9)	71 (17.0)	97 (23.2)	242 (57.9)

^aUntil 25/04/2021.

4. Discussion

The aim of our study was to examine the LOE within clinical Cardiovascular Medicine research in Saudi Arabia. Our systematic review examined 418 articles in clinical Cardiovascular Medicine. Reviewed articles had at least one author affiliated with a Saudi institution and included a population from Saudi Arabia. All articles were published between September 1986 and March 2021, the number of articles has increased throughout the years.

Our results indicate that there is a lack of high-level evidence within clinical Cardiovascular Medicine in Saudi Arabia. The majority of articles were graded as level IV studies. Notably, the proportion of articles classified as level I, II, III, or IV remained constant across the three decades. We did note a significant increase in the number of publications after 2011, marking the year the Journal of the Saudi Heart Association re-emerged as an online journal. Examining how and why the switch of Saudi journals to online platforms impacts readership and research may be crucial to understanding this trend.

The results of our study reflect those found by others on research from Saudi Arabia, both within and outside Cardiovascular Medicine. Saquib et al. conducted a bibliometric analysis examining the types of CVDs research performed in Saudi Arabia from 1987 to 2017. They found that experimental design constituted only 3% of all research studies, and less than 9% of all studies included a hypothesis-testing design [7]. These results are in agreement with our findings, as only 18.8% of studies reviewed had a LOE of I or II. The results of our study also reflect those found in LOE analyses of other medical disciplines within Saudi Arabia. Studies on the LOE of Saudi Arabian publications in the fields of gastroenterology, plastic surgery, orthopaedic surgery, oncology, and neurosurgery found between 80 and 91% of the articles they reviewed had a LOE grade of IV [11–15].

The LOE found in Saudi Arabia is low compared to those produced in North America and Europe. A study found most published orthopaedic studies in Saudi Arabia had a LOE of IV in contrast to North American studies, which contained a higher proportion of high-level studies (p < 0.05) [14]. The consistent presentation of low-grade evidence in research across medical disciplines in Saudi Arabia—especially when contrasted with the high-level evidence present in European and North American publications— brings to question whether there are national barriers impeding the production of high-quality evidence. The literature indicates that despite the increase in funding towards biomedical research in Saudi Arabia, local health practitioners still face many challenges that impede their ability to conduct high-level clinical research [7,18,19]. A cross sectional survey, conducted in 2019, suggested that the primary challenge was the inability to find the necessary human resources to conduct high-quality research [18]. Higher level studies often employ study coordinators to absorb the heavy task of enrolling, assessing, and following patients. Further challenges include the availability of data extraction technologies and support for statistical designs and interpretation. The lack of human resources available in academic and clinical settings within Saudi Arabia may be why researchers are more prone to utilise simpler and less rigorous methodologies, graded as low LOE. Interestingly, Alzahrani also noted that academic staff were not encouraged to publish and conduct research as much as those from European or North American institutions [19]. These challenges may impede the production of high-quality evidence-based research in Saudi Arabia.

Saudi Arabia faces many challenges that impede the ability of clinicians and academics to produce high-level publications; however, it should be noted that low LOE is not an issue exclusive to Saudi Arabia. There is a lack of high-level evidence-based research within clinical Cardiovascular Medicine worldwide. Researchers examined the proportion and LOE of research that supported American College of Cardiology/American Heart Association (ACC/AHA) and European Society of Cardiology (ESC) recommendations between 2008 and 2018. They found the proportion of those supported by level I & II evidence categories remains small [20]. Furthermore, they found the proportions of level I & II evidence did not increase between 2008 and 2018. Van Dijk et al. found comparable results when examining evidence levels supporting European Society of Cardiology guideline recommendations [21].

CVDs remain an urgent public health concern throughout Saudi Arabia. The production of high-level, evidence-based research on the Saudi Arabian population is key to crafting national guidelines that can promote efficient and locally targeted recommendations. Funding should be prioritised for well-designed studies that would provide higher LOE, address priority questions and represent adequate sample of the population in Saudi Arabia.

This study has a few limitations. First, several studies were in fact a cross-sectional analysis of a given disease over the span of a selected time period without any follow-up of the outcome and without temporal precedence of the risk factors; although, a retrospective cohort study or retrospective review, was specified as the design by the authors. We recategorized these studies based on sometimes limited methodological descriptions to the best of our understanding. This might have led to errors and influenced the LOE classification of each study. Second, we did not examine the quality of each study in-depth, which may influence the LOE classification of each article. We considered using other validated scales that evaluate in-detail elements of each study, but many studies did not provide enough detail enabling such rigorous scrutiny.

5. Conclusion

The overall number of LOE I studies in clinical Cardiovascular Medicine in Saudi Arabia is very low. Despite increased number of published articles, we did not see a trend toward producing higher evidence studies over the last three decades. Future research in Saudi Arabia should focus on producing stronger evidence that represents the local and national priorities.

Abbreviations

CVDs

Cardiovascular Diseases

PRISMA

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

LOE

Level of Evidence

Appendix.

Supplementary material link: https://www.j-saudi-heart.com/cgi/viewcontent.cgi?filename=2&article=1264&context=jsha&type=additional&preview_mode=1.