Standard echocardiography versus handheld echocardiography for the detection of subclinical rheumatic heart disease: protocol for a systematic review

Abstract

Introduction

Rheumatic heart disease (RHD) is a preventable and treatable chronic condition which persists in many developing countries largely affecting impoverished populations. Handheld echocardiography presents an opportunity to address the need for more cost-effective methods of diagnosing RHD in developing countries, where the disease continues to carry high rates of morbidity and mortality. Preliminary studies have demonstrated moderate sensitivity as well as high specificity and diagnostic odds for detecting RHD in asymptomatic patients. We describe a protocol for a systematic review on the diagnostic performance of handheld echocardiography compared to standard echocardiography using the 2012 World Heart Federation criteria for diagnosing subclinical RHD.

Methods and analysis

Electronic databases including PubMed, Scopus, Web of Science and EBSCOhost as well as reference lists and citations of relevant articles will be searched from 2012 to date using a predefined strategy incorporating a combination of Medical Subject Heading terms and keywords. The methodological validity and quality of studies deemed eligible for inclusion will be assessed against review specific Quality Assessment of Diagnostic Accuracy Studies 2 criteria and information on metrics of diagnostic accuracy and demographics extracted. Forest plots of sensitivity and specificity as well as scatter plots in receiver operating characteristic (ROC) space will be used to investigate heterogeneity. If possible, a meta-analysis will be conducted to produce summary results of sensitivity and specificity using the Hierarchical Summary ROC method. In addition, a sensitivity analysis will be conducted to investigate the effect of studies with a high risk of bias.

Ethics and dissemination

Ethics approval is not required for this systematic review of previously published literature. The planned review will provide a summary of the diagnostic accuracy of handheld echocardiography. Results may feed into evidence-based guidelines and should the findings of this review warrant a change in clinical practice, a summary report will be disseminated among leading clinicians and healthcare professionals in the field.

PROSPERO registration number

CRD42016051261.

Strengths and limitations of this study.

• We will evaluate the accuracy of handheld echocardiography for detecting subclinical rheumatic heart disease in endemic areas, making the proposed review relevant to current global agendas.

• We will not impose a search filter or any limits in terms of language during the literature search so as to minimise the chance of missing studies.

• Data extraction will be performed by two independent reviewers, thereby reducing the risk of bias.

• Accuracy measures (sensitivity and specificity) may be influenced by underestimated burden of disease estimates (incidence and prevalence) due to the scarcity of good quality epidemiological data.

• Variation in diagnostic criteria for handheld echocardiography may affect data synthesis.

Introduction

Background

Rheumatic heart disease (RHD) is a permanent heart valve condition resulting from an abnormal immune reaction to group A streptococcal infection typically occurring in childhood.¹ If left untreated, disease progression can result in irreversible heart valve damage, cardiac failure, stroke and premature death.^{2 3} Significantly, RHD is a preventable and treatable chronic condition which mostly affects disadvantaged populations across the world.² Even though the disease has mostly been eradicated in North America and Europe, barring a few indigent pockets, it remains prolific in areas of the Middle East, the South Pacific, Africa as well as Central and South Asia.²

The continued persistence of RHD contributes to considerable amounts of preventable morbidity and mortality, particularly among adolescents and young adults.⁴ This adds additional strain to what are often already overburdened health systems with endemic regions, which are typically poorly resourced, bearing the brunt of the disease.^{1 5} Furthermore, the accurate detection of subclinical RHD in children and adolescents remains hampered by the cost of diagnostic machinery and scarcity of trained personnel.⁶ Alternative RHD screening tests, which are both accurate and affordable, are therefore needed in many endemic areas. The value of such a screening test is that significantly more cases of subclinical RHD might be detected, thereby reducing the time to commencement of secondary prophylaxis and thus, in turn, improving long-term outcomes.⁷

Recently, handheld echocardiography has become widely available with a variety of clinical uses.⁸ Similarly, diagnostic accuracy has already been demonstrated in a number of studies assessing its value as a screening tool, despite some limitations such as lack of Doppler capabilities. Due to the non-invasive, safe, portable and relatively inexpensive nature of handheld echocardiography, the device has been presented in recent publications as a promising alternative to standard echocardiography in resource-limited and remote settings.^{4 8} To test this assertion, the diagnostic accuracy of handheld echocardiography needs to be evaluated using a systematic approach. This review, therefore, proposes to evaluate the accuracy of handheld echocardiography for the detection of RHD in children and adolescents within a screening setting. We seek to generate new quantitative evidence for clinicians and guideline developers to establish evidence-based guidelines for diagnosing RHD with handheld echocardiography. Ultimately, this will improve the management of patients with RHD, as effective treatment of subclinical RHD requires accurate and timely diagnosis.

Primary objective

To determine the diagnostic accuracy of handheld echocardiography compared with standard echocardiography (two-dimensional (2D), continuous-wave and colour-Doppler echocardiography) performed by an experienced imager in conjunction with the 2012 World Heart Federation (WHF) criteria for the detection of any RHD in children and adolescents.

Secondary objective

To investigate potential sources of variation in relation to age, gender, geographical location, echocardiographic criteria and echocardiographer expertise in detecting subclinical RHD with handheld echocardiography.

Methods and analysis

The protocol was prepared according to the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guidelines. A PRISMA Protocol checklist is completed and included in online supplementary appendix 1.⁹

Inclusion and exclusion criteria

We will include all primary observational studies which compare the diagnostic accuracy of handheld echocardiography to the reference standard; standard echocardiography performed by an experienced imager and in conjunction with the 2012 WHF criteria. Eligible studies can be of a cross-sectional, cohort or diagnostic case–control design, provided both cases and controls have been sampled from the same population. Studies which report on, or contain the data necessary to extract information on the proportions of true positives (TP), false positives (FP), true negatives (TN) and false negatives (FN) will be included. Studies which enrolled only those with a confirmed RHD diagnosis will be excluded on account of the potential for overestimation of sensitivity. Descriptive studies such as case studies/series will also be excluded from this review. Studies in which we are unable to generate two-by-two tables, as well as different studies which report on duplicate data will not be considered for inclusion in this review.

We will consider all studies in which samples of study participants are either, a randomly, or consecutively selected series of individuals from populations in which RHD is prevalent worldwide for inclusion. For the purposes of this review, children and adolescents will be defined as being between the ages of 5 and 17 years (age range: ≥5 years to <18 years). More specifically, participants will be considered children if they are between 5 and 9 years of age and adolescents if they are between 10 and 17 years of age.

We will include studies evaluating the accuracy of handheld echocardiography for RHD detection. There will be no restrictions regarding the type of handheld device used or the aptitude of person performing the cardiac ultrasound; however, these data will be recorded and analysed accordingly. Studies will be deemed eligible for inclusion if the reference standard constituted the interpretation of echocardiographic findings using the 2012 WHF criteria when echocardiographic assessment by 2D, continuous-wave and colour-Doppler echocardiography was performed by a cardiologist or cardiac sonographer. We will exclude all studies published before 2012 to omit any study which does not use standard echocardiography in conjunction with the 2012 WHF criteria as the reference standard. We will consider all studies which evaluate any RHD (definite and borderline) as the condition of interest for inclusion in this review. All case definitions will be consistent with the 2012 WHF criteria.¹⁰

Search strategy

A comprehensive electronic literature search of PubMed, Scopus, Web of Science and EBSCOhost will be conducted to identify relevant literature. No restrictions in terms of language will be applied during the search. Searches will however be limited to only include articles published from 2012 up until the present. All sources will be systematically searched using a combination, where relevant, of both free text words and Medical Subject Heading terms. Search strategies will be tailored to meet the requirements of each electronic database as in table 1 below. Search terms will include synonyms for ‘rheumatic heart disease’, ‘echocardiography’ and ‘handheld’. A list of all articles identified through the literature search will be compiled and references managed using Mendeley software. In addition, a manual search of all eligible articles’ reference lists, articles citing eligible articles as well as relevant review articles will be carried out to identify any additional literature not identified by the comprehensive electronic literature search. Abstracts from any relevant conference proceedings will also be searched for among appropriate websites and followed up on if eligibility requirements are sufficiently met. Finally, experts in the field will be contacted for additional information where necessary.

Table 1.

Search strategy

Database	Search terms	Limits
PubMed	((((((((((((((((Hand-held) OR handheld) OR hand held) OR hand-carried) OR hand carried) OR HAND) OR HCU) OR HHCU) OR pocket size) OR pocket sized) OR portable) OR miniaturisation) OR miniaturised) OR focused) OR focus)) AND ((((‘Echocardiography’[Mesh]) OR echocardiography) OR echocardiographic) OR cardiac ultrasound)) AND (((‘Rheumatic Heart Disease’[Mesh]) OR rheumatic heart disease) OR RHD) MeSH terms will be exploded during the search	Limited to 2012–2017
Scopus	Hand-held OR handheld OR hand held OR hand-carried OR hand carried OR HAND OR HCU OR HHCU OR pocket size* OR portable OR miniatur* OR focus* Echocardiograph* OR cardiac ultrasound Rheumatic Heart Disease OR RHD #1 AND #2 AND #3	Limited to 2012–2017
ISI Web of Science	Hand-held OR handheld OR hand held OR hand-carried OR hand carried OR HAND OR HCU OR HHCU OR pocket size OR pocket sized OR portable OR Miniaturisation OR Miniaturised OR focused OR focus Echocardiography OR Echocardiographic OR cardiac ultrasound Rheumatic Heart Disease OR RHD Combine #1 AND #2 AND #3	Limited to 2012–2017 and filtering out Medline
EBSCOHost	S1. Hand-held OR handheld OR hand held OR hand-carried OR hand carried OR HAND OR HCU OR HHCU OR pocket size OR pocket sized OR portable OR Miniaturisation OR Miniaturised OR focused OR focus S2. Echocardiography OR Echocardiographic OR cardiac ultrasound S3. Rheumatic Heart Disease OR RHD S1 AND S2 AND S3	Limited to 2012–2017

EBSCO, Elton B Stephens Company; HAND, handheld cardiac ultrasound; HCU, handheld cardiac ultrasound; HHCU, hand-held cardiac ultrasound; ISI, Institute for Scientific Information; MeSH, Medical Subject Heading terms; RHD, rheumatic heart disease.

Selection of studies for inclusion

The titles and/or abstracts of all articles identified by the literature search will be screened independently by two reviewers. Based on the predefined inclusion and exclusion criteria any clearly ineligible studies will be excluded. Following this, the full-text versions of all potentially eligible studies will then be reviewed by two independent reviewers to assess their eligibility. Any discrepancies regarding eligibility will be resolved through discussion and consensus with a third reviewer.

Data extraction and management

Using a predefined data extraction form, two reviewers will independently extract the following information from all studies meeting the criteria for inclusion:

• Study identifiers: author(s), year of publication, journal;

• Study characteristics: study design, study country/setting/context, study population/participants, sample size, participant recruitment procedures, participant demographics and RHD prevalence (pretest probability);

• Reference standard and index test details:

  • General: test positive or negative;
  • Specific: individual findings on cardiac ultrasound;
  • Expertise of person(s) performing and/or interpreting tests: expert versus non-expert;
  • Diagnostic criteria: test threshold(s);
  • Number of missing or unavailable test results.

• Diagnostic test outcome measures: sensitivity, specificity, positive and negative predictive values, number of TP, FP, TN and FN.

If necessary, any disagreements will be resolved through discussion with a third reviewer until a consensus is reached. Any data missing from the reports of included studies will be requested from study authors. In cases where studies have used different diagnostic criteria for handheld echocardiography, attempts will be made to standardise them to mirror the 2012 WHF criteria as closely as possible. The information garnered through the data extraction process will be used to determine each study’s quality as well as for synthesising evidence.

Risk of bias and quality assessment

The Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool (see table 2) will be used to assess the risk of bias and concerns regarding applicability of all included studies.¹¹ The tool encompasses four domains which have been tailored to meet the specific requirements of the review. Two reviewers will independently assess the risk of bias in all included studies according to the revised QUADAS-2 criteria. Any discrepancies will be resolved through discussion until consensus is reached and with the assistance of a third reviewer if necessary. Both text and graphics will be used to demonstrate the results.

Table 2.

Design-specific criteria to assess methodological quality

Categories	Domains
	1. Patient selection	2. IT	3. RS	4. Flow and timing
Description	Briefly describe the methods of patient selection:	Describe the IT (HAND), how it was conducted and interpreted:	Describe the RS (STAND) how it was conducted and interpreted:	Describe patients that did not receive HAND, and/or STAND or who were excluded from the 2×2 table. Describe the time interval and any interventions between the HAND and STAND.
Indicator questions (yes, no, unclear)	Was a consecutive or random sample of patients enrolled?	Were the HAND results interpreted without knowledge of the results of STAND?	Was STAND likely to correctly classify the target condition?	Was there an appropriate time interval between HAND and STAND?
	Was a case–control design avoided?	Was a prespecified threshold used?	Were the STAND results interpreted without knowledge of the HAND results?	Did all patients receive STAND and was it the same RS?
	Did the study avoid inappropriate exclusions?			Were all patients included in the analysis?
*Risk of bias (low, high, unclear)	Based on the indicator questions, could the selection of patients have introduced bias?	Based on the indicator questions, could the conduct or interpretation of HAND have introduced bias?	Based on the indicator questions, could STAND, its conduct or its interpretation have introduced bias?	Based on the indicator questions, could the patient flow and timing have introduced bias?
Concerns Regarding Applicability (low, high, unclear)	Describe included patients (prior testing, presentation, intended use of HAND and setting): Based on the description of included patients, are there concerns that the included patients do not match the review question?	Are there concerns that HAND, its conduct, or interpretation differ from the review question?	Are there concerns that the target condition as defined by STAND does not match the review question?

*Criteria for grading risk of bias: If all indicator questions for a single domain are answered ‘ yes’, then the risk of bias will be judged as being ‘ low’; if any indicator question is answered ‘ no’, then the potential for bias will be flagged and the review authors will be required to judge the risk of bias with the assistance of the senior author (MEE); if all or most indicator questions were answered ‘ no’, then the risk of bias will be judged as being ‘ high’ and indicator questions are can only be answered as ‘unclear’ when the data are insufficient to allow for the formulation of a judgement.

Adapted from Whiting et al.¹¹

IT, index test; RS, reference standard.

Subgroup and sensitivity analyses

Subgroup analysis may be performed, considering specific characteristics of the studies, such as echocardiography protocol, training background of the examiner, age and geographical location.

We will conduct a sensitivity analysis to investigate the effect of variations in criteria on the overall accuracy of diagnosis. In addition, we will explore the effect of excluding studies with a high risk of bias on the accuracy of summary estimates, sensitivity and specificity. We will not investigate publication bias.

Statistical analysis and data synthesis

We will first analyse data descriptively by plotting the sensitivity and specificity (including 95% CIs) of all included studies in both forest plots and receiver operating characteristic (ROC) space. These plots will be generated using the Review Manager software package.¹² If there are sufficient data, we will conduct a meta-analysis to produce summary results of sensitivity and specificity. Because we anticipate that studies will have different positivity thresholds due to the use of different sets of diagnostic criteria, we will pool the results using the hierarchical summary receiver operating characteristic (HSROC) method. Meta-analysis will be performed using SAS V.9.4/STATA V.14.2 software.¹³ We will also explore, through metaregression, the relationship of test accuracy with categorical or continuous covariates such as test threshold.¹⁴

Investigations of heterogeneity will initially begin by visually examining the forest and ROC plots for heterogeneity in sensitivity and specificity. We will then analyse the possible sources of heterogeneity as covariates in the statistical models. Potential sources of heterogeneity to be investigated as categorical variables include; age (children vs adolescents), sex (male vs female), geographical location (high vs low/middle-income countries), diagnostic criteria (single vs multiple views and different thresholds) and echocardiographer expertise (expert vs non-expert).

Presenting and reporting of results

The study selection process will be summarised in the form of a flow diagram detailing the reasoning behind all exclusions. Results will be reported in accordance with the PRISMA guidelines.¹⁵

Dissemination

The planned review will provide a summary of the diagnostic accuracy of handheld echocardiography. Results may feed into evidence-based guidelines and will therefore be disseminated to members of the WHF criteria working group. Should the findings of this review warrant a change in clinical practice, a summary report will be circulated among leading clinicians and healthcare professionals in the field.