Diagnostic accuracy of Doppler twinkling artifact for identifying urolithiasis: a systematic review and meta-analysis

Abstract

Objective

The goal of this study was to perform a comprehensive meta-analysis to assess the overall diagnostic value of Doppler twinkling for the diagnosis of urolithiasis.

Methods

We systematically searched the PubMed, EMBASE, and Cochrane Library databases from inception through May 31, 2021. Studies including patients with urolithiasis who underwent color flow Doppler sampling to highlight the twinkling artifact and computed tomography were included. Diagnostic test meta-analysis was performed with a bivariate model. We used summary receiver operating characteristic curves to summarize the overall diagnostic performance. The weighted sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio were calculated.

Results

Sixteen studies involving 4572 patients were included in the systematic review and meta-analysis. The weighted sensitivity was 0.86 (95% confidence interval [CI] 0.72–0.94), specificity 0.92 (95% CI 0.75–0.98), positive likelihood ratio 11.3, negative likelihood ratio 0.2, and diagnostic odds ratio 75.5.

Conclusion

The Doppler twinkling artifact has good diagnostic value for the diagnosis of urolithiasis and should be used as a complementary tool in the diagnosis of urolithiasis.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40477-022-00759-z.

Introduction

Urolithiasis is a common problem in general practice. Prevalence rates vary from 7 to 13% in North America, 5 to 9% in Europe, and 1 to 5% in Asia [1]. Many factors affect nephrolithiasis, including genetic predilection, age, sex, diet, occupation, and lifestyle [2]. Nephrolithiasis occurs more often in men than women, affecting 10.6% vs. 7.1% in the United States (US) population [3]. Kidney stone formation is a common urological problem, with a lifetime prevalence of approximately 10% in men and 6% in women, and its prevalence has been increasing in many developed countries, with a recurrence rate of nearly 60% within 10 years after initial treatment [4].

Many diagnostic methods have been thought to aid in identifying urolithiasis, including plain radiography, intravenous pyelourethrography, ultrasonography (US), and computed tomography (CT) [5]. Unenhanced CT scanning is broadly considered the current gold standard for the diagnosis of urolithiasis [6]. Noncontrast CT is the most sensitive (up to 98%) and specific (96–100%) method for the detection of urinary stones [7]. A history of nephrolithiasis with the onset of flank pain should prompt the ordering of a noncontrast CT of the abdomen and pelvis to assess for the presence of urinary stones. Moreover, because of its ability to detect nonurologic causes of abdominal pain, CT has become the primary imaging modality for the confirmation of urinary calculi [8]. Nevertheless, one disadvantage of CT is that the associated radiation exposure is increasingly recognized as a public health issue [9], as radiation exposure can damage the genetic material in cells and result in radiation-induced cancers years later or in heritable disease in the descendants of exposed individuals, and it can possibly lead to various developmental effects under specific conditions [10]. Abdominal ultrasound, by contrast, does not involve radiation and has been proven to identify urinary stones using the general criterion of observing a hyperechoic lesion with posterior acoustic shadow [11]. Hydroureteronephrosis also significantly improves the detection of urinary stones of B-Mode ultrasound imaging [12]. However, this phenomenon used alone has limited sensitivity [13], especially for small stones (53% of stones are less than 5 mm) [14].

Ultrasound using Doppler twinkling artifacts (TA) is reportedly an efficient tool for detecting urinary tract calculi [15–18]. The twinkle artifact was first described by the French radiologist Rahmouni and colleagues and can be seen with color Doppler ultrasound, appearing as a rapidly changing mixture of red and blue behind a rough interface object such as a calcification [19]. Moreover, the chemical composition of the calculus is related to the creation of the artifact. Chelfouh et al. [20] discovered that calcium oxalate dehydrate and calcium phosphate stones always produce a TA sign, whereas calculi composed of calcium oxalate monohydrate and urate lack a TA sign. Laher et al. [21] previously reviewed the validity of TA as a diagnostic tool for the presence of urolithiasis using a systematic review and meta-analysis. They reported that the TA sign may be useful as a complementary tool in the diagnostic workup of patients with suspected urolithiasis. Laher et al. [21] evaluated 22 articles regarding studies involving 4389 participants and determined the accuracy of the TA sign for detecting urolithiasis and its pertinent findings in the emergency department using various modalities for confirmation. Their pooled sensitivity (88.16%) for the TA sign was 86%, whereas the pooled specificity was 79.22%). Several recent studies evaluated the diagnostic accuracy of twinkle artifacts for identifying urolithiasis in comparison with CT, but the results were inconclusive [17, 22]. Therefore, in this study, we performed a systematic review and determined the diagnostic accuracy of Doppler TA for diagnosing urolithiasis.

Methods

Study design

The systematic review and meta-analysis was performed according to the protocol registered with PROSPERO (CRD42021258577) [23]. The study results are reported in accordance with the Preferred Reporting Items for a Systematic Review and Meta-Analysis (PRISMA) statement [24]. No ethical approval or informed consent was required.

Search strategy and selection criteria

We searched PubMed, EMBASE, and the Central Register of Controlled Trials (CENTRAL) for relevant studies published up to May 31, 2021. The following search algorithms were used: (Twink* Artefact) OR (Twink* Artifact). No language, publication date, or publication status restrictions were imposed. We also screened ClinicalTrials.gov and other relevant publications, including reviews, systematic reviews, and meta-analyses, for additional relevant studies. In addition, the references of the selected articles were manually screened by two reviewers to achieve a more comprehensive search. Studies were included in this systematic review if they fulfilled all of the following criteria: (i) an evaluation of the diagnostic performance of the Twinkling artifact; (ii) attempted a diagnosis of urolithiasis; and (iii) reported data regarding the accuracy of the test (e.g., events, sensitivity, or specificity).

Study selection and data extraction

Records were independently screened by two investigators (PN, KN) according to title and abstract to identify potential studies based on the inclusion/exclusion criteria. Disagreements were resolved by discussion and/or the participation of a third author (KK). Each potentially relevant study was reviewed in full by the same investigators. Data were extracted by two investigators (PN, KN) using standard data extraction forms. The following data were independently extracted by two reviewers: study details (authors, year of publication, country of origin, study design, and sample size), patient characteristics (type, location of calculi, size of calculi), and diagnostic test results (true positive, TP; true negative, TN; false positive, FP; false negative, FN, sensitivity, and specificity). The risk of bias of the included studies was evaluated and rated independently by two reviewers using the Diagnostic Precision Study Quality Assessment Tool (QUADAS-2) recommended by the Cochrane Collaboration [25].

Statistical analysis

The heterogeneity of the studies was established by χ² analysis, with inconsistency values (I²) greater than 50% considered to be moderate heterogeneity and I² values greater than 75% defined as high heterogeneity. Outcomes with I² values greater than 50% were submitted to sensitivity analysis (i.e., hypothetical removal of studies) [26]. The traditional I² statistic is not recommended for quantifying heterogeneity in sensitivity and specificity because it is a univariate measure that does not account for potential threshold effects. To investigate whether a factor is associated with test accuracy, exploratory analyses can be performed by visual inspection of forest plots and SROC plots [27]. A bivariate model analysis was performed to assess the summary estimates of sensitivity, specificity, diagnostic odds ratio, positive likelihood ratio (LR) and negative likelihood ratio (LR–). Pooled data were given with 95% confidence intervals (CIs) and displayed using forest plots and SROC plots [27]. In that case, pooled estimates of the diagnostic parameters were determined using a random effects model (DerSimonian–Laird method) [27].

Results

Literature search

A total of 479 records were identified from the literature search, and 100 records were identified through other sources (Fig. 1). After removal of duplicates, 418 studies were screened by title and abstract. Of 36 potentially relevant articles, 16 [15–18, 22, 28–38] were included in the meta-analysis.

Fig. 1.

Study flow

Study and patient characteristics

The major characteristics of the 16 included studies involving 4572 patients are illustrated in Table 1. Six studies (37.5%) were performed in the Middle East, 5 studies (31.2%) in North America, 2 studies (12.5%) in Europe and China, and 1 study (6%) in Egypt. Nine studies (56.2%) were retrospective, 6 studies (37.5%) were prospective, and 1 study (6%) was cross-sectional.

Table 1.

Characteristics of included studies

Study	Country	Study design	Sample size (N)	Location of calculi	Size of calculi with SD (mm)	Intervention	Reference standard	Mean age (SD) in years
Adel et al. (2019) [29]	Pakistan	Cross-sectional study	221	Urinary tract calculi	NA	US performed by 3-year experience trained sonographer using curved probe	unenhanced CT	45.9 (16.3)
Dillman et al. (2011) [31]	United States	Retrospective study	74	Kidneys	7.5 (1.3–26.5)	Three board-certified abdominal radiologists reviewed color Doppler images together and formed opinion by consensus	unenhanced CT	52.0 (18.0)
Hanafi et al. (2019) [34]	Iran	Prospective study	100	Kidneys	1.8–5	all of the patients were examined by a radiologist familiar with renal ultrasound and urinary tract	unenhanced CT	13.4 (42.4)
Kielar et al. (2012) [22]	Canada	Prospective study	55	Kidneys, ureters, bladder	2.6 (1–9)	Right after the CT scan, the patient directly underwent a limited sonographic scan of the kidneys, ureters, and bladder by a trained sonographer using a curved low-frequency probe	unenhanced CT	49.0 (28.0–81.0)
Korkmaz et al. (2014) [35]	Turkey	Retrospective study	60	Kidneys, upper, middle, lower ureter	3.9 (2–5)	All patients were examined by an experienced urinary radiologist using US and CDUS devices with a broadband curved-array transducer	unenhanced CT	38.0 (3.1)
Liu et al. (2019) [16]	China	Prospective study	2268	Right, left upper, middle, lower ureter	10 (2)	Two 5-year-experienced in ultrasonography of the kidney, ureter, and bladder doctors performed color Doppler ultrasound and B-ultrasound on each patient	CT	48.2 (13.7)
Masch et al. (2015) [15]	United States	Retrospective blinded cohort study	85	Kidneys	3.6 (1.0–20.0)	Ultrasound examinations of the abdomen were performed with mean PRF color-flow velocity setting of 85 cm/s	Unenhanced CT	46.0 (13.0–92.0)
Puttmann et al. (2021) [36]	United States	Retrospective study	293	Kidneys, ureters	5.0 (2.0–10.0)	The presence of nephrolithiasis was confirmed either by visualization on CT scan within 3 months of the ultrasound, visualization at time of surgery [ureteroscopy or extracorporeal shock wave therapy (ESWL)], or patient reported passing the stone	CT	9.0 (12.1)
Salmaslıoğlu et al. (2018) [38]	Turkey	Prospective study	72	Kidneys, ureters, bladder	8.1 (6.4)	All patients were examined by a standardized US scanning procedure	Unenhanced CT	45.8 (14.7)
Verhagen et al. (2018) [18]	United Kingdom	Retrospective observational study	31	Kidneys	5.0 (1.0–35.0)	Clinical US examinations had been performed by radiologists (all with > 7 years of dedicated paediatric radiology experience) or senior pediatric sonographers (all with > 4 years of dedicated paediatric experience)	unenhanced CT	13.0 (2.5)
Winkel et al. (2012) [17]	Denmark	Prospective study	105	Kidneys, ureters, bladder	3.9 (1.0–20.0)	US examinations were done by one of two skilled physicians using a convex 5 MHz transduce	unenhanced CT	50.0 (19.0–91.0)
Abdel-Gawad et al. (2016) [28]	United States	Prospective study	815	Kidneys, ureters	7.3 (2.4)	CDU was performed by 2 senior radiologists who were blinded to the results of the previously conducted NCCT and US	unenhanced CT	37.2 (11.0)
Ahmad and Abdallah (2014) [39]	Egypt	Prospective study	71	Kidneys, ureters, bladder	5.0–20.0	Following CT, patients immediately underwent ultrasonography of the abdomen and pelvis using a color Doppler scanner equipped with a 2–5 MHz convex probe	unenhanced CT	41.6 (21.0–69.0)
Letafati et al. (2020) [6]	Iran	Prospective study	99	Kidneys	NA	Within 24 h of CT imaging, US study of the abdomen was performed by a radiologist with 10 years of experience who was blind to the patient’s CT scan findings, using a 2–5 MHz convex probe	unenhanced CT	38.7 (16.3)
Sen et al. (2016) [9]	Turkey	Prospective study	106	Right, left upper, middle, lower ureter	10.9 (4.1)	US examination was performed by an experienced sonographer using a convex 2–5 MHz probe at kidney preset	Unenhanced CT	44.9 (15.1)
Wang et al. (2019) [40]	China	Retrospective study	117	Right, left upper, middle, lower ureter	7.2 (2.1)	The patients who were referred to the ultrasound department underwent B-mode ultrasonography and color Doppler ultrasonography of the kidneys and ureters, respectively. Next, color Doppler ultrasonography was performed at the same site under optimized conditions for identifying the TA, with reference to our previous study	CT	40.0 (12.0)

CT computed tomography, NA not available

Eight studies (50%) examined the urinary tract, 5 studies (31.2%) examined only the kidneys, and 3 studies (18.7%) examined the ureter. Nine studies (56.2%) reported a stone size greater than 5 mm, 5 studies (31.2%) reported a stone size less than 5 mm, and 2 studies (12.5%) did not mention the size of the calculus. Seven studies (43.7%) were performed by board-certified radiologists, 5 studies (31.2%) were performed by physicians/trained sonographers, 2 studies (12.5%) involved images reviewed by radiologists, 1 study (6.2%) was reviewed by urologists, and 1 study (6.2%) did not identify the reviewer(s). The reference standard for the diagnosis of calculi was unenhanced CT in 16 studies (81.2%) and CT in 3 studies (18.7%). Two studies (12.5%) investigated pediatric patients, and 14 studies (87.5%) investigated adults (see Table 1).

Diagnostic performance of Doppler twinkling artifacts for identifying urolithiasis

A meta-analysis was performed to appraise the accuracy (sensitivity, specificity) of Doppler TA for identifying urolithiasis (Fig. 2, Table 2). The pooled sensitivity was 86.2 (95% CI 72.4–93.7) for the diagnostic accuracy for TA. The pooled specificity of TA was 92.3 (95% CI 75.2–97.9). The diagnosis odds ratio was 75.5 (95% CI 11.6–492.7). The pooled LR+ was 11.3 (95% CI 3.0–42.2). All 6 studies yielded a relevant negative likelihood ratio of less than 0.5, and the pooled LR− was 0.2 (95% CI 0.1–0.3). Moderate to high heterogenicity was observed for the TA diagnosis test, as shown by forest plots (Fig. 2) and SROC curves (Fig. 3).

Fig. 2.

Forest plots showing the sensitivity and specificity of Twinkling artifact. The study of Ahmad and Abdallah [39] and Wang et al. [40] provided the sensitivity values without event numbers

Table 2.

Diagnostic accuracy of Twinkling artifact

Diagnostic parameter	Value
Sensitivity (95% CI)	86.2 (72.4–93.7)
Specificity (95% CI)	92.3 (75.2–97.9)
Diagnostic OR (95% CI)	75.5 (11.6–492.7)
LR+ (95% CI)	11.3 (3.0–42.2)
LR− (95% CI)	0.2 (0.1–0.3)

LR Likelihood Ratios, OR odds ratio

Fig. 3.

Diagnostic accuracy of Twinkling artifact

Risk of bias

The risk of bias in 6 studies (37.5%) was considered low, whereas 7 studies (43.75%) were considered to have a high risk of bias, and the remaining 3 studies (18.75%) were considered to be of concern (Fig. 4 and online Supplementary Material).

Fig. 4.

Risk of Bias of included studies

Discussion

Main findings

This systematic review and meta-analysis examined recent data on the diagnostic value of Doppler TA in diagnosing urolithiasis. Sixteen studies were included, and the weighted sensitivity, specificity, and AUC were 0.86, 0.92, and 0.42, respectively. The diagnostic OR was 75.5, with strong and moderate effects of positive and negative likelihood ratios.

Comparison with previous studies

Laher et al. [21] reviewed the validity of TA as a diagnostic tool for determining the presence of urolithiasis using a systematic review and meta-analyses utilizing various modalities for confirmation, whereas our research focused particularly on CT as a reference standard. Their pooled sensitivity (88.16%) for the TA sign was similar to our study (86%), whereas the pooled specificity (79.22%) differed (92%). This difference could be because they found only 4 studies that reported on the specificity of the TA sign, although we identified 7 studies.

The sensitivity and specificity of the TA sign exhibited high heterogeneity compared with the same modality, noncontrast CT, which could have been due to differences in sonographer skill with regard to the setting and generation of the ultrasound machine, causing high false-positive and false-negative rates [29]. Adel et al. [29] examined all patients complaining of renal colic who had ultrasound followed by confirmatory CT scan and suggested that ultrasound TA alone would miss some stones due to its very low sensitivity (54%). Masch et al. [15] reviewed all ultrasound reports that contained the term “twinkle” and matched with suitable confirming CT scans. They found that the TA sign had low specificity (40%) with a high false-positive rate due to sonographer technique and ultrasound setting, along with high echogenicity of the adjacent renal sinus and renal arterial calcifications, as did Dillman et al. [31]. Moreover, the size of the stone could have an influence on the comparison between TA sign and noncontrast CT. Letafati et al. [6] evaluated the accuracy of TA in detecting renal stones smaller than 4 mm and found rather low sensitivity (76.8%) but high specificity (100%); therefore, they concluded that TA was a reliable sign for detecting renal calculi smaller than 4 mm. However, this sign could be combined with echogenic focus and posterior acoustic shadow to increase the sensitivity. Winkel et al. [17] also determined that TA incorporation with B-mode ultrasound is valuable in ruling out kidney stones. Likewise, Puttmann et al. [36] studied pediatric renal ultrasound reports predicting the presence or absence of TA associated with a single echogenic focus, with justification by CT within 3 months of Doppler ultrasound. The results showed rather high sensitivity (83%) but low specificity (78%) of TA for detecting nephrolithiasis. The authors identified the impact factors as pediatric renal section and smaller stone size.

Surprisingly, Liu et al. [16] assessed the performance of color Doppler ultrasound with B-mode ultrasound and CT in diagnosing ureteral stones. They reported high sensitivity and specificity of color Doppler ultrasound, at 96.98% and 90.39%, respectively. In addition, the sensitivity and specificity of CT were 99.59% and 94.23%, respectively. Hence, the authors suggested that TA sign should be used as a substitute for CT in patients with renal colic to reduce radiation exposure. Similarly, using CT as a reference standard, Abdel-Gawad et al. [28] reported the high sensitivity (97%) and specificity (99%) of TA in a prospective study of adult patients with renal colic at presentation.

Moreover, some studies have shown that B-mode ultrasound unites with color Doppler ultrasound (CDUS) using TA enhances the sensitivity and specificity, Winkel et al. [17]. Both Korkmaz et al. [35] and Salmaslioglu et al. [38] revealed the superior sensitivity of B-mode ultrasound plus TA in detecting urolithiasis compared with using either B-mode ultrasound alone or CDUS. These studies support our conclusion that using TA as a complementary tool enhances the sensitivity and specificity of diagnosing urolithiasis.

Strengths and limitations

The major strengths of this study include the clear eligibility criteria, a comprehensive search method, the examination of trials published in languages other than English, and independent and duplicate eligibility assessment. Additionally, meta-analyses exhibit increased statistical power due to the expanded sample size, and they can identify small but clinically important effects by combining data from many studies. Our study has some limitations, however. The included studies were different in terms of the size and location of the renal calculi. Due to variations in patient characteristics and techniques used, we identified considerable heterogeneity rates in the included studies. However, we performed sensitivity analyses, which showed no significant differences from the original analyses. Sensitivities, specificities, TPs, and TNs were compared, but these statistics depend on the populations studied, the reference tests used, and the specificity of the diagnostic tests.

Conclusion

Doppler TA has good diagnostic value for the diagnosis of urolithiasis. This study strengthens the concept of using TA as a complementary tool in the diagnosis of urolithiasis.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

Conceived and designed the study: PN, KK, SS, and KN; analyzed and interpreted the data: PN, KK, SS, and KN; drafted the manuscript: PN, KK, SS, and KN. All authors critically revised the article for important intellectual content and approved the final version. The corresponding author attests that all listed authors meet the authorship criteria and that no others meeting the criteria have been omitted.

Funding

University of Phayao under Unit of Excellence on Clinical Outcomes Research and IntegratioN (UNICORN) [Grant number: FF66-UoE004].

Data availability

The authors confirm that the data supporting the findings of this study are available within the article.

Declarations

Conflict of interest

None.

Ethical approval

Not required since no human participants or animals were recruited for the current study.