Urinary Biomarkers in the Evaluation of Primary Hematuria: A Systematic Review and Meta-Analysis

Abstract

Introduction:

Urinary biomarkers are entering the clinical landscape as a non-invasive method to evaluate patients for bladder cancer, however it is currently predominantly used in the surveillance setting. The use of biomarkers in the setting of primary hematuria is not widespread despite initial promising results. This study comprehensively reviews the literature on the diagnostic performance of FDA-approved biomarkers in the evaluation of primary hematuria.

Methods:

According to the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) statement, MEDLINE, EMBASE, ScienceDirect, Cochrane Libraries, HTA database, Google Scholar and Web of Science were searched up to June 2017 for studies assessing the diagnostic accuracy of FDA-approved urinary biomarkers amongst patients presenting with primary hematuria. The quality of included studies was assessed using the QUADAS-2 tool.

Results:

Fourteen studies met the pre-specified eligibility criteria and were included for analysis. The biomarkers assessed in these studies were AssureMDx, Bladder tumor antigen, CxBladder, NMP22, UroVysion and uCyt+. Across these four biomarkers, the sensitivity ranged from 0.67 to 0.95, and specificity from 0.68 to 0.93, respectively. There was significant heterogeneity between the included studies. Limited head-to-head comparison with urine cytology demonstrated that in general, the biomarkers have superior sensitivity but inferior specificity. Overall, the quality of evidence was graded as moderate primarily because of inadequate blinding.

Conclusion:

The current diagnostic performance of biomarkers are inadequate to replace cystoscopy in the primary hematuria setting. However, AssureMDx in particular may have a role as a triage test for cystoscopy but further prospective data is required to validate these findings. Given the current evidence, the use of these markers as an adjunct to cystoscopy for the evaluation of hematuria should be considered investigational.

INTRODUCTION

Bladder cancer (BCa) is the fourth most commonly diagnosed malignancy amongst men in the United States and has the tenth highest incidence amongst women [1]. Patients generally first present to physicians with painless hematuria that initiates a series of investigations evaluating for a urinary tract malignancy, including urinary cytology, imaging and cystoscopy. However, considering the high prevalence of hematuria in the general population, the diagnostic yield of investigating all patients is low [2, 3]. Furthermore, cystoscopy can be an uncomfortable procedure and be financially burdensome to health payers when performed in a large-scale [4, 5]. Hence, there has been interest in the urological field to develop improved, non-invasive methods for bladder cancer diagnosis.

Although yet to be widely employed in routine clinical practice, there have been a range of protein- and cell-based urinary biomarkers that have demonstrated the potential as a non-invasive test to diagnose BCa. AssureMDx (MDxHealth, Irvine, CA, USA), Bladder tumor associated antigen (BTA) (Polymedco Inc. Cortlandt Manor, NY, USA), CxBladder (Pacific Edge Ltd., Dunedin, New Zealand), NMP22 (Matritech. Inc., Newton, MA, USA), UroVysion (Abbott Molecular Inc., Ill., USA) and Immunocyt/uCyt+ (DiagnoCure, Inc., Québec, Canada). have all received Food and Drug Administration (FDA) approval for use in this setting. The diagnostic performance of these biomarkers has predominantly been assessed in the setting of surveillance for recurrent tumors where the results have been promising. However, it is not clear whether the same degree of discriminating ability translates to the setting of primary evaluation of hematuria as studies have shown that the presence of red blood cells can increase the number of false positive results [6]. Therefore, we performed a comprehensive systematic review and meta-analysis to assess the diagnostic performance of FDA approved urinary biomarkers in the setting of primary hematuria evaluation.

METHODS

Search strategy

The Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) framework was used for this review. The search attempted to identify all articles which evaluated the accuracy of an FDA approved biomarker in detecting primary bladder cancer compared to cystoscopy amongst individuals presenting with hematuria for evaluation.

The search was conducted in scientific literature databases (MEDLINE, EMBASE, ScienceDirect, Cochrane Libraries, HTA database and Web of Science) using a range of keywords include “bladder cancer”, “bladder carcinoma” or “bladder neoplasm”; “hematuria”; “urinary biomarkers” or “biomarkers” and each of the FDA approved biomarkers. Major international urological and oncological meetings (for example, American Urological Association, European Association of Urology and American Society of Clinical Oncology) were searched for relevant abstracts. The reference list of relevant studies was checked for additional relevant articles. The “related articles” and “find similar” features on PubMed and Ovid were used to find further publications of interest. Citation alerts were placed on included studies to identify any recent articles. There were no restrictions places on language or date of publication.

The search results were first screened by title and abstract for relevance prior to full-text review by two independent authors (N.S. and M.B.) with a senior author consulted to resolve any disagreements (B.K.). Both retrospective and prospective trials were included if participants had no prior diagnosis of bladder cancer, had presented with hematuria for primary evaluation and had been tested with both an FDA-approved urinary biomarker and cystoscopy. Studies which included patients with recurrent disease or those presenting with other symptoms (e.g. voiding symptoms) had to separately report the results for cases who presented with hematuria for primary evaluation to be eligible for inclusion.

Data extraction

The same authors as above independently extracted data from included articles using a data collection form developed a priori. Data collected included publication details, demographic details, details of biomarker tested including cut-off and the results of the biomarker test in diagnosing bladder cancer using cystoscopy as the gold-standard.

Assessment of methodological quality

The quality of the literature was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool [7]. This was done by two independent authors (N.S. and M.B.) with any disagreements resolved by consultation with a senior author (B.K.).

Statistical analysis

Using a 2×2 table that was constructed for each study during the data extraction process, pooled estimates for sensitivity and specificity were calculated for each biomarker using a bivariate mixed-effects regression model [8]. For head-to-head comparisons of biomarkers and urine cytology, a bivariate mixed-effects regression model was used with an indicator variable for the tests. Hierarchical summary receiver-operating characteristic (HSROC) curves were not constructed because each biomarker generally defined a positive test in the same manner. Subgroup analysis was performed on the type of hematuria. Analysis was conducted in R (version 3.4, R Foundation for Statistical Computing, Vienna, Austria) using the ‘mada’ package [9].

RESULTS

Quantity of evidence identified

The study selection process is depicted in the PRISMA flow diagram (Fig. 1). The initial search returned 5,455 abstracts for review, including 172 duplicate records. Following title and abstract screening, 137 articles were reviewed in full of which 17 met the eligibility criteria and were included for analysis [10–26]. The characteristics of the included studies are highlighted in Table 1.

Fig.1.

PRISMA flow chart.

Table 1.

Characteristics of included studies

Author	Year	Location	Hematuria cohort	Biomarker	Biomarker Cut-off	Age [range]	Cancers diagnosed on the reference test	Positive’ Biomarker test	All cancers diagnosed on biomarkers	TP	FN	FP	TN
Akagashi	2001	Japan	121	NMP22 (quantitative)	9.2	NR	12	52	11	11	1	41	73
Arora	2009	India	53	NMP22 (qualitative)	NA	59 [range 33–83]	38	33	30	30	8	3	12
Cha	2012	Germany/Italy	1182	uCyt	NA	65 [range 18–93]	245	328	202	202	43	126	811
Chong	1999	Singapore	47	BTA (qualitative)	NA	NR	12	20	8	8	4	12	23
Gonzalo Rodriguez	2008	Spain	15	NMP22 (qualitative)	10	59.8 [range 27–85]	12	10	10	10	2	0	3
Kirollos	1997	UK	24	BTA (qualitative)	NA	NR	1	7	1	1	0	6	17
Lotan	2008	International	1166	NMP22 (qualitative)	NA	59 [range 18–96]	74	203	43	43	52	160	932
Miyanaga	1999	Japan	309	NMP22 (quantitative)	12	NR	22	88	20	20	2	68	221
Moonen	2005	Netherlands	28	NMP22 (qualitative)	NR	NR	4	6	4	4	0	2	22
Oge	2001	Turkey	37	NMP22 (quantitative)	10	NR	27	24	20	20	7	4	6
O’Sullivan	2012	Australasia	485	CxBladder	69 [IQR 59–77]	66	116	33	54	12	62	354
Sanchez-Carbayo	2001	Spain	112	NMP22 (quantitative)	10	65.5 [range 23 to 87]	43	35	28	28	19	7	45
Sarosdy	2006	United States	497	UroVysion	NR	63 [range 40–97]	51	124	35	35	16	89	333
Schmitz-Drager	2008	Germany	293	uCyt	NA	59 [range 24–89]	27	53	22	23	4	32	221
van Kessel	2016	Netherlands	154	AssureMDx	0.1917196	68 [range 38–91]	74	85	72	72	2	13	67
van Kessel	2017	Netherlands	200	AssureMDx	0.1917196	NR	97	105	90	90	7	15	88
Zippe	1999	USA	180	NMP22 (quantitative)	10	NR	18	63	18	18	0	32	130

Risk of bias and quality assessment

Figure 2 depicts the risk of bias assessment for the included studies. Overall, the quality of included studies was moderate. The studies were largely retrospective in nature and generally did not clearly report whether the cystoscopy operator was blinded to the results of the biomarker test; it could be reasonable to assume that in these cases the operators were not blinded and thus may introduce a degree of performance bias. Most studies did not clearly indicate the order of testing nor the time interval between the index and reference test. One study also recruited patients about to undergo cystoscopy, it was not clear whether this included all patients that presented with hematuria for evaluation [15]. One study was judged as being a high-risk of bias for the patient selection domain because the sample was recruited non-consecutively [24]. Another study was judged to be a high-risk of bias for the index test because the cut-off was not pre-specified [25].

Fig.2.

Risk of bias assessment.

Diagnostic performance of biomarkers

The sensitivity and specificity for each study is shown in Fig. 3 and the summary diagnostic performance for each biomarker is shown in Appendix 1.

Fig.3.

Sensitivity and 1-specificity of included studies.

AssureMDx

Two studies assessed the accuracy of AssureMDx in the diagnostic work-up of primary hematuria [24, 25]. The pooled sensitivity and specificity was 0.95 [95% CI 0.87–0.98] and 0.85 [95% CI 0.79–0.89], respectively. There was no observed heterogeneity in the studies’ sensitivity (χ² = 0.93, p = 0.34) or specificity (χ² = 0.01, p = 0.91). The summary diagnostic odds ratio (DOR), positive likelihood ratio (posLR) and negative likelihood ratio (negLR) was 114.7, 6.6 and 0.07, respectively. Compared to the other biomarkers in this study, AssureMDx had the highest DOR and lowest negLR.

Bladder tumor antigen (qualitative)

Two studies using qualitative BTA were included for analysis [13, 15]. The summary sensitivity and specificity was 0.67 [95% CI 0.40–0.85] and 0.68 [95% CI 0.55–0.79], respectively. This was the lowest sensitivity and specificity of all the biomarkers included in this study. There was no observed heterogeneity in the studies’ sensitivity (χ² = <0.1, p > 0.99) or specificity (χ² = 0.11, p = 0.73). The summary DOR, posLR and negLR was 5.2, 2.1 and 0.5, respectively.

CxBladder

A single study using CxBladder met the eligibility criteria [19]. The reported sensitivity and specificity was 0.82 [95% CI 0.71–0.89] and 0.85 [95% CI 0.81–0.88]. The calculated DOR, posLR and negLR was 25.7, 5.5 and 0.21, respectively.

Qualitative NMP22

There were four eligible studies which assessed the accuracy of qualitative NMP22 [11, 14, 16, 18]. The summary sensitivity and specificity was 0.70 [95% CI 0.46–0.87] and 0.85 [95% CI 0.83–0.87]. There was observed heterogeneity in the studies’ sensitivity (χ² = 17.8, p < 0.01) but not its specificity (χ² = 1.1, p = 0.77). The summary DOR, posLR and negLR was 15.2, 4.6 and 0.36, respectively.

Quantitative NMP22

Five studies which evaluated quantitative NMP22 were included for analysis [10, 17, 20, 21, 23]. The summary sensitivity and specificity was 0.79 [95% CI 0.63–0.90] and 0.76 [95% CI 0.67–0.93], respectively. There was significant heterogeneity in the studies’ sensitivity (χ² = 15.4, p = 0.004) and specificity (χ² = 15.0, p = 0.005). The summary DOR, posLR and negLR was 12.9, 3.3 and 0.29, respectively.

uCyt+

Two studies used uCyt+ as a biomarker in the primary hematuria setting [12, 22]. The summary sensitivity and specificity was 0.83 [95% CI 0.78–0.87] and 0.87 [95% CI 0.85–0.89], respectively. There was no significant heterogeneity in the sensitivity of uCyt+ in the studies (χ² = <0.1, p = >0.99) or their specificity (χ² = 1.8, p = 0.19). The summary DOR, posLR and negLR was 31.7, 6.2 and 0.20, respectively. uCyt+ had the second highest DOR of the included biomarkers.

UroVysion

One study meeting the eligibility criteria used UroVysion [26]. The summary sensitivity and specificity was 0.69 [95% CI 0.55–0.80] and 0.78 [95% CI 0.75–0.83], respectively. The calculated DOR, posLR and negLR was 8.6, 3.3 and 0.4, respectively. The performance characteristics were similar to that of BTA in this setting.

Subgroup analysis: Macroscopic vs microscopic hematuria

The difference in diagnostic performance for cases of macro- and microscopic hematuria could only be compared for qualitative NMP22 and uCyt+ because these were the only markers in which at least one study reported test results by the extent of hematuria. There was no difference in the sensitivity or specificity of qualitative NMP in macroscopic and microscopic hematuria: 61.7% vs 47.3% and 82.9% vs 86.4%, respectively (p = 0.12). There was also no difference in the sensitivity or specificity based on the type of hematuria using uCyt+: 88.2% vs 80.0% and 80.4% vs 88.9%, respectively (p = 0.26). Two studies reported on microscopic hematuria [17, 21] exclusively and another two reported only on macroscopic hematuria [13, 14].

Head-to-head (within-study) comparison of biomarkers to urine cytology

Nine studies compared the performance of index biomarkers and urine cytology (Table 2) [11–13, 15, 17–19, 21, 22]. The majority of the studies classified atypical or inconclusive cytology results as being negative. The specificity of cytology was superior to all biomarkers except for qualitative NMP22 which was marginally non-significant. Except for BTA, the majority of the biomarkers demonstrated superior sensitivities compared to cytology but only CxBladder and uCyt achieved statistical significance.

Table 2.

Within study comparison of biomarkers and urine cytology

Biomarker	Studies	Sensitivity				Specificity
		Cytology	Biomarker	Difference	p value	Cytology	Biomarker	Difference	p value
BTA	2	0.67 (0.40 to 0.85)	0.67 (0.40 to 0.85)	0 (–0.27 to 0.18)	>0.99	0.98 (0.60 to 0.89)	0.69 (0.55 to 0.79)	–0.29 (–0.43 to –0.19)	<0.01
CxBladder	1	0.56 (0.44 to 0.68)	0.82 (0.71 to 0.89)	0.26 (0.15 to 0.33)	<0.01	0.92 (0.89 to 0.94)	0.85 (0.81 to 0.88)	–0.07 (–0.11 to –0.04)	<0.01
NMP22 Qualitative	2	0.67 (0.52 to 0.80)	0.79 (0.65 to 0.89)	0.12 (0.01 to 0.27)	0.21	0.98 (0.84 to 0.99)	0.84 (0.69 to 0.93)	–0.14 (–0.15 to 0)	0.08
NMP22 Quantitative	2	0.44 (0.24 to 0.66)	0.78 (0.35 to 0.96)	0.34 (–0.09 to 0.52)	0.14	0.99 (0.87 to 1.0)	0.82 (0.69 to 0.90)	–0.17 (–0.30 to –0.09)	<0.01
uCyt	2	0.46 (0.41 to 0.52)	0.82 (0.77 to 0.87)	0.36 (0.31 to 0.41)	<0.01	0.95 (0.93 to 0.96)	0.87 (0.85 to 0.89)	–0.12 (–0.10 to –0.06)	<0.01

DISCUSSION

This is the first study to provide a comprehensive overview using robust methodology of the current evidence in using urinary biomarkers in the evaluation of primary hematuria. The finding from the meta-analysis demonstrate that the sensitivity of biomarkers range from 0.67 to 0.95, and specificity from 0.68 to 0.87. These results are consistent with a previously published systematic review and meta-analysis on the use of biomarkers in all settings [27]. The reported range for the sensitivity and specificity in their study was 0.57–0.82 and 0.74–0.88, respectively. Current guidelines from the American Urological Association does not recommend the use of any biomarkers, including cytology, for initial evaluation of asymptomatic microscopic hematuria and instead rely on cystoscopy and imaging [28]. Aside from urine cytology, the guideline panel only considered NMP22, BTA stat and UroVysion at the time and justified their recommendations by suggesting that the risk of false positives outweighed any benefit. The reported specificities for these markers in the guidelines ranged from 62% to 93% whereas the estimated specificity of markers in this study was mostly at least 85% and this tended to be higher for the subgroup of microscopic hematuria patients. Although this suggests a potential use of these markers as a “triage test” for cystoscopy, the specificities of these markers are inferior to urine cytology based on the limited subset of studies which measured both in the same patients. However, it should be noted that none of the studies compare AssureMDx or UroVysion to cytology and the majority of studies failed to provide a clear definition of a ‘positive’ cytology. Furthermore, AssureMDx demonstrated a low negLR (0.07) suggests a large and often conclusive decrease in the likelihood of disease especially when the pre-test probability is low, such as with microscopic hematuria. Therefore, AssureMDx could potentially be used as a ‘triage’ test to determine the need for cystoscopy. Reducing the number of cystoscopies performed has both individual and societal benefits by avoiding the morbidity of an invasive procedure and reducing the consumption of healthcare resources that can then be used for other purposes. However, the estimates of test accuracy in this study are from a small, heterogeneous sample that could be influenced by performance and detection bias. These major shortcomings in the literature demand that further high quality studies be conducted to provide clarity in this space. Nonetheless, our findings are consistent with previous studies that observed that urine cytology had superior specificity compared to available urine markers [29, 30]. Given the higher cost of biomarkers compared to cytology, it is imperative that they demonstrate superior diagnostic performance. Overall, our results support the current stance of professional societies that these tests lack satisfactory evidence to be used routinely in clinical practice, particularly in place of cystoscopy for diagnosis. However if prospective studies can validate the persistent high NPV of some of these markers, there may be room to consider them as a triage test, particularly in the evaluation of microscopic hematuria where the incidence of bladder cancer is 2% or less. There may be other uses for biomarkers in clinical practice which were not evaluated in this study, such as prognosis or adjudication of atypical or inconclusive cytology [31, 32].

There were no eligible study criteria that compared the diagnostic performance of more than one biomarker and therefore it is not possible to confidently conclude whether any of the biomarkers are superior to others. Based on the diagnostic odds ratio which is commonly used as an estimation of a test’s discriminative ability [33], we can infer that AssureMDx may be the most overall useful test (Table 3). Only one of the included studies assessed the performance of more than one FDA-approved biomarker –CxBladder, qualitative and quantitative NMP22 –and found that the sensitivity for high-grade tumors in each of these tests was 97%, 38% and 69%, respectively [19]. A small number of studies have performed head-to-head comparisons of biomarkers in a diverse population (i.e. not only cases presenting for evaluation of primary hematuria) and a meta-analysis of these studies found that qualitative NMP22 was likely superior to quantitative NMP22, but there was no difference in the comparisons between BTA and UroVysion; and uCyt+ and UroVysion [27]. Given the small sample size and heterogeneity between studies, it is critical for further prospective studies to be performed in our population of interest to discern whether there are significant differences in the diagnostic performance between biomarkers. The results from this study and the literature do not currently provide strong evidence that biomarkers can be used to replace cystoscopy in the diagnostic evaluation of hematuria.

Table 3.

Post-test probabilities of bladder cancer in asymptomatic microscopic hematuria (pre-test cancer probability: 2.1% [34])

Biomarker	Positive LR	Negative LR	Post-test probability with positive test (%)	Post-test probability with negative test (%)
AssureMDx	6.6	0.07	12.40	0.15
BTA	2.1	0.5	4.31	1.06
CxBladder	5.5	0.21	10.55	0.45
NMP22 (qualitative)	4.6	0.36	6.61	0.62
NMP22 (quantitative)	3.3	0.29	8.98	0.77
uCyt+	6.2	0.2	11.74	0.43
UroVysion	3.3	0.4	6.61	0.85

The findings of this systematic review and meta-analysis should be interpreted within the context of its limitations. The summary estimations for diagnostic performance for each biomarker is derived from a small number of heterogeneous studies that limits our confidence in these estimates and generalizability to the wider population. Furthermore, the bias introduced by incomplete blinding may confound the observed diagnostic performance. There was insufficient data to determine whether diagnostic accuracy changes across grade and/or stage of tumor, which would be important for decision-making and counselling of patients. The previous meta-analysis suggests that performance increases with stage and grade [27]. Similarly, it is important to study the impact of patient characteristics on diagnostic accuracy.

CONCLUSION

There are only a limited number of studies that have evaluated the use of a FDA-approved urinary biomarker in the evaluation of primary hematuria. These studies have provided limited evidence of diagnostic performance to justify the routine use of these markers in place of cystoscopy for the evaluation of hematuria. Although these multidimensional markers that incorporate transcriptomics and patient level information are promising, more prospective data is needed to determine if they can truly function as triage tests.

CONFLICTS OF INTEREST

None.

FUNDING

NJS has received support from the Cloverfields Foundation and The Institute for Prostate and Urologic Cancers (University of Minnesota).

Appendix 1

Summary sensitivity and specificity for each biomarker