Diagnostic accuracy of apparent diffusion coefficient (ADC) value in differentiating malignant from benign solid liver lesions: a systematic review and meta-analysis

Abstract

Objectives:

We undertook a systematic review and meta-analysis of the diagnostic performance of mean apparent diffusion coefficient (ADC) values derived by diffusion-weighted (DW)-MRI in the characterization of solid benign and malignant liver lesions, and to assess their value in discriminating these lesions in daily routine practice.

Methods:

A systematic review of PubMed, Embase, Scopus, and Web of Science was conducted to retrieve studies that used ADC values for differentiating solid benign/dysplastic nodules and malignant liver lesions. A bivariate random-effects model with pooled sensitivity and specificity values with 95% CI (confidence interval) was used. This meta-analysis was performed on the per-lesion basis. Summary receiver operating characteristic (SROC) plot and area under curve (AUC) were created.

Results:

A total of 14 original articles were retrieved. The combined (95% CI) sensitivity and specificity of mean ADC values for differentiating solid benign from malignant lesions were 78% (67–86%) and 74% (64–81%), respectively. The pooled (95% CI) positive and negative LRs were respectively 3 (2.3–3.8) and 0.3 (0.21–0.43). The DOR (95% CI) was 10 (7–15). The AUC (95% CI) of the SROC plot was 82% (78–85%). Reporting bias was negligible (p value of regression test = 0.36). Mean size of malignant lesions and breathing pattern of MRI were found to be sources of heterogeneity of pooled sensitivity.

Conclusion:

ADC measurement independently may not be an optimal diagnostic imaging method for differentiating solid malignant from solid benign hepatic lesions. The meta-analysis showed that ADC measurement had moderate diagnostic accuracy for characterizing solid liver lesions. Further prospective and comparative studies with pre-specified ADC thresholds could be performed to investigate the best MRI protocol and ADC threshold for characterizing solid liver lesions.

Advances in knowledge:

ADC measurement by DW-MRI does not have a good diagnostic performance to differentiate solid malignant from solid benign lesions. Therefore, we suggest not using ADC values in clinical practice to evaluate solid liver lesions.

Introduction

Quantification of apparent diffusion coefficient (ADC) through diffusion-weighted MRI (DW-MRI) has been noted in a number of studies in non-invasive assessment of focal liver lesions (FLLs).^1–4 ADC measures random motion (diffusion) of water molecules in the tissues. In malignant tissues that are highly cellular, free water molecule diffusion is restricted and this consequently reflects itself as reduced ADC in comparison to higher ADC values measured in benign tumors. Therefore, quantification of ADC has gained attention in differentiating malignant from benign FLLs as an advanced MRI sequence that abates the need for intravenous contrast agent administration.⁵

A common finding among most primary studies is that mean ADC values of malignant hepatic lesions (hepatocellular carcinoma (HCC), metastases, cholangiocarcinoma) is significantly lower than that of benign cystic and solid hepatic lesions (e.g. simple cysts, hydatid cyst, abscess, hemangioma, hepatocellular adenoma (HCA), and focal nodular hyperplasia (FNH)).^6–11 This observation, however, has been questioned by some investigators. Considering the way that benign lesions (i.e. cystic, solid, or hemangioma) are selected to be analyzed in each study, such statistically significant difference of ADC values between malignant and benign hepatic lesions may not be seen uniformly in all studies. For instance, considerable overlap of ADC values was reported between malignant (HCC and metastases with mean ADC value of 1.52 × 10⁻³ mm²/s) and solid benign lesions including HCA (1.49 × 10⁻³ mm²/s) and FNH (1.79 × 10⁻³ mm²/s).¹² The authors inferred that ADC measurement might not be helpful in differentiating malignant from solid benign hepatic lesions.¹² The findings of another study resemble the results of the latter study indicating that no significant difference existed in terms of mean ADC value between FNH and HCC/metastases.⁸ These findings are supported by more evidence that is relevant.^2,13 On the contrary, a separate study with discrepant results indicated that even when only solid benign lesions included, difference in ADC value between malignant (HCC, metastases, cholangiocarcinoma) and solid benign masses was still statistically significant at three diffusion gradients (b values of 100, 600, and 1000 s/mm²).¹⁴

A difficult challenge in real practice when characterizing a FLL is to distinguish solid benign from malignant masses.^14,15 Diffusion properties of cysts and hemangiomas are different from solid benign masses. Mean ADC values of simple cysts have been reported in a range of 2.45⁶ to values as high as 3.63 × 10⁻³ mm²/s.¹⁶ In a former study including 166 hemangiomas, mean ADC value of these lesions was reported as 2.26 × 10⁻³ mm²/s.¹² It has been suggested that inclusion of benign cystic lesions (e.g. simple cysts, hydatid cyst, and abscess) can surreptitiously overestimate diagnostic accuracy of ADC value. Even inclusion of hemangiomas, as predominantly solid lesions that inherently may have higher ADC value owing to blood (fluid) content, can lead to similar conclusion about the usefulness of ADC measurement in discrimination of malignant vs benign FLLs.¹⁴ Therefore, it may be more appropriate not to combine ADC values of solid benign and cystic benign lesions together in a meta-analysis.

To the best of our knowledge, 5 meta-analyses published in 2010,¹⁷ 2012,¹⁸ 2015,¹⁹ 2016,²⁰ and 2018²¹ with respectively 14, 6, 7, 8, and 6 primary studies in the quantitative synthesis have addressed this topic. The mentioned articles included primary studies that investigated cystic lesions and hemangiomas, beside solid benign lesions, as benign FLLs. These reported usefulness of ADC measurement in differentiating malignant vs benign hepatic lesions. However, each meta-analysis has some specific limitations that may render conclusive results regarding the role of ADC measurement in this regard difficult. One report evaluated only studies that included Asian patients.²⁰ Another meta-analysis included studied that applied intravoxel incoherent motion (IVIM) MRI technique.²¹ Notable heterogeneity in diagnostic accuracy values was observed in all five reports.^17–21 Two reports did not find main sources of heterogeneity.^18,19 One report that investigated only Asian population indicated that the type of benign lesions (cystic vs solid lesions) was a potent source of inconsistency. Subgroup analysis for solid hepatic nodules yielded much lower summary sensitivity (60%) in comparison to all benign lesions (93%); but this subgroup analysis did not affect diagnostic specificity.²⁰ In a similar fashion, subgroup analysis in another report published in 2010 introduced solid benign lesion as a potent source of heterogeneity for both sensitivity and specificity of ADC measurement.¹⁷ It seems that the literature currently lacks a robust conclusion about this topic. Therefore, it is essential to have an updated review and meta-analysis of the available evidence regarding the role of ADC measurement in discriminating solid lesions of the liver.

Objective

We intended to systematically review the literature and synthesize the pooled sensitivity, specificity, and accuracy values of ADC to differentiate solid benign and malignant hepatic lesions, excluding benign cystic lesions as well as hemangiomas that can erroneously overestimate the accuracy of ADC measurement in this context. In our opinion, the synthesis of the existing studies would be useful to assess the value of DW-MRI and ADC values in discrimination these lesions in daily routine practice.

Methods and materials

This report was written based the recommendations made by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2009 checklist.²²

Eligibility criteria

The inclusion criteria consisted of retrospective or prospective studies that investigated solid malignant and benign hepatic lesions in adult patients of either gender with or without liver parenchymal abnormalities (cirrhosis, steatosis, or chronic hepatitis). The studies should have measured mean ADC value via DW-MRI (as the index test) and supplied the data in a way that a two-by-two contingency table can be reconstructed to calculate to differentiate malignant from benign lesions. Studies that employed dual-b-value or multi b-value (maximum value of 1000 s/mm²) DW-MRI regardless of field strength (1.5 or 3 T), the techniques used (single-shot echoplanar imaging (SE-EPI), spinecho (SE)) and any breathing pattern during MRI acquisition (i.e. free-breathing, breath-hold, or respiratory-triggered) were eligible to be included. The methods sections of the articles were reviewed to ensure region of interest (ROI) placement contained the lesion and avoided necrotic areas. Malignant lesions included HCC, metastatic lesions, cholangiocarcinoma, and high-grade dysplastic nodules. The main benign lesions included HCA, FNH, low-grade dysplastic nodules, and nodular regenerative hyperplasia. Studies that included cystic lesions (simple cyst, hydatid cyst), hemangioma, hypervascular lesions (e.g. hypervascular nodules) were not included. We also excluded studies with insufficient data where number of lesions or ADC values of each group was not clearly reported despite trying to contact the authors for the missing data. In case that chemotherapy was administered before DW-MRI or quantitative evaluation (i.e. ADC measurement) had not been done, the study was deemed ineligible for inclusion. Studies that valid reference standard methods (e.g. biopsy or surgery and histopathological examination, or imaging characteristics with sufficient follow-up time to determine benignity of the lesions) were not used for definitive diagnosis were not included. For suspected benign lesions, such as hepatic adenoma, follow-up imaging with CT or MRI at 6 to 12 months has been suggested to evaluate stability of the lesions and their growth pattern.²³ No language restriction was imposed. Studies reported in languages other than English were translated to English using Google Translation service and consultation with an official translation center, if required. Only published reports in the journals (original articles) with available full text of the report were considered eligible. Review articles, case reports, case series, editorials, conference abstracts, and letters were not included.

Information sources

The following electronic databases were systematically searched: PubMed, Embase, Scopus, and Web of Science Core Collection via Clarivate Analytics. The gray literature included Google Scholar. Additionally, manual scanning of the reference list of the eligible citations was done to find any relevant studies.

Search

The electronic databases were searched (20 July 2020) using both controlled vocabulary (for PubMed (Medical Subject Headings, MeSH) and Embase (Emtree)) and free text words without any language restriction. MeSH terms used in PubMed were “liver neoplasms”, “carcinoma, hepatocellular”, “cholangiocarcinoma”, “focal nodular hyperplasia”, “adenoma, liver cell”, and “diffusion magnetic resonance imaging”. Emtree terms used were 'liver tumor', 'nodular hyperplasia', 'diffusion weighted imaging', and 'apparent diffusion coefficient'. Appendix 1 presents search strategy for PubMed. The search results from all sources were entered into the EndNote X8 software (Thomson Reuters, New York, NY). The search was updated in December 2020 to ensure no additional study has been published. No further studies were identified in the updated search.

Study selection

Two reviewers independently screened the records retrieved by the search process at the title and abstract levels to determine how many studies are relevant to the review question. Afterwards, number of the remaining records were compared and any disagreement was solved by discussion, reviewing the full text of the title, and if required comments of a third author. At the next stage, the remaining records were reviewed in their entirety and if the inclusion criteria were met, the required data were extracted.

Data collection process

A data gathering checklist was developed considering the variables of interest. The included full-text articles were reviewed independently by two authors and the required data were extracted. Since several studies measured ADC values in solid benign lesions other than the target conditions reviewed here (e.g. hemangioma, and cystic lesions) we tried to contact the corresponding authors to ask for individual patient data. This was done in an effort to reconstruct two-by-two table for subset of lesions that meet our inclusion criteria.

Data items

The variables of interest included “first author name”, “year of publication”, “country”, “study design (retrospective, cross-sectional, prospective)”, “MRI vendor”, “magnetic field strength (T)”, “DW-MRI technique”, “breathing pattern”, “ADC measurement technique (placing region of interest vs volumetric)”, “b-values (s/mm²)”, “slice thickness (mm)”, “number of malignant and benign lesions”, “mean size of the lesions (in mm)”, “malignant masses (HCC, metastases, cholangiocarcinoma”, “benign masses (hepatocellular adenoma, FNH, dysplastic nodules”, “mean ADC value of malignant and benign lesions (mm²/s)”, and “diagnostic ADC threshold (cut-off) values (×10⁻³ mm²/s)”. A two-by-two contingency table was constructed. In this table, true positive (TP) and false positive (FP) represented the number of patients with malignant lesion diagnosed correctly and incorrectly, respectively, by the applied diagnostic ADC value threshold in that particular study. Accordingly, FN (false negative) represented malignant lesions with mean ADC values over the applied mean ADC threshold, and TN (true negative) represented benign lesions with mean ADC values higher than the applied ADC threshold. We were not able to determine a pre-specified ADC threshold since this can only be performed when individual patient-level data of all primary studies are available.

Risk of bias in individual studies

For quality appraisal and determining the risk of bias in individual studies was determined using the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2).²⁴ This is a validated tool that has been constructed specifically for judging risk of bias in systematic review of diagnostic accuracy studies. Using the revised version of this tool introduced in 2011, the reviewer assesses four domains (patient selection, index test, reference standard, and flow and timing) for possible risk of bias. For each domain, signaling questions should be answered by the reviewers considering the information provided in the primary article to assess the risk of bias. The answer options for each domain are “Yes” which indicates low risk of bias, “No” which indicates high risk of bias, and “unclear risk of bias” when the information presented in the document are insufficient to determine the potential of risk of bias. In addition, applicability concerns are judged for three domains (patient selection, index test, and reference test) and answers are reported similarly (Yes, No, and Unclear). Finally, for each document seven items are required to be completed (risk of bias in four domains and applicability concerns in three domains). Herein, two authors first performed the quality assessments separately and then in a joint session resolved any disagreements. The signaling questions suggested by the QUADAS-2 were used for all domains except for one signaling question used to determine the risk of bias in “flow and timing” domain. This question concerns with application of the same reference standard for all patients. We decided to omit this question since diagnosis of all hepatic lesions, especially benign ones, may not necessarily require histopathological examination and imaging follow-up and characteristics features can be used to diagnose some benign lesions.²³ Additionally, we did not find evidence in the included studies that ADC calculation and its result affected the decision to perform or not to perform lesion biopsy. Only published data were used for determination of risk of bias in the studies.

Synthesis of results

First, summary mean ADC values with 95% confidence interval (95% CI) were calculated by random-effects model in malignant and benign lesions groups. Then, weighted mean differences (WMD) and standardized mean difference (SMD, Cohen’s d) of the mean ADC values between malignant and benign lesions were calculated (based on the DerSimonian and Laird (D + L) random-effects model).

Then, two-by-two contingency table containing TP, FP, FN, and TN values was constructed, the sensitivity, specificity, positive likelihood ratio (LR), and negative likelihood ratio (LR) estimates as well as diagnostic odds ratio (DOR) were computed by a bivariate random-effects approach²⁵ and forest plots were created showing the point estimates, 95% CIs of the point estimates with combined values. Since the studies did not share a common threshold for ADC, it is recommended to assess variation of the thresholds (i.e. threshold effect). This assessment is necessary to ensure pooled values of sensitivity and specificity are calculated appropriately. For this reason, a Spearman correlation coefficient (ρ) test between the logit of sensitivity and the logit of false positive rate (1-specificity) was calculated (p values less than 0.05 indicate significant threshold effect).²⁶

The hierarchical summary receiver operating characteristic (SROC) curve of ADC value and the area under this curve (AUC, area under curve) were used to assess the overall diagnostic accuracy of mean ADC measurement in differentiating malignant from benign solid hepatic lesions. A bivariate boxplot, which depicts interdependence between sensitivity and specificity, was drawn to assess degree of heterogeneity and detection of outliers. The data collected from every single study were eligible to be included in the meta-analysis regardless of the overall quality of that particular study. In order to address the issue of study quality, we decided to perform subgroup analyses with taking into account the domains of the QUADAS-2 that were judged to have unclear/high risk of bias in some reports. The analyses were carried out by Stata v. 12 (StataCorp, College Station, TX) statistical package.

Subgroup analysis

Where possible and sufficient data were available, we intended to determine the effect that particular variables, as potential source of heterogeneity, can have on sensitivity and specificity estimates. To achieve this goal, univariable meta-regression with subgroup analyses were carried out.

Assessing reporting bias

Reporting bias was investigated with creation of Deeks’ funnel plot and conducting a regression test.²⁷ Evidence of asymmetry in the regression line and a p-value < 0.05 in the regression test are indicative for possible reporting bias.

Results

Study selection

A total of 2687 records were identified by searching the electronic bibliographic databases and the gray literature. Of this, 715 records were duplicates. Then, 1699 citations were excluded during screening of the titles and abstracts. Of 273 records remained, 259 ones were excluded due to various reasons such as not reporting mean ADC value, not including solid benign lesions, etc. Finally, 14 records met the inclusion criteria (Figure 1).

Figure 1.

Illustrated flow chart showing the process of literature search and study selection.

Study characteristics

Table 1 presents characteristics of the included studies.^{2,8,14,28–38} Nine studies^{2,14,28,31,33,34,36–38} calculated ADC cut-off value and provided respective sensitivity and specificity values to differentiate malignant lesions from benign lesions/dysplastic nodules. In two studies,^29,32 the individual patent data were available and ADC threshold was calculated by modeling the area under the receiver operating curve (ROC). Corresponding author of another article provided the sensitivity and specificity values.³⁰ This resulted in a total of 12 studies that two-by-two tables were constructed for them. Despite contact with other authors for individual patient-level data, this was not successful and the required data were not available.

Table 1.

Characteristics of the included studies comparing mean ADC values of malignant and solid benign liver lesions

Author, year	Design	MRI vendor, filed strength (T)	b value (s/mm2)	Slice thickness (mm)	DW-MRI sequence; breathing pattern	Malignant lesions	Solid benign lesions	Mean (±SD) ADC of malignant vs solid benign (×10−3 mm2/s)	Mean size of malignant vs solid benign lesions (mm)
Aslan et al28	Retro	Siemens, 1.5	0, 500, 1000	5	SS-SE-EPI; free-breathing	96 lesions (27 HCC, 65 metastases, 3 CCA, one gallbladder carcinoma)	ten lesions (10 FNH)	0.97 (±0.21) vs 1.31 (±0.23)	38.9 vs 38.3 mm
Di Martino et al29	Retro	Siemens, 1.5	0, 50, 400, 800	5	SS-EPI; breath-hold	90 HCC	29 lesions (23 regenerative nodules, 4 DNs, one necrosis, one fibrosis)	0.97 (±0.20) vs 1.21 (±0.31)	15 vs 19.41
Filipe et al8	Retro	Siemens, 1.5	50, 700	8	SS-EPI; respiratory- triggered	30 lesions (12 HCC and 18 metastases)	14 lesions (10 FNH and 4 HCA)	1.16 (0.22) vs 1.54 (0.42)	46.9 vs 35.9
Girometti et al2	Retro	Siemens, 1.5	50, 500, 800	6	SS-EPI; respiratory- triggered	34 lesions (22 HCC, 12 metastases)	16 lesions (12 FNH/HCA, 3 DNs, one atypical hemangioma)	1.08 (±0.36) vs 1.25 (±0.26)	21.88 vs 25.81
Inchingolo et al30	Retro	GE, 1.5	0, 800	8	SS-EPI; free-breathing	42 lesions (29 HCC and 13 HGDN)	11 LGDN	1.32 (±0.26) vs 1.54 (±0.31)	21.8 vs 21.4
Muhi et al31	Retro	GE, 1.5	500, 1000	four to 6	SS-SE-EPI; respiratory- triggered	86 HCC	12 DNs	0.79 (±0.25) vs 1.0 (±0.22)	NR
Mungai et al32	Retro	Siemens, 1.5	50, 400, 800	5	SS-SE-EPI; navigator-triggered	192 lesions (81 HCC, 77 metastases, and 34 CCA)	110 lesions (72 FNH and 38 HCA)	0.96 (±0.23) vs 1.25 (±0.28)	NR
Onur et al14	Pros	GE, 1.5	100, 600, 1000	5	SS-SE-EPI; free-breathing	69 lesions (28 HCC, 33 metastases, 8 CCA)	26 lesions (14 FNH, 4 HCA, 2 AML, one teratoma, two nodular regenerative hyperplasia, three alveolar echinococcosis)	1.08 (±0.22) vs 1.52 (±0.47)	51.26 vs 27.14
Sandrasegaran et al33	Retro	Siemens, 1.5	0, 50, 400	6	SS-EPI; breath-hold	45 lesions (41 HCC and four metastases)	nine lesions (6 FNH and 3 HCA)	1.10 (±0.33) vs 1.21 (±0.87)	36.08 vs 36.0 mm
Sandrasegaran et al34	Retro	Siemens, 1.5	0 or 50, 400 or 500, 800	6	SS-EPI; free-breathing	41 HCC	29 benign hepatic nodules	0.94 (±0.24) vs 1.02 (±0.57)	44 vs 14 mm
Sutherland et al35	Retro	Siemens, 1.5	100, 400, 800	8	Breath-hold	eleven lesions (2 CCA and nine metastases)	47 lesions (40 FNH, 4 HCA, two focal steatosis, one hyalinized scar)	1.15 vs 1.19	24.7 vs 26.5 mm
Xu et al36	Retro	Siemens, 1.5	0, 500	7	SS-SE-EPI; breath-hold	40 HCC	19 DNs	1.28 (±0.25) vs 1.53 (±0.33)	35 vs 25 mm
Yang et al37	Pros	GE, 1.5	0, 600	6	SS-EPI; respiratory- triggered	32 lesions (22 HCC, seven metastases, 2 CCA, one hemangioendothelioma)	25 lesions (8 FNH, 1 HCA, 1 AML, four inflammatory pseudotumor, eight solitary necrotic nodules, two pseudolipoma.1 ectopic adrenal adenoma)	1.29 (±.21) vs 1.59 (±0.47)	NR
Zarghampour et al38	Retro	Siemens, 1.5	0, 750	8	SS-EPI; breath-hold	60 HCC	81 HCA	1.40 (±0.16) vs 1.07 (±0.23)	39.64 vs 26.52 mm

AML, Angiomyolipoma; CCA, Cholangiocarcinoma; DN, Dysplastic nodule; FNH, Focal nodular hyperplasia; HCA, Hepato cellular adenoma; HCC, Hepatocellular carcinoma; HGDN, High-grade dysplasticnodule; LGDN, Low-grade dysplastic nodule; NR, Not reported; Pros, Prospective; Retro, Retrospective; SS-EPI, Single-shot echoplanar imaging; SS-SE-EPI, Single-shot spin-echo echo-planar imaging.

Except in one study³⁸ that volumetric method was used to measure ADC, other studiesapplied placement of region of interest (ROI) for ADC measurement. The primary sites of 106 metastatic lesions out of 225 ones werereported. These included gastrointestinal tract (53 lesions), pancreas (7 lesions), lung (13 lesions), breast (15 lesions), larynx (2lesions), kidney (6 lesions), ovaries and endometrium (6 lesions), and neuroendocrine tumors (4 lesions).

Risk of bias within studies

Figure 2 shows risk of bias assessment in each study using the QUADAS-2 quality assessment tool. Regarding “patient selection” domain, 10 studies had low risk of bias in this domain. In one study,²⁸ the sampling method was unclear. In three studies,^31,36,38 considering the main objective of the current review, we judged this domain as “high risk of bias” as patients with already established conclusive diagnosis (benign and malignant lesions) using pathology department reports had been selected. The included studies had minimal concern for applicability regarding patient selection since all studies included malignant and benign solid liver lesions. Although in one study,²⁸ one patient with gallbladder carcinoma had been included, there were 96 hepatic malignant lesions and this single patient was not considered significant to skew ADC measurement of malignant lesions.

Figure 2.

Risk of bias (in four domains) and applicability concerns for the findings according to the QUADAS-2 quality appraisal tool in 14 diagnostic accuracy studies. The output of the software presents all domains and applicability concerns; as observed, the “Flow and Timing” domain was of low risk and there was no concern regarding applicability judgements in the studies.

In the “index test” domain, although ADC measurement is mainly an objective method and in most studies,^{2,28–34,36–38} the radiologists were blinded to definite diagnosis of hepatic lesions, 13 studies were judged as having “high risk of bias” as a pre-specified ADC threshold had not been applied. In one study, the ADC threshold was not reported.³⁶ There was low concern for applicability in the “index test” domain as sufficient details regarding diffusion-weighted imaging, ADC measurement, and ADC map creation were provided.

In the “reference standard” domain, all studies provided details regarding methods used for establishing the definitive diagnoses of the target condition either by histopathological examination (percutaneous biopsy or surgery) or follow-up serial imaging findings, and tumor marker measurements. Minimum follow-up time mentioned in the reports was considered to be enough to establish benign nature of the lesions. Since design of most studies was retrospective review of DW-MRI images and the final diagnosis of the solid lesions by pathologists or serial imaging had been made earlier without knowledge of ADC measures, it was inferred that the results of reference standard tests were interpreted without knowledge of ADC measurements. However, in two prospective studies,^14,37 it was not clear that whether the radiologists that examined imaging features on follow-up of the lesions that were not diagnosed by histopathological examination were blinded to ADC measurements or not. So, these two studies were determined to have “unclear risk of bias” in this domain. There were no concerns regarding applicability of the “reference standard” in any of the studies since established and validated methods were used to categorize the lesions as benign or malignant. In the “flow and timing” domain, all studies were judged to have “low risk of bias” since all the included lesions were analyzed in a two-by-two table and there was no evidence that the patients received treatments (e.g. chemotherapy or radiotherapy) between index test and the reference standard that could distort the results of either test.

Synthesis of results

14 included studies provided data for a total of 868 malignant lesions (579 HCC, 225 metastases, 49 CCA, 13 HGDN, 1 gallbladder carcinoma, 1 hemangioendothelioma) and 438 benign lesions (160 FNH, 135 HCA, 12 FNH/HCA, 49 DNs, 23 regenerative nodules, 29 benign hepatic nodules, and 30 other less common (e.g. necrosis, fibrosis, scar) benign lesions). Mean ADC values ranged from 0.79 to 1.40 × 10⁻³ mm²/s in malignant lesions group (pooled average = 1.11; 95%CI = 1.00 to 1.21) and from 1.00 to 1.59 × 10⁻³ mm²/s in benign lesions group (pooled average = 1.29; 95%CI = 1.20 to 1.38). Pooled average ADC value was significantly lower in malignant lesions group compared to pooled average ADC value of benign lesions (WMD= – 0.20, 95%CI= – 0.36 to – 0.03; p = 0.01 and SMD= – 0.67, 95%CI= – 1.18 to – 0.16; p = 0.01); Supplementary Figures1 and 2

.

Diagnostic performance

ADC threshold values ranged from 0.81 to 1.60 × 10⁻³ mm²/s (Table 2). The combined (95% CI) sensitivity and specificity of mean ADC values for differentiating solid malignant from solid benign hepatic lesions were respectively 78% (67 to 86%) and 74% (64 to 81%); Figure 3. The pooled (95% CI) positive and negative LRs were respectively 3 (2.3 to 3.8) and 0.3 (0.21 to 0.43). The DOR (95% CI) was 10 (7 to 15). The AUC (95% CI) of the SROC was 82% (78 to 85%); Figure 4. Bivariate boxplot (Supplementary Figure 3) shows that 11 studies are clustered within the median distribution and three studies as outliers.^31,33,34. Sensitivity analysis by excluding these three outliers did not change overall diagnostic statistics; recalculated AUC = 82% (Supplementary Figure 4).

Table 2.

ADC threshold values and sensitivity and specificity values of 14 studies to differentiate solid malignant from solid benign hepatic lesions

Author, year	TP	FP	FN	TN	ADC threshold (×10−3 mm2/s)	Sensitivity	Specificity
Aslan et al28	88	4	8	6	1.33	91.7	60
Di Martino et al29	54	6	36	23	0.99	60	79.3
Filipe et al8	22	3	8	11	1.26	73.3	78.6
Girometti et al2	21	3	13	13	1.09	61.8	81.2
Inchingolo et al30	38	3	4	8	1.41	90.5	72.7
Muhi et al31	45	0	41	12	0.81	52.3	100
Mungai et al32	147	21	45	89	1.06	76.6	80.9
Onur et al14	57	6	12	20	1.23	82.6	76.9
Sanderasegaran et al33	24	5	21	4	1.04	53.3	44.4
Sanderasegaran et al34	39	20	2	9	1.20	95.1	31
Sutherland et al35	6	11	5	36	1.04	54.5	76.6
Xu et al36	23	4	17	15	NR	57.5	78.9
Yang et al37	31	12	1	13	1.60	96.9	52
Zarghampour et al38	64	15	17	45	1.24	79	75

ADC, apparent diffusion coefficient; FN, False negative; FP, False positive; NR, Not reported; TN, True negative; TP, True positive.

Figure 3.

Coupled forest plots for sensitivity (left) and specificity (right) of mean ADC values by diffusion-weighted MRI in differentiating solid malignant from solid benign liver lesions. ADC, apparent diffusion coefficient.

Figure 4.

SROC plot of mean ADC (apparent diffusion coefficient) values in differentiating solid malignant from solid benign liver lesions (14 studies); each circle indicates a single sensitivity-specificity point of a particular study. ADC, apparent diffusion coefficient; AUC, Area under curve; SENS, Sensitivity; SPEC, Specificity; SROC, summary receiver operating characteristic.

Threshold effect

The Spearman correlation coefficient between sensitivity and FP rate was 0.43 (p = 0.13). This indicates that no significant ADC threshold effect was present.

Subgroup analysis

In order to assess possible effects that different covariates can have on summary sensitivity and specificity estimates, we used multiple univariable meta-regression analyses. The effects of 12 covariates (study design, size of the lesions, malignant lesion subtype, benign dysplastic nodules, sample size, MRI vendor type, DW-MRI sequence, breathing pattern, maximal b value, slice thickness, and method of patient selection) were explored and the findings are represented in Table 3 and Supplementary Figure 5. In order to dichotomize lesion size and number of lesions (sample size), median values were used. Among studies that mean size of malignant lesions was greater than 36 mm, pooled sensitivity (83%) was significantly higher than in those with malignant lesions measured less than 36 mm (67%); p = 0.02. Also, pooled sensitivity of studies that applied non-breathhold technique (i.e. free-breathing or respiratory triggered or navigator-triggered) was 84% which was significantly higher than studies where MRI was obtained by breath-hold technique 63% (p < 0.001). In studies that DW-MRI was obtained by General Electric (GE) MRI scanners (GE Healthcare, Milwaukee, WI, USA), the combined sensitivity was relatively greater (85% vs 75%) than studies where images were obtained by Siemens scanners (Siemens Medical Solutions, Erlangen, Germany). Besides these covariate, other explored covarying factors were not recognized as potential causes for between-study heterogeneity of sensitivity and specificity values. Regarding the quality of the studies and risk of bias, there was no concern in applicability of three domains and only the studies differed in patient selection. However, this covariate did not contribute significantly to sensitivity or specificity estimates.

Table 3.

Results of multiple univariable meta-regression to determine possible sources of heterogeneity of the sensitivity and specificity estimates

Covariates		No. of primary studies	Pooled sensitivity (95% CI)	p value	Pooled specificity (95% CI)	p value
Study design	Retrospective	12	75% (65 to 85%)	0.37	75% (66 to 83%)	0.23
	Prospective	2	91% (80 to 100%)		65% (42 to 89%)
Mean size of malignant lesions	<36 mm	5	67% (51 to 84%)	0.02	77% (67 to 88%)	0.83
	≥36 mm	6	83% (73 to 93%)		65% (52 to 77%)
Mean size of benign lesions	<25.71 mm	5	80% (68 to 93%)	0.92	68% (55 to 81%)	0.08
	>25.71 mm	5	67% (51 to 84%)		75% (64 to 87%)
Only HCC in malignant group	Yes	5	74% (57 to 90%)	0.12	74% (61 to 87%)	0.17
	No	9	80% (70 to 91%)		74% (64 to 84%)
Dysplastic nodules as benign lesions	Yes	3	70% (47 to 93%)	0.19	84% (69 to 99%)	0.87
	No	11	80% (70 to 89%)		71% (62 to 80%)
Sample size	<65 lesions	7	73% (58 to 88%)	0.7	72% (59 to 84%)	0.26
	≥65 lesions	7	81% (70 to 92%)		74% (63 to 85%)
MRI vendor	Siemens	10	75% (63 to 87%)	0.08	73% (63 to 84%)	0.22
	GE	4	85% (72 to 98%)		77% (61 to 93%)
Diffusion-weighted sequence	SS-SE-EPI	5	75% (60 to 91%)	0.18	81% (71 to 91%)	0.79
	SS-EPI	8	80% (69 to 92%)		68% (56 to 79%)
Breathing pattern of DW-MRI	Breath-hold	5	63% (46 to 81%)	<0.001	77% (65 to 90%)	0.37
	Non-breath-hold	9	84% (76 to 92%)		72% (61 to 83%)
Maximal b value for DWI	≥800	9	80% (69 to 90%)	0.57	75% (65 to 84%)	0.26
	<800	5	75% (58 to 92%)		72% (58 to 86%)
Slice thickness	≤6 mm	9	80% (69 to 91%)	0.19	72% (61 to 83%)	0.33
	7 or 8 mm	5	75% (57 to 92%)		76% (64 to 89%)
Patient selection	High/unclear risk of bias	4	74% (55 to 99%)	0.21	79% (66 to 92%)	0.58
	Low risk of bias	10	79% (69 to 90%)		71% (61% to 81%)

DWI, diffusion-weighted imaging; HCC, hepatocellular carcinoma; SS-EPI, single-shot echoplanar imaging.

Reporting bias

Visual inspection of the Deek’s funnel plot (Figure 5) and the associated regression test (p = 0.36) indicated that reporting bias is negligible in this review.

Figure 5.

Deek’s funnel plot to investigate reporting bias; ESS = effective sample size.

Discussion

In this review, we intended to investigate the diagnostic accuracy of ADC measurement for differentiating solid benign from solid malignant hepatic lesions. As mentioned earlier, there is uncertainty in the literature about overlap of ADC values between benign and malignant liver lesions. Since many studies in the literature include cystic lesions and hemangiomas in comparative studies with malignant lesions, diagnostic accuracy of ADC measurement may be overestimated. Therefore, there is concern that previous meta-analyses that included non-solid lesions, especially hemangiomas, resulted in over optimistic conclusions about the role that ADC plays for differentiating malignant from benign lesions.^17,18,20

Here, a total of 14 studies fulfilled the eligibility criteria. When compared with previous pertinent meta-analyses,^17–21 only 2 primary studies,^31,36 were cited by a former meta-analysis.²⁰ There was no overlap between the remaining 12 studies and other published meta-analyses. Of the 14 articles, only 2 studies had a prospective design.^14,37 Most of the studies were not specifically designed to address the central objective of this review and consequently ADC values of the solid lesions were gathered by either extraction some parts of the data or correspondence with authors to ask for ADC values of the subset of solid lesions. In one study, ADC threshold had not been reported.³⁶ None of the studies had applied a pre-specified ADC cut-off point. ADC levels used ranged from 0.81 to 1.60 × 10⁻³ mm²/s. In nine studies, ADC cut-off point ranged from 0.99 to 1.26 × 10⁻³ mm²/s. ROC analyses were done to derive the optimal ADC threshold for studies that raw data were provided by the corresponding authors of four studies.^8,29,32,35 This approach was selected for two reasons. First, we were not able to find a widely accepted ADC threshold. Although ADC values about 1.5 to 1.6 × 10⁻³ mm²/s have been proposed, but no established consensus is available about the best threshold value.³⁹ Second, other eligible studies had applied ROC analyses to derive the optimal ADC cut-point. The meta-analysis showed that diagnostic performance measures (pooled sensitivity of 78%, pooled specificity of 74% and AUC of 82%) of ADC measurement was relatively modest in characterization of solid benign from solid malignant hepatic lesions. The derived positive LR of 3 is less than 10 which is recommended as indicative of a significant increase in the probability of disease presence after a positive test.⁴⁰

Heterogeneity is a common and well-recognized feature of meta-analysis of DTA studies. Heterogeneity is expected owing to natural variation in diagnostic performance statistics (sensitivity and specificity) across positivity cut points.⁴¹ We decided not to use traditional I² (inconsistency index)⁴² to quantify heterogeneity as I² is mainly used for univariate meta-analyses. However, since bivariate meta-analyses is the recommended and preferred method for analyzing DTA studies, there is concern regarding use of this index.^41,43 The more important issue is the exploration of possible causes of between-study heterogeneity. Meta-regression showed size of malignant lesions and breathing pattern during MRI acquisition as significant contributors to heterogeneity of pooled sensitivity.

Three studies included dysplastic nodules.^30,31,36 In two studies, mean ADC values of dysplastic nodules were measured as 1.53 × 10⁻³ mm²/s with b values of 0, 500³⁶ and 1.54 × 10⁻³ mm²/s with applied b values of 0, 800.³⁰ Both studies reported that mean ADC values of dysplastic nodules were significantly higher than malignant lesions. Subgroup analyses did not reveal any significant effect of inclusion of dysplastic nodules.

11 studies^{2,8,14,28–30,33–36,38} provided mean size of the benign and malignant lesions separately. Except in one study²⁹ that range of the size of the malignant lesions was from 5 to 20 mm, other articles included malignant lesions larger than 10 mm. It was observed that mean size of the malignant lesions could affect the pooled sensitivity as this combined value was significantly higher in those studies with mean malignant size of ≥36 mm. An explanation for this observation would be the method of ROI placement. Some authors^8,28,30,34 mentioned that in larger lesions (>20 mm), more than a single ROI was drawn (3 or 4 ROIs) and it is possible that with several ROIs, more accurate ADC measurement could be calculated. In a previous meta-analysis⁴⁴ to explore the role of ADC measurement in discrimination of HCC pathological grades, the authors found that accuracy of this parameter is higher in lesions < 50 mm compared to those measured larger than 50 mm. They added that since larger lesions are more prone to have necrotic areas or hemorrhage, ADC measurement is more accurate in HCCs smaller than 50 mm. However, in the articles reviewed here, it was observed that most authors avoided placing ROIs on necrotic portions, bile ducts, and vessels and tried to place ROIs on homogeneous areas of the tumors. Various techniques for ROI placement and number of ROIs used to average ADC values is an important issue in DW-MRI of hepatic lesions that require more attention in future studies.

The meta-analysis showed that pooled sensitivity of studies that used non-breath-hold DW-MRI acquisition was significantly higher in comparison to those that used breath-hold technique. Cardiac pulsation and breathing are among physiological factors that can affect DW-MRI and correct ADC calculation. Therefore, techniques such as breath-holding has been used to control respiratory movements.^18,39 It has been reported that respiratory-triggered approach for DW-MRI of liver lesions yielded better image quality and higher signal-to-noise ratio (SNR) in comparison to breath-hold technique.⁴⁵ However, the calculated ADC values by either technique were in good agreement. In agreement with these findings, another comparative study demonstrated that non-breath-hold DW-MRI provided higher SNR and contrast-to-noise ratio (CNR) than in breath-hold technique for focal hepatic lesions on 3.0 T vendors.⁴⁶ Most MRI scans were performed on a 1.5 T Siemens MRI scanner (10 out of 14 studies). Studies using GE scanners found greater sensitivity compared with those using Siemens machines, though not statistically significant. MRI manufacturer could be a potential cause for some degree of heterogeneity in combined results. This heterogeneity may arise from differences in scanner hardware, DW image quality, quantitative imaging software, ADC map resolution, sequence parameters, and SNR of the two machines. In previous reports, differences in ADC values quantified by different MRI vendors by similar sequence parameters have been shown and standardization of MRI models for quantitative image analysis has been proposed.⁴⁷

Overall, the reviewed studies were at low risk of bias and were sufficiently well conduced. All of them had been published in peer-reviewed scholarly journals. Since quality of the studies in each QUADAS-2 domain was satisfactory, only “patient selection” was investigated in meta-regression as a cause for heterogeneity. This domain was not found a contributor to between-study heterogeneity. However, none of the studies had selected a pre-specified ADC threshold.

Strengths and limitations

This is the first review of the diagnostic performance of ADC measurement by DW-MRI for characterization of solid liver lesions. We performed a comprehensive search of four major electronic databases with gray literature search. Reporting bias was insignificant and the risk of bias was low in the included studies. We used QUADAS-2 which is a validated tool specifically designed to assess risk of bias in DTA studies. We tried to explore the effect of 12 covariates to find possible factors that contributed to between-study heterogeneity. The systematic search found a large number of records that DW-MRI had been studied in hepatic lesions. However, both solid and non-solid (e.g. cystic and hemangiomas) lesions had been included in several potentially eligible primary studies. Since we did not have access to the raw data of several studies, solid lesions of such studies were not included in the meta-analysis. Also, a wide range of ADC levels had been used to define positive results, but the correlation test showed that threshold effect was not significant. None of the studies had used 3.0 T field strength. In some studies, all patients with HCC or a subset of them had cirrhotic livers.^{2,8,29,30,32–34,37,38} However, since ADC values were not reported separately in cirrhotic and non-cirrhotic livers, we were not able to explore this. ADC values of liver with cirrhosis or steatosis is believed to be lower in comparison to normal liver tissue.^8,30 Because the number of patients with malignant and benign lesions were not uniformly reported, we were not able to extract the number of lesions in each patients and per-patient analysis was not performed. A protocol for conducting the review was prepared before initiating the review. In this protocol, all main steps including the search terms, search strategy in each electronic database, screening of the records, data extraction, and meta-analysis were written with participation of all authors. The protocol was reviewed by the Research Deputy experts of Kermanshah University of Medical Sciences, Kermanshah, Iran and necessary corrections were made. Since subgroup analysis is an important step in the meta-analyses, we noted these analyses considering the study design, MRI vendor, maximal b value, and risk of bias (or its domains) in advance. However, lesion size, DW-MRI sequence, breathing pattern, and slice thickness were introduced into the analyses after inspection of the gathered data.

Conclusions

Based on the present meta-analysis output and the current body of evidence, ADC measurement had relatively moderate diagnostic performance for discriminating solid malignant from solid benign hepatic lesions. ADC measurement is unlikely to be considered sufficiently sensitive or specific to be used alone in this context. With a false positive rate of 26%, it is likely that some patients undergo unnecessary interventions to diagnose incorrectly classified malignant lesions by ADC calculation. Therefore, we think that it is essential that radiologists or oncologists consider other additional imaging modalities to avoid incorrect discrimination of solid hepatic lesions. Future research efforts might focus on high-quality prospective studies with larger and more representative samples of solid hepatic lesions with setting pre-specified ADC thresholds or pre-specified ADC intervals. Additionally, making direct comparisons of ADC accuracy based on variables such as pre-specified ADC thresholds, breathhold vs non-breathhold MRI techniques and size of the solid lesions, MRI manufacturer, number and maximum b factor value, and ROI placement methods will be informative to select the optimal standardized DW-MRI protocols in this context.