LI-RADS Treatment Response versus Modified RECIST for Diagnosing Viable Hepatocellular Carcinoma after Locoregional Therapy: A Systematic Review and Meta-Analysis of Comparative Studies

Abstract

Purpose

To systematically compare the performance of liver imaging reporting and data system treatment response (LR-TR) with the modified Response Evaluation Criteria in Solid Tumors (mRECIST) for diagnosing viable hepatocellular carcinoma (HCC) treated with locoregional therapy (LRT).

Materials and Methods

Original studies of intra-individual comparisons between the diagnostic performance of LR-TR and mRECIST using dynamic contrast-enhanced CT or MRI were searched in MEDLINE and EMBASE, up to August 25, 2021. The reference standard for tumor viability was surgical pathology. The meta-analytic pooled sensitivity and specificity of the viable category using each criterion were calculated using a bivariate random-effects model and compared using bivariate meta-regression.

Results

For five eligible studies (430 patients with 631 treated observations), the pooled per-lesion sensitivities and specificities were 58% (95% confidence interval [CI], 45%–70%) and 93% (95% CI, 88%–96%) for the LR-TR viable category and 56% (95% CI, 42%–69%) and 86% (95% CI, 72%–94%) for the mRECIST viable category, respectively. The LR-TR viable category provided significantly higher pooled specificity (p < 0.01) than the mRECIST but comparable pooled sensitivity (p = 0.53).

Conclusion

The LR-TR algorithm demonstrated better specificity than mRECIST, without a significant difference in sensitivity for the diagnosis of pathologically viable HCC after LRT.

INTRODUCTION

Treatment response following locoregional therapy (LRT) has been considered a strong and valid biomarker for the prediction of survival in patients with hepatocellular carcinoma (HCC) (1,2,3). The monitoring of treatment response is primarily evaluated by dynamic contrast-enhanced CT or MRI (1,4). In this context, several imaging-based criteria including the modified Response Evaluation Criteria in Solid Tumors (mRECIST) or European Association for the Study of the Liver criteria have been used as surrogate endpoints in clinical trials (5,6,7). Unlike traditional whole tumor size-based systems such as the RECIST, both criteria apply the concept of viable tumors, which corresponds to the portion of tumors that shows enhancement after intravenous contrast injection (5,6).

In 2017, liver imaging reporting and data system (LI-RADS) introduced a new algorithm to standardize the reporting of treated observations and is applicable after any type of LRT (8). The LI-RADS treatment response (LR-TR) algorithm extends the definition of viable tumor by adding new imaging features such as washout appearance and enhancement similar to pretreatment, whereas the mRECIST considers arterial phase hyperenhancement (APHE) as the only characteristic of a viable tumor (5,8). Moreover, in the LR-TR algorithm, the treated observation can be classified as an equivocal category when the distinction between viable tumor and the expected posttreatment enhancement is uncertain (8). Therefore, the LR-TR algorithm uses a ternary system that categorizes the treated observations as LR-TR viable (probably or definitely viable), equivocal (equivocally viable), or nonviable category. Given these distinguishable features between the LR-TR algorithm and mRECIST criteria, several studies have compared the diagnostic performance between the two criteria, but reported results vary considerably (9,10,11). Therefore, we conducted a systematic review and meta-analysis using intra-individual comparative studies to compare the diagnostic accuracy between the LR-TR and mRECIST criteria for diagnosing the viability of LRT-treated HCC.

MATERIALS AND METHODS

Our meta-analysis adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (12). This study was registered to the PROSPERO, and the registration number is CRD42021282595. The literature search, evaluation for the eligibility, data extraction, and quality assessment were performed by two of the authors (each having ≥ 9 years of experience in liver imaging) independently, and any conflicts were resolved through consensus.

SEARCH

The database of PubMed MEDLINE and EMBASE was searched to find original articles that performed intra-individual comparisons of diagnostic performance between the LR-TR algorithm and mRECIST for diagnosing viable HCC treated with LRT. The search terms included “LI-RADS”, “LI-RADS Treatment Response”, “CT”, and “MRI” (Supplementary Table 1 in the online-only Data Supplement). The literature search was conducted from January 1, 2017 to August 25, 2021. The search was restricted to human participants and English language studies.

INCLUSION AND EXCLUSION CRITERIA

Studies were included if all of the following criteria were satisfied: 1) Population: patients undergoing LRT for HCC, 2) Index test: multiphasic contrast-enhanced CT or MRI, 3) Reference standard: pathological diagnosis after liver transplantation or hepatic resection, and 4) Outcomes: diagnostic performance of the viable category for each criterion (i.e., LR-TR viable category and mRECIST viable category) for the diagnosis of the viability of LRT-treated HCC. The mRECIST viable category was assigned when APHE was present in and around the treated observation (9,10,11). Studies were excluded if they satisfied any of the following criteria: 1) Case reports, review articles, editorials, scientific abstracts, systematic reviews, and meta-analyses, 2) Studies that were not within the area of interest of this study (i.e., reporting interreader reliability, survival outcomes of LR-TR categories, or performance for LI-RADS diagnostic algorithm), and 3) studies that used clinical composite reference standard for determining tumor viability after LRT (i.e., the viability was assessed using follow-up imaging or cone-beam CT angiography). The presence of overlapping patients between eligible articles was also verified.

DATA ITEMS AND QUALITY ASSESSMENT

The following data were extracted from each eligible study: 1) Study characteristics, 2) Subject characteristics, 3) Characteristics for treated observations, 4) The specific type of LRTs, 5) Imaging modality, 6) MRI characteristics, 7) Number of reviewers and image analysis method, 8) Details of the reference standard for determining tumor viability, 9) Interobserver agreement (κ) for the classification of treated observations according to LR-TR and mRECIST, and 10) Study outcomes (please see the details in Supplementary Materials in the online-only Data Supplement). We used the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool for quality assessment (13).

DATA SYNTHESIS AND STATISTICAL ANALYSIS

The units of analysis were per lesion. Meta-analytic pooled sensitivity and specificity with 95% confidence intervals (CIs) of the viable category using each criterion (LR-TR algorithm and mRECIST) were calculated using a bivariate random-effects model, and a coupled forest plot was obtained. A hierarchical summary receiver operating characteristic (HSROC) curve with 95% confidence and prediction regions was plotted. The pooled sensitivity and specificity of each viable category were compared using bivariate meta-regression. For available studies, the pooled sensitivity and specificity for a combination of LR-TR viable and equivocal (LR-TR viable/equivocal) categories were compared to those for the mRECIST viable category. Additional subgroup analysis was performed according to the imaging modality (MRI vs. CT).

Heterogeneity was evaluated using Cochran Q test or Higgins inconsistency index (I²) test, with p < 0.10 or I² > 50% indicating the presence of substantial heterogeneity, respectively. The presence of a threshold effect was evaluated by the visual assessment of the coupled forest plots of sensitivity and specificity, and the Spearman correlation coefficient between the sensitivity and false-positive rate (14). A correlation coefficient > 0.6 was assumed to indicate a significant threshold effect (14). When substantial heterogeneity was noted, meta-regression analysis was performed to investigate the causes. The following covariates were used in the regression analysis: 1) The most common etiology of liver disease (hepatitis C vs. hepatitis B), 2) MRI contrast agent (hepatobiliary contrast agent [HBA] only vs. extracellular contrast agent [ECA] or both), 3) Type of LRT (transarterial radioembolization [TARE] only vs. others), 4) Image analysis method (multiple independent reviewers vs. multiple reviewers with consensus), and 5) percentage of viable HCC among treated observations (≥ 50% vs. < 50%).

Publication bias was evaluated by visual assessment of funnel plot and Deeks’ asymmetry test. Stata version 16.0 (StataCorp LP, College Station, TX, USA) was used for statistical analysis, with p < 0.05 considered statistically significant.

RESULTS

STUDY SELECTION AND CHARACTERISTICS

A total of 459 articles were screened after adjusting for duplicates (Fig. 1). Of these, 394 articles were removed after reviewing the titles and abstracts. Sixty articles were further excluded after full-text scrutiny, and a total of five eligible articles were finally included in the meta-analysis (9,10,11,15,16).

Fig. 1. Flow diagram of the article selection process.

LR-TR = LI-RADS Treatment Response, LI-RADS = Liver Imaging Reporting and Data Systems, mRECIST = modified Response Evaluation Criteria in Solid Tumors

The characteristics of the included studies are summarized in Table 1 (430 patients with 631 treated observations). All five included studies were of retrospective design (9,10,11,15,16). Hepatitis B was the dominant etiology of liver disease in three studies (9,10,16) and hepatitis C in two (11,15). One study included only patients treated with TARE (15) and one included only patients treated with conventional transarterial chemoembolization (TACE) (11). The remaining three studies included patients treated with different types of LRT, but conventional TACE was used the most commonly (9,10,16). Three studies used MRI only (11,15,16) and two used both CT and MRI (9,10). For MRI contrast agent, two studies used only HBA (gadoxetate disodium) (10,16), whereas three used both HBA (gadoxetate disodium or gadobenate dimeglumine) and ECA (gadoterate meglumine or gadobutrol) (9,11,15). All five studies used only pathological diagnosis by hepatic resection or explantation as the reference standard for viable HCC (9,10,11,15,16).

Table 1. Study Characteristics.

Study	Study Design	Number of Patients	Patient Age, Years*	Dominant etiology of Liver Disease	Number of Treated Observation	Size of Observation, mm*	Type of Locoregional Therapy (%)	Imaging Modality	MRI Field Strength	MRI Contrast Agent	Image Analysis	Reference Standards for Viable HCC
King et al. (15)	Retrospective	57	66.9 ± 8.9	Hepatitis C	77	41 ± 33	TARE (100)	MRI	1.5T or 3T	ECA	Multiple independent reviewers	Pathology (explant or resection)
Seo et al. (9)	Retrospective	114	54.0 ± 6.9	Hepatitis B	206	NA	TACE (78.6), RFA (16.5), or DEB-TACE (2.9)	CT (113 patients with 203 HCCs) or MRI (53 patients with 84 HCCs)	1.5T or 3T	HBA or ECA	Multiple independent reviewers	Pathology (explant)
Bae et al. (10)	Retrospective	165	62 ± 9	Hepatitis B	237	22 (3–95), median (range)	TACE (67.5), RFA (22.0), or PEIT (4.6)	CT (165 patients) and MRI (165 patients)	1.5T or 3T	HBA	Multiple independent reviewers	Pathology (explant)
Kierans et al. (11)	Retrospective	52	63.1 (25–74), mean (range)	Hepatitis C	71	21.1 (8–50), mean (range)	TACE (100)	MRI	1.5T or 3T	ECA or HBA	Multiple reviewers with consensus	Pathology (explant)
Youn et al. (16)	Retrospective	90	57 (38–84), mean (range)	Hepatitis B	105	19.4 ± 11.0 (size of viable HCC)	TACE (57.0), RFA (23.8), or DEB-TACE (2.9)	MRI	1.5T or 3T	HBA	Multiple reviewers with consensus	Pathology (explant or resection)

^*Unless otherwise stated, data represent mean ± standard deviation.

DEB = drug-eluting beads, ECA = extracellular contrast agent, HBA = hepatobiliary contrast agent, HCC = hepatocellular carcinoma, NA = not available, PEIT = percutaneous ethanol injection therapy, RFA = radiofrequency ablation, TACE = transarterial chemoembolization, TARE = transarterial radioembolization

QUALITY OF INCLUDED STUDIES

The overall quality of the included studies is presented in Supplementary Fig. 1 (in the online-only Data Supplement). In the reference standard domain, one study (9) had a high risk of bias because the histopathological evaluation was performed with correlation to preoperative CT or MRI. In addition, in two studies (11,15), it was unclear whether the reference standard results were interpreted without knowledge of the index test results. Regarding the flow and timing domain, one study (15) was considered to have a high risk of bias as only a small number of patients (9/57) received a reference standard, and the time interval between the index test and reference standard was not mentioned. In one study (15), it was unclear whether the index test results were interpreted without knowledge of the reference standard results, resulting in a risk of bias in the index test domain. In the patient selection domain, two studies with a retrospective design and small sample size were judged to be at high risk of bias (11,15).

DIAGNOSTIC PERFORMANCE IN THE VIABLE CATEGORY USING EITHER LR-TR OR MRECIST

For the LR-TR viable category, the pooled sensitivity and specificity for diagnosing viable HCC were 58% (95% CI, 45%–70%; I², 77%; Cochran Q test, p < 0.10) and 93% (95% CI, 88%–96%; I², 0%; Cochran Q test, p = 0.51), respectively (Table 2, Fig. 2). For the mRECIST viable category, the pooled sensitivity and specificity for diagnosing viable HCC were 56% (95% CI, 42%–69%; I², 74%; Cochran Q test, p < 0.10) and 86% (95% CI, 72%–94%; I², 82%; Cochran Q test, p < 0.10), respectively (Table 2, Fig. 2). In the mRECIST viable category, the HSROC curves with 95% confidence and prediction regions showed a large difference between the two regions, indicating substantial heterogeneity between studies (Supplementary Fig. 2 in the online-only Data Supplement). Compared to the mRECIST viable category, the LR-TR viable category provided significantly higher pooled specificity (93% vs. 86%; p < 0.01) but comparable pooled sensitivity (58% vs. 56%; p = 0.53) for diagnosing viable HCC. No significant threshold effect between sensitivity and specificity was noted for both LR-TR and mRECIST viable categories (rho ≤ 0.4; p ≥ 0.28). Deeks’ funnel plot and asymmetry test revealed no significant publication bias across the studies for both criteria (p ≥ 0.35) (Supplementary Fig. 3 in the online-only Data Supplement).

Table 2. Accuracy of the Viable Category Using LR-TR and mRECIST Criteria for Diagnosing Pathologically Viable HCC.

Study	Total Number of Observations	LR-TR Viable						mRECIST Viable
		Number of Observations				Sensitivity (95% CI)	Specificity (95% CI)	Number of Observations				Sensitivity (95% CI)	Specificity (95% CI)
		TP	FP	FN	TN			TP	FP	FN	TN
King et al. (15)	12	2	0	0	10	100 (16, 100)	100 (69, 100)	2	5	0	5	100 (16, 100)	50 (19, 81)
Seo et al. (9)	84	42	3	16	23	72 (59, 83)	88 (70, 98)	41	3	17	23	71 (57, 82)	88 (70, 98)
Bae et al. (10)	237	56	8	51	122	52 (42, 62)	94 (88, 97)	56	7	51	123	52 (42, 62)	95 (89, 98)
Kierans et al. (11)	71	17	3	30	21	36 (23, 51)	88 (68, 97)	17	5	29	19	37 (23, 52)	79 (58, 93)
Youn et al. (16)	105	46	2	27	30	63 (51, 74)	94 (79, 99)	41	3	32	29	56 (44, 68)	91 (75, 98)
Higgins I2 for study heterogeneity						77	0					74	82
Meta-analytic summary estimate using the bivariate model						58 (45, 70)	93 (88, 96)					56 (42, 69)	86 (72, 94)

CI = confidence interval, FN = false negative, FP = false positive, HCC = hepatocellular carcinoma, LR-TR = Liver Imaging Reporting and Data System Treatment Response, mRECIST = modified Response Evaluation Criteria in Solid Tumors, TN = true negative, TP = true positive

Fig. 2. Coupled forest plots of sensitivity and specificity of the LI-RADS Treatment Response (A) and mRECIST viable category (B) for diagnosing viable hepatocellular carcinoma.

CI = confidence interval, LI-RADS = Liver Imaging Reporting and Data Systems, mRECIST = modified Response Evaluation Criteria in Solid Tumors

SUBGROUP ANALYSIS

Four studies reported the performance of the LR-TR equivocal category for diagnosing viable HCC (9,10,11,16). The pooled sensitivity and specificity for a combination of LR-TR viable and equivocal (LR-TR viable/equivocal) categories were 61% (95% CI, 48%–73%) and 86% (95% CI, 75%–93%), respectively. In comparison with the mRECIST viable category, LR-TR viable/equivocal categories showed a higher pooled sensitivity (61% vs. 56%) and a similar pooled specificity (86% vs. 86%), although not statistically significant (p = 0.77).

Regarding the imaging modality, all included studies used MRI (9,10,11,15,16) and two of these studies used CT as well (9,10). These two studies reported the results of CT and MRI separately (9,10), which were used as independent datasets for subgroup analysis. For the LR-TR viable category, studies using MRI had higher pooled sensitivity (57% vs. 41%) than those using CT, but lower pooled specificity (93% vs. 96%) (p = 0.11). A similar trend was observed in the mRECIST viable category, where studies using MRI had higher pooled sensitivity (55% vs. 38%) than those using CT, but lower pooled specificity (87% vs. 97%) (p = 0.05).

META-REGRESSION ANALYSIS

Meta-regression analysis showed that the most common etiology of liver disease and type of LRT were significantly associated with the study heterogeneity for the mRECIST viable category (p = 0.01) (Table 3). The pooled sensitivity and specificity were significantly higher for studies with hepatitis B as the most common etiology of underlying liver disease compared to those with hepatitis C (59% vs. 43% and 92% vs. 69% for sensitivity and specificity, respectively). In addition, a study that included only patients treated with TARE showed a significantly higher sensitivity (100% vs. 54%) than other studies, but lower specificity (50% vs. 91%). The other covariates did not show statistical significance.

Table 3. Meta-Regression Analysis of the Accuracy of the Viable Category Using LR-TR and mRECIST Criteria.

Criterion	Covariates (Number of Studies)		Sensitivity (95% CI)	Specificity (95% CI)	p-Value
LR-TR viable	Most common etiology of liver disease				0.21
		Hepatitis B (n = 3)	62 (53, 71)	93 (88, 97)
		Hepatitis C (n = 2)	39 (22, 57)	91 (81, 100)
	MRI contrast agent				0.62
		Hepatobiliary agent (n = 2)	58 (39, 76)	94 (90, 98)
		Extracellular agent or both (n = 3)	59 (39, 78)	90 (82, 98)
	Type of LRT				> 0.99
		TARE only (n = 1)	100 (100, 100)	100 (100, 100)
		Others (n = 4)*	57 (57, 57)	92 (92, 92)
	Image analysis				0.46
		Multiple independent reviewers (n = 3)	64 (48, 80)	94 (89, 98)
		Multiple reviewers with consensus (n = 2)	50 (33, 68)	91 (84, 99)
	Percentage of viable HCC				0.50
		≥ 50% (n = 2)	58 (33, 84)	95 (90, 99)
		< 50% (n = 3)	58 (43, 73)	90 (84, 97)
mRECIST viable	Most common etiology of liver disease				0.01
		Hepatitis B (n = 3)	59 (50, 69)	92 (87, 97)
		Hepatitis C (n = 2)	43 (24, 63)	69 (50, 88)
	MRI contrast agent				0.08
		Hepatobiliary agent (n = 2)	54 (39, 69)	94 (90, 98)
		Extracellular agent or both (n = 3)	55 (38, 73)	78 (67, 89)
	Type of LRT				0.01
		TARE only (n = 1)	100 (100, 100)	50 (14, 86)
		Others (n = 4)*	54 (43, 65)	91 (86, 97)
	Image analysis				0.25
		Multiple independent reviewers (n = 3)	68 (47, 89)	85 (72, 99)
		Multiple reviewers with consensus (n = 2)	48 (30, 66)	86 (71, 100)
	Percentage of viable HCC				0.09
		≥ 50% (n = 2)	58 (10, 100)	84 (66, 100)
		< 50% (n = 3)	55 (41, 69)	87 (75, 100)

^*Conventional TACE was used the most commonly in the four studies.

CI = confidence interval, HCC = hepatocellular carcinoma, LRT = locoregional therapy, LR-TR = Liver Imaging Reporting and Data System Treatment Response, mRECIST = modified Response Evaluation Criteria in Solid Tumors, TACE = transarterial chemoembolization, TARE = transarterial radioembolization

INTEROBSERVER AGREEMENT FOR CLASSIFICATION

Two studies (9,10) compared lesion-based interobserver agreement for classification between LR-TR (ternary classification, i.e., LR-TR viable, equivocal, or nonviable) and mRECIST (binary classification, i.e. viable or nonviable). Because of the small number of studies, only qualitative synthesis was performed for interobserver agreement. Interobserver agreement for LR-TR was moderate to substantial for CT (κ, 0.69) and MRI (κ, 0.56–0.69). Interobserver agreement for mRECIST was substantial for both CT (κ, 0.74–0.80) and MRI (κ, 0.64–0.71).

DISCUSSION

This meta-analysis of intra-individual comparative studies found that the pooled specificity of the LR-TR viable category for diagnosing pathologically viable HCC was significantly higher than that of the mRECIST viable category, but the pooled sensitivity was comparable. When combining the LR-TR viable and equivocal categories for diagnosing viable HCC, the pooled sensitivity tended to be higher for LR-TR than mRECIST, whereas the pooled specificity was similar between the two criteria. Although mRECIST is primarily intended to determine perpatient response (5), we evaluated it as a per-lesion criterion for direct comparison with LR-TR algorithm on the basis of pathologic diagnosis.

Our results showed that the LR-TR viable category had high specificity but suboptimal sensitivity for diagnosing pathologically viable HCC following LRT, in line with previous meta-analyses (17,18,19). Compared to the mRECIST viable category, the LR-TR viable category demonstrated significantly higher pooled specificity (93% vs. 86%) which is mainly attributable to the presence of the equivocal category. Because treated observations tend to be assigned the equivocal rather than the viable category when tumor viability is uncertain, the presence of the equivocal category in the LR-TR algorithm results in increased specificity of the viable category. Although this improved specificity may come at the cost of the viable category sensitivity, the decreased sensitivity could be compensated by applying two novel diagnostic features of viable tumors, i.e., washout appearance and enhancement similar to pretreatment. Among the two added features, the washout appearance provided a modest sensitivity and diagnostic odds ratio for diagnosing viable HCC in recent meta-analyses (18,19). Notably, when LR-TR viable and equivocal categories were combined, LR-TR showed higher pooled sensitivity and similar specificity when compared to mRECIST. Thus, the combined LR-TR viable/equivocal may be more useful than mRECIST in situations where residual tumor viability is important in determining the next treatment option (e.g., repeat LRT or curative surgery) at a fixed time point after LRT.

In the meta-regression analyses, the etiology of underlying liver disease and type of LRT were the significant factors affecting the study heterogeneity of the mRECIST viable category. Studies with hepatitis B as the predominant etiology showed higher sensitivity and specificity than those with hepatitis C as the predominant etiology. Prior studies suggested that hepatitis B-related HCCs may do worse with LRT compared to hepatitis C-related HCCs (20,21). Further studies on the association between the etiology of liver disease and the evaluation of tumor response to LRT are needed. In addition, the study with only TARE showed higher sensitivity but lower specificity than other studies. The fact that persistent tumoral and peritumoral APHE after radiation therapy are often seen and can mimic viable HCCs (22) may explain more false-positive diagnoses and lower specificity after TARE.

This meta-analysis has several limitations. First, mRECIST was originally intended to assess overall disease status at the per-patient level (5), but was used as a per-lesion criterion in our meta-analysis. Because the LR-TR algorithm is lesion-based and our study aimed to compare the per-lesion accuracy between the two criteria via radio-pathologic correlation, we adopted only the notion of viable tumors from mRECIST for the interpretation of HCC viability. However, further studies based on per-patient analysis are needed. Second, the number of included studies was small (n = 5) and all included studies were retrospective in design. Third, substantial study heterogeneity especially for the mRECIST viable category precluded the generation of robust meta-analytic estimates. To minimize this limitation, we explored its causes and found that the etiology of underlying liver disease and type of LRT influenced the study heterogeneity. Fourth, datasets from all included studies contain multiple observation per patient (clusters), which may lead to biased estimates of sensitivity and specificity. However, adjustment for such clustering effects was not performed in our study.

In conclusion, the specificity of the viable category for the diagnosis of pathologically viable HCC was significantly higher when applying LR-TR than mRECIST, but the sensitivity was comparable. When combining the LR-TR viable and equivocal categories, the sensitivity tended to be higher for LR-TR than for mRECIST, but the specificity was similar between the two criteria. The etiology of underlying liver disease and type of LRT significantly affected study heterogeneity for the mRECIST viable category.

Supplementary Materials

The online-only Data Supplement is available with this article.

Supplementary Table 1

Search Queries

Supplementary Fig. 1

Results of quality assessments of the articles according to QUADAS-2 criteria.

Supplementary Fig. 2

HSROC curves for the accuracy of LI-RADS treatment response (A) and mRECIST viable category (B).

Supplementary Fig. 3

Deeks' funnel plot to evaluate publication bias regarding LI-RADS treatment response (A) and mRECIST viable category (B).