Performance of the PRO-C3 collagen neo-epitope biomarker in non-alcoholic fatty liver disease

Background & Aim

There is an unmet need for non-invasive biomarkers in non-alcoholic fatty liver disease (NAFLD) that can diagnose advanced disease and identify patients suitable for clinical trials. The PRO-C3 collagen neo-epitope is a putative direct marker of fibrogenesis. We assessed the performance of PRO-C3 in a large, well-characterised international NAFLD cohort and report the development and validation of 2 novel panels for the diagnosis of advanced fibrosis (F≥3) in NAFLD, including a simplified clinical score which eliminates the need for online calculators.

Methods

Plasma PRO-C3 levels were determined in a prospectively recruited international cohort of 449 patients with biopsy diagnosed NAFLD across the full disease spectrum (F0: n = 90; F1: 100; F2: 92; F3: 101; F4: 66). The cohort was divided into a discovery group (n = 151) and a validation group (n = 298). Logistic regression was performed to establish complex (FIBC3) and simplified (ABC3D) diagnostic scores that accurately identify advanced fibrosis. Performance for each was compared to established non-invasive fibrosis scoring systems.

Results

Plasma PRO-C3 levels correlated with grade of histological steatohepatitis (r_s = 0.367, p ≪0.0001) and stage of fibrosis (r_s = 0.462, p ≪0.0001), exhibiting similar performance to current fibrosis scores such as FIB4 for the detection of F≥3 fibrosis. FIBC3 exhibited substantially improved accuracy (AUROC 0.89 and 0.83 in the discovery and validation sets, respectively) and outperformed FIB4 and other similar diagnostic panels. The simplified version, ABC3D, was concurrently developed and had comparable diagnostic accuracy (AUROC 0.88 and 0.81 in the discovery and validation sets, respectively).

Conclusion

Plasma PRO-C3 levels correlate with severity of steatohepatitis and fibrosis stage. The FIBC3 panel is an accurate tool with a single threshold value that maintains both sensitivity and specificity for the identification of F≥3 fibrosis in NAFLD, eliminating indeterminate results and outperforming commonly used non-invasive tools. A greatly simplified version (ABC3D) that is readily amenable to use in the clinic has been validated and shown to perform with similar accuracy, and may prove a useful tool in routine clinical practice.

Lay summary

We performed a comprehensive, independent evaluation of a collagen biomarker (PRO-C3) to detect and quantify liver fibrosis in patients with non-alcoholic fatty liver disease (NAFLD). We report the development of 2 diagnostic panels using PRO-C3 to identify patients with advanced fibrosis, one optimal but more complex to calculate (FIBC3), the other easier to use (ABC3D) whilst still performing well.

Graphical abstract

Highlights

• •Plasma PRO-C3 levels correlate with severity of steatohepatitis and fibrosis stage.
• •FIBC3 panel achieves good sensitivity and specificity for the identification of F≥3 fibrosis in NAFLD.
• •FIBC3 panel uses a single threshold value, eliminating indeterminate results and outperforming other non-invasive tools.
• •A simplified version (ABC3D) is readily amenable to use in clinical practice.

Introduction

Non-alcoholic fatty liver disease (NAFLD) represents a continuum of liver injury, ranging from steatosis affecting ≫5% of hepatocytes, to fatty liver in the presence of inflammation and hepatocyte ballooning (non-alcoholic steatohepatitis [NASH]), with or without fibrosis, through to cirrhosis.¹ Fibrosis has emerged as the key histological determinant of long-term prognosis.[2], [3], [4], [5] Liver biopsy is generally viewed as an imperfect reference standard for grading disease activity and staging fibrosis in NASH and is not scalable to NAFLD given the magnitude of the “at risk” population.[6], [7], [8] Therefore, the increasing prevalence of NAFLD necessitates a shift from histology towards the development of non-invasive assessments. This challenge not only impacts routine clinical management but also the clinical investigation of potential new therapies to prevent the progression of fibrotic NASH to cirrhosis. The current regulatory pathway to registration requires demonstration of improvement in histological features of NASH.⁹ Consequently, a significant majority of the potential participants in recent clinical trials in NASH have been requested to undergo liver biopsy as part of the screening process. Biopsy is not without burdens and risks for patients, investigator sites and study sponsors. Rates of screening failure of up to 70% have been reported after biopsy.¹⁰ Hence, minimally invasive techniques are sought to identify study participants with the highest probability of demonstrating qualifying histological features on biopsy. The current study addresses the performance of the PRO-C3 biomarker within the FDA BEST (Biomarkers, EndpointS and other Tools) defined diagnostic context of use.¹¹

Blood-based non-invasive tests for fibrosis can be dichotomised into “indirect makers”, including simple non-invasive fibrosis scores derived from clinical and biochemical indices, such as the fibrosis-4 (FIB4) score and the NAFLD fibrosis score (NFS),[12], [13], [14], [15], [16] and “direct biomarkers” that measure collagen deposition or matrix turnover.[17], [18] The majority of non-invasive tests exhibit high negative predictive value, implying that they are best employed to exclude patients without advanced fibrosis (Kleiner ≤F2). However, many issues exist with currently available biomarkers. For example, FIB4 and NFS provide “indeterminate” results in a quarter of patients¹⁹ and although elastography based techniques such as Fibroscan™ (vibration controlled transient elastography [VCTE]) have a competitive diagnostic accuracy, they require specialist equipment, are operator dependent and exhibit low success rates in obese patients.²⁰ Magnetic resonance elastography can accurately diagnose fibrosis in patients with NAFLD.[21], [22] However, it is expensive and not widely available in most centres. A mandate therefore exists for improved biomarkers.

Research exploiting knowledge of collagen structure and protease-protein interactions have resulted in the design of a specific ELISA that measures ADAMS2 mediated collagen cleavage during the formation of type III collagen in fibrogenesis.[23], [24] Previous studies have shown that measuring formation of type III collagen neo-epitopes (PRO-C3) as a single diagnostic marker or by incorporation into a diagnostic panel can provide a reasonably accurate assessment of disease stage and activity, but to date the diagnostic panels require complex mathematical calculations necessitating the use of an online App.[25], [26], [27], [28], [29], [30] Similarly, NFS and FIB4 require the use of online calculators to generate a result. This may be onerous in a busy clinical environment, limiting adoption in the primary care setting.[31], [32] A simplified but accurate fibrosis assessment algorithm would therefore help physicians to risk stratify patients without recourse to an online calculator.

In the current study, we seek to: i) assess the performance of PRO-C3 as a NASH-fibrosis biomarker within the BEST diagnostic context of use; ii) develop and validate a novel biomarker panel incorporating PRO-C3 and determine its performance in comparison to established clinical scores and previously reported biomarker panels; and iii) develop and validate a simplified clinical tool that is both accurate and clinically accessible immediately.

Materials and methods

Study design and participants

Fig. 1 shows the flow of patients through the study. Participants were recruited at 7 specialist European centres. Patients eligible for inclusion were ≥18 years, with suspected NAFLD undergoing a diagnostic liver biopsy on clinical grounds. Patients were excluded if they had evidence of coexistent liver disease or consumed greater than 30 g of alcohol per day for males or greater than 20 g per day for females. The human biological samples were sourced ethically following receipt of informed consent from each patient and their research use was in accordance with the terms of the informed consents under an IRB/EC approved protocol at participating centres.

Fig. 1.

Patient flow for analysis inclusion.

Clinical and laboratory assessments

Gender, age and body mass index (BMI; weight (kg)/height (m²)) were recorded for all patients at time of index liver biopsy. Patients were classified as having type 2 diabetes mellitus (T2DM) if HbA1c was ≫6.5% or they were receiving dietary, oral hypoglycaemic drug or insulin treatment for T2DM. Blood tests taken at the time of liver biopsy were used to calculate the simple non-invasive scores. The FIB4 score, APRI (aspartate aminotransferase to platelet ratio index), NFS, ADAPT (Age, Diabetes, PRO-C3 and platelets panel) score and BARD (BMI, aspartate aminotransferase to alanine aminotransferase ratio [AAR], T2DM) score were calculated and applied as previously described.[13], [29], [33], [34], [35] PRO-C3 and additional biomarkers PRO-C6, PRO-C4, C4M were assessed using competitive ELISAs (Nordic Bioscience A/S, Denmark) measured by experienced technicians unaware of any associated clinical data.[23], [36]

Histological assessment

Liver biopsies were performed at each centre as per unit protocol. Target biopsy length was ≥15 mm. Biopsies were stained with haematoxylin and eosin and Masson's trichrome. Histological diagnosis, grade of steatosis and scoring for NAFLD activity and fibrosis stage were performed by expert liver pathologists at each study site according to the NASH Clinical Research Network (CRN) classification.³⁷ To reduce the element of inter-observer variability, over half of all biopsies (254, 57%) in our study were centrally reviewed by an expert member of the Elucidating Pathways of Steatohepatitis (EPoS) Histopathology Group (DT). A weighted kappa coefficient of 0.90 for fibrosis stage was established, demonstrating a very high level of inter-observer agreement.

Statistical analysis

The primary endpoint of the study was to predict the presence of advanced fibrosis (stages 3–4). The combined cohort of 449 patients was randomly separated into approximately 1/3 (n = 151) (discovery cohort) and 2/3 (n = 298) of patients (validation cohort) for model building and validation. Continuous variables were compared using the t test and categorical variables using Fisher’s exact test. The Kruskal-Wallis test was used to perform comparisons between mean marker levels followed by Dunn’s multiple comparison tests. In the discovery cohort, significant variables on univariate analysis (p ≪0.05) were included in the backward stepwise multiple logistic regression analysis to identify independent factors associated with fibrosis. Variables with p ≪0.05 by multivariate analysis were used to construct scoring systems (FIBC3 and ABC3D) to predict advanced fibrosis. Optimal cut-offs for each component of ABC3D were selected using the Youden index (J-Index) which attributes equal value to sensitivity and specify. Cross-validation was performed using the leave-one-out method to facilitate the calculation of over-fit bias reduced estimates. We calculated reduced bias estimates of predicted probability. This involved removing each individual subject and re-estimating the model parameters and then classifying the subject based on the new parameters. This enabled us to interrogate a suspicious positive or negative validation subject.

The diagnostic accuracies of both scoring systems were determined by calculating the area under the receiver operating characteristic curve (AUROC, the c-statistic) and its 95% CIs. The 5-point fibrosis scales presented both spectrum effect and ordinal scale issues. To overcome this, we calculated the Obuchowski measure using the package “nonbinROC” version 1.0.1 (https://CRAN.R-project.org/package=nonbinROC) using the R statistical analysis software platform.³⁸ This is a measure of the probability that our fibrosis index will correctly rank 2 randomly chosen patient samples from different fibrosis stages according to the weighting scheme, with a penalty score of 1 for incorrect scoring.³⁹ The method of DeLong, DeLong and Clarke-Pearson was used to compare AUROCs.⁴⁰ Validation was performed in (1) the validation dataset (n = 298) and (2) in the full dataset (n = 449). Using the ROC curve for the final model, a cut-off point was selected using the Youden index (J-Index). ROC curves were also calculated for the established diagnostic scores, AAR, FIB4, APRI, NFS, BARD and the recently described ADAPT score.[10], [24], [27], [28], [29] All statistical analyses were performed using SPSS software version 24.0 (SPSS Inc, Chicago, USA), R and SAS version 9.1 (SAS Institute Inc., Cary, NC, USA).

Results

Characteristics of patient population

Table 1 summarises the clinico-demographic details of the study population. The 449 patients were pooled from 7 international centres (Table S1). No country of origin/centre effect was detected in the analysis (p = 1.000).

Table 1.

Baseline demographic and clinical characteristics of participants.^{^}

Variable	All patients (n = 449)	Discovery cohort (n = 151)	Validation group (n = 298)	p value
Age (years)	52 ± 13	51.6 ± 13	51.5 ± 13	0.957
Gender (male)	263 (59%)	94 (62%)	169 (57%)	0.260
BMI (Kg/m2)	32.6 ± 6.8	32.9 ± 7.1	32.4 ± 6.4	0.608
T2DM	216 (48%)	74 (49%)	142 (48%)	0.786
ALT (U/L)	69 ± 41	66 ± 39	71 ± 42	0.166
High ALT (≫40 U/L)	340 (76%)	112 (74%)	228 (77%)	0.585
AST (U/L)	47 ± 26	47 ± 26	48 ± 26	0.339
Albumin (g/dl)	44 ± 5	44 ± 4	44 ± 5	0.780
Platelets (X109/L)	230 ± 72	225 ± 61	233 ± 77	0.448
Cholesterol (mg/dl)	7 ± 14	7 ± 10	7.1 ± 16	0.630
Triglycerides (mg/dl)	3.8 ± 17	3.6 ± 16	3.9 ± 18	0.758
Collagen PRO–C3 (ng/ml)	18.9 ± 15	18.1 ± 14	19.3 ± 15	0.438
Collagen PRO–C6 (ng/ml)	9.6 ± 4.4	9.3 ± 4	9.8 ± 4.7	0.501
PRO–C4 (ng/ml)	266 ± 142	253 ± 147	273 ± 139	0.067
C4M (ng/ml)	27.3 ± 10	26.8 ± 10.1	27.6 ± 9.8	0.374
C3M (ng/ml)	11.6 ± 4	11.6 ± 4.8	11.6 ± 4.2	0.644
Fibrosis Stage (0/1/2/3/4)	90/100/92/101/66	36/28/27/34/26	54/72/65/67/40	0.309
Steatosis (0/1/2/3)	10/149/171/110	6/50/56/35	4/99/115/75	0.342
Ballooning (0/1/2)	112/188/138	38/60/49	74/128/89	0.791
Lobular Inflammation (0/1/2/3)	48/219/147/24	18/78/43/8	30/141/104/16	0.578
NAS	4 ± 2	4 ± 2	4 ± 2	0.848
FIB4	1.53 ± 1.07	1.55 ± 1.08	1.52 ± 1.06	0.483
AAR	0.76 ± 0.31	0.79 ± 0.34	0.75 ± 0.30	0.428
NAFLD Fibrosis Score	–1.304 ± 1.796	–1.182 ± 1.797	–1.367 ± 1.795	0.303
APRI	0.68 ± 0.48	0.68 ± 0.51	0.68 ± 0.46	0.718
ADAPT Score	6.3 ± 2.2	6.3 ± 2.3	6.4 ± 2.2	0.652
BARD Score	2 ± 1	2 ± 1	2 ± 1	0.428
Centrally reviewed biopsies	254 (57%)	79 (52%)	175 (59%)	0.622

Mann-Whitney/ t tests were used to test for significant differences within continuous variables and Chi-Square test was used for categorical variables.

ALT, alanine aminotransferase; AST, aspartate aminotransferase; BMI, body mass index; T2DM, type 2 diabetes mellitus.

^{^}The table shows the mean ± SD for continuous variables, number (%) for binary variables, and number per group for categorical variables.

PRO-C3 levels correlated with steatohepatitis and fibrosis stage

Across all histological features (steatosis, lobular inflammation, hepatocyte ballooning, fibrosis), PRO-C3 was positively associated with increasing NAFLD severity (Fig. 2). In the discovery cohort (n = 151), PRO-C3 correlated with the NAFLD activity score (NAS) (rho = 0.304, p ≪0.0001) and fibrosis stage (rho = 0.422, p ≪0.0001). Confirming that PRO-C3 is primarily a fibrosis marker, the correlation with fibrosis stage remained significant when controlling for NAS however the converse did not hold true. Indeed, PRO-C3 exhibited the strongest correlation with fibrosis stage when compared to a number of other putative extracellular matrix turnover biomarkers (PRO-C6 (rho = 0.355), PRO-C4 (rho = 0.279), C4M (rho = 0.177), p ≪0.05).

Fig. 2.

PRO-C3 and its association with non-alcoholic fatty liver disease severity (complete cohort n = 449).

Spearman’s correlation coefficient r_s measures the strength and direction of association between 2 variables. Independent samples were compared using the Kruskal-Wallis test. All data are represented as medians, with variation in expression shown in Tukey plots. P values ≪0.05 were considered significant.

In the discovery cohort (n = 151) an optimal PRO-C3 cut-off level for the detection of advanced fibrosis was determined. PRO-C3 ≫15.5 ng/ml had an AUROC of 0.73 for the detection of advanced fibrosis ≥F3 (sensitivity 60%, specificity 74%, accuracy 68%). This was replicated in the validation cohort (n = 298) (AUROC = 0.78, sensitivity 72%, specificity 71%, accuracy 71%) (Table S2). The sensitivity and specificity for fibrosis across a range of PRO-C3 thresholds are reported for the overall cohort (Table S3).

Development of panels incorporating PRO-C3 that are diagnostic for advanced fibrosis

To identify other clinical factors that readily predict the presence of fibrosis, additional analyses were conducted. Table 2 shows the results of univariate and multivariate analyses preformed in the discovery cohort. Using backward logistic regression, 5 variables remained significantly associated with advanced fibrosis: age, BMI, T2DM, platelets and PRO-C3. No multi-collinearity was identified between variables used in the model. Variables were assessed for all 2-way interactions with no significant outcomes (p ≫0.05). These 5 variables were incorporated into a model that distinguished advanced fibrosis (F3-4) from mild fibrosis (F0-F2). The diagnostic panel “FIBC3” was calculated from the regression formula for prediction of severity of fibrosis: -5.939 + (0.053*Age) + (0.076*BMI) + (1.614*T2DM) – (0.009*platelets) + (0.071*PRO-C3). FIBC3 correlated strongly with fibrosis stage (rho = 0.630, p ≪0.0001), which remained significant independently of NAS. In the discovery cohort, the AUROC for FIBC3 was 0.89 (95% CI 0.843–0.941, p ≪0.001).

Table 2.

Variables Associated with the Presence of Advanced Fibrosis (stage F3-4) in the Discovery Cohort (n = 151).

	Univariate			Adjusted (Multivariate)
Variable	Odds Ratio	95% CI	p value	Odds Ratio	95% CI	p value
Age	1.088	1.049–1.128	≪0.0001	1.055	1.008–1.103	0.022
Gender	1.172	0.599–2.291	0.643
BMI	1.090	1.035–1.148	0.001	1.079	1.014–1.148	0.017
T2DM	8.570	4.003–18.348	≪0.0001	5.023	1.920–13.140	0.001
ALT	1.002	0.994–1.011	0.611
AST	1.020	1.005–1.034	0.007
Albumin	0.934	0.853–1.021	0.133
Platelets	0.986	0.986–0.979	≪0.0001	0.991	0.982–1.000	0.039
Cholesterol	0.841	0.714–0.990	0.038
Triglycerides	1.024	0.952–1.101	0.520
PRO-C3	1.079	1.039–1.120	≪0.0001	1.074	1.023–1.127	0.004
AST-ALT Ratio	3.072	1.119–8.436	0.029
FIBC3: –5.939 + (0.053*Age) + (0.076*BMI) + (1.614*T2DM) – (0.009*platelets) + (0.071*PRO–C3) ABC3D: Age ≫50 = 1 point, BMI ≫30 = 1 point, platelet Count ≪200 = 1 point, PRO–C3 ≫15.5 = 1 point, Diabetes = 2 points

ALT, alanine aminotransferase; AST, aspartate aminotransferase; BMI, body mass index; T2DM, type 2 diabetes mellitus.

To facilitate adoption in a clinical setting, a simplified score based on the same 5 variables identified as significant on univariate analysis and weighted according to their odds ratio (OR) values was generated. The derived “ABC3D” score comprises: A = Age≫50 years, B = BMI≫30, C = platelet Count≪200, 3 = PRO-C3≫15.5 ng/ml, Diabetes = present. Optimal thresholds for each variable were selected by maximising the Youden index for the corresponding ROC curves. The presence of each factor scored 1 point, except for T2DM which, with an OR of 5, was awarded 2 points to yield a maximum score of 6. In the discovery cohort, the AUROC for ABC3D was 0.88 (95% CI 0.822–0.929, p ≪0.001).

Validation of FIBC3 and ABC3D model accuracy and derivation of diagnostic thresholds for advanced fibrosis

The diagnostic accuracy of these models for the detection of advanced fibrosis was confirmed in a the validation cohort (n = 298) and also in the overall combined cohort (n = 449). Diagnostic accuracy was assessed by the standard AUROC and also the weighted AUROC computed using the Obuchowski measure to account for spectrum effect and ordinal scale.

For FIBC3, the AUROC remained high in both the validation cohort (0.83, 95% CI 0.777-0.880) and the combined cohort (0.85, 95% CI 0.812-0.886). The weighted AUROC was calculated to be 0.77, 0.75 and 0.79 in the combined, discovery and validation cohorts, respectively. Similar results were obtained for ABC3D with AUROC of 0.81 and 0.83 in the validation and combined cohorts, respectivelY (Table 3). Reduced bias estimates of predicted probability were calculated in the discovery and validation cohorts, employing the leave-one-out method of cross-validation as previously described. To assess the added value of including PRO-C3 in the diagnostic model, we removed PRO-C3 from the FIBC3 model. This yeilded AUROCs of (0.80, 0.86 and 0.76) in the total, discovery and validation cohorts, respectively. These improved to (0.85, 0.89 and 0.83) with the inclusion of PRO-C3 in the model.

Table 3.

Diagnostic accuracy of non-invasive tests by detecting Histologic stage F3–F4 and weighted AUROC derived from the Obuchowski measure.

	Combined cohort (n = 449)
Non-invasive test	AUROC	95% CI	Adj AUROC	SD	95% CI
AAR	0.67	0.615–0.716	0.62	0.019	0.581–0.653
APRI	0.75	0.698–0.794	0.68	0.017	0.652–0.717
BARD	0.71	0.664–0.761	0.67	0.017	0.642–0.707
FIB4	0.78	0.732–0.820	0.70	0.015	0.671–0.731
NFS	0.79	0.751–0.838	0.72	0.015	0.694–0.752
ADAPT	0.85	0.815–0.888	0.77	0.014	0.739–0.794
PRO–C3	0.76	0.718–0.811	0.69	0.017	0.660–0.726
FIB–C3	0.85	0.812–0.886	0.77	0.013	0.745–0.797
ABC3D	0.83	0.793–0.868	0.76	0.013	0.730–0.783
p value	≪0.0001
	Discovery cohort (n = 151)
AAR	0.66	0.579–0.751	0.62	0.031	0.555–0.675
APRI	0.75	0.669–0.830	0.69	0.028	0.638–0.748
BARD	0.76	0.683–0.834	0.69	0.028	0.637–0.746
FIB4	0.80	0.726–0.867	0.70	0.026	0.651–0.751
NFS	0.85	0.791–0.911	0.71	0.023	0.669–0.758
ADAPT	0.86	0.800–0.917	0.74	0.025	0.695–0.793
PRO–C3	0.75	0.661–0.831	0.68	0.031	0.617–0.740
FIB–C3	0.89	0.843–0.941	0.75	0.021	0.707–0.789
ABC3D	0.88	0.822–0.929	0.75	0.022	0.704–0.790
p value	≪0.0001
	Validation cohort (n = 298)
AAR	0.66	0.599–0.725	0.62	0.024	0.571–0.663
APRI	0.75	0.686–0.805	0.68	0.021	0.640–0.722
BARD	0.69	0.624–0.749	0.66	0.021	0.623–0.705
FIB4	0.76	0.707–0.819	0.70	0.019	0.644–0.739
NFS	0.76	0.701–0.818	0.73	0.019	0.692–0.766
ADAPT	0.85	0.803–0.896	0.78	0.017	0.749–0.815
PRO–C3	0.78	0.727–0.838	0.70	0.020	0.622–0.741
FIB–C3	0.83	0.777–0.880	0.79	0.017	0.753–0.819
ABC3D	0.81	0.755–0.856	0.76	0.017	0.730–0.795
p value	≪0.0001

*Prevalence advanced fibrosis *combined cohort = 0.37 *Discovery cohort = 0.40 * Validation cohort = 0.36

*DeLong DeLong Clarke test for comparison of AUROC

An optimal FIBC3 threshold value of ≫-0.4 was chosen using the Youden index (sensitivity 83%, specificity 80%, positive predictive value [PPV] 74% and negative predictive value [NPV] 88%). An optimal ABC3D cut-off level for the detection of advanced fibrosis was ≫3. In the validation cohort (n = 298), FIBC3 exhibited a sensitivity of 75%, specificity of 75%, accuracy of 75% (Table 4). In the discovery cohort, ABC3D exhibited a sensitivity of 77%, specificity of 82%, and accuracy of 80%. This was replicated in the validation cohort, where a sensitivity of 66%, specificity of 75% and accuracy of 73% were observed.

Table 4.

Optimal cut-off values for the detection of advanced fibrosis (≥F3) as per Youden index derived in discovery cohort (prevalence 0.40, n = 151) and applied in validation cohort (prevalence 0.36, n = 298).

Panel	AUC	Cut-off	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	Accuracy (%)
FIB-C3	0.89	≫–0.4	83	80	74	88	81
ABC3D	0.88	≫3	77	82	74	84	80
Validation cohort
		Cut-off	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	Accuracy (%)
AAR		≫0.8	46	71	47	70	62
APRI		≫1.5	11	96	63	66	66
BARD		≫2	76	51	47	79	60
FIB4		≫2.67	21	94	67	68	68
NFS		≫0.676	27	95	78	70	71
ADAPT		≫6.3	76	75	63	86	76
FIB–C3		≫–0.4	75	75	62	84	75
ABC3D		≫3	66	75	61	80	73

PPV, positive predictive value; NPV, negative predictive value.

Both FIBC3 and ABC3D performance were superior to simple non-invasive scores in common use, with accuracies of 75% and 73%, respectively. Performance characteristics of FIBC3 and the simplified ABC3D score were comparable to the recently described ADAPT score (Table 4). Comparing AUROCs using the DeLong, DeLong and Clarke-Pearson method confirmed that FIBC3 and ABC3D have similar performance characteristics (p = 0.1422) as do FIBC3 and ADAPT (p = 0.1859). Using the FIBC3 model, the optimal threshold correctly staged 224 out of 298 patients (75%) in the validation cohort, compared to 227 patients (76%) with ADAPT and 217 (73%) with ABC3D. Considering NPV, of 191 patients with mild fibrosis, 144 (75%) were staged correctly using FIBC3 or ABC3D, equal to ADAPT (75%) (Table 5). In the combined cohort (n = 449), 347 of the patients (77%) were correctly staged using FIBC3, which outperformed both FIB4 at 304 (68%) and ADAPT at 341 (76%). The most simple model, ABC3D, had a diagnostic accuracy of 75% correctly classifying 338 cases into mild or severe fibrosis.

Table 5.

Validation cohort divided into mild and severe fibrosis (prevalence 0.39, n = 298).

	F0–2 ‘Rule out’ advanced fibrosis				F3–4 ‘Rule in’ advanced in severe
	Correctly identified	Indeterminate	Incorrectly identified		Correctly identified	Indeterminate	Incorrectly identified
N = 191	n/N (%)	n/N (%)	n/N (%)	N = 107	n/N (%)	n/N (%)	n/N (%)
AAR ≪0.8	135/191 (71)		56/191 (29)	AAR ≫0.8	49/107 (46)		58/107 (54)
APRI ≪0.5	112/191 (59)	72/191 (38)	7/191 (3)	APRI ≫1.5	12/107 (11)	72/107 (67)	23/107 (22)
BARD ≪2	98/191 (51)		93/191 (49)	BARD ≫2	81/107 (76)		26/107 (24)
FIB4 ≪1.3	133/191 (70)	47/191 (25)	8/188 (5)	FIB4 ≫2.67	22/107 (20)	53/107 (50)	32/107 (30)
NFS ≪–1.433	120/191 (64)	63/191 (33)	5/191 (3)	NFS ≫0.676	29/107 (27)	51/107 (48)	27/107 (25)
ADAPT ≪6.3	144/191 (75)		47/191 (25)	ADAPT ≫6.3	83/107 (78)		24/107 (22)
FIBC3 ≪–0.4	144/191 (75)		47/191 (25)	FIBC3 ≫–0.4	80/107 (75)		27/107 (25)
ABC3D ≪3	144/191 (75)		47/191 (25)	ABC3D ≫3	73/107 (68)		34/107 (32)

Performance of FIBC3 and ABC3D in real-world settings

We assessed the performance of FIBC3 and ABC3D in a range of pre-test probability scenarios that may be encountered across primary care and specialist care environments, where the prevalence of advanced fibrosis varies, to see if they were equivalent. The PPV and NPV were calculated across an advanced and mild fibrosis prevalence range between 5–50% (Table 6). We also stratified our validation cohort in different, clinically distinct, sub-populations and observed that performance was maintained across all sub-populations, with a reliable NPV for advanced fibrosis ≫74% (Table S4,5).

Table 6.

Predictive values of cut-offs at different prevalences of advanced and mild fibrosis.

Combined Cohort (n = 449)
Predictive values of cut-offs for different prevalences of advanced fibrosis (F≫3); “Rule in” advanced fibrosis
	FIBC3 ≫–0.4		ABC3D ≫3		FIB4 ≫2.67		NFS ≫0.676		ADAPT ≫6.3
Prevalence of significant fibrosis (%)	PPV (%)	NPV (%)	PPV (%)	NPV (%)	PPV (%)	NPV (%)	PPV (%)	NPV (%)	PPV (%)	NPV (%)
5	15	99	15	98	19	96	22	96	14	98
10	27	97	26	96	33	92	38	92	26	96
15	37	95	36	94	44	87	49	88	36	94
20	46	93	45	91	52	83	57	84	45	92
25	53	91	52	89	59	79	64	80	52	90
30	59	89	58	86	65	74	70	75	58	87
35	65	87	63	83	70	69	74	71	63	85
40	69	84	68	80	75	65	78	66	68	82
45	74	81	73	77	78	60	82	61	72	78
50	77	78	76	73	81	55	84	57	76	75
Predictive values of cut-offs for different prevalences of mild fibrosis (F≪2); “Rule in” mild fibrosis
	FIBC3 ≪–0.4		ABC3D ≪3		FIB4 ≪1.3		NFS ≪–1.433		ADAPT ≪6.3
Prevalence of mild fibrosis (%)	PPV (%)	NPV (%)	PPV (%)	NPV (%)	PPV (%)	NPV (%)	PPV (%)	NPV (%)	PPV (%)	NPV (%)
5	25	99	12	98	11	98	14	98	12	97
10	42	97	23	97	21	95	26	95	22	94
15	53	96	32	95	30	93	36	92	31	91
20	62	94	40	93	38	90	45	90	39	88
25	68	92	47	91	46	87	51	87	46	85
30	73	90	53	88	51	84	57	84	52	81
35	78	88	59	86	57	81	63	80	58	78
40	81	85	64	83	62	78	68	76	63	74
45	84	82	69	80	67	74	72	73	68	70
50	87	79	73	76	71	69	76	68	72	65

PPV, positive predictive value; NPV, negative predictive value.

Performance of PRO-C3, FIBC3 and ABC3D as pre-screening tools prior to liver biopsy to support clinical trial recruitment

As there is also a need for tools to assist in pre-screening patients for clinical trials in NASH, we modelled the performance of PRO-C3 as pre-screening tools for entry into clinical trials of fibrosing steatohepatitis. Two target populations were modelled: (i) “tdNASH”, defined as NAS ≥4 with at least 1 point each for steatosis, hepatocyte ballooning and hepatic inflammation and fibrosis stage ≥F2; and (ii) “tdNASH-Cirrhosis”, defined as above but with fibrosis stage F4. For tdNASH, a PRO-C3 level ≫14.5 ng/ml had an AUROC of 0.68 (sensitivity 59%, specificity 69%, accuracy 64%). This was replicated in the validation cohort (n = 298), AUROC = 0.76, sensitivity 70%, specificity 68%, accuracy 69%. Similarly, a PRO-C3 level ≫16.5 ng/ml identified tdNASH-Cirrhosis with an AUROC of 0.68 (sensitivity 74%, specificity 67%, accuracy 68%). This was replicated in the validation cohort (n = 298), AUROC = 0.76, sensitivity 76%, specificity 61%, accuracy 63% (Table S2). The results for the FIBC3 and ABC3D scores in the complete cohort (n = 449) are shown in Table S6. In general, tests incorporating PRO-C3 performed well. The most accurate test for the detection of tdNASH was FIBC3 ≫-0.4 (71%). Phase II/III clinical trials that are currently recruiting will be informative for the further validation of these findings.

ABC3D to improve the accuracy of NFS and FIB4 scores

Although FIB4 and NFS are useful, the use of 2 cut-off thresholds leads to indeterminate results that fail to classify a substantial proportion of patients. For each diagnostic test we employed a method of sequential testing by applying the low and high cut-off values. The residual cohort of patients with NAFLD and indeterminate scores were then assessed with the ABC3D diagnostic algorithm to detect cases of advanced fibrosis (Tables S7,8). With the application of sequential testing, the accuracy improved from 52% to 70% in the cases involving indeterminate FIB4 scores and from 54% to 77% in the case involving indeterminate NFS scores.

Discussion

NAFLD has an estimated global prevalence of 25%, which is predicted to rise internationally.[41], [42], [43] Associated mortality is directly proportional to fibrosis stage, with patients at ≥F3 being at highest risk.² Current non-invasive tests are suboptimal; therefore, there is a clear need for better diagnostic biomarkers to detect advanced fibrosis. Such tests could potentially aid diagnosis and risk stratification, as well as facilitate clinical trial pre-screening to reduce screening failure rates; all of which fall within the BEST diagnostic context of use.¹¹

At present, the reference standard to assess severity of NAFLD is histological, using the semi-quantitative NASH CRN system.³⁷ However, it is generally accepted that inter- and intra-observer variability, and sampling error due to variability in the extent of fibrosis within the liver, may impair the accuracy and reproducibility of these histological assessments.[37], [44], [45] This implies a paradox that makes addressing the need for biomarkers all the more challenging: the histological reference standard, against which a biomarker is assessed, is inherently imperfect and unable to produce a completely error-free classification with respect to the presence or absence, or severity, of the target condition. Semi-quantitative histological grading conflates anatomical distribution of fibrosis with extent and imposes discrete categorical staging bins on what are continuous variables like collagen deposition.³⁷ This inevitably leads to discrepancies due to inter- and intra-observer judgement, especially at the margins. It also blunts sensitivity as semi-quantitative grades fail to recognise modest differences in severity that do not transition across predefined but arbitrary categorical boundaries. This phenomenon is well illustrated by the breadth of disease that is encompassed by stage F3 fibrosis in the NASH CRN classification³⁷ where histological portal-portal, central-central and/or portal-central bridging is the defining feature, yet no weight is given to density of collagen deposition or the number of “bridging” septae. The situation where an imperfect reference standard is used in place of a perfect standard, introduces “imperfect gold standard bias”. This means that the performance of the new test may be under- or over-estimated and, even if it is in reality a better measure of disease, it never has the potential to generate an AUROC ≫0.90.⁴⁶ Although not unique to liver histopathology, such situations are methodologically challenging to address.⁴⁷

Cognisant of these challenges, we report measurement of PRO-C3 levels in a large international cohort and incorporate this measure into novel diagnostic models that outperform numerous previously described blood-based tests that detect advanced fibrosis.[12], [13], [14], [15], [16], [17], [18]

Utility of PRO-C3 as a single diagnostic biomarker

Although isolated parameters seldom exhibit an adequate level of diagnostic accuracy and are unlikely to be a surrogate for the complex diagnostic information provided by liver biopsy, we assessed how PRO-C3 performed in this context of use. PRO-C3 performed moderately as a biomarker of advanced fibrosis, comparable to simple panels such as FIB4. Similarly, when used to screen patients for clinical trial recruitment, PRO-C3 accurately identified 65% of cases that were histologically eligible for current phase III trial recruitment (NASH with significant fibrosis). This moderate performance as a diagnostic biomarker may partially be explained by the biological process that generate PRO-C3 during collagen deposition, implying that PRO-C3 is most sensitive to active fibrogenesis rather than static collagen accumulation. Supporting this view, preliminary evidence suggests that PRO-C3 may aid the evaluation of patients with active collagen turnover.⁴⁸ In the present study we were unable to assess the value of PRO-C3 as a prognostic test, that could be used to enrich studies for cases at greatest risk of subsequent disease progression, or to monitor change in disease severity.

FIBC3 and ABC3D performance for risk stratification of fibrosing steatohepatitis

In light of the moderate performance of PRO-C3 as a single diagnostic biomarker, we assessed its value as part of a non-invasive fibrosis panel composed of routinely measured clinical and laboratory variables enhanced by inclusion of a single biomarker of fibrogenesis, PRO-C3. We report development and validation of FIBC3. Whilst not the first panel to incorporate these components, many of which are used within ADAPT,²⁹ the current study benefits from detailed development and validation in a large, international patient cohort where careful harmonisation of histological practice, coupled with central reviewing of biopsies, has been undertaken to minimise the potential impact of an imperfect reference standard. Overall, a FIBC3 threshold of ≫-0.4 correctly identified fibrosis status in 77% of patients in the total cohort. However, the diagnostic accuracy of ABC3D, a simplified panel, better adapted for use in clinical practice (at the bedside) rivalled this model with an accuracy of 75% and performed equivalently when assessed across different clinical sub-populations and consistently outperformed all other routinely used scores to which it has been compared. Thus, in contrast to FIB4, NFS or the PRO-C3 based ADAPT score, which require more complex formulas, this simple model can be easily calculated by summing 5 easy to assess clinical items, removing the need to access to a web-based calculator or App to aid patient risk stratification. Furthermore, in contrast to FIB4 or NFS, FIBC3 and ABC3D both have a single, optimised, risk-threshold value, without “indeterminate” results which would require further testing or liver biopsy to clarify disease severity.¹⁹

In the validation cohort, FIBC3 performed best, correctly identifying 75% of patients, with ABC3D more or less equivalent correctly identifying 72% of patients. In the full cohort of 449 patients, the FIBC3 model identified 254 patients as not having advanced fibrosis (at a threshold of less than -0.4) of which 217 were correctly classified. Therefore, in this “low-risk cohort” the FIBC3 model could have correctly avoided a liver biopsy in 85% of patients. Applying the same analysis to ABC3D, 267 patients were identified as ‘low-risk’ (score ≤3). In this cohort, 219 patients were correctly staged, thus potentially correctly avoiding biopsies in 82% of cases. Complex fibrosis panels also exist. They include markers of matrix turnover, such as the Enhanced Liver Fibrosis (ELF™) panel.¹⁸ However, a recent meta-analysis has reported that ELF and NFS have very similar AUCs.⁴⁹ Extrapolating this observation to our data would imply that FIBC3/ABC3D (like the NFS) had comparable, if not better, diagnostic value than the more complex Fibrotest and ELF.

Potential to use ABC3D in primary care

The point performance of diagnostic tests in terms of PPV/NPV are affected by pre-test probability, which reflects the prevalence of disease in a specific clinical setting. The prevalence of advanced fibrosis in the current study cohort was 37% which is much higher than would be expected in a primary care setting. Indeed, population data, albeit limited, have found that 5.6% of the Dutch population have clinically significant fibrosis based on a VCTE liver stiffness ≫8 kPa.⁵⁰ Similarly, based on VCTE thresholds ≥6.8, ≥8.0, and ≥9.0 kPa prevalence estimates in the Spanish population were 9.0%, 5.8%, and 3.6%, respectively.⁵¹ These levels contrast sharply to a tertiary referral centre where the prevalence of advanced liver disease is often well in excess of 10%, and frequently nearer 30%.[52], [53], [54] To model performance across a range of settings, we calculated PPV and NPV for prevalence levels of advanced fibrosis from 5–50%. The NPV for both FIBC3 and ABC3D were similar across a prevalence range of 5–15% and in excess of 90%. To explore performance of the models in specific patient subgroups, we split the cohort by gender, diabetes status, BMI, and patients with elevated or normal alanine aminotransferase levels. FIBC3 and ABC3D maintained high NPV in all subgroups, although sensitivity was lower in patients with a BMI ≪25 and non-diabetics.

Strengths and limitations

FIBC3 and ABC3D were developed using an international cohort of well-characterised, untreated patients with NAFLD, covering a wide spectrum of disease severity. Liver biopsies were read by expert histopathologists that belong to the EPoS consortium pathology group, a group that undertook extensive harmonisation procedures for NAFLD pathological assessment and demonstrated high kappa-value reproducibility.⁴⁵ Moreover, half of the biopsies across all sites were assessed centrally. While this certainly reduces the reader-related variability, it is still dependent on limitations intrinsic to histological classifications such as the semi-quantitative nature of fibrosis scoring and on sampling variability of the procedure. These limitations are common to all biomarkers that use biopsy as the reference standard. Our diagnostic model consists of readily available clinical and laboratory variables that are routinely determined in patients with NAFLD in outpatient appointments. PRO-C3 levels were also measured in a central College of American Pathologists certified lab by staff blinded to the clinical data, before results were sent to a separate, independent centre for statistical analysis. Protein finger print technology has been developed to produce a reliable assay for PRO-C3 measurement.²⁵ Our model, in comparison to previous complex biomarker panels (e.g. ELF or Fibrotest) includes only one variable that is not routinely measured in a clinical setting. To minimise the effects of inter-observer variability in fibrosis staging, half the cohort across all centres had centrally reviewed liver biopsies confirming high inter-observer agreement.

Although we have taken measures to minimise inter-observer variability in the histological scoring, and concordance between liver pathologists was very good, an element of variability cannot be fully excluded. We also acknowledge that percutaneous liver biopsy is prone to sampling error leading to mis-staging of disease severity. However, the key limitation, which is common to all biomarker studies that rely on histology, relates to the nature of the semi-quantitative scoring systems and how this conflates histological localisation of fibrosis and extent of collagen deposition. We also acknowledge that AUROCs are not perfect as a means for assessing diagnostic accuracy. ROC curves attribute equal weight to false positives and false negatives and do not provide information on predictive values, which may be of greater value in a clinical setting.⁵⁵ Our results require further independent validation in other patient populations, to critically assess these models’ ability to discriminate fibrosis stage.

In conclusion, both FIBC3 and ABC3D are simple indices including accessible routine laboratory tests and a single marker of collagen turnover. We have shown that both can accurately differentiate mild to moderate fibrosis from bridging fibrosis and cirrhosis in patients with NAFLD. Given that the ABC3D model is much simpler to compute and can be done at the bedside, the ABC3D diagnostic index has the potential to be widely used for the identification of patients with significant/active fibrosing steatohepatitis who should undergo specialised liver explorations, closer monitoring and possibly, specific therapies. FIBC3 and ABC3D may also be used as pre-screening tools for therapeutic trials, potentially helping to minimise histological severity-related screening failure rates. However, this will require further prospective validation.

Financial support

The research leading to these results has been supported by the EPoS (Elucidating Pathways of Steatohepatitis) consortium funded by the Horizon 2020 Framework Program of the European Union under Grant Agreement 634413 and the Newcastle NIHR Biomedical Research Centre with additional support from GSK.

Conflict of interest

MK, DL and MJN are employed at Nordic Bioscience, a privately-owned company responsible for the development of PRO-C3. MK and DL are stock holders in the company. SK is employed by and holds stock in GSK. The other authors have no relevant potential conflicts of interests and none of the authors have received any payment for the work described in this study.

Please refer to the accompanying ICMJE disclosure forms for further details.

Authors’ contributions

Study Concept and Design QMA, SK; Data collection QMA, MB, DT, JMS, VR, EB, SP, CPO, OG, MY, SM, PB, MJN, MK, DL; Statistical analysis MB; Drafting initial manuscript MB, QMA; All authors reviewed the manuscript, revised the manuscript for important intellectual content and approved the final manuscript.

Supplementary data

Supplementary tables

Supplementary material 1

Supplementary material 2

Supplementary material 3