ROC curves in evaluation of serum fibrosis indices for hepatic fibrosis

Abstract

AIM: Use Receiver operating characteristic (ROC) curves to find out the relationship between serum level of hyaluronic acid (HA), type III procollagen (PCIII), N-terminal procollagen III peptide (PIIINP), laminin (LN), type IV collagen (C-IV) and hepatic fibrosis, as well as to determine their value in clinical practice.

METHODS: 114 serum samples from chronic hepatitis patients were assayed for fibrosis indices including HA, PCIII, PIIINP, LN and IV-C with radioimmunoassay (RIA). Liver biopsy was also performed in all these patients and the biopsy material was examined histopathologically.

RESULTS: ROC curves analysis showed that area under the curve (AUC) of PIIINP, HA, PCIII, C-IV and LN was 0.800, 0.728, 0.727, 0.583 and 0.463, respectively. The analysis also showed that PIIINP (r = 0.452), HA (r = 0.497) and PCIII (r = 0.404) have greater diagnosis performances than C-IV (r = 0.238) and LN (r = 0.128) according to fibrosis staging. The sensitivity of HA plus PIIINP was 55.1%, it was the most sensitive combination. Combined three or more than three indices that based on HA, the specificity was 100%. Using combination assays can improve the specificity, but its sensitivity was not high. Serum fibrosis indices increased as the grade of inflammation aggravated. But only PIIINP and PCIII had significant difference between G1 and G2 (PIIINP: 13.16 ± 8.07 vs 8.32 ± 5.09; PCIII: 164.22 ± 65.69 vs 138.23 ± 77.63). The coefficient correlation of the results of inflammation grade and fibrosis staging to HA was 0.525 and 0.553 respectively, that to PCIII, 0.446 and 0.412, that to LN, 0.234 and 0.194, and that to IV-C, 0.363 and 0.351, respectively.

CONCLUSION: Serum fibrosis indices can indicate tendency of hepatic fibrosis, but it cannot replace liver biopsy. However, as diagnostic markers, more efficient serum fibrosis indices for the diagnosis of hepatic fibrosis need to be explored.

INTRODUCTION

Chronic injury leading to fibrosis in the liver[1-11]. Liver biopsy has traditionally been the standard method for assessing hepatic fibrosis, but the procedure is associated with complications in patient under biopsy and so it is difficult to put into practice. Reports showed that serum fibrosis indices, including PCIII, PIIINP, LN, HA and C-IV and others, can reflect the activity of hepatic fibrosis to some extent[12-19]. Mean ± SD has always been used to express the standard for hepatic fibrosis[20]. In resent years, some scientists have recommended to use Receiver operating characteristic (ROC) curves in determination of indices of hepatic fibrosis in clinical practice[21]. Reports using ROC curves to evaluate fibrosis indices were seen, but histopathological results of the liver have not been used as control. In this study, levels of all the five fibrosis indices were measured in patients with chronic hepatitis B and comparison with biopsy results of the liver was carried out to determine if the measurements of these indices have any clinical value as markers of chronic hepatic fibrosis. ROC analysis was used to determine the sensitivity and specificity of the assays in detecting the liver disease.

MATERIALS AND METHODS

Subjects

During the Sixth National Conference for Infectious and Parasitic Diseases, the protocol of prevention and treatment for virus hepatitis was modified in 2000 (abbreviated as “2000 criteria”)[22]. According to the “2000 criteria”, 114 patients had typical presentations of chronic hepatitis. 99 were males and 15, females. Among them, 75, 30 and 9 showed mild, moderate and severe degree of the disease, respectively. The patients’ histories were mainly collected from the First Affiliated Hospital of School of Medicine, Zhejiang University and several hospitals in Zhejiang Province between July, 1998 and September, 1999. Their age ranged between 16 and 57 years and the disease course was from one to 30 years. All patients showed positive in HBV markers (HBVM) and the diagnosis was made by liver biopsy according to the “2000 criteria”[22].

Histology

Biopsy fragments of the livers were fixed in 10% neutralized formadehyde, embedded in paraffin, and then stained with hematoxylin and eosin. Reticulation fibrosis stain and the Sirius red method were used specially for staining fibrous tissue components. Histological assessment of the liver was done according to Wang’s report[23], and the stage of fibrosis was divided into four, expressed as S1 to S4 according to the “2000 criteria”[22]. S1 shows expansion in portal tract areas with fibrosis; S2, fibrosis around portal tract areas with fibrosis segregation formation, while maintaining lobule structure; S3, formation of fibrosis segregation and disorder of lobule structure without hepatic cirrhosis, and S4, early stage or confirmed cirrhosis. S0 shows no fibrosis.

Determination of serum fibrosis indices

The serum specimens were divided into five proportions and stored at -20 °C. The assay of the levels of serum HA, PCIII, PIIINP, IV-C and LN was done by RIA. The kits of HA, IV-C and LN were provided by the Shanghai Navy Medical Institute. The kit of PCIII was provided by the Chongqing Tumor Institute. The kit of PIIINP was provided by the Shanghai Orion Diagnostic Reagent Corporation (produced by Finland Orion Corporation). The operations were performed according to the user’s manual.

Statistical analysis

Results were expressed as mean ± standard deviation (^-x ± s) and compared when necessary. The relationship between noninvasive markers and stage of histological liver fibrosis was analyzed by the Spearman rank-correlation test and nonparametric one-way ANOVA for nonparametric data. Tests were considered statistically significant at P < 0.05. Sensitivity of the assays was plotted against the false positivity (1-specificity) using ROC curves using SPSS 10.5 statistical program. Comparison of AUC was performed, which compares the AUC to the diagonal line of no information (AUC 0.5). The pathologist was blind to the results of serum indices in the study subjects. In order to determine the specificity and sensitivity of the assays, S ≥ 1 for one with hepatic fibrosis was arbitrarily defined. Take (1-specificity) as x-axis, and take sensitivity as y-axis. If AUC = 1.0, the index is an ideal test, and if AUC < 0.5, the index has no diagnostic value.

RESULTS

ROC curves of five serum fibrosis indices

From Figure 1 the AUC of serum fibrosis indices in ROC curves is in the order of PIIINP > HA > PCIII > C-IV > LN. It means that PIIINP, HA and PCIII are more useful than C-IV and LN in terms of the fibrosis index for diagnosis. PIIINP is the most sensitive index among the five indices, but its specificity is not as high as HA and PCIII. Take both sensitivity and specificity into account, the cut-off point was selected according to max number of sensitivity add specificity. Table 1 shows the cut-off point, sensitivity, specificity and correct diagnosis index of the five indices.

Figure 1.

ROC curves of five serum fibrosis indices

Table 1.

ROC curves of five serum fibrosis indexes

Indices	AUC	P Vaule	cut-off point	Sensitivity	Specificity	r Vaule
PIIINP	0.800	0.000	≥ 5.61 μg/L	82.7%	62.5%	0.452
HA	0.728	0.003	≥ 145.2 ng/mL	62.2%	87.5%	0.497
PCIII	0.727	0.004	≥ 137.4 μg/L	59.2%	81.2%	0.404
C- IV	0.583	0.287	≥ 74.2 μg/L	55.1%	68.7%	0.238
LN	0.463	0.636	≥ 156.65 ng/mL	37.8%	75.0%	0.128

Right diagnosis index (r) = sensitivity + specificity -1

The value of combination of the five indices for diagnosis

HA is the generally used index for diagnosis. From Figure 1, HA is the best one among the five indices. So HA was taken as a basic index and was used in combination with other indices for the diagnosis of hepatic fibrosis. Each index result of ROC analysis is shown in Table 1. Table 2 shows the sensitivity and specificity of combination diagnosis. Among two index combination groups HA + PIIINP is better than HA + PCIII and HA + C-IV and HA+LN. In combinations of the above three indices, specificity is all 100%. The best sensitivity was seen in HA + C-IV + PIIINP and HA + C-IV+ PCIII combinations.

Table 2.

Parameter of combination diagnosis with several fi-brosis indexes

Combination	Sensitivity (%)	Specificity (%)
HA + PIIINP	55.1	87.5
HA + PCIII	50.0	87.5
HA + LN	27.6	100
HA + C-DIV	42.9	100
HA + LN + C-IV	21.4	100
HA + LN + PIIINP	24.5	100
HA + LN + PCIII	24.5	100
HA + C-IV + PIIINP	38.8	100
HA + C-IV + PCIII	38.8	100
HA + LN + C-IV + PIIINP	19.4	100
HA + LN + C-IV + PCIII	19.4	100

The relationship between hepatic fibrosis indices and inflammation grades

Table 3 shows that numerical value of serum fibrosis indices, excluding LN, increased along with the development of inflammation grades. Only PIIINP and PCIII, but not others, have significant difference between G2 and G1. Values of PIIINP, PCIII, HA and C-IV in the stage G3 and G4 have significant differences as compared to those in the stage G1.

Table 3.

The relationship between hepatic fibrosis indexes and inflammation grades

G	n	PIIINP(μg/L)	HA(ng/mL)	PCIII(μg/L)	C-IV(μg/L)	LN(ng/mL)
1	45	8.32 ± 5.09	144.78 ± 123.31	138.23 ± 77.63	73.89 ± 23.61	155.43 ± 55.48
2	36	13.16 ± 8.07b	211.26 ± 187.17	164.22 ± 65.69b	6.26 ± 52.56	138.76 ± 43.42
3	27	15.61 ± 7.05b	476.26 ± 296.44b	190.06 ± 75.10b	109.75 ± 42.12b	161.52 ± 34.25
4	6	15.60 ± 5.41b	562.08 ± 274.47b	261.68 ± 127.77b	123.10 ± 41.60b	154.60 ± 20.03

^bP < 0.01; Compare to G1

Correlation between serum fibrosis indices and histological classification (r)

Table 4 shows the details of relationship between serum fibrosis indices and liver inflammation grades and fibrosis staging.

Table 4.

Correlation between serum fibrosis indexes and his-tology classification

	PIIINP	HA	PCIII	C-IV	LN
Inflammation grads	0.463b	0.523b	0.446b	0.363b	0.234
Fibrosis stages	0.403b	0.553b	0.412b	0.351b	0.194

^bP < 0.01

DISCUSSION

Hepatic fibrosis is a result in loss of normal liver cell function due to the disorganized over-accumulation of extracellular matrix (ECM) components in the liver[24-29]. It is clear that the increase production and decrease degradation of ECM components are responsible for the altered ECM metabolism in fibrotic liver. So the metabolism production of ECM in serum can be regarded as the indices of hepatic fibrosis. Different serum fibrosis indices represent different ECM metabolism. Such as PIIINP and PCIII reflect collagen metabolism[30], and HA reflects hepatic fibrosis activity and liver injury[31,32]. LN reflects basement membrane transformation and has some relation to portal hypertension[32,33].We have reported an integral project for the diagnosis of hepatic fibrosis and serum fibrosis index spectrum for the serodiagnosis of hepatic fibrosis[34]. But along with the development of the technology of testing and statistic, new evaluation for serum hepatic fibrosis indices is necessary. ROC curves and AUC of ROC curves can assess the value of one test[35,36]. ROC curves can do comparison of several diagnostic techniques for one disease. In this way, clinicians can get help for screening out the most suitable scheme. So ROC analysis has become an important method for the assessment of diagnostic markers for hepatic fibrosis.

From Figure 1 and Table 1 it is clear that PIIINP is the most sensitive index among the five indices. However, its specificity is comparatively low. Also a conclusion can be got from Figure 1 and Table 1 that HA is the most specific index. This is consistent with the results of our previous study on the relationship between serum fibrosis indices and liver histological changes[37], and is also consistent with other reports[38,39]. The cut-off point is used in our study to confirm the critical value of serum fibrosis indices when they are used in the diagnosis of hepatic fibrosis. Not only the cut-off point meets the requirement of study on clinical epidemiology, but also it can get more reliable result. One may get conclusion from the figure that using only one index to diagnose fibrosis may be prejudicial. From the ROC curves we can see, no single serum index has both ideal sensitivity and specificity. Our results also showed that excluding LN, other serum indices have close relation with inflammation. This is also the reason why the specificity is low if only one serum fibrosis index is used in the diagnosis. Ye et al reported that serum PCIII index can exclude the influence of hepatocyte inflammation and necrosis. But their results just took serum alanine aminotransferase (ALT), bilirubin, albumin, HBVDNA and HBVM as standards for estimation. As we know that serum liver function tests and HBVDNA are not very objective indices. Our results based on histological examination of the liver hinted that PCIII also has close relation with inflammation grades of chronic hepatitis. The correlation coefficient of PCIII is just lower than HA and PIIINP. So PCIII examination cannot get rid of interference of inflammation. And PIIINP is more sensitive than PCIII in early hepatic fibrosis[40].

As influenced by the metabolism of connective tissue in the body, one index can only reflect one aspect of synthesis or degradation of ECM. So combined test of several indices for the diagnosis of hepatic fibrosis is the way of choice. Use several noninvasive markers combination to diagnose hepatic fibrosis have been reported[37,41]. However, no identical standard has been made in terms of index selecting. Positive rate of HA in the diagnosis was 91%-94% in liver cirrhosis, so it is the most sensitive index to screen hepatic fibrosis and cirrhosis. According to the cut-off point and take HA as the basis index to combined with other indices to diagnose hepatic fibrosis, our study was carried out. From Table 2 tests of HA plus PIIINP or PCIII are the most sensitive combination with rather high specificity. Although other combinations seem to be more specific, their sensitivity is quite low. The sensitivity of our combinations is lower than other report, but the specificity is higher[42]. Reports already published usually took ^-x ± s of serum indices for the diagnosis of chronic active hepatitis. However, using serum markers to diagnose fibrosis caused by chronic hepatitis without liver histological results for control may be over-estimated the sensitivity of the diagnosis. This study divided different groups by stage according to histological results and utilized ROC curves concatenation variable to ascertain cut-off point and in doing so attention was paid to both sensitivity and specificity. One of the reasons of low sensitivity may be due to patient selection for the study among them most were cases of chronic hepatitis. As we only use fibrosis stage S ≥ 1 as grouping standard, the inflammation in the groups between S = 0 and S ≥ 1 may influence the result of sensitivity.

In conclusion, serum HA, PIIINP,PCIII,C-IV and LN levels reflect some aspects of ECM synthesis and degradation. Although serum fibrosis indices can reflect degree of fibrosis impersonally, the stage classification of fibrosis is too simple. And, as a result, fibrosis in the same stage may have considerable difference. So if quantified measurement of liver fiber with computer analysis is done first[43-45], and then followed by using ROC curves to assess the serum fibrosis indices, as a noninvasive diagnosis technique, serum fibrosis indices will have more significant meaning in clinical evaluation of the liver disease.