Development and validation of a prognostic nomogram for HIV/AIDS patients who underwent antiretroviral therapy: Data from a China population-based cohort

Abstract

Background

Accurate forecast of the death risk is crucial to the administration of people living with HIV/AIDS (PLHIV). We aimed to establish and validate an effective prognosis nomogram in PLHIV receiving antiretroviral therapy (ART).

Methods

All the data were obtained from 2006 to 2018 in the Wenzhou area from China AIDS prevention and control information system. Factors included in the nomogram were determined by univariate and multiple Cox proportional hazard analysis based on the training set. The receiver operating characteristic (ROC) and calibration curves were used to assess its predictive accuracy and discriminative ability. Its clinical utility was also evaluated using decision curve analysis (DCA), X-tile analysis and Kaplan-Meier curve, respectively in an independent validation set.

Findings

Independent prognostic factors including haemoglobin, viral load and CD4+ T-cell count were determined and contained in the nomogram. Good agreement between the prediction by nomogram and actual observation could be detected in the calibration curve for mortality, especially in the first year. In the training cohort, AUC (95% CI) and C-index (95% CI) were 0.93 (0.90, 0.96) and 0.90 (0.85, 0.96), respectively. In the validation set, the nomogram still revealed excellent discriminations [AUC (95% CI): 0.95 (0.91, 1.00)] and good calibration [C-index (95% CI): 0.92 (0.82–1.00)]. Moreover, DCA also demonstrated that the nomogram was clinical beneficial. Additionally, participants could be classified into three distinct (low, middle and high) risk groups by the nomogram.

Interpretation

The nomogram presents accurate and favourable prognostic prediction for PLHIV who underwent ART.

Funding

This work was supported by Zhejiang Basic Public Welfare Research Project (LGF19H260011), Wenzhou Basic Public Welfare Research Project (Y20180201), the Initial Scientific Research Fund (KYQD170301), the Major Project of the Eye Hospital Wenzhou the Major Project of the Eye Hospital Wenzhou Medical University (YNZD201602). Part of this work was also funded by National Natural Science Foundation of China (81670777) and Science and Technology Innovation Activity Plan and New Talents Plan for College Students in Zhejiang Province (2019R413073). The funders had no roles in study design, data collection, data analysis, interpretation and writing of the report.

Research in context

Evidence before this study

Although a proportion of people living with HIV (PLHIV) with access to antiretroviral therapy (ART) have a life expectancy similar to the general population, the increased mortality of many PLHIV still can't be ignored and there has been increasing focus on the management of PLHIV. Unfortunately, there is no universal and widely recognized applicable scoring system to predict the mortality of PLHIV who underwent ART. Therefore, we aimed to develop and validate a risk scoring system among patients who underwent ART. This approach would allow us to better identify patients with a high risk of mortality, which would facilitate a more personalized approach to managing these cases.

Added value of this study

We established an effective and accurate nomogram model including haemoglobin (HB), viral load (VL) and CD4+ T-cell count for the evaluation mortality of PLHIV receiving ART in the present study and the superiority of our nomogram also reflects on the clinical utility. This would be very favourable to promote the precise prevention and personalized health management of PLHIV to maximize cost-effectiveness.

Implications of all the available evidence

The nomogram in the present study was completely an optimal prognostic model for evaluating the mortality of PLHIV, containing VL, CD4 and HB, and with satisfying stability and accuracy, for the prediction of PLHIV mortality was successfully developed and carefully evaluated.

1. Introduction

In the past several decades, HIV/AIDS is still one of the leading health problems worldwide [1]. Thanks to the era of universal scaling-up antiretroviral therapy (ART), more and more people living with HIV (PLHIV) nowadays have received ART as suggested by WHO regardless of their CD4+ T-lymphocyte (CD4) counts [1,2]. Although a proportion of PLHIV with access to ART have a life expectancy similar to the general population, the increased mortality of many PLHIV still can't be ignored. So, an in-time accurate forecast of the death risk amongst PLHIV is crucial for clinicians and health care providers to perform effective PLHIV administration [3].

Several studies have shown that some laboratory indicators are closely related to the mortality of PLHIV and other HIV-related comorbidities [4], [5], [6]. For example, Bansi et al. [7] suggest that alanine transaminase (ALT), albumin, alkaline phosphatase (AP) and haemoglobin (HB) to be used to predict mortality risk of PLHIV. Moreover, the platelet-associated index is reported to be closely related to the disease progression of PLHIV and still found to be a predictor of PLHIV combined with community-acquired pneumonia (CAP) mortality [6,8]. In practice, a single indicator is often insufficient in predicting the prognosis of the disease and does not achieve satisfying results. However, the combination of several independent indicators may highly improve predictive effectiveness. Since multiple risk factors of HIV/AIDS-related mortality have been identified, some scoring systems have been formulated to predict mortality [9], [10], [11], unfortunately, there is no universal and widely recognized applicable scoring system to predict the mortality of PLHIV who underwent ART.

In recent years, more and more researches have focused on laboratory testing indicators to predict survival and prognosis of diseases [12,13]. For example, several risk factors have been formulated base on baseline data of PLHIV on ART to assess short-term disease prognosis [14], [15], [16], however, few available predictive models are well validated. As a simpler, more intuitive and advanced approach, nomogram is more and more commonly used in the prognostic analysis. It simplifies the predictive model to the probability of an event. Besides, it also provides a user-friendly interface for clinical usage [17]. Though prognosis nomogram models for predicting the death risk of PLHIV have been developed and validated based on previous studies, the performance is unsatisfied. For instance, Margaret et al. [18] have developed a nomogram model with the concordance index (95% CI) of 0.75 (0.74, 0.81) in the training set and 0.69 (0.59, 0.77) in the validation cohort. This nomogram model can't be unsatisfied, but far from excellent. As far as we know, few previous studies have formulated prognosis models of PLHIV receiving ART with satisfied discrimination and excellent calibration. Depending on common laboratory and clinical indicators of 12 years of follow up from the AIDS Prevention and Control Information System (AIDS-PCIS) in the Wenzhou area, we aimed to develop a simple and effective nomogram scoring system (NSS) to foresee the personalized death risk of PLHIV. To improve the robustness and reliability of our conclusion, a propensity-score matching (PSM) approach was applied at a 1:4 ratio to determine the participants, in which a case (dead PLHIV) was matched by age and gender with 4 controls (alive PLHIV). To make our findings more credible, the enrolled study participants were then randomly split into a training and another separate validation sets, depending on the block, at a ratio of 7:3 without replacement. The association between the presence of death and potential prognostic factors from both clinical and laboratory data were comprehensively analysed within the training set. Besides, the effect on the performance, discrimination and calibration of the nomogram model were thoroughly assessed based on the training and validation set, respectively. Meanwhile, the performance of the NSS has also been assessed by a decision curve analysis. Potential over-fitting has also been well considered in the current study. The individual survival probability of each participant was obtained by the nomogram based on the training set, and extensively validated in a separate validation set.

2. Methods

2.1. Study design

The data used in this study were extracted in 3733 PLHIV from the AIDS Prevention and Control Information System (AIDS-PCIS) [19,20] during the 12 years of follow up in the Wenzhou area. Participation has been approved by the ethics committees of the Wenzhou Centre for Disease Control and Prevention (CDC). All participants identified in the AIDS-PCIS would receive a combination ART regimen containing at least 3 antiretroviral drugs [21]. Follow-up began after starting ART in which each PLHIV was visited by per three months strictly following the current clinical guidelines until December 31, 2018, and the antiviral drugs were distributed free of charge [22]. The inclusion criteria were as follows: 1) Having complete laboratory blood tests before receiving ART; 2) Being visited at least once; 3) Living in Wenzhou area including temporary residents; 4) Over 15 years old; 5) Being identified from January 2006 to December 2018. The survival time was defined as the duration from receiving the first ART to death or December 31, 2018.

At the end of the 12 years’ follow-up, 150 PLHIV died due to HIV/AIDS-related diseases and were determined as the cases in this nested case-control study. To improve the comparison and the stability of the results to some extent, the cases and controls were matched by the age and gender at a ratio of 1:4 using a propensity score matching (PSM) approach [23]. As a result, a total of 750 subjects, consisting of 150 dead and 600 alive PLHIV, were identified in the current study, and classified into 150 blocks, which were then randomly split into a training set and another separate validation set at a ratio of 7:3. The flowchart of recruitment participants is shown in the supplement Fig. S1.

2.2. Clinical information and laboratory reports

Data on demographics and clinical characteristics of participants were collected in a face-to-face investigation manner at their enrolment or extracted from their medical records utilizing a structured questionnaire specifically designed for AIDS-PCIS. The information included age, sex, height, weight, marital status, occupation, history of the sexually transmitted disease (STD), disease stage, the origin of identification, the world health organization (WHO) clinical staging, infection pathway and others. Body mass index (BMI) was calculated using the formula: BMI = weight (kg)/height (m)/height (m).

Information on the laboratory test was obtained from Wenzhou CDC or local hospital. The laboratory testing indicators were consisted of CD4+ T lymphocytopenia (CD4), CD8+ T lymphocytopenia (CD8), viral load (VL), white blood cell (WBC), Platelet (PLT), haemoglobin (HB), serum creatinine (CR), triglyceride (TG), serum total cholesterol (TC), fasting plasma glucose (GLU), alanine aminotransferase (ALT), aspartate transaminase (AST), total bilirubin (TBIL) and others. All of these laboratory assessments were conducted at each visit in the central laboratory of local hospitals or Wenzhou CDC by the same trained technicians strictly following the clinical guidelines.

The routine blood biochemical examination such as TG, TC, GLU, CR, AST, ALT and TBIL were applied using the Beckman AU5800 automatic biochemical analyser (BECKMAN COULTER K., Japan). Other blood tests including WBC, HB and PLT were performed using the Sysmex Xe-2100 automatic blood cell analyzer (SYSMEX CORPORATION, Japan). Moreover, CD4 and CD8 were determined using the BD FACSCalibur flow cytometer (Becton Dickinson Corporation, USA). VL was assessed using nucleic acid sequence-based amplification, NASBA™, NucliSens EasyQ KPC (bioMérieux, France).

2.3. Ethics statement

The data utilized for the present study were extracted from AIDS-PCIS, which was established by National Centre for AIDS/sexually transmitted disease Control and Prevention of Chinese Centre for Disease Control and Prevention (CCDC) to generate the HIV/AIDS epidemic database with continued enrolment of HIV-infected person. The protocol of AIDS-PCIS establishment had been approved by the institutional review boards at CCDC and informed consents were obtained before the subjects’ enrolment. Furthermore, the protocol of the current study was also approved by the ethical review board of Wenzhou Centre for Disease Control and Prevention.

2.4. Statistical analysis

As missing values will lead to bias on the results to some extent, multiple imputations were applied to achieve suitable values for those missing data before the data analysis. A sensitivity analysis was also carried out in the current study to evaluate the effect of missing value filling (Supplement Table S1). To improve the robustness and reliability of our conclusion, the enrolled 150 blocks of participants containing 150 dead and 600 alive PLHIV were split into a training set and another separate validation set in a random manner without replacement at a ratio of 7:3. The comparability of the two sets was then evaluated (Table 1). Continuous variables with normal distribution were presented as the mean ± standard deviation ($\overline{x}$ ± s) and student t-tests were used to infer the differences between the training and validation sets. For those with obviously skewed distribution, median (1st quartile, 3rd quartile) were utilized to describe their features and comparisons in the two sets were carried out with Mann–Whitney U tests. Categorical variables were presented as frequency (proportion) and chi-square or Fisher's exact tests would be applied for their comparisons.

Table 1.

Baseline demographics and clinical characteristics of patients in training cohort and validation cohort.

Variables	Training set (N = 525)	Validation set (N = 225)	P
Discrete variables
Gender			0.119
Men	436(83.0)	197(87.6)
Woman	89(17.0)	28(12.4)
Hepatitis B Virus			0.849
Positive	68(13.0)	28(12.4)
Negative	457(87.0)	197(87.6)
Tuberculosis			0.932
Yes	24(4.6)	9(4.0)
No	488(93.0)	210(93.3)
Missing	13(2.5)	6(2.7)
WHO			0.507
Ⅰ	265(50.5)	125(55.6)
Ⅱ	53(10.1)	17(7.6)
Ⅲ	157(29.9)	65(28.9)
Ⅳ	50(9.5)	18(8.0)
Infection pathway			0.915
NMHR	304(57.9)	134(59.6)
MSM	158(30.1)	65(28.9)
Others	63(12.0)	26(11.6)
Occupation			0.506
Farmer	128(24.4)	62(27.6)
Business people	99(18.9)	45(20.0)
House keeping	124(23.6)	40(17.8)
Worker	73(13.9)	32(14.2)
Others	101(19.2)	46(20.4)
History STD			0.490
Yes	88(16.8)	30(13.3)
No	310(59.0)	137(60.9)
Missing	127(24.2)	58(25.8)
Participants category			0.730
Fixed population	407(77.5)	177(78.7)
Floating population	118(22.5)	48(21.3)
Education			0.097
Illiterate or elementary school	192(36.6)	85(37.8)
Senior middle school	78(14.9)	38(16.9)
Junior high school	207(39.4)	71(31.6)
College and above	48(9.1)	31(13.8)
Marital status			0.523
Married	277(52.8)	113(50.2)
Unmarried	248(47.2)	112(49.8)
Disease stage			0.379
HIV	278(53.0)	127(56.4)
AIDS	247(47.0)	98(43.6)
Origin of identification			0.418
CDC	136(25.9)	49(21.8)
Hospital	317(60.4)	140(62.2)
Others	72(13.7)	36(16.0)
Viral load, copies/mL			0.508
<200	413(78.7)	175(77.8)
200–1000	16(3.0)	4(1.8)
≥1000	96(18.3)	46(20.4)
Continuous variables
Age, year	49.7(37.3,63.5)	48.2(38.2,61.9)	0.624
Body mass index, kg/m2	21.3(19.3,23.7)	21.2(19.1,23.3)	0.347
CD4+ T-lymphocyte count, cells/μL	208.3(86.0,328.0)	203.0(93.9,299.0)	0.430
CD8+ T-lymphocyte count, cells/μL	861.0(535.0,1272.0)	838.0(581.1,1208.0)	0.870
White blood cell, 109/L	5.3(4.2,6.8)	5.1(4.1,6.5)	0.175
Platelet, 109/L	186.0(144.0,224.0)	191.5(145.0,236.0)	0.495
Haemoglobin, g/L	136.7(118.0,149.0)	135.0(119.0,150.0)	0.933
Creatinine, μmol/L	70.0(58.6,81.0)	70.1(58.3,83.0)	0.610
Triglyceride, mmol/L	1.5(1.0,2.4)	1.4(1.0,2.4)	0.576
Total cholesterol, mmol/L	4.1 ± 0.9	4.1 ± 0.9	0.717
Fasting plasma glucose, mmol/L	5.3(4.7,6.6)	5.4(4.6,6.9)	0.916
Aspartate transaminase, U/L	25.0(19.0,34.0)	23.0(19.0,35.0)	0.449
Alanine aminotransferase, U/L	21.6(15.0,34.0)	21.0(15.0,35.0)	0.761
Total bilirubin, μmol/L	10.8(7.7,15.4)	10.3(7.2,15.6)	0.678

Abbreviations: NMHR, Non-marital heterosexual transmission; MSM, men who have sex with men; CDC, centre for disease prevention and control; STD, sexually transmitted disease.

Depending on the training set, univariate and multivariable Cox proportion hazard regression models were used to screen potential prognostic factors and estimate their weights. According to the Akaike information criterion (AIC), we developed a series of different multivariable models. Factors with p-values, which were calculated based on multivariable Cox proportion hazard regression models, over 0.05 were removed from the associated models. Moreover, according to Occam's Law of Razor, the best model for achieving optimal results are models with fewer variables [24]. As a result, a candidate nomogram model with appropriate predictive ability was screened depending on the three most significant risk factors. The predictive performance of nomogram and other models to predict the survival were quantified using the area under the curve (AUC) of the receiver operating characteristic (ROC) analysis and the consistency index (C-index) [25], and comprehensively evaluated based on the training set and validation set, respectively. It was reported that the c-index, a generalization of the AUC, was developed by Royston and Sauerbrei [26] to represent the probability of the concordance between the observed and predicted survival. The performance of the nomogram was also performed with calibration. Moreover, the clinical utilities of the nomogram were also carefully investigated using decision curve analysis (DCA), X-tile analysis and Kaplan-Meier curves in an independent validation set, respectively to make up the limitations of ROC curves, which could not achieve the best sensitivity and specificity at the same time.

All data management and statistical analyses were carried out using Stata/MP 15.1 for windows (Copyright 1985-2017 Stata Corp LLC, College Station, Texas 77845, USA). Figures were drawn with R-Studio for windows (Version 1.2.1335 Copyright 2009–2019 R-Studio, Inc.). All tests were two-sided and P ≤ 0.05 was set as the significant level.

3. Results

3.1. Characteristics of participants

In this PSM based nested case-control study, the characteristics of the 750 PLHIV (525 from the training set and 225 from the validation set) revealed that both sets were similar in all variables (Table 1). During the period of follow-up, a total of HIV/AIDS-related mortality was 73.1 per 1000 person-years in the training set and 72.4 per 1000 person-years in the validation set. The average overall survival for the training and validation sets were 2.73 and 2.76 years, respectively.

3.2. Nomogram screening depending on the training set

On the univariate survival analysis, a total of 13 factors including CD4, VL and HB were detected to be statistically associated with the mortality of PLHIV depending on the univariate survival analysis (Table 2). While in the multivariable Cox proportion hazard regression models in which variables with p-value less than 0.2 were included based on the results of the univariate analysis, we found that only CD4, VL, HB, GLU and CR were directly and independently linked to the HIV/AIDS-related survival time (Table 2). To formulate an optimal nomogram model, the individual and combined performances of these five factors were then comprehensively evaluated using ROC analysis. As could be seen in Fig. 1, the individual AUCs of GLU, CR, CD4, VL and HB were 0.59, 0.51, 0.84, 0.85 and 0.75, respectively. Moreover, ROC curves with the combination1 (CD4, VL, HB, CR, and GLU) and combination2 (CD4, VL, and HB) performed similarly (AUC = 0.94 vs 0.93, χ² = 3.165, p-value = 0.075). To simplify the model by including only those most statistically significant variables, GLU and CR were removed from the model because of their relatively small AUC. So, a nomogram for predicting the mortality of PLHIV was preliminarily constructed with three factors containing CD4, VL, and HB (Figs. 1, 2, and Supplement Table S2). The associated concordance index (c-index) was 0.91 (95% CI, 0.86–0.97), which indicated that about 91% of the probability of individual mortality would be correctly predicted by the nomogram model. Furthermore, as could be seen in Fig. 2, each selected biomarker was assigned a corresponding score according to its value on the nomogram. After calculating the total score, we drew a vertical line using the total score and obtained the individual probability of PLHIV survival. The associated scores for the independent risk factors calculated by the nomogram on the corresponding situation were presented in Supplement Table S2.

Table 2.

Univariate and multivariable Cox hazards analysis of the training cohort.

Variables	Univariate		Multivariate
	HR(95% CI)	P-value	AHR(95% CI)	P-value
Statistically significant factors
WHO
Ⅰ	Ref	Ref	Ref	Ref
Ⅱ	1.442(0.680,3.059)	0.340	1.376(0.569,3.330)	0.478
Ⅲ	2.698(1.675,4.348)	<0.001	1.596(0.944,2.698)	0.081
Ⅳ	5.113(2.977,8.782)	<0.001	1.870(0.997,3.507)	0.051
Disease stage (HIV)	0.109(0.055,0.216)	<0.001	0.832(0.331,2.094)	0.697
Origin of identification
Hospital	Ref	Ref	Ref	Ref
CDC	0.407(0.234,0.706)	0.001	0.648(0.354,1.185)	0.159
Others	0.461(0.231,0.919)	0.028	0.577(0.256,1.305)	0.187
CD4+ T-lymphocyte count, cells/μL	0.990(0.988,0.992)	<0.001	0.995(0.992,0.999)	0.007
CD8+ T-lymphocyte count, cells/μL	0.999(0.998,0.999)	<0.001	1.000(0.999,1.000)	0.134
Viral load, copies/mL	4.882(3.881,6.142)	<0.001	4.025(3.029,5.348)	<0.001
Platelet, 109/L	1.003(1.001,1.005)	0.006	1.001(0.998,1.004)	0.412
Haemoglobin, g/L	0.970(0.963,0.978)	<0.001	0.988(0.979,0.998)	0.014
Aspartate transaminase, U/L	1.005(1.002,1.009)	0.004	0.993(0.984,1.003)	0.159
Body mass index, kg/m2	0.893(0.837,0.952)	0.001	1.019(0.950,1.094)	0.593
Total cholesterol, mmol/L	0.547(0.440,0.681)	<0.001	1.006(0.794,1.273)	0.964
Total bilirubin, μmol/L	1.010(1.001,1.019)	0.031	1.018(0.998,1.038)	0.077
Fasting plasma glucose, mmol/L	1.021(1.003,1.040)	0.023	1.064(1.027,1.101)	<0.001
Statistically non-significant factors
Woman	0.947(0.564,1.592)	0.838
Age	0.997(0.985,1.008)	0.591
TB	1.686(0.848,3.353)	0.136	2.115(0.944,4.738)	0.069
HBV(Negative)	0.819(0.449,1.494)	0.514
Infection pathway
NMHR	Ref	Ref	Ref	Ref
MSM	0.530(0.323,0.869)	0.012	1.046(0.581,1.883)	0.881
Others	0.728(0.395,1.341)	0.309	0.688(0.337,1.403)	0.303
Occupation			0.941(0.828,1.070)	0.356
Farmer	Ref	Ref	Ref	Ref
Business people	0.308(0.142,0.669)	0.003	0.385(0.157,0.941)	0.036
House keeping	0.881(0.514,1.510)	0.645	1.807(1.002,3.259)	0.049
Worker	0.909(0.492,1.681)	0.762	1.224(0.625,2.398)	0.556
Others	1.039(0.623,1.735)	0.882	0.901(0.508,1.596)	0.721
History STD	0.761(0.434,1.335)	0.341
Participants category	0.998(0.644,1.548)	0.993
Education	1.052(0.909,1.218)	0.495
Illiterate or elementary school	Ref	Ref
Senior middle school	0.631(0.269,1.480)	0.290
Junior high school	0.915(0.517,1.619)	0.760
College and above	1.011(0.656,1.557)	0.962
Marital status	1.369(0.932,2.011)	0.110	0.764(0.483,1.209)	0.250
White blood cell, 109/L	0.992(0.924,1.065)	0.831
Creatinine, μmol/L	1.009(0.998,1.019)	0.102	1.013(1.002,1.024)	0.023
Alanine aminotransferase, U/L	1.003(0.998,1.007)	0.235
Triglyceride, mmol/L	1.024(0.890,1.179)	0.738

Abbreviations: NMHR, Non-marital heterosexual transmission; MSM, men who have sex with men; CDC, center for disease prevention and control; STD, sexually transmitted disease.

Fig. 1.

The ROC curve of the prognostic nomogram, CD4, VL, HB, CR, GLU and combined group in the training and validation set.

Notes: The ROC curve for training cohort (a) and validation set (b).

Abbreviations: VL, viral load; HB, haemoglobin; GLU, fasting plasma glucose; Combine, CD4, VL, HB, CR and GLU; CR, creatinine; ROC, receiver operating characteristic.

Fig. 2.

Nomogram of laboratory index for predicting HIV/AIDS-related survival in the initiated of ART.

Note: The scores of each variable were added to obtain the total score, and a vertical line was drawn on the total score to obtain the corresponding probability of death.

Abbreviations: HB, haemoglobin.

3.3. Validation of the nomogram based on the validation set

To verify the efficacy of the nomogram to predict survival of PLHIV, we conducted comprehensive validations of the nomogram based on the validation set as follows: First, we determined the c-index with 0.92 (95% CI, 0.82–1.00). Second, as could be seen in Fig. 1, its associated AUC in the ROC analysis achieved as high as 0.95 (95% CI, 0.91–1.00). Third, calibration curves also displayed high consistency in the prediction of PLHIV's survival time, especially in the first year after receiving ART (Fig. 3).

Fig. 3.

Calibration curves for predicting overall survival rate by the nomogram in the training and validation set.

Notes: Calibration curves of the prognostic nomogram for 1-year overall survival (a), 2-year overall survival (b) and 3-year overall survival (c) in the training set; calibration curves for 1-year overall survival (d), 2-year overall survival (e), and 3-year overall survival (f) in the validation set.

3.4. Comparison between nomogram and laboratory test indicators

To further validate and compare the superiority of the laboratory indicators and the nomogram in assessing the survival of PLHIV, we plotted the ROC curves of CD4, HB, GLU, VL, CR, a full combined model containing these five indicators together and the nomogram (Fig. 1) in the training set and validation set, respectively. The ability of each model was assessed by AUC (95% CI). Among them, the AUCs for the nomogram in either the training set (AUC = 0.93) or the validation set (AUC = 0.95) were all significantly higher than those of each laboratory indicator (P < 0.05), which reflects the high diagnostic value of the nomogram model. Moreover, no significant differences in the AUCs between the nomogram and the full combined model were observed in the training set (AUC = 0.93 vs 0.94) and validation set (AUC = 0.95 vs 0.95). So, to simplify the model and achieve the same prediction effect simultaneously, we determined the final nomogram as the model containing VL, CD4 and HB (Fig. 2).

3.5. DCA curve analysis

As was shown in Fig. 4, whether in the training set or validation set, the nomogram and the full combined model all performed outstandingly in various predictors regardless of the threshold, which ensured to achieve maximum clinical benefit. Overall, the DCA curve indicated that a nomogram was feasible to make valuable and profitable judgments. Furthermore, among the three detected factors included in the nomogram, we also observed that VL was more beneficial than the other two routine clinical laboratory indicators in the prediction of PLHIV survival.

Fig. 4.

The Decision Curves Analysis curve of the prognostic nomogram, CD4, VL, HB and combined group in the training and validation set.

Notes: The DCA curve for Training cohort (a) and validation set (b). Combined group: CD4, VL, HB, CR and GLU; the horizontal axis represents the threshold value, which is the reference probability of whether a patient receives treatment, and the vertical axis represents the net benefit rate after the advantages minus the disadvantages. Under the same threshold probability, the larger net benefit implies that patients can obtain the maximum benefit using the diagnosis of this model. The closer the curve in the DCA graph is to the top, the higher the value of the model diagnosis will be.

Abbreviations: DCA, decision curve analysis.

3.6. Performance of the nomogram in stratifying risk of patients

In the training set, the total prognostic scores calculated by the nomogram were categorized into three risk groups to predict mortality: low-risk (<112), moderate-risk (112∼) and high-risk (≥166) following the cut-off points detected by the X-tile analysis (Fig. 5). As compared to the low- or moderate risk population, HR (95% CI) for the moderate- and high-risk categories were 11.33 (5.49, 23.38) and 6.23 (4.11, 9.44), respectively. In the validation set, the comparisons of the cumulative probability of survival in each category could be found in Fig. 6. The HR (95% CI) for the moderate and high-risk categories were 20.50 (4.73–88.81) and 11.12 (5.53–22.36) as compared to people with the low- or moderate risk. The Kaplan-Meier curves in both the training and validation sets all clearly disclosed that the nomogram was stable to forecast the probability of survival in PLHIV after receiving ART.

Fig. 5.

X-tile analysis of the total risk score in the train cohort.

Fig. 6.

Survival curves stratified by the score calculated by the nomogram scoring system [low-risk (<112), moderate-risk (112∼) and high-risk (≥166)] in the validation cohort.

4. Discussion

Although PLHIV may live longer than before with the widespread usage of ART, accurate prediction of the survival for PLHIV remains to be necessary and a major challenge for health-care providers since it can effectively improve PLHIV's survival. Besides, a prediction helps develop appropriate healthcare for PLHIV and guidelines for HIV/AIDS administration [27]. For clinicians or disease control staff, it is also highly beneficial in the personalized administration of PLHIV and optimizing the delivery of limited health resources [28]. Based on this nested case-control study from a HIV/AIDS cohort in the Wenzhou area, we comprehensively assessed the relationship between some routine laboratory test markers and the mortality of PLHIV. An optimal prognostic nomogram model, containing VL, CD4 and HB, and with satisfying stability and accuracy, for the prediction of PLHIV mortality was successfully developed and carefully evaluated.

Available pieces of evidence suggest that a nomogram can be effective in predicting the prognosis of patients and it's simple and intuitive features are easy for clinical staff to interpret [29,30]. To the best of our knowledge, the present study is the first report on developing a nomogram to predict the survival probability for PLHIV using some baseline laboratory testing indicators. In the multivariable Cox proportion hazard regression model, we observed that CD4, HB and VL were all independently associated with the mortality of PLHIV. When the three variables mentioned above were incorporated into a nomogram model, we achieved a bootstrapped corrected concordance index as 0.914 in the training set. A satisfying agreement with good calibration (c-index = 0.917) and high accuracy (AUC = 0.95) was also observed in the independent validation set.

In the present study, the nomogram did not include clinical signs and symptoms such as the WHO clinical stage, which had been reported to be associated with the mortality of PLHIV. The possible reason was that the laboratory indicators were more sensitive in predicting mortality than those clinical signs and symptoms. A previous study has shown that establishing a haematological index to assess the status of disease progression in PLHIV may be more useful to clinicians than available methods [31]. In recent years, many studies have reported that some laboratory tests are closely related to the death of PLHIV. Engsig et al. [32] found that virally suppressed HIV-positive individuals on ART who do not achieve a CD4 count >200 cells/µL have substantially increased long-term mortality. Shoko et al. [33] also suggested that VL was a better predictor to forecast PLHIV mortality than CD4. A strong association of CD4 at the start of therapy with subsequent survival in PLHIV was also detected [34]. Meanwhile, HB was also independently associated with the survival and progression of PLHIV [7]. Fortunately, our results disclosed that VL, CD4 and HB were also significantly linked to the survival of PLHIV and well consistent with these published findings. Elevated VL and decreased CD4 levels were markers of immune dysregulation and ongoing inflammatory processes, which might induce increased mortality of PLHIV. As can be seen in Supplement Fig. S2, the higher level of VL the lower survival probability of PLHIV (χ² = 380.358, P < 0.001). Meanwhile, we also find that elevated CD4 level is significantly associated with reduced death risk of PLHIV (Supplement Fig. S3a), which is consistent with previous reports [35]. Furthermore, haemoglobin level was also observed to be independently associated with the mortality of PLHIV (Supplement Fig. S3b). Possible explanations are as follows: (1) Chronic inflammation contributes to the development of anaemia is observed not only in the general population but also in PLHIV. Previous reports suggest that anaemia is associated with the mortality of PLHIV because of the haematological abnormality [36], [37], [38], [39], [40], [41]. (2) PLHIV with long-term anaemia is commonly due to an unhealthy lifestyle, more likely to have poor treatment compliance, which will largely increase the death risk to some extent. Although ART is an effective measure to reduce the prevalence of anaemia, a large number of PLHIV are still in the trouble of unresolved anaemia persistently or developing to anaemia [42], this is consistent with our findings. Furthermore, our findings also revealed that VL was the most important predictor with an excellent predictive ability (AUC = 0.8536). It also proved that HIV RNA would be an optimal predictor of long-term mortality than other laboratory markers [33]. However, as a single predictor is highly susceptible to offset, its prediction accuracy would be ineffective, which would greatly limit its results extrapolation [21]. To overcome or avoid the limitation of a single predictor and achieve higher predictive accuracy, we combined three detected predictors to construct a nomogram model in the present study. Our data confirmed that the nomogram was much better in the prediction of PLHIV survival than that of any single predictor in both the training set (AUC = 0.93) and the validation set (AUC = 0.95). Besides, the DCA curve has been commonly used to evaluate the efficacy of specific clinical approaches in many studies [43,44]. In the present study, we also used the DCA curve to check out the potential clinical effects of the nomogram and our findings showed that the nomogram was more useful than any other indicator in predicting HIV/AIDS related mortality.

Several strengths could be found in the present study. First, our data suggested that the nomogram was a good option to effectively forecast the death risk of PLHIV. Second, to obtain better performance in the actual prediction, we also calculated a total score of the death risk by the nomogram. PLHIV could be classified into different categories according to the cut-off points determined by X-tile analysis. This would be very favourable to promote the precise prevention and personalized health management of PLHIV to maximize cost-effectiveness. Furthermore, our findings came from a PSM-based nested case-control study, in which the cases and controls originated from the same population-based cohort. This study design would be advantageous to reduce the selection bias, improve statistical efficiency, and make our conclusion more robust and reliable.

Limitations also existed in this study. First, the nomogram model was established mainly based on the baseline level of VL, CD4 and HB. As the levels of these three laboratory markers could not maintain stable throughout the whole follow-up period, it would partly affect the accuracy of the prediction, and perhaps the reason of the differences between the predicted and observed mortalities, especially in the 2- or 3- year survivals after receiving ART (Fig. 3). Fortunately, the differences between the predicted 1-year mortality and the observed one in both the training and validation sets were very similar, which indicated that the baseline laboratory marker-based nomogram was more suitable for the prediction of short-term death risk in PLHIV after receiving ART. Second, several ARTs had been provided to the identified PLHIV during the study period. This would also affect the performance of the nomogram to some extent. As we know, China carried out free ART strategies to HIV/AIDS throughout their lives. However, as the largest developing country with a large number of PLHIV, China could not afford too many kinds of ART strategies, and the commonly used antiviral drugs were limited, which would reduce the impacts due to ART strategies on our findings to some extent. Third, the participants were only from Wenzhou, which might limit the extrapolation of our findings to the whole PLHIV.

In conclusion, our findings suggest that the nomogram constructed by routine laboratory markers can serve accurate and favourable prognostic prediction in the mortality of PLHIV. This may be very beneficial in promoting the precise prevention and personalized administration of PLHIV.

CRediT authorship contribution statement

Xiangqing Hou: Conceptualization, Formal analysis, Writing - original draft, Writing - review & editing, Data curation. Dayong Wang: Conceptualization, Writing - original draft, Data curation. Jingjing Zuo: Data curation. Jushuang Li: Data curation. Tao Wang: Data curation. Chengnan Guo: Data curation. Fang Peng: Writing - review & editing, Data curation. Dehua Su: Data curation. Lina Zhao: Data curation. Zhenmiao Ye: Data curation. Hemei Zhang: Data curation. Chao Zheng: Data curation. Guangyun Mao: Conceptualization, Formal analysis, Writing - original draft, Data curation.

Declaration of Competing Interest

All authors declare no potential conflicts of interest.

Data for reference

The datasets generated and/or analysis during the current study are not publicly available due to protect the privacy of people living with HIV/AIDS but are available from corresponding author on reasonable request.