Development and validation of risk models to predict the 7‐year risk of type 2 diabetes: The Japan Epidemiology Collaboration on Occupational Health Study

Abstract

Aims/Introduction

We previously developed a 3‐year diabetes risk score in the working population. The objective of the present study was to develop and validate flexible risk models that can predict the risk of diabetes for any arbitrary time‐point during 7 years.

Materials and Methods

The participants were 46,198 Japanese employees aged 30–59 years, without diabetes at baseline and with a maximum follow‐up period of 8 years. Incident diabetes was defined according to the American Diabetes Association criteria. With routine health checkup data (age, sex, abdominal obesity, body mass index, smoking status, hypertension status, dyslipidemia, glycated hemoglobin and fasting plasma glucose), we developed non‐invasive and invasive risk models based on the Cox proportional hazards regression model among a random two‐thirds of the participants, and used another one‐third for validation.

Results

The range of the area under the receiver operating characteristic curve increased from 0.73 (95% confidence interval 0.72–0.74) for the non‐invasive prediction model to 0.89 (95% confidence interval 0.89–0.90) for the invasive prediction model containing dyslipidemia, glycated hemoglobin and fasting plasma glucose. The invasive models showed improved integrated discrimination and reclassification performance, as compared with the non‐invasive model. Calibration appeared good between the predicted and observed risks. These models performed well in the validation cohort.

Conclusions

The present non‐invasive and invasive models for the prediction of diabetes risk up to 7 years showed fair and excellent performance, respectively. The invasive models can be used to identify high‐risk individuals, who would benefit greatly from lifestyle modification for the prevention or delay of diabetes.

Introduction

Type 2 diabetes affects various populations around the world1. Globally, the number of adults with diabetes was estimated to 415 million in 2015, and is projected to increase by 55%, to a total of 642 million in 20401. Japan is one of the top 10 countries with the highest number of adults with type 2 diabetes1. Its prevalence has been projected to rise from 7.9% in 2010 to 9.8% by 2030 in the Japanese adult population2. To combat the increasing burden of diabetes and its complications, identifying high‐risk individuals is important in the prevention of diabetes or delaying its progression.

More than 100 risk assessment tools were developed worldwide to identify people at the risk of developing diabetes3, 4. However, these risk models might not be applied to external populations, particularly if ethnicities and countries differ from the derivation populations3, 4. In Japan, a few risk models have been developed5, 6, 7, 8, 9, 10 using data from health checkups at hospital5, 6, 8 or local community7, 9 settings. Among these, some were developed utilizing a small sample (n < 2,000)6, 7, and excluded individuals aged >40 years7, 8, 9. Furthermore, some models included variables that were not routinely collected at regular health checkups (e.g., family health history and exercise)5, 6, 7, 8, limiting the wider use of these prediction tools.

Using checkup data of the Japan Epidemiology Collaboration on Occupational Health (J‐ECOH) Study, we previously developed a 3‐year diabetes risk score10. The risk score, however, can only predict risk in a short‐ and fixed‐time period. To overcome this limitation, the present study aimed to develop and validate non‐invasive and invasive risk prediction models that can more flexibly predict the risk of diabetes at any time‐point within 7 years, based on the J‐ECOH Study data with an extended follow‐up period. We also created risk calculators and charts to make these models easier to use in practice.

Methods

The J‐ECOH Study is an ongoing cohort study among workers from 12 companies in Japan, and has been described in our previous studies10, 11, 12. Briefly, participants in the J‐ECOH Study underwent a health examination each year under the Industrial Safety and Health Act. They underwent anthropometric measurements, physical examination and laboratory examination (blood sugar, blood lipids, etc.) at annual health examinations. Additionally, a questionnaire that covered medical history, health‐related lifestyle and work environment was completed. So far, the annual health examination data between 2008 and 2016 have been collected from 11 companies.

The J‐ECOH Study was announced in each company using posters. Verbal or written informed consent was not obtained, but the participants were given the opportunity to refuse to participate, according to the Japanese Ethical Guidelines for Epidemiological Research13. The study obtained ethics approval from the ethics committee of the National Center for Global Health and Medicine, Japan.

In the present study, the baseline data mainly comprised data from the 2008 health checkup. If the 2008 dataset had large amounts of missing data, then the data collected for the 2009 or 2010 (two companies) health checkups were treated as the baseline data. The outcome was ascertained using the annual health examination data after the baseline examination through March 2016.

Participants

Of the 75,857 participants aged 30–59 years, we excluded people who self‐reported receiving treatment for diabetes (n = 2,496), lacked data on diabetes treatment status (n = 1,171), blood glucose (n = 6,064), glycated hemoglobin (HbA1c; n = 566) or had blood drawn while they were non‐fasted (n = 7,218) at baseline. Furthermore, we excluded people with fasting plasma glucose (FPG) ≥126 mg/dL (n = 1,570) or HbA1c ≥6.5% (n = 599) at baseline. Participants with self‐reported cancer (n = 484) or cardiovascular disease (n = 599) at baseline were also excluded. Of the remaining 55,090 participants, we excluded those with the following missing variables used in developing the risk prediction model for diabetes: smoking status, waist circumference, body mass index (BMI), hypertension status and dyslipidemia status (n = 7,000). After further excluding participants without subsequent health checkups (n = 1,794) or who attended but received neither glucose measurement nor HbA1c measurement (n = 98), 46,198 participants, comprising 39,276 men and 6,922 women, remained.

Two‐thirds of the eligible participants stratified by worksite and sex were randomly allocated to the derivation cohort (25,927 men and 4,573 women), saving the remaining one‐third for the validation cohort (13,349 men and 2,349 women). The derivation cohort was used to derive risk models for estimating diabetes risk and validated using the validation cohort.

Predictor variables

We selected and categorized the following predictor variables as we did for predicting the 3‐year diabetes risk10: sex, age (30–39, 40–49 or 50–59 years), BMI (<21, 21–<23, 23–<25, 25–<27, 27–<29 or ≥29 kg/m²), abdominal obesity (waist circumference ≥90 cm for men and ≥80 cm for women), smoking status (never, former or current), hypertension status, dyslipidemia status, FPG level (<100, 100–<110 or 110–<126 mg/dL) and HbA1c level (<5.6, 5.6–<6.0 or 6.0–<6.5%). In a sensitivity analysis, BMI and age were treated as continuous variables. Data collection methods, which have been described in detail in previous papers10, 11, 12, are provided in the Appendix S1.

Outcome

Incident diabetes was ascertained using the data obtained from annual health checkups after the baseline health checkup. Diabetes was defined as a FPG level of at least 126 mg/dL or a random plasma glucose level of at least 200 mg/dL, an HbA1c level of at least 6.5%, or receiving antidiabetic treatment14. Participants were considered to have type 2 diabetes if they met the above definition of diabetes.

Statistical analysis

Characteristics of participants were expressed as percentages and means for categorical and continuous variables, respectively. The χ‐test for categorical variables and t‐test for continuous variables were used to examine the differences in baseline characteristics between participants in the derivation and validation cohorts.

The 7‐year risk prediction models of diabetes were developed using the Cox proportional hazards regression analysis, with a backward selection procedure to determine predictors (P < 0.05). The coefficients of each predictor and baseline survivor function were used to develop risk models, as in other studies15, 16. We initially developed a non‐invasive prediction model (containing sex, age, abdominal obesity, BMI, smoking status and hypertension status), and subsequently the invasive prediction models (containing dyslipidemia, either HbA1c or FPG, or both).

Predictive performance of prediction models was assessed by examining measures of discrimination and calibration. Discrimination is the ability of the risk model to differentiate between people who develop diabetes during the study and those who do not. This measure is quantified by calculating the time‐dependent area under receiver operating characteristic (ROC) curve (AUROC). In addition, integrated discrimination improvement and net reclassification improvement were computed to show the improved performance of the invasive models as compared with the non‐invasive model for predicting diabetes17. Calibration refers to the agreement between the predicted and observed 7‐year risk of diabetes. This was assessed for each decile of predicted risk by plotting the observed risk vs the predicted risk18, 19. More spread between the deciles was associated with a better discriminating model. Finally, discrimination and calibration of the prediction models were assessed in the validation cohort to check internal validity. Furthermore, risk calculators and charts (see Figures [Link], [Link], [Link], [Link], [Link]) were created using these models.

All statistical analyses were carried out using SAS version 9.3 (SAS Institute, Cary, NC, USA). A two‐sided P < 0.05 was considered statistically significant.

Results

In the derivation cohort, 2,216 participants (2,055 men and 161 women) developed diabetes during follow up. In the validation cohort, 1,169 participants (1,085 men and 84 women) developed diabetes. The incidence rates of diabetes were 12.5 and 12.8 per 1,000 person‐years, respectively. Table 1 shows that the mean age, waist circumference, FPG and HbA1c, as well as the prevalence of smoking, hypertension and dyslipidemia showed no significant difference between the validation and derivation cohorts.

Table 1.

Baseline characteristics of study participants in the derivation and validation cohorts, Japan Epidemiology Collaboration on Occupational Health Study, 2008–2015

Characteristics	Derivation cohort	Validation cohort	P‐value
No. participants	30,500	15,698
Age (years)	45.4 ± 7.7	45.5 ± 7.6	0.09
Women (%)	15.0	15.0	0.93
BMI (kg/m2)	23.3 ± 3.2	23.2 ± 3.2	0.03
Waist circumference (cm)	82.3 ± 8.8	82.2 ± 8.9	0.24
Smoking status (%)
Current smoker	36.7	37.3	0.41
Past smoker	20.6	20.2
Never smoker	42.7	42.5
Hypertension (%)	18.2	18.2	0.99
Dyslipidemia (%)	44.4	43.8	0.17
FPG (mg/dL)	96.5 ± 9.0	96.4 ± 9.0	0.37
HbA1c (%)	5.5 ± 0.4	5.5 ± 0.4	0.74

Data are mean ± standard deviation unless otherwise indicated. BMI, body mass index; FPG, fasting plasma glucose; HbA1c, glycated hemoglobin.

Table 2 shows the coefficients associated with each predictor of diabetes. The non‐invasive prediction model revealed that increased risk of diabetes is associated with sex (male), higher BMI, older age, abdominal obesity, smoking and hypertension. By contrast, the invasive prediction model containing dyslipidemia, HbA1c and FPG showed that the coefficients associated with older age, higher BMI and hypertension attenuated, sex and abdominal obesity were no longer related with the risk of diabetes. Thus, sex and abdominal obesity were excluded from this model.

Table 2.

Multivariate regression coefficients (standard errors) of diabetes risk prediction models in the derivation cohort, Japan Epidemiology Collaboration on Occupational Health Study, 2008–2015

	No. participants	No. cases	Non‐invasive model		Invasive model including FPG		Invasive model including HbA1c		Invasive model including FPG and HbA1c
			β (SE)	P	β (SE)	P	β (SE)	P	β (SE)	P
Sex
Women	4,412	161	Reference
Men	23,872	2,055	0.365 (0.089)	<0.01	–	–	–	–	–	–
Age (years)
30–<40	8,569	390	Reference		Reference		Reference		Reference
40–<50	11,380	842	0.380 (0.061)	<0.01	0.157 (0.062)	<0.01	0.131 (0.062)	<0.01	0.063 (0.063)	<0.81
50–<60	8,335	984	0.961 (0.062)	<0.01	0.447 (0.063)	<0.01	0.446 (0.063)	<0.01	0.248 (0.063)	<0.01
BMI (kg/m2)
<21	6,819	201	Reference		Reference		Reference		Reference
21–<23	7,565	395	0.396 (0.087)	<0.01	0.167 (0.086)	<0.01	0.304 (0.088)	<0.01	0.121 (0.087)	0.12
23–<25	6,927	525	0.677 (0.085)	<0.01	0.297 (0.083)	<0.01	0.434 (0.086)	<0.01	0.181 (0.084)	0.03
25–<27	3,987	484	1.016 (0.092)	<0.01	0.525 (0.087)	<0.01	0.611 (0.089)	<0.01	0.320 (0.089)	<0.01
27–<29	1,819	274	1.152 (0.111)	<0.01	0.694 (0.098)	<0.01	0.694 (0.099)	<0.01	0.464 (0.099)	<0.01
≥29	1,167	337	1.715 (0.114)	<0.01	1.131 (0.099)	<0.01	1.019 (0.099)	<0.01	0.758 (0.101)	<0.01
WC† (cm)
<90	22,500	1,310	Reference
≥90	5,784	906	0.182 (0.065)	<0.01	–	–	–	–	–	–
Smoking status
Never smoker	12,301	718	Reference		Reference		Reference		Reference
Past smoker	5,767	523	0.162 (0.060)	<0.01	0.044 (0.059)	0.34	0.199 (0.059)	<0.01	0.074 (0.059)	0.16
Current smoker	10,216	975	0.325 (0.051)	<0.01	0.356 (0.051)	<0.01	0.202 (0.051)	<0.01	0.221 (0.052)	<0.01
Hypertension
No	23,475	1,467	Reference		Reference		Reference		Reference
Yes	4,809	749	0.471 (0.049)	<0.01	0.251 (0.050)	<0.01	0.550 (0.050)	<0.01	0.375 (0.051)	<0.01
Dyslipidemia
No	16,164	782			Reference		Reference		Reference
Yes	12,120	1,434			0.325 (0.047)	<0.01	0.208 (0.048)	<0.01	0.158 (0.049)	<0.01
FPG (mg/dL)
<100	19,783	437			Reference				Reference
100–<110	6,947	682			1.257 (0.063)	<0.01			0.937 (0.065)	<0.01
110–<126	1,554	1,097			2.950 (0.062)	<0.01			2.221 (0.068)	<0.01
HbA1c (%)
<5.6	16,589	300					Reference		Reference
5.6–<6.0	10,138	842					1.348 (0.068)	<0.01	1.019 (0.069)	<0.01
6.0–<6.5	1,557	1,074					3.083 (0.069)	<0.01	2.295 (0.076)	<0.01

In invasive models, sex and waist circumference (WC) were not statistically significant, and thus were excluded. ^†WC was 80 cm for women. BMI, body mass index; FPG, fasting plasma glucose; HbA1c, glycated hemoglobin; SE, standard error.

The time‐dependent ROC curve of risk models for predicting the development of diabetes within 7 years are shown in Figure 1. The AUROC in the derivation cohort increased from 0.73 (95% confidence interval [CI] 0.72–0.74) for the non‐invasive prediction model to 0.89 (95% CI 0.89–0.90) for the prediction model containing both HbA1c and FPG. When age and BMI were treated as continuous variables, the predictive performance was similar, with an AUROC of 0.74 (95% CI 0.73–0.75) for the non‐invasive prediction model, and 0.89 (95% CI 0.89–0.90) for the prediction model containing both HbA1c and FPG.

Figure 1.

Receiver operating characteristic curves for each risk model in predicting 7‐year diabetes risk, Japan Epidemiology Collaboration on Occupational Health Study, 2008–2015. (a) In the derivation cohort, the area under the receiver operating characteristic curve were 0.73 (95% confidence interval [CI] 0.72–0.74) for the non‐invasive model, 0.86 (95% CI 0.85–0.87) for the model including fasting plasma glucose (FPG), 0.85 (95% CI 0.84–0.86) for the model including glycated hemoglobin (HbA1c) and 0.89 (95% CI 0.89–0.90) for the model including both FPG and HbA1c. (b) In the validation cohort, the corresponding values were 0.73 (95% CI 0.68–0.77) for the non‐invasive model, 0.86 (95% CI 0.82–0.89) for the model including FPG, 0.85 (95% CI 0.82–0.88) for the model including HbA1c and 0.90 (95% CI 0.87–0.92) for the model including both FPG and HbA1c.

The invasive models showed improved integrated discrimination and reclassification performance, as compared with the non‐invasive prediction model (Table 3). The net reclassification improvement was 0.50 (95% CI 0.47–0.53) for the prediction model containing HbA1c, 0.56 (95% CI 0.53–0.59) for the prediction model containing FPG, and 0.74 (95% CI 0.71–0.77) for the model containing both HbA1c and FPG, as referenced to the non‐invasive prediction model. With regard to integrated discrimination improvement, the values were 0.17 (95% CI 0.16–0.18) for the prediction model containing HbA1c, 0.18 (95% CI 0.17–0.19) for the prediction model containing FPG and 0.26 (95% CI 0.25–0.27) for the model containing both HbA1c and FPG. Calibration appeared good between predicted risk and observed risk (Figure 2).

Table 3.

Discriminative ability of invasive risk models in comparison with the non‐invasive model in the derivation and validation cohorts, Japan Epidemiology Collaboration on Occupational Health Study, 2008–2015

	NRI (95% CI)	IDI (95% CI)
Derivation cohort
Non‐invasive model	Reference	Reference
Invasive model including HbA1c	0.50 (0.47–0.53)	0.17 (0.16–0.18)
Invasive model including FPG	0.56 (0.53–0.59)	0.18 (0.18–0.19)
Invasive model including HbA1c and FPG	0.74 (0.71–0.77)	0.26 (0.25–0.27)
Validation cohort
Non‐invasive model	Reference	Reference
Invasive model including HbA1c	0.46 (0.42–0.51)	0.16 (0.15–0.17)
Invasive model including FPG	0.53 (0.49–0.59)	0.17 (0.16–0.19)
Invasive model including HbA1c and FPG	0.71 (0.66–0.76)	0.24 (0.23–0.26)

FPG, fasting plasma glucose; HbA1c, glycated hemoglobin; IDI, integrated discrimination improvement; NRI, net reclassification improvement.

Figure 2.

Calibration plot for type 2 diabetes risk models in the derivation cohort, by deciles of predicted risk, Japan Epidemiology Collaboration on Occupational Health Study, 2008–2015. (a) Non‐invasive risk model. (b) Invasive risk model including both glycated hemoglobin and fasting plasma glucose.

These prediction models performed well in the validation cohort, with an AUROC of 0.73 (95% CI 0.68–0.77) for the non‐invasive prediction model and 0.89 (95% CI 0.87–0.92) for the prediction model containing both HbA1c and FPG (Figure 1). The calibration plots also showed a good agreement between the predicted and observed risks (Figure 3).

Figure 3.

Calibration plot for type 2 diabetes risk models in the validation cohort, by deciles of predicted risk, Japan Epidemiology Collaboration on Occupational Health Study, 2008–2015. (a) Non‐invasive risk model. (b) Invasive risk model including both glycated hemoglobin and fasting plasma glucose.

Discussion

Based on a large‐scale working population‐based cohort study in Japan, two types of models were developed to predict the risk of diabetes within 7 years: the non‐invasive prediction model (containing sex, age, abdominal obesity, BMI, smoking status and hypertension status) and the invasive prediction models (containing dyslipidemia, either HbA1c or FPG, or both). The non‐invasive prediction model showed a fair performance for predicting diabetes, whereas the invasive prediction models showed excellent performance. These prediction models also performed well in the validation cohort.

We previously reported that a 3‐year diabetes risk score was developed based on the logistic regression models10. In the same study population with extended follow up, the risk models were developed to predict the 7‐year diabetes risk using the Cox proportional hazards regression model to account for loss to follow up. The prediction models in the present study can also be used to predict the 3‐year diabetes risk by replacing the value of the baseline survival function at 7 years with the value at 3 years. The performance of our models in predicting the 3‐year diabetes risk (data not shown in the table; an AUROC of 0.74 for the non‐invasive prediction model and 0.91 for the invasive prediction model containing both HbA1c and FPG) was slightly improved, as compared with the previous 3‐year diabetes risk score (an AUROC of 0.72 for the non‐invasive prediction model and 0.89 for the invasive prediction model containing both HbA1c and FPG)10. We also created risk calculators and charts, useful in estimating the future risk of diabetes. Taken together, the present risk models have more utilities than our previous ones10.

The non‐invasive prediction model showed fair predictive ability, with an AUROC of 0.73, which was within the reported range based on previous studies carried out in Japan (AUROC ranged between 0.68 and 0.77)5, 6, 7, 8 and other countries (AUROC ranged between 0.62 and 0.87)4. As expected, our invasive model including both HbA1c and FPG showed a convincing performance for predicting diabetes. The AUROC value (0.89) was equal to or greater than that in the previously published models including both HbA1c and FPG, which ranged from 0.80 to 0.895, 8, 20. Furthermore, our calibration plot for the invasive model showed improved agreement between the observed outcomes and predictions. In case both FPG and HbA1c were not measured during the health checkup, we also created another two invasive models including either FPG or HbA1c, with slightly decreased AUROC values (0.85 for the prediction model containing FPG and 0.86 for the prediction model containing HbA1c). Given the high performance of these invasive models, they are suitable for identifying at‐risk individuals for diabetes at settings where the data on FPG or HbA1c are available (i.e., annual health checkup in Japan). Unlike the existing risk models in Japan5, 6, 7, 8, our models were derived from routinely collected health checkup data from a working population. Therefore, these models can be easily incorporated into strategies for diabetes prevention at worksites. Furthermore, our sample size is large, which ensures the precision in the estimate of diabetes risk. These advantages make our models highly applicable in the working population for diabetes prevention.

The large population‐based cohort, long‐term follow up and sufficient number of diabetes events were strengths of the present study. In addition, a comprehensive assessment of the multiple measures was used for the diagnosis of incident diabetes. However, several limitations warrant mention. First, our participants were mainly from large companies. Thus, caution should be exercised when applying the risk models to people working in small companies or other populations. Future study should validate the present risk models in these populations. Second, because data about socioeconomic status, lifestyle (except for smoking) and family health history, such as diabetes and CVD, were not collected, these potential predictors were not added in our prediction models. However, the performance of our models is comparable with the previous published models for predicting diabetes. Third, we cannot distinguish between type 1 and type 2 diabetes. However, as new cases of type 1 diabetes are rare after 30 years‐of‐age, we expect that virtually all incident cases in this cohort correlate with type 2 diabetes. We also did not have data on other types of diabetes, such as gestational diabetes. Given that just 38 cases of diabetes occurred among young women aged 30–39 years in the present study, and that just 2% of pregnant women are known to develop gestational diabetes21, we believe that the impact of gestational diabetes, if any, was negligible in the present study.

In conclusion, the present non‐invasive and invasive models for the prediction of diabetes risk up to 7 years showed fair and excellent performance, respectively. The invasive models can be used to identify high‐risk individuals, who would benefit greatly from lifestyle modification for the prevention or delay of diabetes.

Disclosure

The authors declare no conflict of interest.

Supporting information

Figure S1 | Predicted risk of diabetes within 7 years based on non‐invasive model (risk calculator).

Figure S2 | Predicted risk of diabetes within 7 years based on invasive model (risk calculator).

Figure S3 | Predicted risk of diabetes within 7 years based on non‐invasive model (risk chart).

Figure S4 | Predicted risk of diabetes within 7 years based on invasive model (risk chart, including dyslipidemia and glycated hemoglobin).

Figure S5 | Predicted risk of diabetes within 7 years based on invasive model (risk chart, including dyslipidemia and fasting plasma glucose).

Appendix S1 | Data collection method.

J Diabetes Investig 2018; 9: 1052–1059