Diabetes Risk Scores for Hispanics Living in the United States.: A systematic review

Abstract

Aim:

Despite the proliferation of risk scores, few have been validated in Hispanic populations. Undiagnosed diabetes is more prevalent among racial/ethnic minorities in the United States (U.S.). The aim of this study is to systematically review published studies that developed risk scores to identify undiagnosed Type 2 Diabetes Mellitus based on self-reported information that were validated for Hispanics in the U.S.

Methods:

The search included PubMed, EMBASE, Cochrane and CINAHL from inception to 2016 without language restrictions. Risk scores whose main outcome was undiagnosed Type 2 diabetes reporting performance measures for Hispanics were included.

Results:

We identified three studies that developed and validated risk scores for undiagnosed diabetes based on questionnaire data. Two studies were conducted in Latin America and one in the U.S. All three studies reported adequate performance (area under the receiving curve (AUC) range between 0.68 and 0.78). The study conducted in the U.S. reported a higher sensitivity of their risk score for Hispanics than whites. The limited number of studies, small size and heterogeneity of the combined cohorts provide limited evidence of the validity of risk scores for Hispanics.

Conclusions:

Efforts to develop and validate risk prediction models in Hispanic populations in the U.S are needed, particularly given the diversity of this fast growing population. Healthcare professionals providing should be aware of the limitations of applying risk scores developed for the general population on Hispanics.

1. Background

The prevalence of diabetes is increasing worldwide [1–3]. Diabetes and its vascular complications are the seventh leading cause of disability worldwide and contribute to the deaths of two million adults per year [4]. Type 2 diabetes is the most common type affecting 90–95% of those with diabetes and may be asymptomatic for years [5]. Uncontrolled diabetes leads to microvascular (e.g., neuropathy, nephropathy, retinopathy) and macrovascular (e.g., myocardial infarction, stroke) complications and may increase mortality risk [6, 7]. Timely lifestyle interventions and clinical treatments can help manage hyperglycemia [8–11], reducing the risk of vascular complications. However, evidence-based interventions may not reach those with undiagnosed diabetes.

In the United States (U.S.), approximately 1 in 4 people with diabetes is undiagnosed and could be targeted for early intervention. Universal screening is recommended for high-risk adults but there is limited information on the implementation or effectiveness of national recommendations to identify those with diabetes [12]. Only half of those individuals meeting American Diabetes Association’s (ADA) screening recommendation criteria report being screened, while the screening rate for those not meeting these criteria is 30% [13]. Actual diagnostic testing involving laboratory measures is less prevalent in racial/ethnic minorities, those with lower socioeconomic status, and those who lack health insurance [14]. In 2011–2012, 49% of Hispanics with diabetes living in the U.S. were undiagnosed, compared to 32.3% among non-Hispanic blacks and 33.5% among non-Hispanic whites [15]. There is also significant variability in diabetes prevalence within Hispanic groups, ranging from 10.2% among South Americans and 13.4% among Cubans to 17.7% for Central Americans, 18.0% for Dominicans and Puerto Ricans, and 18.3% for Mexican-Americans [16]. In the Hispanic Community Health Study/Study of Latinos (SOL), one third of Hispanics with diabetes were not aware of having it, with Puerto Ricans being more likely know of their diabetes diagnosis (70%) [16]. Further efforts to identify Hispanics most at risk of having undiagnosed diabetes will help target appropriate interventions to improve health outcomes in this diverse and growing population.

Diabetes risk scores are prediction models that identify significant risk factors in the population in which they are developed. The accuracy of their predictions depends on the availability and completeness of demographic, anthropometric, clinical data, and diagnostic tests for the target population [21–24]. Limited data for Hispanic populations present multiple challenges. Electronic databases may not include detailed information on factors that increase risk of diabetes for Hispanics such as country of origin [16], length of residency in the U.S. [25, 26], stress [27] and depression [28, 29]. Questionnaire assessments may provide some of these variables but data on biomedical tests and clinical factors such as family history of diabetes, gestational diabetes or polycystic ovary syndrome depend on individual access to care. Another crucial aspect in the accuracy of risk scores is the selection of the cut-point used to define a positive test result. Lower cut-points increase the sensitivity of the model but decrease its specificity. For diabetes screening, moderate sensitivity (60%) but high specificity (90%), repeated every three years are recommended to balance disease detection avoiding false-positive results [30]. There are several risk scores to identify undiagnosed diabetes based on readily available information that have been developed and validated in the U.S. and other countries [17–19, 31]. The purpose of this study is to identify diabetes risk scores for undiagnosed diabetes that have been developed and evaluated in Hispanic populations. We assess whether risk scores developed and validated in one cohort perform equally in other cohorts; we explore consideration of risk factors specific to Hispanics in the models, and examine methodological issues in the development, validation and comparison of diabetes risk scores in terms of their sensitivity and specificity.

2. Methods

2.1. Search strategy

We conducted a systematic literature search using PubMed, EMBASE, CINAHL and Cochrane Library from date of inception through December 31, 2016. The search strategy was based on type 2 diabetes, screening and the development and validation of a prediction tool: risk score/ assessment/algorithm/prediction/model. Subject specific and medical subject headings (MeSH) terms such as “Diabetes Mellitus Type 2”, “Prediabetes”, “screening”, “risk scores”, “algorithms” and other broad terms were included in the search. The detailed search strategy with all terms for the four databases is included in Appendix A. We also screened reference lists of previous systematic reviews of diabetes risk scores. As a validity check, 120 references from other reviews [17, 18, 31–34] were selected as an exemplary sample. A total of 16,249 references were retrieved from all four sources after removal of duplicates and the inclusion of manually searched articles. Our final sample included 103 of the 120 exemplary articles. Research library staff at our institution assisted with all electronic searches.

2.2. Inclusion criteria

The articles selected for this review met the following inclusion criteria: 1) were published in a peer reviewed journal, 2) used any study design with evidence of random selection of adult participants, 18 years or older, 3) were based on participants’ data collected via questionnaire, 4) developed and validated a risk score to identify undiagnosed diabetes, 5) reported sensitivity, specificity, and area under the curve (AUC) as outcome measures specifically for Hispanics, and 6) the final instrument did not include genetic risk factors, or invasive laboratory measures. No language restriction was applied in the search.

2.3. Article selection

Three investigators (LJ, AA, JS) independently reviewed titles and abstracts. Each database had two independent reviewers. Discrepancies were resolved by a third investigator (AC) before continuing with full paper reviews. A data extraction form (Appendix B) was used for each eligible article. The initial examination of titles discarded the majority of articles because the main outcome was other than screening for Type 2 diabetes, or because the studies targeted specific populations, such as patients who had another disease, and not the general population. The reviewers examined 1,247 abstracts of which 43 were selected for full-paper evaluation. Eighteen articles studied risk factors for undiagnosed diabetes but did not develop a score or risk assessment tool; nine predicted development of diabetes but did not evaluate undiagnosed diabetes separately, and two included invasive measures in their prediction algorithms. We also excluded six studies in which the main outcome combined undiagnosed diabetes with other forms of glucose intolerance. Only three of the remaining eight studies reported performance measures specific to Hispanics (Figure 1).

Figure 1.

Flow diagram of articles selected for review of diabetes risk scores for Hispanic populations in the U.S. (Figure adapted from Liberati et al., 2009 [65])

2.4. Assessment of study quality

In order to assess the quality of the risk score, we made sure the risk score was validated. Risk scores perform better for the populations they were developed in and their performance needs to be evaluated in a different population or setting [35]. Validation of a risk score can be internal, employing the part of the sample used to develop the score; temporal, using the same sample after a selected time period; or ideally, external, using a similar but not identical population [36]. In practice, simpler models that are easy to interpret and implement are preferred. When the purpose of the risk score is to identify individuals at high risk of diabetes for intervention, specific absolute risks are not necessary. However, accurate information of individual absolute risks based on modifiable risk factors will be useful to convey the benefit of the intervention to participants.

2.5. Performance of the risk scores

The predictive performance of a risk score can be evaluated using calibration and discrimination measures [35]. Calibration measures the agreement of the study’s observed risk with the model’s predicted risk. Discrimination (c-statistic) measures the ability of the model to assign a higher predicted probability to those with the event compared with those without the event. The area under the receiver operating curve (AUC) may be used to assess risk score performance, such that a score of one indicates full concordance and a score of 0.50 chance agreement. Other test performance measures include sensitivity (true positive rate), specificity (true negative rate), positive predictive value or the probability that the disease is present when the test is positive, and the negative predictive value or the probability that the disease is not present when the test is negative [37].

3. Results

We identified three studies that met inclusion criteria [38–40]. All three studies developed and validated a diabetes risk score for the prediction of undiagnosed type 2 diabetes; two were conducted in Latin America (Peru and Brazil) [38, 40] and the third in the U.S.[39]. The total number of participants in the three cohorts was 7,466. Approximately 53% were Hispanics, the age ranged from 20–74 years of age. All three studies reported AUCs as a performance measure to assess discrimination. Only one study (Peru) [38] compared the performance of their newly developed score with other scores. General characteristics of all three studies are shown in Table 1.

Table 1-.

Diabetes risk score characteristics for included articles

	Peru [38]	Brazil [40]	USA [39]
Year published	2016	2009	1995
Year data development	2004-2005	1999-2000 c.	1976-1980
Year data validation	2010-2014	2005	1976-1980
Sample size	2472	1224	3770*
Target age group	≥ 35	> 35	20 - 74
Diabetes criteria	FPG ≥ 126 md/dL	FPG > 126 md/dL	OGTT (WHO)
# risk factors	3	3	6
Scoring criteria	0 -5	0 - 48	decision tree
AUC	0.68	0.77	0.78
Sensitivity	70	76	80
Specificity	59	67	61

^*Total number of participants. The study did not report number of Hispanic participants. A similar study analyzing diabetes trends using NHANES II reported approximately 2.9% of Hispanics

3.1. Study characteristics

The main purpose of all three studies was to develop and validate a simple and inexpensive tool to identify individuals with undiagnosed diabetes. The Peru and U.S. studies [38, 39] used nationally representative data and the study from Brazil used two urban populations for development and validation [40]. Diabetes was defined as fasting plasma glucose ≥126mg/dL for the Peruvian and Brazilian risk scores. The study from the U.S. [39] reported using the oral glucose tolerance test 2-h post challenge, following World Health Organization criteria [41]. Only participants who reported having no history of physician-diagnosed diabetes were included. The performance for all three scores was adequate, with the AUC ranging between 0.68 and 0.78. In the study from the U.S. the authors reported a sensitivity of 80% and specificity of 61% for its combined analysis of Hispanics, Blacks and Native Americans compared to a sensitivity of 78% and specificity of 65% among whites [39].

3.2. Study populations

Herman and colleagues [39] used data from the Second National Health and Nutrition Examination Survey (NHANES II, 1976–1980) that examined a nationally representative sample of the U.S. population. NHANES II classified individuals as Hispanics based on three questions: race (White, Black, Other); state or foreign country where the participant was born, and national origin or ancestry as reported by the participant (Central, South America; Chicano; Cuban; Mexican; Mexicano; Mexican-American; Puerto Rican; Other Spanish). All Hispanics were combined into a single group to develop the score.

The study from Peru [38] used data from a national population-based survey (2004–2005) designed to study chronic conditions. Peru is a middle-income country with a multiethnic society. Mestizos, a combination of Amerindian and European (mostly Spanish) ancestries, represent over half of the national population (59.5%); Amerindians (mostly Quechua) represent 27.2%; Whites (4.9%) and Blacks and other (8.3%) [43]. The authors did not report data on ethnicity nor was it included as a risk factor to develop the score.

Pires de Sousa and colleagues’ [40] study used data from two urban populations in Brazil collected between 1999 and 2005. Vitoria is a capital city with a population of 265,874 according to the 1996 Brazilian census. The racial makeup of its population is 52% white, 39% pardo (triracial heritage: European, Amerindian and West African), 7% black and 2% other [44]. The other urban location was Ouro Preto, a smaller city with a population of 37,603 and similar racial composition according to the 1996 Brazilian census. Data for the city of Vitoria were collected in 1999–2000 with a follow-up of the same cohort 5 years later. Dates were not provided for data collection in Ouro Preto. The authors included ethnicity as a risk factor when developing the diabetes risk score but it was not retained in the final model.

3.3. Risk factors considered in the risk scores

The sample size from all three studies was adequate to develop the models, based on the recommendation of a minimum of 10 events per variable to develop a predictive model [45]. All three scores tested demographic, behavioral and anthropometric risk factors. Age was included in all three scores. A complete list of risk factors considered and included in the scores is shown in Table 2.The risk factors considered by all three studies were similar. While the study from the U.S. used only self-reported information, the studies from Brazil and Peru included laboratory measurements that were not significant in their adjusted models and were not included in their final risk scores [38–40].

Table 2.

Risk factors considered and odds ratio with 95% confidence interval for those included in risk scores

Risk factor	Peru [38]	Brazil [40]	USA* [39]
	OR (95%CI)	OR (95%CI)
Demographic
Age
≥45 (versus <45 years)			●
≥55 (versus <55 years)	● 1.85 (1.30–2.63)
45–54 versus 35–44 year		● 2.10 (1.15–3.84)
55 or more versus 35–44 year		3.41 (1.89–6.15)
Gender	❍	●	●
Ethnicity		❍	❍
Education	❍		❍
Behavioral
Sendentarism/PA	❍	❍	●
Smoking	❍	❍
Alcohol	❍
General Health Assessment			❍
History of diabetes			❍
Family History Diabetes	● 2.34 (1.04–5.31)		●
Macrosomic infant			●
Anthropometric
BMI	❍
30 or more versus ≤30			●
25–29.9 versus ≤25		● 1.61 (0.90–2.87)
30 or more versus ≤25		6.06 (3.49–10.52)
Waist circumference
90.0 to <99.9 cm (versus <90 cm)	● 2.09 (1.09–4.02)
100+ cm (versus <90 cm)	4.07 (2.60–6.40)
Waist to height ratio	❍
Waist to hip ratio		❍
Hypertension	❍	● 1.87 (1.18–2.97)	❍
Systolic blood pressure	❍	❍
Diastolic blood pressure	❍	❍
Total cholesterol	❍	❍
HDL	❍	❍
LDL		❍
CT ≥ 240 versus <240		❍
TG		❍
Uric acid		❍
Creatinine		❍

^●Included

^❍Considered

^*Diabetes risk score used a classification tree. No odds ratios were reported

4. Discussion

This review identified three studies that developed and validated risk scores for undiagnosed diabetes in cohorts from Latin America and the U.S. The small number of studies; the limited number of participants, and the heterogeneity of the cohorts did not allow for a meta-analysis, providing limited evidence of the validity of undiagnosed diabetes risk scores among Hispanics. Nonetheless, findings from this study provide evidence for the need to develop and validate prediction risk models for Hispanics living in the U.S. who are at increased risk of diabetes.

The model developed by Herman and colleagues [39] has been used by the American Diabetes Association (ADA) to help identify those at risk of diabetes in the U.S.[46]. Although the performance of this model was adequate overall and among Hispanics, Native Americans and African-Americans, data used for this study dates from 1976–1980, before oversampling of Hispanics started with NHANES III [42]. We identified three other diabetes risk scores for undiagnosed diabetes that used more recent waves from NHANES [47–49]. These studies were not included in the review because the authors did not report performance measures of their final instruments for Hispanics. The Patient Self-Assessment Score developed by Bang and colleagues was developed using data from 1999–2004 and was validated using data from NHANES 2005–2006 and cohort data from two community studies [47]. Another study by He and colleagues used NHANES data from 2005–2006 for development and validation [49]. Although both studies included race as a risk factor when developing their models, race was not retained in the final scores for undiagnosed diabetes. Both studies suggested that the performance of their scores needed to be evaluated for demographic subgroups. Cichosz and colleagues used NHANES data from 2005–2010 to develop an improved screening risk score based on extended predictive features [48]. Their final model included educational level and race/ethnicity, and performed better than the patient self-assessment developed by Bang [47]. However, its performance in population subgroups was not investigated.

In Latin America, Garcia-Alcala and colleagues [50] applied the Finish Diabetes Risk Score (FINDRISC) in a convenience sample to identify individuals with diabetes in Puebla, Mexico. Another study developed a score in rural Honduras using questionnaire data and point-of-care capillary glucose tests that were applied in clinics [51]. Recently, a Colombian diabetes risk score that combined undiagnosed diabetes and impaired glucose regulation as their main outcome was developed and validated using a sample of 2060 individuals in northern Colombia [52]. The relevance of these studies for application to specific national-origin Hispanic groups in the U.S. has not been explored.

The scarcity of research on diabetes risk prediction for Hispanics in Latin America and in the U.S. is a major limitation. Little is known about the recalibration of instruments already developed, particularly in national-origin subgroups. This suggests that the development of new tools in these large and growing populations who are at high risk for diabetes is warranted. Screening thresholds of common glycemic markers also need further investigation within Hispanic populations, particularly for diagnosing diabetes. A study conducted among low-income, elderly Mexicans showed disagreement between OGTT and HbA1c measurements, suggesting possible misclassification when using HbA1c alone [53]. Other studies have confirmed significantly higher levels of HbA1c for Hispanics compared to non-Hispanic whites in the U.S. [54, 55].

Some of the limitations of this systematic review highlight the need for future research in Hispanic populations. Although this study used a comprehensive search without language restrictions, some articles may have been missed, particularly those recently published and others from Latin America not indexed by major scientific publication engines or those with inconsistent use of the term “Hispanic”. The heterogeneity among Hispanic populations is another limitation, detailed subgroup analyses of these populations were not available. Recent cohort studies that examine chronic diseases among Hispanics such as SOL in the U.S. [56] and others that combine populations from Latin America such as CESCAS I [57], INTERHEART [58] and ELSA in Brazil [59] will be important for development and validation of future risk scores for diabetes that account for subgroup variation and the inclusion of risk factors unique to Hispanics living in the U.S. and elsewehere.

It is well documented that on average Hispanics have better health upon arrival to the U.S. compared to their American counterparts [60, 61]. Although Hispanics tend to have better longevity than their socioeconomic status would predict [62], the overall health of Hispanics declines as they spend more time living in the U.S. leaning towards that of natives, or even worse [63, 64]. Migration and onward integration are major life experiences and present challenges such as discrimination, language proficiency, stress and depression. These factors may contribute to adverse health outcomes among Hispanics and their inclusion in screening tools deserves further investigation.

Diabetes risk scores are important to the prevention and timely management of a range of diseases and health complications [17–19]. However, none have been developed and validated explicitly for Hispanic populations living in the U.S. This systematic review of the literature on risk scores aimed to identify undiagnosed diabetes among Hispanic populations living in the U.S. has revealed that only a handful of studies published to date have reported performance measures specific for Hispanics. Further, only one such study developed a diabetes risk score that was validated for Hispanics in the U.S. and this used data from 1976–1980 with all Hispanic groups combined. This highlights the lack of evidence regarding the applicability of such tools for Hispanics living in the U.S. This review underscores the urgent need to develop and validate simple and inexpensive tools to identify undiagnosed diabetes for Hispanics in the U.S. who constitute a large, diverse and growing population at high risk for diabetes.