Psychometric Properties of the Hypoglycemia Fear Survey-II for Adults With Type 1 Diabetes

Abstract

OBJECTIVE

To perform the first comprehensive psychometric evaluation of the Hypoglycemia Fear Survey-II (HFS-II), a measure of the behavioral and affective dimensions of fear of hypoglycemia, using modern test-theory methods, including item-response theory (IRT).

RESEARCH DESIGN AND METHODS

Surveys completed in four previous studies by 777 adults with type 1 diabetes were aggregated for analysis, with 289 subjects completing both subscales of the HFS-II and 488 subjects completing only the Worry subscale. The aggregated sample (53.3% female, 44.4% using insulin pumps) had a mean age of 41.9 years, diabetes duration of 23.8 years, HbA_1c value of 7.7%, and 1.4 severe hypoglycemic episodes in the past year. Data analysis included exploratory factor analysis using polychoric correlations and IRT. Factors were analyzed for fit, trait-level locations, point-measure correlations, and separation values.

RESULTS

Internal and test-retest reliability was good, as well as convergent validity, as demonstrated by significant correlations with other measures of psychological distress. Scores were significantly higher in subjects who had experienced severe hypoglycemia in the past year. Factor analyses validated the two subscales of the HFS-II. Item analyses showed that 12 of 15 items on the Behavior subscale, and all of the items on the Worry subscale had good-fit statistics.

CONCLUSIONS

The HFS-II is a reliable and valid measure of the fear of hypoglycemia in adults with type 1 diabetes, and factor analyses and IRT support the two separate subscales of the survey.

The Hypoglycemia Fear Survey (HFS), first published in 1987 (1), originally was developed to measure behaviors and worries related to fear of hypoglycemia (FOH) in adults with type 1 diabetes. Both the original HFS (HFS-I) and the revised version (HFS-II) are composed of two subscales, the Behavior (HFS-B) and Worry (HFS-W). HFS-B items describe behaviors in which patients may engage to avoid hypoglycemic episodes and/or their negative consequences (e.g., keeping blood glucose [BG] levels above 150 mg/dL, making sure other people are around, and limiting exercise or physical activity). HFS-W items describe specific concerns that patients may have about their hypoglycemic episodes (e.g., being alone, episodes occurring during sleep, or having an accident). Since it was first published, the HFS, in its original form and subsequent revisions, has been used in >60 published studies and numerous clinical trials and has been translated into over 50 languages. The survey also is commonly used in studies of people with type 2 diabetes to better understand their experiences of hypoglycemia (2). In addition, the authors have developed alternative versions of the HFS for use with pediatric patients with type 1 diabetes and their parents, as well as spouses and significant others of adults with diabetes (3–5).

Despite its widespread use, the only comprehensive study to address the psychometric properties of the U.S. HFS was published >15 years ago and focused on the HFS-I, which is no longer in use (6). Since then, based on subsequent studies, the scale has been significantly revised, leaving only 18 of 33 items unchanged from the original version from 1987. Although there is some evidence for the reliability and validity of the HFS-II (2,7), no comprehensive study of its psychometric properties has been conducted with a large sample. In addition, psychometric results of previous studies included only classical test-theory methods that have not acknowledged the categorical nature of HFS data. Modern test procedures, such as item-response theory (IRT), are better suited to analyze categorical data and can provide valuable information about item quality, including response option use, fit, and endorsement difficulty (8).

For researchers and clinicians interested in using the HFS-II, this lack of supporting psychometric data presents many obstacles not the least of which is the inability to interpret scores in a clinically meaningful way. The purpose of the current study was to conduct a comprehensive psychometric investigation of the HFS-II U.S. English adult version using a large aggregate sample of data collected at our laboratory over the past decade and using modern test-theory methods. The following were the hypotheses tested: 1) that the HFS-II is a reliable and valid measure of FOH and 2) that factor analysis will yield a two-factor structure reflecting the two subscales of the survey.

RESEARCH DESIGN AND METHODS

This was a secondary data analysis study using HFS-II survey data collected from four different National Institutes of Health–funded projects conducted at the University of Virginia Center for Behavioral Medicine Research between 1998 and 2009. Details of the methods for several of these projects have been previously published (9,10). All four projects included participants who had type 1 diabetes for at least 1 year and were willing to perform BG measurements at least two times every day. Study 1 had an additional inclusion criterion of having a history of two or more episodes of severe hypoglycemia during the past year. Study 4, a survey of drivers with diabetes, had additional criteria of possessing a legal driver’s license and driving at least 10,000 miles per year. In each study, participants completed a battery of questionnaires, including the HFS-II. Although studies 1, 2, and 3 included the entire HFS-II scale (i.e., both HFS-B and HFS-W subscales), study 4 only used the HFS-W subscale. Participants in all studies also completed a diabetes history questionnaire, including an item to assess frequency of severe hypoglycemia over the past year. Severe hypoglycemia was defined as a hypoglycemic episode during which BG was so low that self-treatment was not possible because of mental confusion or stupor and external assistance was required. Glycosylated hemoglobin (HbA_1c) measures were available for three studies. Finally, all four studies included additional psychological questionnaires, including the Beck Depression Inventory (11), the Beck Anxiety Inventory (12), the Modified State-Trait Personality Inventory (13), and the Short-Form (SF) 12 Quality-of-Life Inventory (14). These measures were used to assess the construct validity of the HFS-II. Only baseline data (i.e., before any intervention) was included for the purposes of the current study.

Sample characteristics

The total number of participants with HFS-II data was 777, with 289 participants completing the HFS-B subscale and 488 participants completing only the HFS-W subscale. Across the studies, there were no differences in sex composition, mean HbA_1c values, mean age, or ethnicity. There were differences in mean duration of diabetes [F(3,791) = 10.22, P < 0.0005; minimum–maximum = 19.6–26.0 years] and years of education [F(2,661) = 10.9, P < 0.0005; minimum–maximum = 14.9–15.9 years], but these did not appear to be clinically meaningful. For the aggregate sample, mean (SD) age was 41.9 (12.6) years, diabetes duration was 23.8 (12.5) years, HbA_1c was 7.7 (1.4) %, and education was 15.6 (2.5) years. A total of 53.3% of the subjects were female, 44.4% used insulin pumps, 63.1% reported no severe hypoglycemia in the past year, and 95.9% were white.

Data analysis

Descriptive statistical analyses, correlations, group comparisons, and Cronbach α reliability analyses were performed with PASW 18.0.2 (SPSS, Chicago, IL). Because factor analyses have not previously been conducted on the HFS-II, the dimensional structure of the survey was examined with exploratory factor analysis in Mplus software (15), using polychoric correlations for handling the categorical nature of the data. Model fit was estimated using a χ² goodness-of-fit statistic, root mean squared error of approximation (RMSEA), standardized root mean squared residuals (SRMRs), and the Tucker-Lewis Index (TLI). Currently, there is much debate in the field about cutoffs for fit indices (16), but, in general, RMSEA and SRMR values <0.10 indicate acceptable fit (17,18) and TLI values >0.90 indicate acceptable fit (19).

IRT methods were used to assess the rating scale of HFS-II items. HFS-B and HFS-W subscales were analyzed using the Rasch Partial Credit Model (PCM) in Winsteps (20). Several statistics from PCM analyses assess item quality and responses. These include 1) infit and outfit statistics that indicate the degree of fit between item endorsement scores and the person’s level of the trait being measured, in this case FOH; 2) point-measure correlations, which are similar to item-total correlations, with a range of −1 to 1 on the PCM logit scale (low and negative values being problematic); and 3) separation values that indicate the degree of distinguishable trait levels measured by the scale, with high values desirable (e.g., a separation of “2” indicates that only two levels of the trait are distinguishable and a separation of “3” indicates three levels, etc). Separation is computed as the ratio of the test SD corrected for estimation error to root mean squared error (RMSE).

RESULTS

Mean HFS scores

Table 1 summarizes the mean total HFS-II, HFS-B, and HFS-W scores for each study and for the aggregate sample. Scores were significantly higher on the total HFS-II and HFS-W in study 1, in which inclusion criteria required that individuals had experienced two or more episodes of severe hypoglycemia in the previous year compared with other studies {F(2,282) = 8.46, P < 0.0005 (T₁₂[143.4] = 2.43, P = 0.016, T₁₃[128.3] = 3.83, P < 0.0005) and F(3,773) = 26.47, P < 0.0005 (T₁₂[152.1] = 3.84, P < 0.0005, T₁₃[141.5] = 5.40, P < 0.0001, T₁₄[174.3] = 7.27, P < 0.0001), respectively}. There were no differences in HFS-B (M_i = 18.0, SD = 9.4; M_p = 17.3. SD = 9.2) or HFS-W scores (M_i = 22.1, SD = 14.6; M_p = 22.1, SD = 13.9) as a result of insulin regimen (injection versus pump) [F(1,259) = 0.32, P = 0.572 and F(1,747) = 0.00, P = 0.951, respectively]. HFS-B scores were significantly higher for female subjects (M = 19.0, SD = 9.8) than for male subjects (M = 16.8, SD = 8.5) [F(1,281) = 4.07, P = 0.045]. HFS-W scores were also significantly higher for female subjects (M = 23.4, SD = 14.5) than for male subjects (M = 21.0, SD = 14.0) [F(1,773) = 5.395, P = 0.02]. Years of education were negatively correlated with HFS-B (r = −0.28, P = 0.0001) and HFS-W scores (r = −0.10, P = 0.016). HFS-II scores did not correlate with age or duration of diabetes.

Table 1.

HFS-II score means (SD) by study

Score	All studies	Study 1	Study 2	Study 3	Study 4
n	777*	80	96	113	488
HFS-II
Total	44.1 (21.7)	52.2 (24.8)	43.5 (20.6)	39.3 (18.7)
Item	1.3 (0.7)	1.6 (0.8)a,b	1.3 (0.6)a	1.2 (0.6)b
HSB-B
Total	17.9 (9.3)	18.8 (10.5)	18.1 (9.6)	17.2 (8.1)
Item	1.2 (0.6)	1.3 (0.7)	1.2 (0.6)	1.2 (0.5)
HSB-W
Total	22.3 (14.4)	34.0 (16.6)	25.2 (13.5)	22.1 (12.7)	19.9 (13.4)
Item	1.2 (0.8)	1.9 (0.9)a,b,c	1.4 (0.8)a,e	1.2 (0.7)b	1.1 (0.8)c,e

Means sharing the same subscript denote statistically significant differences; all P < 0.0169 for total HFS-II comparisons and P < 0.0085 for HFS-W comparisons.

*Total n for total HFS-II and HFS-B is 279 and 289, respectively.

Reliability analyses

Cronbach α coefficients indicated a high level of internal consistency for the total HFS-II (α = 0.94), HFS-B (α = 0.85), and HFS-W (α = 0.94) scores. Test--retest data were available from one study (study 3) that used a repeated-baseline design, with 2 months between data collection. Temporal reliability was 0.74 for the total HFS-II, 0.81 for the HFS-B, and 0.63 for the HFS-W scales.

Validity analyses

As Table 2 shows, both the HFS-B and HFS-W correlated positively with measures of psychological distress, including the Modified State-Trait Personality Inventory Anxiety, Anger, and Depression Subscales, as well as the Beck Depression Inventory and Beck Anxiety Inventory. HFS-B and HFS-W scores correlated negatively with health-related quality of life, including the SF-12 Physical and Mental scores.

Table 2.

Correlations and mean (SD) HFS-II scores by measures of validity

Correlations	HFS-B	HFS-W	Total HFS
n	283	777	279
Beck Depression Inventory	0.35*	0.45*	0.43*
Beck Anxiety Inventory	0.51*	0.55*	0.61*
Modified State-Trait Personality Inventory
Anxiety	0.30*	0.38*	0.39*
Depression	0.26†	0.29*	0.31*
Anger	0.21‡	0.22‡	0.24‡
SF-12
Physical	−0.42*	−0.26‡	−0.32†
Mental	−0.41*	−0.63*	−0.58*
History of severe hypoglycemia
In the past year
Total	19.9 (10.2)*	27.5 (15.4)†	49.6 (23.2)†
Item	1.3 (0.7)	1.5 (0.9)	1.5 (0.7)
Not in the past year
Total	16.6 (8.3)	19.3 (12.8)	40.1 (19.7)
Item	1.1 (0.6)	1.1 (0.7)	1.2 (0.6)
Glycosylated hemoglobin (%)
<7.5
Total	16.6 (9.4)‡	25.7 (15.5)	42.1 (21.8)
Item	1.1 (0.6)	1.4 (0.9)	1.3 (0.7)
≥7.5
Total	19.2 (9.1)	27.5 (14.5)	46.3 (21.7)
Item	1.3 (0.6)	1.5 (0.8)	1.4 (0.7)

*P < 0.001;

†P < 0.01;

‡P < 0.05.

Table 2 also shows that both HFS-B and HFS-W scores were significantly higher for those who had experienced severe hypoglycemia in the past year compared with those who had not [F(1,281) = 8.956, P = 0.003 and F(1,770) = 63.037, P < 0.0005, respectively], with effect sizes 0.35 for the HFS-B and 0.58 for the HFS-W. Median HFS-B item scores were 1.27 for those who experienced SH and 1.0 for those who did not experience SH, with interquartile range scores of 0.87 and 0.63, respectively. Median HFS-W item scores were 1.44 for those with a history severe hypoglycemia and 0.94 for those with no history in the past year, with interquartile range scores of 1.13 and 0.89, respectively. HFS-B scores were higher for those with HbA_1c ≥7.5% compared with those with HbA_1c <7.5% [F(1,268) = 5.34, P = 0.022], with an effect size of 0.28. HFS-W scores did not differ, with an effect size of 0.12. Median HFS-B scores were 1.13 for those with HbA_1c ≥7.5% and 1.0 for those with HbA_1c <7.5%, with interquartile range scores of 0.87 and 0.73, respectively.

Factor analyses

Table 3 summarizes factor analyses results. Fit estimates for the one-factor solution were questionably acceptable [χ²(79) = 1,288.4, P < 0.0001; RMSEA = 0.14, SRMR = 0.13, TLI = 0.92] but improved in the two-factor solution [χ²(102) = 973.8, P < 0.0001; RMSEA = 0.10, SRMR = 0.08, TLI = 0.95]. As expected, this two-factor solution reflected the two HFS-II subscales. The Δ χ²(23) = 314.6 between these two models indicates that the two-factor solution fit better. A three-factor model also was tested and fit the data [χ²(112) = 630.1, P < 0.0001; RMSEA = 0.08, SRMR = 0.06, TLI = 0.98], with only four HFS-B items loading onto a third factor. Based on study hypotheses and the fit of the two-factor solution reflecting the HFS-II subscales, IRT analysis was performed on the two-factor solution.

Table 3.

Exploratory factor analysis results

Item	Item content	One-factor solution	Two-factor solution		PCM ranking*
		Factor 1	Factor 1	Factor 2
HFS-B1	Ate large snack	0.24	0.33	−0.02	B-3
HFS-B2	Kept BG >150 mmol/L	0.4	0.42	0.07	B-6
HFS-B3	Reduced insulin when BG low	0.12	0.04	0.1	B-1
HFS-B4	Measured BG six or more times per day	0.09	0.2	−0.08	B-2
HFS-B5	Take someone with me when out	0.58	0.76	−0.09	B-7
HFS-B6	Limited out-of-town travel	0.78	0.85	0.01	B-10
HFS-B7	Limited driving	0.68	0.72	0.06	B-9
HFS-B8	Avoided visiting friends	0.86	0.86	0.11	B-14
HFS-B9	Stayed home more than liked	0.79	0.81	0.08	B-11
HFS-B10	Limited physical activity	0.56	0.67	0	B-8†
HFS-B11	Made sure others were around	0.69	0.81	−0.01	B-8†
HFS-B12	Avoided sex	0.52	0.56	0.06	B-12
HFS-B13	Kept BG high in social situations	0.64	0.87	−0.13	B-5
HFS-B14	Kept BG high during important tasks	0.65	0.84	−0.08	B-4
HFS-B15	Had others check on me	0.59	0.67	0.04	B-13
HFS-W1	Not recognizing low BG	0.65	0.16	0.57	W-2
HFS-W2	Not having food available	0.43	0.04	0.43	W-5
HFS-W3	Passing out in public	0.76	0.13	0.7	W-17
HFS-W4	Embarrassing myself in social situation	0.75	0.24	0.61	W-14
HFS-W5	Having hypoglycemic episode alone	0.81	0.32	0.61	W-4
HFS-W6	Appearing drunk or stupid	0.79	0.25	0.64	W-15
HFS-W7	Losing control	0.81	0.16	0.73	W-11
HFS-W8	No one to help during hypoglycemia	0.85	0.35	0.63	W-7
HFS-W9	Having hypoglycemia while driving	0.79	−0.21	0.94	W-6
HFS-W10	Making mistakes or having accidents	0.85	−0.28	1.04	W-9†
HFS-W11	Getting bad evaluation	0.72	0.06	0.7	W-16
HFS-W12	Difficulty thinking clearly	0.73	−0.01	0.75	W-9†
HFS-W13	Feeling lightheaded or dizzy	0.61	0.18	0.51	W-12
HFS-W14	Injuring myself or others	0.81	−0.11	0.9	W-13
HFS-W15	Permanent injury to health	0.74	−0.15	0.86	W-8
HFS-W16	Low BG interfering with important things	0.73	0.08	0.71	W-3
HFS-W17	Becoming hypoglycemic while sleeping	0.63	0.17	0.53	W-1
HFS-W18	Becoming upset and difficult	0.68	0.18	0.58	W-10

*PCM ranking of the measures is on a scale from 1 (the most frequently strongly endorsed item on each scale) to the highest number on each scale representing the least frequently strongly endorsed item.

†Some rankings indicate tied measures.

In the two-factor solution, 13 of 15 HFS-B items loaded onto the first factor, with coefficients ranging from 0.33 to 0.87 (see Table 3). The only items that did not load within this two-factor solution range were HFS-B3 and HFS-B4. All 18 HFS-W items loaded onto the second factor, with coefficients ranging from 0.43 to 1.04. The correlation between these two subscales was r = 0.60 (P < 0.0005). These results support two distinct, yet correlated, dimensions of FOH.

IRT analyses

Optimal assessment is indicated by matching between person trait-level scores and item endorsement scores. Nearly all item endorsement scores matched with individual person trait scores within the range of the highest to lowest scores for both people and items. The following scores are reported in logits. For HFS-B item endorsement scores, M = 0.00 (by definition) and SD = 0.58 and for person HFS-B trait-level scores, M = −0.86, SD = 0.83. For HFS-W item endorsement scores, M = 0.00 (by definition) and SD = 0.44 and for person HFS-W trait-level scores, M = −1.01 and SD = 1.26. For both scales, the mean person trait-level score was slightly lower than mean item endorsement score, and person trait-level variability was greater than item variability for the respective scales, which was acceptable. Items were rank-ordered separately for each subscale from the most frequently strongly endorsed item to the least frequently strongly endorsed item, on average (see Table 3).

HFS-B subscale

Twelve HFS-B items showed excellent fit (item responses were as expected), with proper ranges of infit values (0.64–1.28) and outfit values (0.67–1.09). Point-measure correlations for these items ranged from 0.40 to 0.72, which is excellent. For HFS-B3 and HFS-B4, infit and outfit values indicated less fit than expected, had low point-measure correlations, and were the two most frequently strongly endorsed items, on average. For HFS-B8, outfit was better than expected, and it was the least frequently strongly endorsed item with lower variability compared with other HFS-B items. Separation for the HFS-B was 7.92, indicating nearly eight statistically distinguishable trait levels and excellent discrimination quality.

HFS-W subscale

Seventeen HFS-W items showed excellent fit, with ranges of infit values (0.77–1.25) and outfit values (0.72–1.27). Point-measure correlations ranged from 0.49 to 0.74. HFS-W2 had the highest outfit and infit values. No other statistics (item endorsement score, point-measure correlation) were extreme, so concern for this item was negligible. Separation for the HFS-W was 9.35, indicating over nine statistically distinguishable trait levels and excellent discrimination quality. See Table 3 for item endorsement score level rankings.

CONCLUSIONS

The survey demonstrated strong internal consistency, with reliability coefficients ≥0.85 for the total HFS-II and subscales. Temporal reliability was adequate, ranging from 0.81 (HFS-B) to 0.63 (HFS-W), although it is important to note that FOH would not be expected to stay completely stable over time. Previous studies have shown that levels of FOH can increase after traumatic experiences with severe hypoglycemia (21) and decrease after interventions that reduce the frequency of hypoglycemia (22,23).

Several findings support the validity of the HFS-II, including significantly higher scores for both subscales in patient populations expected to have higher levels of FOH, such as those who experience more frequent severe hypoglycemia. In addition, HFS-B scores, but not HFS-W scores, were higher in patients with poorer glycemic control, indicative of more hyperglycemia. Female subjects also had significantly higher HFS-II scores, replicating numerous studies that found higher levels of emotional and physical symptoms in female subjects across a broad range of self-report instruments. Construct validity was additionally supported by the positive correlations between the HFS-II and other valid measures of psychological distress, including anxiety and depression. As expected, higher HFS-II scores were negatively related to quality of life, with the HFS-W subscale more strongly related to mental and emotional quality of life and the HFS-B subscale more strongly related to physical quality of life.

Overall, psychometric properties were very good. Factor analyses supported a two-factor solution over a one-factor solution, with item loadings justifying the structure and separate scoring of the HFS-B and HFS-W subscales. A recent Swedish study (24) found a three-factor solution for the HFS-I in type 1diabetic patients, with the third factor comprising HFS-W items describing worries about being alone during hypoglycemia. The current study tested a three-factor solution for the HFS-II but did not replicate the Swedish findings. The three-factor solution for the HFS-II did provide slightly better fit, but the third factor comprised a few items on the HFS-B, not the HFS-W, subscale. It is almost always the case that increasing the number of factors results in better fit, although this does not necessarily result in better understanding of a scale. Unfortunately, the HFS-B subscale was not included in some of the datasets analyzed for this study. Additional analysis of datasets with more HFS-B data are needed to determine whether this factor will be confirmed and whether it improves the utility of the HFS-II.

Rasch PCM analyses indicated very good results for the items in the two HFS-II subscales. Both subscales had excellent separation, reasonable item-to-person matching, and good item-fit statistics, indicating that they are appropriate measures of the constructs represented in the two-factor solution. Two items on the HFS-B (reduce insulin when BG is low and measure BG six or more times per day) yielded very low factor loadings, and the Rasch PCM analysis indicated that these items were, on average, the most frequently strongly endorsed. Although sometimes this type of result may indicate appropriate removal of these items from the instrument, this is not recommended for two reasons. The first is clinical significance. IRT analysis indicated that these items tended to be strongly endorsed by most individuals; therefore, the failure to strongly endorse these items may be important information to have about a person’s behavioral reactions to low BG levels. Second, maintaining these items, and keeping the HFS-II consistent across studies and translations, allows researchers to compare results more easily across various populations, translations, and research.

It also is important to consider the clinical implications of these findings. The mean scores and comparisons across different patient populations generated by this study may help to guide clinical interpretations of HFS-II scores. For example, higher HFS-II scores, on both subscales but especially on the HFS-W subscale, are expected in patients with a recent history of severe hypoglycemia. On the other hand, patients in poorer glycemic control did not exhibit higher HFS-W subscale scores but did exhibit higher HFS-B scores. In addition, whereas HFS-W scores were more strongly related to lower quality of life in terms of emotional distress, HFS-B scores were more strongly related to lower quality of life in terms of physical burden and impairment. The original purpose of the two subscale construction of the HFS was to assess both the behavioral and the affective dimensions of patient experience and reaction to hypoglycemia. These results suggest that the instrument accomplishes this goal. These results also support the need to administer the entire HFS-II scale in order to assess these different dimensions of FOH and not just the HFS-W subscale, which has been done in some studies.

It still is premature, however, to generalize these findings to translated versions of the HFS-II, and it is a limitation of this study that only U.S. subjects were included. Additional research is needed to validate other versions of the scale and to compare FOH across different cultures and countries. Other limitations of this study include the substantially lower number of HFS-B surveys for analysis and the exclusion of individuals with type 2 diabetes, which was beyond the scope of this article. Future studies are needed of the psychometric properties of the HFS-II in type 2 diabetes, as well as in pediatric patients and their parents. Nonetheless, the results of this first study indicate that the U.S. English version of the HFS-II is a psychometrically valid and reliable instrument for adults with type 1 diabetes, with potential clinical as well as scientific utility for the assessment of FOH.