Abstract
Introduction
Response rates for the Behavioral Risk Factor Surveillance System (BRFSS) have declined in recent years. The response rate in 1993 was approximately 72%; in 2006, the response rate was approximately 51%. To assess the impact of this decline on the quality of BRFSS estimates, we compared selected health and risk factor estimates from BRFSS with similar estimates from the National Health Interview Survey (NHIS) and the National Health and Nutrition Examination Survey (NHANES).
Methods
We reviewed questionnaires from the 3 surveys and identified a set of comparable measures related to smoking prevalence, alcohol consumption, medical conditions, vaccination, health status, insurance coverage, cost barriers to medical care, testing for human immunodeficiency virus, and body measurements (height and weight).
We compared weighted estimates for up to 15 outcome measures, including overall measures and measures for 12 population subgroups. We produced design-appropriate point estimates and carried out statistical tests of hypotheses on the equality of such estimates. We then calculated P values for comparisons of NHIS and NHANES estimates with their BRFSS counterparts.
Results
Although BRFSS and NHIS estimates were statistically similar for 5 of the 15 measures examined, BRFSS and NHANES estimates were statistically similar for only 1 of 6 measures. The observed differences for some of these comparisons were small, however.
Conclusion
These surveys produced similar estimates for several outcome measures, although we observed significant differences as well. Many of the observed differences may have limited consequences for implementing related public health programs; other differences may require more detailed examination. In general, the range of BRFSS estimates examined here tends to parallel those from NHIS and NHANES, both of which have higher rates of participation.
Introduction
The federal government allocates substantial resources each year for collection of state and national data to monitor trends and changes in the health of the U.S. population. Scientists, health care professionals, and policy makers use such data to understand current and emerging trends in public health, to provide a basis for the establishment and evaluation of health policies and programs, and to assess where best to apply limited public health resources. It is, therefore, imperative for health statistics and other population estimates obtained from these surveys to be of the highest possible quality.
The Behavioral Risk Factor Surveillance System (BRFSS), the world's largest ongoing random-digit–dialed (RDD) telephone survey, is conducted by the health departments in the 50 states as well as those in the District of Columbia, Puerto Rico, Guam, and the Virgin Islands, with assistance from the Centers for Disease Control and Prevention (CDC) (1). Estimates obtained from the BRFSS are based on sound methods for conducting surveys and performing statistical analyses; comparison of these estimates with those from other national surveys is important for measuring the validity and reliability of the estimates. Such comparisons are especially important because RDD surveys are facing an increasing number of newly emerging operational challenges.
The rate of response to telephone surveys has traditionally served as a proxy indicator of the survey's data quality. Link et al point out that telephone survey response rates have recently declined (2). BRFSS response rates declined between 1993 and 2006 (Figure). Several researchers have presented convincing arguments on the negative correlation between rate of nonresponse and survey data quality (3-5). RDD surveys face a unique and growing problem: an increasing number of people rely on cell phones as the only means of telephone communication. Most RDD surveys traditionally sample only from landline telephone numbers (6). Also of concern is the change in BRFSS sampling methods. As of 2004, all zero-listed telephone banks (sets of telephone numbers that include only unlisted or nonresidential numbers) were eliminated from the sampling frame to increase efficiency; this change has increased undercoverage of the population by up to 2% (7).
Figure.
Response rates for the Behavioral Risk Factor Surveillance System, 1993–2006 (Reference 1).
This study compares a number of key estimates obtained from BRFSS with related estimates from the National Health Interview Survey (NHIS) and the National Health and Nutrition Examination Survey (NHANES). These surveys have structural differences in design protocols, modes of data collection, and postsurvey procedures for data adjustment. NHIS and NHANES also have higher rates of participation than BRFSS. A comparison of the estimates from the 3 surveys can render valuable insights into the quality of the estimates. The estimates included in these comparisons are related to smoking prevalence, alcohol consumption, medical conditions, vaccination, health status, insurance coverage, cost barriers to medical care, testing for human immunodeficiency virus (HIV), and body measurements (height and weight).
Gentry and colleagues compared estimates of chronic drinking obtained from the National Institute on Alcoholism and Alcohol Abuse and estimates of current smoking obtained from NHIS with BRFSS estimates and concluded that their similarity was statistically significant (8). A more comprehensive study by Nelson et al examined the comparability of national estimates from the 1997 BRFSS and NHIS (9). Nelson et al concluded that the 2 surveys provided similar estimates for most of the overall measures examined. This research reexamines these findings after nearly a decade and makes new comparisons with estimates from NHANES and NHIS.
Methods
Survey designs and data
The data for this research were obtained from the 2004 public-use versions of BRFSS, NHIS, and NHANES. Although technical and methodological details of these surveys are readily available online and in print form, a brief overview of these surveys is provided here for reference purposes (10-12).
BRFSS relies on a monthly, state-based RDD sample design in which virtually all households with landline telephones have a nonzero probability of selection. States use disproportionate stratified sampling in which listed residential telephone numbers are sampled at a higher rate than unlisted residential telephone numbers. One adult aged 18 years or older is chosen at random for interview from each selected household; annual sample sizes vary from state to state. In 2004, state sample sizes ranged from 2656 in Alaska to 18,587 in Washington, with a median state-level sample size of 5903 and a weighted national response rate of 47.2%. BRFSS data from the 50 states and the District of Columbia are aggregated to provide a national sample and are weighted to reflect the sample design and to compensate for differential nonresponse and undercoverage. The resulting weights are ratio-adjusted (poststratified) within cells indexed by age and sex and, in certain states, by race/ethnicity.
NHIS is a national survey conducted annually by interviewers of the U.S. Census Bureau for the National Center for Health Statistics (NCHS) of CDC and is designed to track major trends in illness, disability, and coverage of certain health care services in the U.S. population. The annual questionnaire consists of 3 components: the family core, the sample adult core, and the sample child core. The family core collects information on all members of the household aged 17 years or older who are at home during the interview. Proxy data are collected on children and adults who are not at home. In addition, 1 sample child and 1 sample adult from each household are randomly selected, and information on each is collected with the sample child core and sample adult core questionnaires. This survey, which is conducted year-round, is based on a multistage design that starts with selection of a sample of 358 primary sampling units (PSUs) from approximately 1900 geographically designed PSUs in the country. In the second stage, all occupied houses in selected area segments in each PSU are targeted for interview. The sample for the 2004 survey consisted of 94,460 people in 36,579 households. Interviewers obtained data for the sample adult component of the questionnaire from 31,326 of the 37,388 adults eligible for the interview. The household-level response rate was 86.9%, and the conditional response rate for the sample adult component was 83.8% (calculated by dividing the number of completed sample adult interviews by the total number of eligible sample adults). Weights for NHIS data are derived from the 2000 census and adjusted by age, sex, and race/ethnicity.
NHANES, which is also conducted by NCHS, aims to assess the health and nutritional status of adults and children in the United States as a basis for setting national standards for physical measurements such as height and weight. Basic survey data are obtained in the homes of study participants; data from detailed physical examinations are obtained for a selected subset of participants at mobile examination centers. Incentives range from $20 to $100 and are given for completion of surveys and examinations and as compensation for expenses related to participating in the survey. Similar to NHIS, NHANES begins with selection of PSUs. Clusters of households are selected to be screened for specific demographic characteristics. Then a sample of eligible households is selected, and 1 or more individuals per household are interviewed. During the 2003–2004 NHANES, of the 12,761 people selected, 10,122 were interviewed, and 9643 had a physical examination. Because of the multistage design for NHANES, a complex weighting procedure is used to compensate for unequal selection probabilities and for poststratification to the Current Population Survey estimates of the U.S. population.
Analysis
We reviewed questionnaires used for BRFSS, NHIS, and NHANES and identified a set of comparable measures related to smoking prevalence, alcohol consumption, medical conditions, vaccination, health status, insurance coverage, cost barriers to medical care, testing for HIV, and body measurements. (The exact questions used in this study are provided in the Appendix.) We identified 15 measures from NHIS (14 reported measures and 1 calculated measure, body mass index [BMI]) and 6 measures from NHANES (5 reported measures and 1 calculated measure, BMI) for comparison with their BRFSS counterparts.
We combined some response categories to create conforming scales between the 3 surveys. We also used only subsets of respondents with matching age categories for certain comparisons. For example, we grouped people with borderline diabetes with people who reported a diagnosis of diabetes. Also, we combined the response categories of fair health and poor health to create 1 category. To eliminate confounding effects attributable to different age requirements for these surveys, we excluded from analysis all respondents aged younger than 18 years. Finally, we restricted analysis of health care coverage to participants aged 18 to 64 years, and we restricted questions on influenza and pneumococcal vaccinations to participants aged 65 years or older.
A few wording differences between the 3 survey instruments could not be addressed completely by combining response categories. For instance, the 2004 BRFSS asked for the number of times during the past 30 days the person had 5 or more drinks in a single occasion, while NHIS allows a choice of unit for time (per day, week, month, or year).
To determine which estimates were statistically different, we used the SAS-callable version of SUDAAN (Research Triangle Institute, Research Triangle Park, North Carolina) to produce design-appropriate point estimates and 95% confidence intervals. We calculated corresponding P values for comparisons of NHIS and NHANES estimates with their BRFSS counterparts under the null hypothesis of equality of point estimates. All test statistics were produced using z tests for proportions for weighted data. We used a nominal α level of .05 (P < .05), adjusted for multiple comparisons, to determine significant differences between 2 corresponding surveys.
In addition to comparing overall estimates, we compared the following 12 demographic categories:
Age: 18 to 34, 35 to 54, and 55 years or older.
Education: less than a high school diploma, high school diploma or some college, and college diploma or more.
Sex: male or female.
Race/ethnicity: Hispanic, non-Hispanic white, non-Hispanic black, and non-Hispanic other.
Results
The data show varying levels of agreement between the 3 surveys (Tables 1–4). BRFSS and NHIS overall estimates were statistically similar for 5 of the 15 items examined: current smoker, diabetes, BMI, average number of alcoholic drinks per occasion, and no health insurance. BRFSS and NHANES estimates were statistically similar for 1 of the 6 items analyzed: self-reported height. BRFSS and NHIS were significantly different for 10 items; for 9 of these, BRFSS estimates were significantly higher than NHIS estimates. Only 1 BRFSS estimate (for binge drinking) was significantly lower than the NHIS estimate. In contrast, BRFSS estimates were lower than NHANES estimates for 4 of the 5 items for which differences were found; diabetes was the sole exception.
Table 1.
Summary Results From 6 Questions Common to BRFSS, NHIS, and NHANES Among Respondents Aged 18 or Older
| Question/Characteristic | BRFSS | NHIS | NHANES | ||
|---|---|---|---|---|---|
| Ever smoked cigarettes | % (95% CI) | % (95% CI) | z Test (P)a | % (95% CI) | z Test (P)a |
| Overall | 44.0 (43.7-44.4) | 42.4 (41.6-43.1) | 4.08 (<.001) | 48.0 (45.8-50.2) | −3.79 (<.001) |
| Aged 18-34 y | 36.8 (36.1-37.5) | 33.2 (31.9-34.4) | 4.88 (<.001) | 42.1 (37.3-47.1) | −2.30 (.02) |
| Aged 35-54 y | 44.8 (44.2-45.3) | 43.5 (42.3-44.6) | 2.05 (.04) | 49.2 (46.5-51.9) | −3.39 (.001) |
| Aged ≥55 y | 50.9 (50.4-51.5) | 50.6 (49.4-51.8) | 0.50 (.62) | 55.6 (51.9-59.1) | −2.70 (.007) |
| Male | 49.9 (49.4-50.5) | 48.3 (47.3-49.4) | 2.66 (.008) | 53.4 (50.0-56.8) | −2.14 (.03) |
| Female | 38.5 (38.1-38.9) | 36.8 (35.9-37.8) | 3.21 (.001) | 42.4 (39.7-45.2) | −3.00 (.003) |
| White non-Hispanic | 48.0 (47.6-48.3) | 47.0 (46.1-47.8) | 2.05 (.04) | 50.6 (48.9-52.2) | −3.18 (.001) |
| Black non-Hispanic | 36.7 (35.6-37.8) | 33.3 (31.6-35.1) | 3.26 (.001) | 34.6 (27.7-42.2) | 0.61 (.54) |
| Hispanic | 32.5 (31.2-33.8) | 28.2 (26.6-29.9) | 4.01 (<.001) | 42.8 (36.9-48.9) | −3.55 (<.001) |
| Other non-Hispanic | 36.9 (34.9-38.9) | 30.2 (27.3-33.3) | 3.61 (<.001) | 48.2 (35.7-60.9) | −1.84 (.06) |
| <High school diploma | 49.8 (48.6-51.0) | 46.6 (45.0-48.3) | 3.07 (.002) | 58.8 (51.0-66.1) | −2.48 (.01) |
| High school diploma or some college | 48.0 (47.5-48.5) | 45.7 (44.7-46.8) | 3.95 (<.001) | 55.2 (50.1-60.3) | −3.01 (.003) |
| ≥College diploma | 34.8 (34.2-35.4) | 35.7 (34.6-36.8) | −1.41 (.16) | 43.0 (40.2-45.8) | −6.01 (<.001) |
| Currently smoke cigarettes | % (95% CI) | % (95% CI) | z Test (P)a | % (95% CI) | z Test (P)a |
| Overall | 20.7 (20.4-21.0) | 20.9 (20.3-21.6) | −0.64 (.52) | 23.6 (21.4-25.9) | −2.70 (.007) |
| Aged 18-34 y | 25.2 (24.5-25.8) | 23.6 (22.5-24.8) | 2.35 (.02) | 30.3 (26.8-34.1) | −2.90 (.004) |
| Aged 35-54 y | 23.3 (22.8-23.7) | 23.8 (22.9-24.8) | −1.07 (.28) | 24.5 (20.4-29.1) | −0.59 (.55) |
| Aged ≥55 y | 12.7 (12.4-13.1) | 14.2 (13.4-15.0) | −3.41 (.001) | 10.8 (9.3-12.4) | 2.68 (.007) |
| Male | 23.0 (22.6-23.5) | 23.5 (22.7-24.4) | −0.93 (.35) | 25.8 (22.8-29.1) | −1.85 (.06) |
| Female | 18.5 (18.2-18.8) | 18.6 (17.8-19.3) | −0.10 (.92) | 21.3 (18.5-24.4) | −2.01 (.04) |
| White non-Hispanic | 21.4 (21.0-21.7) | 22.3 (21.6-23.1) | −2.23 (.03) | 23.6 (21.1-26.3) | −1.77 (.07) |
| Black non-Hispanic | 21.9 (21.0-22.9) | 20.4 (18.8-22.1) | 1.55 (.12) | 21.0 (16.1-26.9) | 0.36 (.72) |
| Hispanic | 16.8 (15.8-17.8) | 15.0 (13.8-16.3) | 2.19 (.03) | 27.3 (22.3-32.9) | −4.12 (<.001) |
| Other non-Hispanic | 19.8 (18.3-21.4) | 16.5 (14.0-19.5) | 2.00 (.04) | 21.7 (15.0-30.2) | −0.52 (.60) |
| <High school diploma | 29.1 (28.1-30.2) | 26.8 (25.3-28.4) | 2.39 (.02) | 33.3 (29.0-38.0) | −1.93 (.05) |
| High school diploma or some college | 24.4 (24.0-24.8) | 24.1 (23.3-25.0) | 0.67 (.50) | 30.6 (26.1-35.6) | −2.79 (.005) |
| ≥College diploma | 10.8 (10.5-11.2) | 13.6 (12.7-14.4) | −5.90 (<.001) | 18.9 (16.9-21.1) | −7.99 (<.001) |
| Ever told have diabetes | % (95% CI) | % (95% CI) | z Test (P)a | % (95% CI) | z Test (P)a |
| Overall | 8.1 (7.9-8.2) | 8.1 (7.8-8.5) | −0.20 (.84) | 6.1 (5.1-7.4) | 3.45 (.001) |
| Aged 18-34 y | 1.4 (1.3-1.6) | 1.5 (1.3-1.8) | −0.62 (.54) | 1.1 (0.6-2.1) | 0.97 (.33) |
| Aged 35-54 y | 6.4 (6.2-6.7) | 6.4 (5.9-7.0) | 0.07 (.95) | 5.9 (4.6-7.6) | 0.76 (.45) |
| Aged ≥55 y | 17.2 (16.8-17.7) | 17.3 (16.4-18.2) | −0.10 (.92) | 16.6 (13.5-20.2) | 0.39 (.70) |
| Male | 8.3 (8.0-8.6) | 8.3 (7.8-8.8) | 0.17 (.87) | 5.2 (4.2-6.4) | 5.62 (<.001) |
| Female | 7.8 (7.6-8.0) | 7.9 (7.5-8.4) | −0.51 (.61) | 7.1 (5.9-8.6) | 1.03 (.30) |
| White non-Hispanic | 7.6 (7.5-7.8) | 7.7 (7.3-8.1) | −0.09 (.93) | 5.8 (4.5-7.5) | 2.57 (.01) |
| Black non-Hispanic | 11.4 (10.7-12.2) | 10.7 (9.8-11.8) | 1.11 (.27) | 8.4 (6.8-10.3) | 3.42 (.001) |
| Hispanic | 7.6 (7.0-8.4) | 8.2 (7.3-9.1) | −0.91 (.36) | 6.1 (3.7-9.8) | 1.08 (.28) |
| Other non-Hispanic | 8.5 (7.4-9.8) | 8.5 (6.7-10.8) | 0.01 (.99) | 5.8 (2.6-12.1) | 1.29 (.20) |
| <High school diploma | 13.0 (12.2-13.8) | 12.4 (11.5-13.4) | 0.94 (.35) | 12.2 (9.5-15.7) | 0.50 (.62) |
| High school diploma or some college | 8.3 (8.0-8.5) | 8.0 (7.6-8.5) | 0.83 (.41) | 6.3 (5.0-7.8) | 3.12 (.002) |
| ≥College diploma | 5.7 (5.4-6.0) | 6.1 (5.6-6.7) | −1.30 (.20) | 4.7 (3.6-6.2) | 1.66 (.10) |
| Self-reported height | Inches (95% CI) | Inches (95% CI) | z Test (P)a | Inches (95% CI) | z Test (P)a |
| Overall | 67.1 (67.1-67.1) | 67.0 (66.9-67.0) | 3.88 (<.001) | 67.1 (66.8-67.4) | 0 (>.99) |
| Aged 18-34 y | 67.5 (67.4-67.5) | 67.3 (67.2-67.4) | 2.34 (.02) | 67.4 (67.2-67.7) | 0.22 (.82) |
| Aged 35-54 y | 67.3 (67.3-67.4) | 67.1 (67.0-67.2) | 4.25 (<.001) | 67.4 (67.0-67.7) | −0.23 (.82) |
| Aged ≥55 y | 66.4 (66.4-66.5) | 66.4 (66.3-66.4) | 1.30 (.19) | 65.9 (65.6-66.2) | 3.61 (<.001) |
| Male | 70.0 (70.0-70.1) | 69.9 (69.9-70.0) | 3.33 (.001) | 69.8 (69.5-70.0) | 2.20 (.03) |
| Female | 64.3 (64.3-64.3) | 64.2 (64.2-64.3) | 3.13 (.002) | 64.4 (64.1-64.6) | −0.50 (.62) |
| White non-Hispanic | 67.5 (67.4-67.5) | 67.2 (67.2-67.3) | 6.10 (<.001) | 67.4 (67.2-67.7) | 0.23 (.82) |
| Black non-Hispanic | 67.1 (67.0-67.2) | 67.0 (66.9-67.2) | 0.74 (.46) | 67.1 (66.9-67.3) | 0.18 (.86) |
| Hispanic | 65.5 (65.3-65.6) | 65.7 (65.6-65.9) | −2.93 (.003) | 65.1 (64.6 -65.5) | 1.81 (.07) |
| Other non-Hispanic | 66.4 (66.2-66.6) | 65.4 (65.1-65.6) | 5.75 (<.001) | 66.1 (65.6-66.6) | 1.14 (.26) |
| <High school diploma | 65.8 (65.7-66.0) | 66.1 (66.0-66.2) | −2.93 (.003) | 66.0 (65.6-66.4) | −0.79 (.43) |
| High school diploma or some college | 67.0 (67.0-67.0) | 67.0 (66.9-67.0) | 1.34 (0.18) | 67.1 (66.8-67.4) | −0.66 (.51) |
| ≥College diploma | 67.7 (67.7-67.8) | 67.3 (67.3-67.4) | 7.83 (<.001) | 67.4 (67.0-67.7) | 2.16 (.03) |
| Self-reported weight | Pounds (95% CI) | Pounds (95% CI) | z Test (P)a | Pounds (95% CI) | z Test (P)a |
| Overall | 173.0 (172.7-173.3) | 170.9 (170.4-171.5) | 6.71 (<.001) | 176.6 (174.2-178.9) | −3.20 (.001) |
| Aged 18-34 y | 168.5 (167.9-169.1) | 166.3 (165.3-167.4) | 3.60 (<.001) | 171.5 (167.4-175.5) | −1.55 (.12) |
| Aged 35-54 y | 178.0 (177.5-178.4) | 174.6 (173.7-175.5) | 6.59 (<.001) | 183.2 (180.6-185.9) | −4.16 (<.001) |
| Aged ≥55 y | 171.4 (171.0-171.9) | 170.8 (169.9-171.7) | 1.17 (.24) | 174.7 (171.7-177.6) | −2.31 (.02) |
| Male | 191.0 (190.6-191.5) | 189.2 (188.5-189.9) | 4.36 (<.001) | 192.4 (190.2-194.6) | −1.27 (.20) |
| Female | 155.3 (155.0-155.6) | 153.4 (152.8-154.1) | 4.96 (<.001) | 160.3 (156.6-164.0) | −2.85 (.004) |
| White non-Hispanic | 173.7 (173.4-174.0) | 171.3 (170.6-171.9) | 6.60 (<.001) | 176.7 (174.0-179.3) | −2.39 (.02) |
| Black non-Hispanic | 183.0 (182.0-184.0) | 178.9 (177.4-180.4) | 4.41 (<.001) | 189.9 (183.8-196.0) | −2.40 (.02) |
| Hispanic | 166.5 (165.5-167.6) | 168.0 (166.8-169.3) | −1.80 (.07) | 168.9 (164.9-172.9) | −1.22 (.22) |
| Other non-Hispanic | 161.3 (159.8-162.9) | 152.6 (150.3-154.9) | 6.16 (<.001) | 158.7 (148.4-169.0) | 0.53 (.60) |
| <High school diploma | 170.6 (169.6-171.5) | 168.9 (167.7-170.1) | 2.05 (.04) | 174.2 (170.0-178.4) | −1.80 (.07) |
| High school diploma or some college | 174.4 (174.0-174.8) | 172.2 (171.4-173.0) | 4.66 (<.001) | 181.0 (178.1-183.9) | −4.78 (<.001) |
| ≥College diploma | 171.6 (171.1-172.1) | 170.0 (169.2-170.9) | 3.08 (.002) | 175.3 (172.0-178.6) | −2.35 (.02) |
| BMI (calculated from height and weight) | BMI (95% CI) | BMI (95% CI) | z Test (P)a | BMI (95% CI) | z Test (P)a |
| Overall | 27.0 (26.9-27.0) | 27.0 (26.9-27.1) | −1.12 (.26) | 27.6 (27.2-27.9) | −3.20 (.001) |
| Aged 18-34 y | 26.0 (25.9-26.1) | 26.0 (25.8-26.2) | 0.32 (.75) | 26.4 (25.9 -27.0) | −1.69 (.09) |
| Aged 35-54 y | 27.5 (27.5-27.6) | 27.6 (27.4-27.7) | −0.39 (.69) | 28.4 (28.0 -28.8) | −4.01 (<.001) |
| Aged ≥55 y | 27.2 (27.2-27.3) | 27.4 (27.3-27.5) | −2.39 (.02) | 28.2 (27.7-28.7) | −4.31 (<.001) |
| Male | 27.4 (27.4-27.5) | 27.4 (27.4-27.6) | −0.34 (.73) | 27.8 (27.5-28.1) | −2.70 (.007) |
| Female | 26.5 (26.4-26.6) | 26.6 (26.5-26.7) | −1.31 (.19) | 27.3 (26.6-28.0) | −2.43 (.015) |
| White non-Hispanic | 26.7 (26.7-26.7) | 26.8 (26.7-26.9) | −2.04 (.04) | 27.3 (26.8-27.8) | −2.58 (.01) |
| Black non-Hispanic | 28.6 (28.4-28.7) | 28.7 (28.4-28.9) | −0.54 (.59) | 29.8 (29.0-30.6) | −3.19 (.001) |
| Hispanic | 27.6 (27.4-27.8) | 27.6 (27.4-27.8) | −0.30 (.77) | 28.1 (27.4-28.8) | −1.43 (.15) |
| Other non-Hispanic | 25.7 (25.4-26.0) | 24.7 (24.4-25.0) | 4.47 (<.001) | 25.4 (24.1-26.7) | 0.41 (.68) |
| <High school diploma | 27.9 (27.8-28.1) | 27.5 (27.3-27.7) | 3.27 (.001) | 28.2 (27.5-28.8) | −0.83 (.41) |
| High school diploma or some college | 27.2 (27.2-27.3) | 27.3 (27.2-27.4) | −1.19 (.23) | 28.2 (27.9-28.5) | −6.77 (<.001) |
| ≥College diploma | 26.2 (26.1-26.2) | 26.4 (26.3-26.5) | −3.88 (<.001) | 27.1 (26.6-27.7) | −3.74 (<.001) |
BRFSS indicates Behavioral Risk Factor Surveillance System; NHIS, National Health Interview Survey; NHANES, National Health and Nutrition Examination Survey; CI, confidence interval; BMI, body mass index.
P values represent comparison with BRFSS.
Table 4.
Summary Results From 2 Questions Common to BRFSS and NHIS Among Respondents Aged 65 years or Older
| Question/Characteristic | BRFSS% (95% CI) | NHIS% (95% CI) | z Test(P) |
|---|---|---|---|
| Had an influenza vaccination in the past 12 months | |||
| Overall | 67.6 (66.9-68.2) | 64.6 (63.2-66.0) | 3.64 (<.001) |
| Male | 68.9 (67.7-70.0) | 64.1 (61.9-66.2) | 3.81 (<.001) |
| Female | 66.6 (65.8-67.5) | 65.0 (63.3-66.7) | 1.68 (.09) |
| White non-Hispanic | 70.2 (69.6-70.9) | 67.3 (65.8-68.9) | 3.36 (.001) |
| Black non-Hispanic | 49.6 (46.5-52.7) | 45.4 (41.0-49.8) | 1.54 (.12) |
| Hispanic | 57.9 (53.6-62.0) | 54.6 (49.3-59.7) | 0.97 (.33) |
| Other non-Hispanic | 65.0 (58.4-71.0) | 62.2 (50.0-72.9) | 0.42 (.68) |
| <High school diploma | 61.5 (59.7-63.2) | 58.7 (56.0-61.4) | 1.69 (.09) |
| High school diploma or some college | 68.0 (67.2-68.9) | 65.0 (62.8-67.1) | 2.61 (.009) |
| ≥College diploma | 71.0 (69.6-72.4) | 71.1 (68.4-73.7) | −0.08 (.94) |
| Ever had a pneumonia vaccination | |||
| Overall | 63.4 (62.7-64.1) | 56.8 (55.3-58.3) | 7.66 (<.001) |
| Male | 61.8 (60.6-63.0) | 54.3 (51.7-56.8) | 5.30 (<.001) |
| Female | 64.5 (63.6-65.4) | 58.7 (56.9-60.6) | 5.46 (<.001) |
| White non-Hispanic | 66.9 (66.3-67.6) | 60.9 (59.3-62.5) | 6.67 (<.001) |
| Black non-Hispanic | 43.0 (40.0-46.1) | 38.5 (34.0-43.2) | 1.61 (.11) |
| Hispanic | 46.9 (42.4-51.5) | 33.7 (29.0-38.9) | 3.85 (<.001) |
| Other non-Hispanic | 53.7 (46.5-60.8) | 37.4 (27.9-48.0) | 2.58 (.01) |
| <High school diploma | 55.2 (53.3-57.0) | 49.3 (46.5-52.0) | 3.46 (.001) |
| High school diploma or some college | 65.1 (64.2-66.0) | 58.5 (56.2-60.8) | 5.23 (<.001) |
| ≥College diploma | 65.5 (64.0-66.9) | 62.9 (59.9-65.8) | 1.54 (.12) |
BRFSS indicates Behavioral Risk Factor Surveillance System; NHIS, National Health Interview Survey; CI, confidence interval.
The magnitude of differences varied widely. For example, the relative difference between BRFSS and NHIS estimates of height is less than 1% (derived by dividing the difference [0.14 inches] between the BRFSS and NHIS overall point estimates by the NHIS overall point estimate [66.96 inches]). For weight, the relative difference is 1.2% (2.13%/170.90%). The relative difference in overall estimates of ever having smoked a cigarette is 4.0% (1.68%/42.35%) and of having received an influenza vaccination in the past 12 months, 4.6% (2.95%/64.63%). At the other end of the spectrum, however, the relative differences in BRFSS and NHIS estimates are 94.4% (7.35%/7.79%) for experiencing cost barriers to medical care, 35.1% (3.48%/9.91%) for having asthma, 33.9% (4.16%/12.28%) for reporting poor or fair health, 26.4% (9.15%/34.61%) for having ever had an HIV test, and 11.5% (6.54%/56.84%) for having ever had a pneumonia vaccination. In contrast, the estimate for binge drinking is 8.3% (0.39 drinks/4.70 drinks) lower for BRFSS than for NHIS.
As a relative percentage of the NHANES estimates, BRFSS estimates show less variation. For diabetes, BRFSS estimates are 31.3% (1.92%/6.14%) higher, but they are 12.2% (2.89%/23.60%) lower for current smoking, 8.3% (3.96%/47.99%) lower for ever having smoked a cigarette, 2.1% lower for BMI (0.58/27.55), and 2.0% (3.55 lb/176.58 lb) lower for self-reported weight.
BRFSS and NHIS estimates for diabetes prevalence were similar for all 12 population subgroups. The following tabulation shows the number of subgroups for which BRFSS and NHIS estimates were statistically similar:
| Survey Question | No. of Subgroups Similar/No. Subgroups Surveyed |
|---|---|
| No health insurance | 10/12 |
| Binge drinking | 8/12 |
| BMI | 7/12 |
| Influenza vaccination | 6/9 |
| Number of alcohol drinks | 5/12 |
| Current smoker | 5/12 |
| Self-reported height | 3/12 |
| Pneumonia vaccination | 2/9 |
| Self-reported weight | 2/12 |
| Ever smoked | 2/12 |
The estimates for fair or poor health status, asthma, having been tested for HIV, and not receiving medical care because of cost were significantly different between all subgroups examined.
Similarity between BRFSS and NHANES estimates among the 12 subgroups was greatest for self-reported height (9 subgroups), followed by diabetes (8 subgroups), current smoker (6 subgroups), self-reported weight (5 subgroups), BMI (4 subgroups), and ever having smoked a cigarette (2 subgroups).
The estimates provided by the 3 surveys for the 4 demographic characteristics (age, sex, race/ethnicity, and education) are not entirely consistent despite being indexed to population totals; estimates for educational attainment differ most notably.
Discussion
Some of the estimates from the 2004 BRFSS, NHIS, and the 2003–2004 NHANES vary overall and within certain subgroups. While a number of these differences are small and may not warrant more examinations, other differences — especially those that reflect a relative difference of more than 20% — may require further investigations. The BRFSS estimates tended to be higher than NHIS estimates but lower than NHANES estimates. For the 6 items examined in all 3 surveys, 1 of the BRFSS estimates (self-reported height) was identical to NHANES, 2 fell between those of NHIS and NHANES (ever smoke cigarettes and self-reported weight), and 3 were statistically identical to NHIS (current smoking status, diabetes, and BMI).
The extent of similarity between the 2004 BRFSS and NHIS estimates was less than what was reported by Nelson and colleagues (9) for the 1997 comparisons. They found statistically significant differences between overall estimates for 6 of 14 items examined; BRFSS estimates were lower for current smoking, height, and BMI and higher for pneumococcal vaccination, cost as a barrier to medical care, and reporting fair or poor health.
Several explanations may account for the inconsistencies
BRFSS estimates and estimates from the other 2 surveys, including 1) differences in questionnaire wording, 2) mode of survey administration, 3) sampling design differences and related postsurvey adjustments such as weighting procedures, and 4) use of incentives or proxy data.
Differences in questionnaire wording represent some of the greatest challenges in comparing survey results. The wording of the BRFSS and NHIS questionnaires was identical for only 4 of the items examined (ever smoked cigarettes, ever had a pneumonia vaccination, health status, and asthma) and for 1 of the NHANES items (ever smoked cigarettes). Some questions that seemed to be measuring the same concept differed in ways that may have led to differences in their final estimates. For example, the questions used to assess binge drinking are dissimilar in wording, time frame, and response categories. BRFSS asks, "Considering all types of alcoholic beverages, how many times during the past 30 days did you have 5 or more drinks on an occasion?"; NHIS asks, "In the past year, on how many days did you have 5 or more drinks of any alcoholic beverages?" Although both surveys attempt to measure the same concept, differences in wording and time frame could explain the small but statistically significant differences in estimates that we observed.
Even with identical wording, however, different modes of survey administration can produce varying results and magnify differences in measurement (13,14). BRFSS is a telephone survey, whereas NHIS and NHANES are conducted face-to-face. For BRFSS the interaction is aural and limited to voice communication, whereas NHIS and NHANES rely on both aural and visual cues. Such differences in the patterns of interviewer–respondent communication and situational contexts may explain some of the variation in survey estimates.
Differences in weighting adjustment to compensate for design-imposed differential selection probabilities may also explain some of the observed differences in survey estimates. The comparison of weighted estimates for basic demographic characteristics, such as race/ethnicity and education, shows that even when weights are applied, differences remain between survey respondents. These differences may lead, in part, to some of the inconsistencies in health and risk estimates between the 3 surveys. In addition, the 3 surveys use different sources to obtain population control totals for weight calibrations. BRFSS uses population estimates from Claritas (Claritas Inc, San Diego, California), a private data vendor that uses census projections as part of its process for developing yearly population estimates, whereas NHIS uses census projections and NHANES relies on the Current Population Survey estimates. These sources for population control totals vary in their analysis of the U.S. population by sex, age, and race/ethnicity. In addition, the weighting procedures for both NHIS and NHANES account for differences in the race/ethnicity distribution in each state, whereas BRFSS includes this type of adjustment in selected states only. Education was the socioeconomic indicator that differed most markedly between the 3 surveys. BRFSS is revising its weighting methodology to account for educational differences; the new weight will be applied to public release data sets in 2010 (15).
Other factors, perhaps minor, may also contribute to differences in survey estimates. For instance, BRFSS and NHIS do not offer any incentives for survey participation, but NHANES relies on incentives to secure higher rates of response. The use of incentives may alter a participant's survey response or attract a different mix of participants (16). Also, differences between BRFSS and NHIS estimates may be affected by the use of proxy data in NHIS. Although BRFSS does not allow proxy respondents, some NHIS estimates (e.g., cost as a barrier to medical care, health status) can be based on proxy data. BRFSS and NHIS provide significantly different estimates for both of these measures. Hays et al report that agreement between self and proxy responses are often good for measures of function that are directly observable but poor for subjective measures (17). Moreover, Nelson and colleagues suggest that proxy-derived data are reliable for demographic and body measures but are less reliable for questions on medications and alcohol consumption (18).
Although the many differences in design, procedures, and context of these surveys may confound our ability to single out the proximal causes of differences in survey estimates noted in this research, the comparison of estimates for similar concepts is important. These periodic comparisons are particularly important because BRFSS, NHIS, and NHANES are critical components of the U.S. public health system, with each providing essential data for policy makers, researchers, and the public alike. The periodic examination of when and why measurements of similar concepts vary is equally important.
These research findings also need to be interpreted in light of tangible implications. Although many of the estimates examined here differed statistically, their programmatic implications and impact on actionable policy are subject to debate. What constitutes a "significant" difference is ultimately up to the people who use the data. Perhaps the good news for BRFSS is that despite declines in survey participation rates during the past decade, BRFSS estimates do not appear to be radically different from similar estimates produced by NHIS and NHANES. In fact, for the 6 common items in these surveys, BRFSS estimates were either similar to NHIS or NHANES estimates or were found between the estimates of these 2 surveys.
Table 2.
Summary Results From 5 Questions Common to BRFSS and NHIS Among Respondents Aged 18 or Older
| Question/Characteristic | BRFSS | NHIS | Statistic |
|---|---|---|---|
| Fair or poor health status | % (95% CI) | % (95% CI) | z Test (P) |
| Overall | 16.4 (16.2-16.7) | 12.3 (11.8-12.7) | 15.45 (<.001) |
| Aged 18-34 y | 9.1 (8.6-9.6) | 4.5 (4.0-5.0) | 13.37 (<.001) |
| Aged 35-54 y | 14.7 (14.2-15.1) | 10.5 (9.9-11.1) | 10.92 (<.001) |
| Aged ≥55 y | 26.7 (26.2-27.2) | 22.9 (22.0-23.9) | 6.97 (<.001) |
| Male | 15.2 (14.7-15.6) | 11.4 (10.8-12.0) | 10.19 (<.001) |
| Female | 17.6 (17.3-18.0) | 13.1 (12.5-13.7) | 12.64 (<.001) |
| White non-Hispanic | 13.7 (13.5-14.0) | 11.4 (10.9-12.0) | 7.78 (<.001) |
| Black non-Hispanic | 19.9 (19.0-20.8) | 18.0 (16.6-19.5) | 2.15 (.03) |
| Hispanic | 27.9 (26.7-29.2) | 13.0 (11.9-14.1) | 17.57 (<.001) |
| Other non-Hispanic | 14.1 (12.8-15.4) | 9.3 (7.4-11.8) | 3.64 (<.001) |
| <High school diploma | 39.0 (37.8-40.2) | 25.9 (24.5-27.4) | 13.86 (<.001) |
| High school diploma or some college | 16.5 (16.2-16.9) | 11.9 (11.3-12.5) | 13.29 (<.001) |
| ≥College diploma | 7.3 (7.0-7.6) | 6.1 (5.6-6.7) | 3.73 (<.001) |
| Ever told have asthma | % (95% CI) | % (95% CI) | z Test (P) |
| Overall | 13.4 (13.1-13.6) | 9.9 (9.5-10.3) | 14.09 (<.001) |
| Aged 18-34 y | 15.3 (14.8-15.9) | 10.3 (9.6-11.1) | 10.41 (<.001) |
| Aged 35-54 y | 12.8 (12.4-13.2) | 9.6 (9.0-10.2) | 9.07 (<.001) |
| Aged ≥55 y | 12.2 (11.8-12.5) | 9.9 (9.3-10.6) | 5.69 (<.001) |
| Male | 11.6 (11.3-12.0) | 8.5 (7.9-9.1) | 8.64 (<.001) |
| Female | 15.0 (14.7-15.4) | 11.2 (10.7-11.8) | 11.53 (<.001) |
| White non-Hispanic | 13.5 (13.2-13.8) | 10.2 (9.7-10.8) | 11.18 (<.001) |
| Black non-Hispanic | 14.8 (14.0-15.6) | 11.3 (10.2-12.6) | 4.63 (<.001) |
| Hispanic | 11.8 (11.0 -12.7) | 7.5 (6.6-8.5) | 6.71 (<.001) |
| Other non-Hispanic | 14.0 (12.7-15.5) | 7.7 (6.1-9.7) | 5.42 (<.001) |
| <High school diploma | 14.3 (13.5-15.1) | 10.7 (9.8-11.8) | 5.39 (<.001) |
| High school diploma or some college | 13.6 (13.2-13.9) | 10.0 (9.4-10.7) | 9.96 (<.001) |
| ≥College diploma | 12.8 (12.4-13.2) | 9.5 (8.9-10.1) | 8.55 (<.001) |
| Ever had HIV test | % (95% CI) | % (95% CI) | z Test (P) |
| Overall | 43.8 (43.4-44.2) | 34.6 (33.9-35.3) | 21.51 (<.001) |
| Aged 18-34 y | 51.6 (50.8-52.4) | 43.9 (42.5-45.3) | 9.56 (<.001) |
| Aged 35-54 y | 44.1 (43.5-44.7) | 40.6 (39.4-41.8) | 5.24 (<.001) |
| Aged ≥55 y | 23.6 (22.9-24.5) | 16.5 (15.6-17.4) | 11.73 (<.001) |
| Male | 41.2 (40.6-41.9) | 31.7 (30.7-32.7) | 15.96 (<.001) |
| Female | 46.2 (45.7-46.8) | 37.3 (36.4-38.3) | 16.06 (<.001) |
| White non-Hispanic | 40.3 (39.9-40.7) | 31.5 (30.6-32.4) | 17.95 (<.001) |
| Black non-Hispanic | 62.6 (61.4-63.9) | 51.4 (49.5-53.2) | 9.87 (<.001) |
| Hispanic | 45.9 (44.4-47.30 | 38.2 (36.5-39.9) | 6.80 (<.001) |
| Other non-Hispanic | 43.3 (41.1-45.6) | 32.7 (29.5-36.1) | 5.21 (<.001) |
| <High school diploma | 41.8 (40.3-43.3) | 30.7 (29.1-32.4) | 9.84 (<.001) |
| High school diploma or some college | 43.1 (42.6-43.7) | 32.8 (31.8-33.7) | 18.32 (<.001) |
| ≥College diploma | 45.6 (44.9-46.3) | 39.2 (38.0-40.3) | 9.72 (<.001) |
| Binge drinking past 30 days | % (95% CI) | % (95% CI) | z Test (P) |
| Overall | 4.3 (4.2-4.4) | 4.7 (4.5-4.9) | −2.91 (.004) |
| Aged 18-34 y | 4.2 (4.1-4.4) | 4.2 (3.9-4.5) | 0.51 (.61) |
| Aged 35-54 y | 4.1 (4.0-4.3) | 5.0 (4.6-5.4) | −3.97 (<.001) |
| Aged ≥55 y | 5.3 (4.9-5.8) | 6.2 (5.3-7.1) | −1.80 (.07) |
| Male | 4.7 (4.6-4.9) | 5.0 (4.8-5.3) | −1.86 (.06) |
| Female | 3.0 (3.0-3.2) | 3.6 (3.2-3.9) | −2.42 (.02) |
| White non-Hispanic | 4.3 (4.2-4.4) | 4.7 (4.5-5.0) | −2.75 (.006) |
| Black non-Hispanic | 4.8 (4.2-5.3) | 5.7 (4.8-6.6) | −1.73 (.08) |
| Hispanic | 3.8 (3.5-4.1) | 4.1 (3.7-4.6) | −1.07 (.28) |
| Other non-Hispanic | 5.3 (4.3-6.4) | 4.2 (2.6-5.8) | 1.19 (.23) |
| <High school diploma | 5.1 (4.6-5.6) | 5.8 (5.1-6.5) | −1.55 (.12) |
| High school diploma or some college | 4.6 (4.4-4.8) | 4.9 (4.6-5.3) | −1.73 (.08) |
| ≥College diploma | 3.4 (3.3-3.6) | 3.8 (3.4-4.1) | −1.97 (.05) |
| Average number of alcoholic drinks per occasion | No. (95% CI) | No. (95% CI) | z Test (P) |
| Overall | 2.4 (2.4-2.5) | 2.5 (2.4-2.5) | −1.06 (.29) |
| Aged 18-34 y | 3.2 (3.1-3.3) | 3.2 (3.1-3.3) | 0 (>.99) |
| Aged 35-54 y | 2.3 (2.2-2.3) | 2.3 (2.2-2.3) | −0.71 (.48) |
| Aged ≥55 y | 1.7 (1.6-1.7) | 1.8 (1.7-1.8) | −2.85 (.004) |
| Male | 2.9 (2.8-2.9) | 2.9 (2.8-3.0) | −0.80 (.42) |
| Female | 1.9 (1.9-1.9) | 1.9 (1.9-2.0) | −2.68 (.007) |
| White non-Hispanic | 2.3 (2.3-2.3) | 2.4 (2.3-2.4) | −3.79 (<.001) |
| Black non-Hispanic | 2.3 (2.2-2.4) | 2.4 (2.2-2.5) | −0.74 (.46) |
| Hispanic | 3.4 (3.2-3.6) | 3.0 (2.9-3.2) | 3.33 (.001) |
| Other non-Hispanic | 2.6 (2.4-2.8) | 2.2 (2.1-2.4) | 2.39 (.02) |
| <High school diploma | 3.6 (3.4-3.8) | 3.1 (3.0-3.3) | 3.52 (<.001) |
| High school diploma or some college | 2.6 (2.6-2.6) | 2.7 (2.6-2.8) | −1.57 (.12) |
| ≥College diploma | 2.0 (1.9-2.0) | 2.0 (2.0-2.1) | −2.12 (.03) |
BRFSS indicates Behavioral Risk Factor Surveillance System; NHIS, National Health Interview Survey; CI, confidence interval; HIV, human immunodeficiency virus.
Table 3.
Summary Results From 2 Questions Common to BRFSS and NHIS Among Respondents Aged 18 to 64 years
| Question/Characteristic | BRFSS% (95% CI) | NHIS% (95% CI) | z Test (P) |
|---|---|---|---|
| No medical care due to cost | |||
| Overall | 15.1 (14.8-15.4) | 7.8 (7.4-8.2) | 30.36 (<.001) |
| Aged 18-34 y | 17.6 (17.0-18.2) | 7.6 (7.0-8.4) | 21.52 (<.001) |
| Aged 35-54 y | 14.6 (14.2-15.0) | 8.0 (7.5-8.6) | 19.52 (<.001) |
| Aged 55-64 y | 10.9 (10.4-11.5) | 7.4 (6.6-8.3) | 6.78 (<.001) |
| Male | 13.0 (12.5-13.4) | 6.8 (6.3-7.4) | 17.34 (<.001) |
| Female | 17.3 (16.9-17.7) | 8.7 (8.2-9.3) | 25.51 (<.001) |
| White non-Hispanic | 12.7 (12.4-13.0) | 7.6 (7.1-8.0) | 18.98 (<.001) |
| Black non-Hispanic | 20.3 (19.4-21.3) | 9.7 (8.8-10.8) | 15.11 (<.001) |
| Hispanic | 21.7 (20.6-22.9) | 8.3 (7.5-9.3) | 17.84 (<.001) |
| Other non-Hispanic | 16.2 (14.8-17.8) | 4.8 (3.6-6.3) | 11.07 (<.001) |
| <High school diploma | 27.6 (26.4-28.9) | 11.4 (10.4-12.6) | 18.75 (<.001) |
| High school diploma or some college | 16.5 (16.1-16.9) | 8.8 (8.3-9.3) | 22.09 (<.001) |
| ≥College diploma | 8.4 (8.0-8.8) | 5.0 (4.6-5.5) | 10.85 (<.001) |
| No health insurance coverage | |||
| Overall | 18.7 (18.4-19.1) | 19.3 (18.7-20.0) | −1.67 (.09) |
| Aged 18-34 y | 25.9 (25.2-26.6) | 27.0 (25.9-28.2) | −1.67 (.10) |
| Aged 35-54 y | 15.4 (15.0-15.9) | 15.8 (15.1-16.6) | −0.93 (.35) |
| Aged 55-64 y | 11.3 (10.8-11.9) | 11.7 (10.6-12.9) | −0.65 (.52) |
| Male | 20.3 (19.7-20.8) | 21.3 (20.4-22.2) | −1.77 (.08) |
| Female | 17.2 (16.8-17.6) | 17.5 (16.8-18.2) | −0.64 (.52) |
| White non-Hispanic | 13.6 (13.3-13.9) | 14.1 (13.4-14.7) | −1.29 (.20) |
| Black non-Hispanic | 23.2 (22.2-24.3) | 22.9 (21.2-24.7) | 0.34 (.73) |
| Hispanic | 37.2 (35.8-38.6) | 42.8 (41.0-44.5) | −4.87 (<.001) |
| Other non-Hispanic | 19.2 (17.6-21.0) | 20.5 (17.5-23.9) | −0.68 (.50) |
| <High school diploma | 43.6 (42.2-45.2) | 41.1 (39.2-43.0) | 2.10 (.04) |
| High school diploma or some college | 19.8 (19.4-20.3) | 20.4 (19.6-21.2) | −1.10 (.27) |
| ≥College diploma | 8.1 (7.7-8.5) | 8.8 (8.0-9.6) | −1.51 (.13) |
BRFSS indicates Behavioral Risk Factor Surveillance System; NHIS, National Health Interview Survey; CI, confidence interval.
Appendix
Comparison of Questions Used in Study of Population Estimates Generated by 3 National Health Surveys: BRFSS, NHIS, and NHANES
| Domain | BRFSS | Comparison Questiona |
|---|---|---|
| Ever smoked | Have you smoked at least 100 cigarettes in your entire life? | — |
| Current smoker | — | NHIS: Do you now smoke cigarettes every day, some days, or not at all? NHANES: Do you now smoke cigarettes every day or some days? |
| Diabetes | Have you ever been told by a doctor that you have diabetes? | NHIS: Other than during pregnancy, have you EVER been told by a doctor or health care professional that you have diabetes or sugar diabetes? NHANES: Other than during pregnancy, have you ever been told by a doctor or health care professional that you have diabetes or sugar diabetes? |
| Influenza vaccination | During the past 12 months, have you had a flu shot? | NHIS: During the past 12 months, have you had a flu shot? A flu shot is usually given in the fall and protects against influenza for the flu season. |
| Pneumonia vaccination | Have you ever had a pneumonia shot? This shot is usually given only once or twice in a person's lifetime and is different from the flu shot. It is also called the pneumococcal vaccine. | — |
| No health insurance coverage | Do you have any kind of health care coverage, including health insurance, prepaid plans such as HMOs, or government plans such as Medicare? | NHIS: I have recorded you are (covered by/not covered by) health insurance. Is this correct? |
| Do not receive medical care due to cost | Was there a time in the past 12 months when you needed to see a doctor but could not because of cost? | NHIS: During the past 12 months, was there any time when [you/someone in the family] needed medical care, but did not get it because [you/the family] couldn't afford it? |
| Health status | Would you say that in general your health is excellent, very good, good, fair, or poor? | — |
| Asthma | Have you ever been told by a doctor, nurse, or other health professional that you had asthma? | — |
| Testing for human immunodeficiency virus | Have you ever been tested for HIV? Do not count tests you may have had as part of a blood donation. | NHIS: The next questions are about the test for HIV (the virus that causes AIDS). Except for tests you may have had as part of blood donations, have you ever been tested for HIV? |
| Height | About how tall are you without shoes? | NHIS: Total height in inches. NHANES: Standing height (cm). |
| Weight | About how much do you weigh without shoes? | NHIS: How much do you weigh without shoes? NHANES: Weight (kg). |
| Average number of alcoholic drinks per occasion | On the days when you drank, about how many drinks did you drink on the average? (response range: 1-76) | NHIS: In the past year, on those days that you drank alcoholic beverages, on the average, how many drinks did you have? |
| Binge drinking | Considering all types of alcoholic beverages, how many times during the past 30 days did you have 5 or more drinks on an occasion? (response range: 1-76) | NHIS: In the past year, on how many days did you have 5 or more drinks of any alcoholic beverages? NHANES: How many days per week, per month, or per year did you have 5 or more drinks on a single day? |
BRFSS indicates Behavioral Risk Factor Surveillance System; NHIS, National Health Interview Survey; NHANES, National Health and Nutrition Examination Survey.
Text of questions for NHIS is shown only when different from BRFSS.
Footnotes
The opinions expressed by authors contributing to this journal do not necessarily reflect the opinions of the US Department of Health and Human Services, the Public Health Service, the Centers for Disease Control and Prevention, or the authors’ affiliated institutions. Use of trade names is for identification only and does not imply endorsement by any of the groups named above. URLs for nonfederal organizations are provided solely as a service to our users. URLs do not constitute an endorsement of any organization by CDC or the federal government, and none should be inferred. CDC is not responsible for the content of Web pages found at these URLs.
Suggested citation for this article: Fahimi M, Link M, Schwartz DA, Levy P, Mokdad A. Tracking chronic disease and risk behavior prevalence as survey participation declines: statistics from the Behavioral Risk Factor Surveillance System and other national surveys. Prev Chronic Dis 2008;5(3). http://www.cdc.gov/pcd/issues/2008/jul/07_0097.htm. Accessed [date].
Contributor Information
Mansour Fahimi, Marketing Systems Group. At the time of the study, Dr Fahimi was affiliated with RTI International, Rockville, Maryland; Email: mfahimi@m-s-g.com, 565 Virginia Dr, Fort Washington, PA 19034, Phone: (240) 477-8277.
Michael Link, Division of Adult and Community Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia.
Ali Mokdad, Division of Adult and Community Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia.
Deborah A Schwartz, RTI International, Rockville, Maryland.
Paul Levy, Research Triangle Institute, Cary, North Carolina.
References
- 1.Mokdad AH, Stroup DF, Giles HW. Public health surveillance for behavioral risk factors in a changing environment. Recommendations from the Behavioral Risk Factor Surveillance Team. MMWR Recomm Rep. 2003;52(RR-9):1–12. [PubMed] [Google Scholar]
- 2.Link MW, Mokdad AH, Kulp D, Hylon A. Has the National Do Not Call Registry helped or hurt survey research efforts? Public Opin Q 2006;70(5):794–805. [Google Scholar]
- 3.Groves RM. Nonresponse rates and nonresponse bias in household surveys. Public Opin Q 2006;70:646–675. [Google Scholar]
- 4.Curtin R, Presser S, Singer E. The effect of response rate changes on the Index of Consumer Sentiment. Public Opin Q. 2000;64:413–428. doi: 10.1086/318638. [DOI] [PubMed] [Google Scholar]
- 5.Keeter S, Miller C, Kohut A, Groves R, Presser S. Consequences of reducing nonresponse in a national telephone survey. Public Opin Q. 2000;64:125–148. doi: 10.1086/317759. [DOI] [PubMed] [Google Scholar]
- 6.Blumberg SJ, Luke JV, Cynamon ML. Telephone coverage and health survey estimates: evaluating the need for concern about wireless substitution. Am J Public Health. 2006;96(5):926–931. doi: 10.2105/AJPH.2004.057885. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Brick JM, Waksberg J, Kulp D, Starer A. Bias in list-assisted telephone samples. Public Opin Q 1995;59:218–235. [Google Scholar]
- 8.Gentry EM, Kalsbeek WD, Hogelin GC, Jones JT, Gaines KL, Forman MR, et al. The behavioral risk factor surveys: II. Design, methods, and estimates from combined state data. Am J Prev Med. 1985;1(6):9–14. [PubMed] [Google Scholar]
- 9.Nelson DE, Powell-Griner E, Town M, Kovar MG. A comparison of national estimates from the National Health Interview Survey and the Behavioral Risk Factor Surveillance System. Am J Public Health. 2003;93(8):1335–1341. doi: 10.2105/ajph.93.8.1335. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Behavioral Risk Factor Surveillance System (BRFSS) Atlanta (GA): Centers for Disease Control and Prevention; http://www.cdc.gov/brfss . [Google Scholar]
- 11.National Health Interview Survey (NHIS) Atlanta (GA): Centers for Disease Control and Prevention; http://www.cdc.gov/nchs/nhis.htm . [Google Scholar]
- 12.National Health and Nutrition Examination Survey (NHANES) Atlanta (GA): Centers for Disease Control and Prevention; http://www.cdc.gov/nhanes . [Google Scholar]
- 13.Dillman DA, Sangster RL, Tanari J. Understanding differences in people's answers to telephone and mail surveys. In: Braverman MT, Slater JK, editors. New directions for evaluation series, 70 (advances in survey research) San Francisco (CA): Jossey-Bass; 1996. [Google Scholar]
- 14.Tourangeau R, Smith T. Asking sensitive questions: the impact of data collection, question format, and question context. Public Opin Q 1996;60:275–304. [Google Scholar]
- 15.Frankel M, Battaglia M, Link M. A new weighting system for BRFSS. Presentation at the 2007 Behavioral Risk Factor Surveillance System annual conference.March 2007; Decatur, GA. [Google Scholar]
- 16.Fahimi M, Chromy J, Whitmore W, Cahalan M. Efficacy of incentives in increasing response rates. Proceeding from the Sixth International Conference on Social Science Methodology.2004; Amsterdam, Netherlands. [Google Scholar]
- 17.Hays RD, Vickrey BG, Hermann BP, Perrine K, Cramer J, Meador K, et al. Quality of life research. Amsterdam: Springer Netherlands; 1994. pp. 159–168. [DOI] [PubMed] [Google Scholar]
- 18.Nelson LM, Longstreth WT, Jr, Koepsell TD, Checkoway H, van Belle., G Completeness and accuracy of interview data from proxy respondents: demographic, medical, and life-style factors. Epidemiology. 1994;5(2):204–217. doi: 10.1097/00001648-199403000-00011. [DOI] [PubMed] [Google Scholar]