Growing Cell-Phone Population and Noncoverage Bias in Traditional Random Digit Dial Telephone Health Surveys

Abstract

Objective

Examine the effect of including cell-phone numbers in a traditional landline random digit dial (RDD) telephone survey.

Data Sources

The 2007 California Health Interview Survey (CHIS).

Data Collection Methods

CHIS 2007 is an RDD telephone survey supplementing a landline sample in California with a sample of cell-only (CO) adults.

Study Design

We examined the degree of bias due to exclusion of CO populations and compared a series of demographic and health-related characteristics by telephone usage.

Principal Findings

When adjusted for noncoverage in the landline sample through weighting, the potential noncoverage bias due to excluding CO adults in landline telephone surveys is diminished. Both CO adults and adults who have both landline and cell phones but mostly use cell phones appear different from other telephone usage groups. Controlling for demographic differences did not attenuate the significant distinctiveness of cell-mostly adults.

Conclusions

While careful weighting can mitigate noncoverage bias in landline telephone surveys, the rapid growth of cell-phone population and their distinctive characteristics suggest it is important to include a cell-phone sample. Moreover, the threat of noncoverage bias in telephone health survey estimates could mislead policy makers with possibly serious consequences for their ability to address important health policy issues.

Traditional random digit dial (RDD) telephone surveys have been a popular tool for population-based data collection in the United States RDD surveys are cost-effective relative to in-person interviewing, have sound probability sampling characteristics (Mitofsky 1970; Waksberg 1978; Lepkowski 1988; Casady and Lepkowski 1993;), and benefit from high penetration of landline telephones, which reached over 95 percent in the late 1990s and early 2000s (Blumberg et al. 2007). For state health surveys, RDD telephone surveys have been the predominant method, as over 90 percent of the states and territories conduct health surveys within their jurisdictions using this method (see http://www.shadac.org/content/state-survey-research-activity).

Over the past several years, the utility of RDD telephone surveys has been questioned due to another trend in telephone usage: the increased popularity of cell phones. More and more telephone users are switching from regular landline telephones to cell phones, thereby reducing the population covered by landline telephones. The Consumer Expenditure Survey estimated cell-only (CO) households in the United States at <1 percent in 2000 but around 7 percent in the later quarters of 2004 (Keeter and Kennedy 2006; Tucker, Brick, and Meekins 2007;). The National Health Interview Survey (NHIS) estimated about 6.7 percent of adults residing in CO households in the first half of 2005 increasing to 18.4 percent in the second half of 2008 (Blumberg and Luke 2009a). This has raised concerns about the validity of traditional RDD health surveys among state health researchers (Brick, Edwards, and Lee 2007; State Health Access Data Assistance Center 2009;).

Growth of the Cell-Phone Population

Survey data collection methods inevitably reflect ongoing changes in communication technology and the diffusion of new technologies, as population-based data collection relies on communication between people (Dillman 2002). The choice of survey modality, consequently, may introduce different types of survey errors (Groves 1990). For RDD telephone surveys of the general population, a growing number of people who have only cell phones increases the likelihood of noncoverage error. To the extent that RDD surveys systematically exclude people without landline telephones and those with landline telephones are different in important ways from those without landline telephones, then traditional RDD survey estimates may not represent the general population.

Noncoverage bias is a product of the noncoverage rate and the distinctive characteristics of the CO population: a population shown to be disproportionately young, male, mobile, single, and living in rental housing when compared with adults living in households with landline telephones (Blumberg, Luke, and Cynamon 2006; Brick et al. 2006; Keeter and Kennedy 2006; Brick et al. 2007; Link et al. 2007; Tucker, Brick, and Meekins 2007; Blumberg and Luke 2008; Keeter, Dimock, and Christian 2008; Blumberg and Luke 2009a;). For health surveys, noncoverage bias may distort any number of estimates. In fact, there is ample evidence from NHIS studies that CO adults show distinctive characteristics on health measures with higher rates of health risk behaviors such as smoking and binge drinking, HIV tests, and experiencing financial difficulties with obtaining health care, and lower rates of health insurance coverage, while reporting better health status (Blumberg, Luke, and Cynamon 2006; Blumberg and Luke 2008; Blumberg and Luke 2009a;).

Adjusting the landline sample through weighting may mitigate noncoverage problems, as suggested by Keeter (2006) and Blumberg and Luke (2009b). However, because not all the relevant variables that explain telephone usage can be included in weighting, noncoverage bias for at least some items may be unavoidable. Nevertheless, the extent to which exclusion of CO households may bias estimates in an RDD telephone survey practice and the adequacy of adjusting for noncoverage bias are unknown.

Additionally, concerns have been raised about cell-mostly (CM) adults who live in households with both landline- and cell-phone services but use their cell phones for all or most of their calls. While this group is technically included in the landline RDD frames, they may be difficult to reach over landline telephones. Moreover, these adults report that they are more likely to drop their landline phones in the future than other phone usage groups (Keeter, Dimock, and Christian 2008). According to NHIS, the percentage of adults who live in CM households was estimated to be 15.4 percent in the second half of 2008 (Blumberg and Luke 2009a). With such a large percentage of adults in the CM group, it is clearly important to understand the characteristics of the CM adults relative to the other telephone usage groups.

DATA AND METHODOLOGY

California Health Interview Survey (CHIS) and Its Cell-Phone Supplemental Sample

A supplemental sample of adults living in households with cell phones only was implemented in the 2007 CHIS. CHIS is a biennial RDD telephone survey of California's population conducted by the UCLA Center for Health Policy Research in collaboration with the California Department of Public Health, the California Department of Health Care Services, and the Public Health Institute. Data collection was conducted by Westat under contract with UCLA between June 2007 and March 2008. CHIS interviews are conducted in five languages: English, Spanish, Chinese (Mandarin and Cantonese), Korean, and Vietnamese. A total of 49,249 adults completed the interview over regular landline telephones, among which 46,007 responded in English.

The supplemental state-wide CO sample component was included to address potential noncoverage bias by sampling and contacting cell-phone numbers. The feasibility of conducting cell-phone surveys was evaluated and confirmed in a CHIS 2005 pilot study (Brick, Edwards, and Lee 2007). The main cell-phone sample was drawn using the latest Telcordia database. Following the U.S. Telephone Consumer Protection Act, all sampled cell-phone numbers were manually dialed by the interviewing staff. Once contact was made with the dialed cell-phone sample, those living in households with landline telephones were screened out because they were covered by the landline telephone frame; the remainder was comprised of households with cell phones only. In CO households with one adult, no within-household sampling was required. In households with more than one adult, sampling adults depended on whether other household members shared the sampled cell phone. If adults shared the cell phone, the same within-household sampling method used in the landline survey was implemented (CHIS 2009a). That is, the screener respondent (SR) was randomly selected for the adult interview with a probability equal to the inverse of the number of adults in the household. In case the SR was not sampled, then one adult other than the SR was selected for the adult interview using the next birthday method. If the cell phone was not shared, then the SR is sampled. Monetary incentives were provided for those who completed interviews (U.S.$5 for the screener and U.S.$25 for the full-length adult interview) as a token of appreciation and to reimburse cell-phone air time. A total of 825 adults living in CO households completed the interview.

Response rates between the two samples were comparable. The landline sample had an AAPOR RR4 response rate of 35.5 percent in the screener interview conducted with a household informant, and a 59.4 percent rate in the main interview with a selected individual. Respective response rates for the CO sample were 30.5 and 52.0 percent (CHIS 2009b). It was shown that the response rates in CHIS do not directly indicate nonresponse bias (Lee et al. 2009).

Two questions were included in the landline survey to determine telephone usage in the population and to examine health characteristics by telephone usage. The first item asked whether the respondents had a working cell phone at the time of interview. Those with working cell phones were then asked, “Of all the telephone calls that you receive, are all or almost all calls received on cell phones, some received on cell phones and some on regular phones, or very few or none on cell phones?” These items follow the identical logic reported in the 2007 NHIS questionnaire changes (Blumberg and Luke 2008). Information from these items allowed us to divide the total adult sample, after excluding 3,138 cases whose telephone usage was not ascertained, into four mutually exclusive telephone usage status groups: CO with n=825, CM with n=6,253, landline-mostly (LM) n=26,088, and landline-only (LO) with n=13,763. CM and LM are dual users, those who live in households with both landline- and cell-phone services. As noted above, the CM users are dual users who rely on their cell phone for all or almost all of their calls (Boyle, Lewis, and Tefft 2009), and LM is actually defined as any dual user that is not CM.

Weighting the CO survey data is methodologically challenging due to the lack of control totals for CO adults. NHIS is a potential source for national estimates, but its sampling error is too large to provide reliable population control totals at the state level, although model-based population totals became available recently (Blumberg et al. 2009). For this study, after assigning design weights to the CO sample so that it reflects differential sample designs and selection probabilities relative to the landline sample, the two samples were combined. The full-adjustment weights were calculated by projecting the combined sample to the California population totals. Age, gender, race/ethnicity, county of residence, education level, household composition, and home ownership were controlled through ratio-raking adjustment in this process. The same adjustment was applied to the landline sample to account for noncoverage bias. Therefore, the combined and landline samples were each assigned with respective design and full-adjustment weights. A study by Brick, Edwards, and Lee (2007) and a CHIS report (CHIS 2009c) include details on this weighting method.

The weighted population total of the CO sample was 3.5 million, which was about 13.2 percent of the total adult California noninstitutionalized civilian population in 2007. This figure corresponds well with national CO adult estimates of 12.6 and 14.5 percent from the first and second half of 2007 from NHIS (Blumberg and Luke 2008). However, the NHIS model-based estimate for California CO adults was only 8.4 percent with a confidence interval of 7.7–9.1 percent (Blumberg et al. 2009). This estimate appears lower than what would be expected from such an urban state, making California one of the states with lowest CO rates.

The weighted percentages of CO, CM, LM, and LO adults in California from CHIS 2007 are 13.8, 17.2, 43.7, and 25.4 percent, respectively. The NHIS report for the second half of 2007 provides the best comparison estimate; using these data, we can estimate that the corresponding rates are roughly 14.8, 14.3, 50.2, and 19.5 percent, respectively, for the U.S. adult population who has telephone services and whose telephone usage status is ascertained (Blumberg and Luke 2008). These match CHIS estimates reasonably well.

We first examine potential noncoverage bias and the effectiveness of adjustment weighting. As noncoverage bias is the difference between estimates coming from a landline sample and a combined landline and CO samples, we estimate it by comparing fully adjusted landline sample estimates with the fully adjusted combined sample estimates. Here, we assume that the combined sample is unbiased with respect to the noncoverage due to the inclusion of the CO sample and the landline sample biased. Of course, the combined sample estimates assumed to be unbiased in this study may in fact be biased in the same direction as the landline sample estimates. As the landline and CO samples used different sample designs, we apply design weights for calculating estimates unadjusted for noncoverage bias, and the final weights for fully adjusted estimates. Effectiveness of the weighting is examined by comparing changes in the differences between the landline and combined samples when the design versus the full-adjustment weights are applied.

The analysis then focuses on comparing demographic characteristics and health measures by four distinct telephone usage status groups described previously: CO, CM, LM, and LO. We examine how different the CM group is from the other three in terms of various health characteristics controlling for demographic characteristics typically used in weighting adjustments for RDD surveys. Full-adjustment weights are applied in this set of analysis.

All analyses were conducted using SAS 9.2 and WesVar accounting for complex sample design, and the significance of differences was determined by t-tests. Whenever dependent samples were compared, the covariance between the samples was taken into the analyses.

RESULTS

Noncoverage Bias and Effectiveness of Weighting Adjustments

Without the CO sample, telephone surveys using only landline numbers are widely believed to yield estimates that are potentially biased. Therefore, a measure of noncoverage bias in landline surveys would be the extent to which estimates differ from comparable estimates from samples including both landline- and cell-phone numbers.

Columns 1A and 2A in Table 1 present estimates from combined and landline samples using respective full-adjustment weights. The difference between these two estimates corresponds to noncoverage bias. As a number of demographic variables are controlled through the weighting adjustments described previously, the two samples show very few statistically significant differences on the demographic variables. Differences in the presence of children in the family, marital status, employment, citizenship, and interview language are statistically significant, but not substantive, all around 1 percent point.

Table 1.

Demographic and Health-Related Characteristics from Combined California Health Interview Survey (CHIS) Landline and Cell-Only Sample and CHIS Landline Sample Using Final Adjustment Weights and Base Weights

	1. Combined Sample (Landline+Cell-Only) (n=50,067)				2. Landline Sample (n=49,242)
	A. Full-Adjustment Weight		B. Design Weight		A. Full-Adjustment Weight		B. Design Weight
Characteristics	%	SE	%	SE	%	SE	%	SE	1A Versus 2A	1B Versus 2B
Demographics
Age
Age 18–24	13.8	0.00	22.5	1.15	13.8	0.22	7.8	0.17	0.0	−14.7***
Age 25–29	8.8	0.06	12.1	0.68	8.8	0.17	4.7	0.11	0.0	−7.4***
Age 30–44	29.6	0.13	21.1	0.80	29.6	0.25	22.7	0.23	0.0	1.6*
Age 45–64	33.3	0.11	30.4	0.74	33.3	0.22	40.9	0.26	0.0	10.5***
Age 65+	14.5	0.00	13.8	0.32	14.5	0.15	23.9	0.22	0.0	10.1***
Male	49.0	0.00	50.4	0.96	49.0	0.26	41.5	0.29	0.0	−8.9***
Race/ethnicity
Hispanic	31.5	0.00	24.4	0.92	31.5	0.28	21.1	0.28	0.0	−3.3***
Non-Hispanic white	47.6	0.00	53.1	0.89	47.6	0.24	60.2	0.31	0.0	7.2***
Non-Hispanic black	5.7	0.00	5.8	0.43	5.7	0.13	5.1	0.11	0.0	−0.7
Non-Hispanic Asian	12.7	0.00	12.9	0.89	12.7	0.17	10.3	0.14	0.0	−2.7**
Non-Hispanic other	2.5	0.00	3.8	0.39	2.5	0.07	3.3	0.10	0.0	−0.5
Family with kids	34.4	0.29	23.7	0.72	35.9	0.20	29.9	0.27	1.5***	6.2***
One-person household	11.3	0.10	12.8	0.44	11.1	0.12	15.8	0.16	−0.1	3.1***
Single	37.6	0.31	50.8	0.92	36.4	0.24	34.3	0.25	−1.2***	−16.5***
Education
Less than high school	16.6	0.05	9.5	0.56	16.6	0.24	10.5	0.17	0.0	1.0
High school	27.0	0.03	27.3	1.09	27.0	0.28	21.9	0.24	0.0	−5.4***
Some college	24.2	0.26	29.9	0.80	23.7	0.22	27.2	0.26	−0.6*	−2.7***
College+	32.3	0.26	33.3	0.93	32.8	0.22	40.3	0.25	0.5	7.1***
Poverty
Poor	13.7	0.28	13.9	0.74	13.6	0.23	10.2	0.15	−0.1	−3.7***
Near poor	17.0	0.33	15.4	0.68	17.3	0.21	14.8	0.21	0.3	−0.6
Not poor	69.3	0.31	70.7	0.89	69.1	0.26	75.0	0.24	−0.2	4.3***
Renter	38.0	0.00	49.2	0.96	38.0	0.23	31.5	0.23	0.0	−17.7***
Unemployed	32.3	0.30	30.5	0.89	32.8	0.24	38.9	0.30	0.6*	8.4***
Non-U.S. citizen	15.8	0.24	11.4	0.77	16.5	0.23	11.0	0.19	0.6*	−0.5
Interviewed in English	84.8	0.21	92.9	0.33	83.7	0.24	89.4	0.17	−1.1***	−3.4***
Live in metropolitan statistical areas	97.6	0.00	97.5	0.17	97.6	0.05	96.9	0.05	0.0	−0.7***
Health condition
Fair/poor health	19.0	0.31	16.1	0.73	19.5	0.18	18.6	0.23	0.5	2.5***
Ever diagnosed with asthma	13.1	0.25	14.3	0.61	13.1	0.16	13.6	0.18	0.0	−0.7
Asthma attack past year	4.1	0.15	4.0	0.30	4.2	0.11	4.7	0.12	0.1	0.7*
Ever diagnosed with diabetes	7.8	0.21	6.3	0.28	7.8	0.13	9.1	0.17	0.0	2.8***
Ever diagnosed with high blood pressure	26.2	0.32	24.9	0.83	26.1	0.19	32.1	0.25	−0.1	7.2***
Ever diagnosed with heart disease	6.4	0.17	5.9	0.31	6.5	0.11	8.9	0.15	0.1	3.0***
Disabled	29.0	0.31	27.6	0.75	29.2	0.21	32.2	0.27	0.2	4.6***
Serious psychological distress last month	3.8	0.18	4.3	0.59	3.6	0.12	3.4	0.09	−0.2	−1.0
Cancer screening
Pap test past 3 years (female)	80.7	0.35	77.2	1.29	81.0	0.28	80.7	0.29	0.3	3.5**
Mammogram past 2 years (female, age ≥30)	63.5	0.46	65.9	1.05	63.6	0.36	70.4	0.36	0.0	4.5***
Colon cancer screening past 5 years (age ≥40)	44.3	0.33	46.4	0.79	44.8	0.26	50.4	0.28	0.5	4.0***
Health risk behavior
Regular physical activity	36.3	0.39	38.6	0.94	36.5	0.24	36.9	0.26	0.2	−1.7
Moderate physical activity	27.2	0.31	28.8	0.86	27.5	0.22	28.8	0.23	0.3	0.1
Vigorous physical activity	17.5	0.31	19.4	0.82	17.4	0.20	16.1	0.18	−0.1	−3.3***
Obese	22.6	0.31	20.5	0.79	22.5	0.27	22.0	0.22	−0.1	1.5
Current smoker	14.4	0.30	17.9	0.95	13.6	0.19	12.1	0.18	−0.7**	−5.8***
No household smoking	91.5	0.21	89.5	0.63	92.0	0.11	90.9	0.15	0.5*	1.4*
Had alcohol past year	69.0	0.36	72.6	0.68	68.4	0.25	69.5	0.22	−0.7*	−3.1***
Binge drinking past year	29.7	0.36	33.8	1.01	28.7	0.27	23.7	0.25	−1.0**	−10.1***
Sexually transmitted disease test past year (age ≤70)	21.8	0.41	27.6	1.27	20.5	0.28	16.0	0.27	−1.3***	−11.5***
Health care access
Currently insured	84.2	0.31	82.8	0.85	84.8	0.24	89.4	0.17	0.6*	6.6***
Visited doctor past year	81.3	0.36	81.1	0.73	81.2	0.21	85.3	0.21	−0.1	4.2***
Visited emergency room past year	18.9	0.31	19.7	0.70	18.6	0.18	19.3	0.21	−0.3	−0.5

^***p<.01,

^**p<.05,

^*p<.1.

We selected 23 health-related characteristics considered important in CHIS. Surprisingly, only six variables show evidence of noncoverage bias, and the evidence is very weak. The only characteristic subject to noncoverage bias >1 percent point is having a sexually transmitted disease test within the past year, where using only the landline sample may have underestimated it by 1.3 percent points. Overall, it appears that the weighted landline sample would have represented the general population reasonably well in CHIS 2007.

The representativeness of the landline sample does not mean that traditional landline sample telephone surveys are free from noncoverage bias. The landline sample estimates in columns 2A in Table 1 discussed above reflect the noncoverage adjustment applied through a complex and sophisticated weighting procedure. The effectiveness of this adjustment can be assessed by comparing the differences between fully adjusted combined and landline sample estimates and design-weighted combined and landline sample estimates. The design weights account for differential sample designs, whereas the full-adjustment weights reflect not only the efforts to correct for potential noncoverage bias but also sample designs (CHIS 2009c). Columns 1B and 2B in Table 1 show unadjusted estimates using design weights, whereas 1A and 2A show those adjusted by full-adjustment weights. Comparing the estimates in these two columns demonstrates how the weighting adjustment alleviates noncoverage bias. The differences between the two samples are substantial and significant without noncoverage adjustment. However, these differences disappear for most variables once the full-adjustment weights are applied, suggesting that the weighting adjustment used in this study is effective in reducing the noncoverage bias. This echoes a recent study by Blumberg and Luke (2009b) which showed potential alleviation of noncoverage bias from statistical adjustments, although their weighting method was simpler than the one in this study.

Relationship between Telephone Usage and Demographic and Health-Related Characteristics

Although the overall noncoverage bias in landline telephone surveys appears to attenuate with complex statistical adjustments at the current state, an examination of various characteristics by telephone usage status will provide a more detailed picture of foreseeable future of telephone surveys. Mirroring most of the previous research, we found that CO adults are mobile and more likely to be male, young, single, and renters, compared with those with landline phones (results not shown). They are also more likely to engage in riskier health behaviors, such as smoking and drinking, and are less likely to have health insurance and access to care. However, CO adults appear to be healthier than their counterparts, as their reports on chronic and physical conditions and obesity are at lower rates.

Because much is known about CO populations, we focused on CM adults who present another threat to telephone surveys and examined a series of 16 demographic and health characteristics in logistic regressions using the telephone usage variable with four categories defined previously where CM is a reference group. Table 2 clearly shows that CM adults are distinctive from CO, LM, and LO adults. CM adults are younger than LM and LO adults but older than CO adults. CM adults are more likely to be male relative to LM and LO groups. CO and LO groups show a lower association with being non-Hispanic white and having a family with kids and a higher association with being a renter than CM adults, whereas LM shows the opposite. Odds ratio of being single was highest for CO, followed by CM, LO, and LM. While LM adults appear to have a higher education level than CM, both CO and LO appear to have the opposite. CM adults are least likely to be unemployed among all groups. These results are consistent with many of those observed by Blumberg and Luke (2008) at a similar time point. Given these results, it is not surprising to find differences in health measures by telephone usage where CM adults do not resemble any other particular telephone usage groups.

Table 2.

Odds Ratios for Various Demographic and Health-Related Characteristics from Simple Logistic Regressions Using Telephone Usage as a Predictor

	Dependent Variables: Demographic Characteristics
	Age≤40	Male	Race: Non-Hispanic White	Family with Kids	Marital Status: Single	Education: Some College or More	Renter	Unemployed
Independent Variables	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio
Intercept	1.896***	1.402***	1.051	0.541***	0.848***	1.662***	0.536***	0.174***
Telephone usage (reference group: cell-mostly)
Cell-only	1.429***	1.014	1.311**	0.510***	2.521***	0.749**	3.830***	1.541***
Landline-mostly	0.247***	0.609***	0.712***	1.287***	0.351***	1.146**	0.601***	2.531***
Landline-only	0.288***	0.534***	1.987***	0.823***	0.800***	0.344***	1.778***	6.236***
	Dependent Variables: Health-Related Characteristics
	Fair/Poor Health	Serious Psych. Distress Past Month	Obese	Current Smoker	Binge Drinking Past Year	Sexually Transmitted Disease Test Past Year (Age≤70)	Currently Insured	Visited Doctor Past Year
Independent Variables	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio
Intercept	0.125***	0.027***	0.261***	0.179***	0.752***	0.346***	5.716***	3.956***
Telephone usage (reference group: cell-mostly)
Cell-only	1.190	2.258***	0.972	1.816***	1.133	1.931***	0.499***	0.832#
Landline-mostly	1.467***	0.953	1.091	0.669***	0.466***	0.462***	1.655***	1.435***
Landline-only	4.097***	2.354***	1.315***	0.984	0.316***	0.836**	0.602***	0.899#

^#p<.1,

*p<.05,

^**p<.01,

^***p<.001.

For eight health-related characteristics in Table 2, we further examined the relationships after controlling for demographic characteristics indicated in the same table. In Table 3, telephone usage shows significant associations with seven of the eight health variables. Even after controlling for age and the other characteristics, the CM adults have a lower likelihood of reporting fair/poor health, compared with the LM and LO groups. The differences shown in the univariate analysis for psychological distress between CM and CO/LO remain significant, where CO/LO are much more likely to experience psychological distress. The analysis confirms the results shown in Table 2 that the current smoking rate is lower among CM than among CO and higher than among LM adults and that binge drinking reports are higher for CM than for LM and LO groups.

Table 3.

Odds Ratios for Various Health-Related Characteristics from Multivariate Logistic Regressions Using Telephone Usage and Demographics as Predictors

	Dependent Variables: Health-Related Characteristics
	Fair/Poor Health	Serious Psych. Distress Past Month	Obese	Current Smoker	Binge Drinking Past Year	Sexually Transmitted Disease Test Past Year (Age≤70)	Currently Insured	Visited Doctor Past Year
Independent Variables	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio	Odds Ratio
Intercept	0.074***	0.033***	0.245***	0.146***	2.325***	2.229***	1.141	2.463***
Telephone usage (reference group: cell-mostly)
Cell-only	0.876	1.605*	0.950	1.422**	1.126	1.415**	0.708**	1.026
Landline-mostly	1.182*	0.936	1.028	0.862*	0.771***	0.844*	0.978	0.937
Landline-only	1.792***	1.371*	1.055	1.034	0.650***	1.052	0.566***	0.700***
Age (years)	1.021***	0.997	1.008***	0.996**	0.962***	0.943***	1.030***	1.018***
Gender: male	1.127*	0.851	1.232***	1.782***	2.061***	0.434***	0.763***	0.442***
Race: non-Hispanic white	0.518***	0.934	0.837***	1.538***	1.646***	0.833**	1.970***	1.349***
Family with kids	1.019	1.152	1.219***	1.120#	0.919#	1.041	1.113	0.945
Marital status: single	1.129*	1.267*	0.953	1.489***	0.950	1.835***	0.881#	0.828**
Education: some college or more	0.488***	0.651***	0.685***	0.578***	0.935	1.034	2.228***	1.444***
Renter	1.769***	1.937***	1.142**	1.742***	0.898*	1.419***	0.494***	0.737***
Unemployed	2.011***	2.554***	0.990	0.930	0.584***	0.923	1.106	1.244***
Living in metropolitan statistical areas	0.955	0.668**	0.864#	0.701***	0.779**	1.196	1.471**	1.248*

^#p<.1,

^*p<.05,

^**p<.01,

^***p<.001.

The results for current insurance coverage and doctor visits are noteworthy. CM adults have significantly higher health insurance coverage than CO and LO groups after controlling for demographic characteristics that may affect current insurance status. The odds of having health insurance are lower for LM compared with CM, although LM's coverage was higher than CM's in Table 2. The odds ratios of LO adults having insurance coverage and doctor visits, compared with that of CM, become more pronounced once controlling for demographic characteristics (0.566 versus 0.602 and 0.899 versus 0.700). Although LM adults showed higher rates of having health insurance and visiting a doctor compared with CM in the univariate analysis, the multivariate analysis suggests that these two groups are not different with respect to those health care access measures.

CONCLUSION

Our study shows that weighting adjustment may be an effective tool to adjust for noncoverage biases associated with the growing cell-phone population. Estimates from the fully adjusted landline sample are very similar to those from the sample adding cell-phone samples. Although the noncoverage bias examined in this study does raise serious concerns, future telephone surveys should give serious considerations to an inclusion of CO populations for four reasons. First, the proportion of those who are not covered by the landline telephone frame in the population is expected to grow in the future. Noncoverage bias examined in this study was relatively small, partly because this proportion was rather small, at <15 percent. Note that it has been reported that the CO rates were over 25 percents in states such as Oklahoma and Utah as of 2007 (Blumberg et al. 2009). Given the large differences between the CO and landline adults and the expected growth of the CO group, relying on landline samples alone in population-based surveys may jeopardize the data quality even if sophisticated weighting adjustments are applied.

Second, the study showed the distinctiveness of the CM adults throughout various characteristics. This distinctiveness remained even after controlling for demographic characteristics commonly used in weighting adjustments. As CM people are not likely to answer the landline telephone calls but to answer cell-phone calls, they may be a source of nonresponse bias and, consequently, a threat to representativeness of telephone surveys using landline samples. Weighting adjustments may not be effective for such cases, because distinctiveness of the CM group in health measures remained significant even after controlling for demographic variables.

Third, there may be subgroups who are more affected by the cell-phone usage trends and less amenable to effectiveness of weighting adjustment. Larger problems were found with noncoverage bias in subgroups, such as young adults and low-income populations (Blumberg and Luke 2007; Blumberg and Luke 2009b;). Moreover, by including cell samples, telephone surveys may gain the statistical power for analyzing some subgroups that may have higher cell-phone penetration. While additional studies on this aspect will show the actual magnitudes, it is imaginable that cell samples are much more productive in obtaining highly mobile populations than landline samples.

Fourth, weighting itself does not guarantee a removal or decrease in noncoverage bias. Its effectiveness depends on the type of weighting methods, variables and their nature controlled under weighting, and the tightness of weighting variables not only with telephone phone usage status but also survey variables.

Given the results of this study and the cell-phone usage trend, it is important that RDD health surveys, including those conducted by state agencies, continue to explore ways to enhance their data quality by introducing cell-phone samples to the data collection. Obviously, including those who use cell phones mainly in telephone survey data collection is important, and it is worthwhile to discuss how to include them. Some RDD telephone surveys utilize full dual-frame samples—both landline- and cell-phone samples without screening by telephone usage (Kennedy 2007; Keeter, Dimock, and Christian 2008;). While this is feasible, there is evidence that those using cell phone more frequently are more accessible and amenable to responding to cell-phone survey interviews (Brick et al. 2006). This suggests that lower response rates may occur in full dual-frame surveys among those who have both landline and cell phones but do not use cell phones frequently. This hypothesis is supported by two studies by Pew Research that found that those using both dual users sampled from a landline frame are different from those sampled from a cell-phone frame (Kennedy 2007; Keeter, Dimock, and Christian 2008;). Of course, data collection costs must also be considered, and including more adults from the cell-phone frame will reduce cost per completed case.

As cell-phone usage grows, noncoverage bias of landline telephone surveys may not be adequately compensated by weighting. Such datasets may produce biased estimates, which may have substantial policy implications. For example, state health telephone surveys typically produce point-in-time estimates of the uninsured rate in those states that average 23 percent lower compared with Current Population Survey (CPS) data (Call, Davern, and Blewett 2007). Although the state survey–CPS difference is due to multiple factors, including landline telephone coverage, it suggests the problem for policy makers when two credible data sources provide widely different estimates of the problem to be addressed. If the policy maker assumes that the CPS estimate is correct, he or she must plan for an uninsured population that is substantially larger than if he or she were to assume that the state survey estimate is more accurate. In that case, policy makers may be discouraged from attempting to address a problem because it appears to be beyond their resources. A large data gap was a contributing factor in the defeat of health care reform in California in 2008 (see analysis by the California Legislative Analyst's Office [2008] for the Health Care Security and Cost Reduction Act, ABX1 1). In a similar fashion, as CO households increase as a proportion of all households, the threat of noncoverage bias in health survey estimates could mislead policy makers with possibly serious consequences for their ability to address important health policy issues. Therefore, cell-phone samples should be included in telephone surveys not only to represent the population but also to assess the trends and magnitudes of potential noncoverage bias. Policy makers depend on good data to help them make good policy.

SUPPORTING INFORMATION

Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.

Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.