A Feasibility Study on Using an Internet-Panel Survey to Measure Perceptions of E-cigarettes in 3 Metropolitan Areas, 2015

Abstract

Objectives:

Internet-panel surveys are emerging as a means to quickly and cost-effectively collect health data, and because of their large memberships, they could be used for community-level surveys. To determine the feasibility of using an internet-panel survey to quickly provide community-level data, we conducted a pilot test of a health survey in 3 US metropolitan areas.

Methods:

We conducted internet-panel surveys in Cleveland, Ohio; New York, New York; and Seattle, Washington, in 2015. Slightly more than 500 people responded to the survey in each city. We compared weighted unadjusted prevalence estimates from the internet-panel data with estimates from the 2014 Health Information National Trends Survey (HINTS) for the following question in each survey: “Compared to smoking cigarettes, would you say that electronic cigarettes are…much less harmful, less harmful, just as harmful, more harmful, much more harmful, or I’ve never heard of electronic cigarettes.” We used multivariable logistic regression to compare associations of respondents’ demographic and health characteristics with perceived harm from e-cigarettes.

Results:

The prevalence of the perception that e-cigarettes are less harmful than smoking cigarettes ranged from 35.9% to 39.9% in the internet-panel sites and was 43.0% in HINTS. Most patterns of beliefs and respondent characteristics in the internet-panel data were consistent with patterns in HINTS. We found inconsistent patterns between internet-panel sites and HINTS by race/ethnicity and education.

Conclusions:

This feasibility study found that internet-panel surveys could quickly produce community-level data for targeted public health interventions and evaluation, but they may be limited in producing estimates among subgroups.

Communities and public health agencies want information on the health characteristics, behaviors, and outcomes of their populations, so they can identify the highest-priority issues and affected populations and then design the most appropriate interventions. Public health agencies are more apt to address public health issues when they have local health information rather than just national or state health information.¹ However, information on the local population may not be available from state or national data sets or surveys, and local data may be difficult to collect because of competing priorities and limited funding.^2,3

Tobacco control and the use of electronic cigarettes (e-cigarettes) are areas in which local data can inform local health communications and public health interventions better than state or national data. With the rise in e-cigarette use⁴ and recent changes in US Food and Drug Administration regulations to further restrict access to minors,⁵ public health agencies need to assess the use of e-cigarettes and monitor changes over time. Most data on e-cigarette use are national.^6,7 Data on local use of e-cigarettes could help public health agencies develop tailored messages, aim them at better-defined target populations, and help evaluate the response to campaigns. However, because surveys with sample designs necessary to produce representative estimates of local areas can be time and cost prohibitive, other methods are needed.

Internet-panel surveys are a potential means to quickly and cost-effectively collect local health data. Internet panels comprise internet users who volunteer to become members of a panel and receive small incentives, such as credits toward airline miles, hotel points, or other rewards, to participate in surveys. Historically, these panels were intended for marketing research purposes. Although they are convenience samples, internet panels hold promise as a source of local health data because they have large memberships and can easily, quickly, and inexpensively provide local data.

The American Association for Public Opinion Research cautions against using internet panels as a substitute for probability sampling to produce precise representative population estimates.^8,9 However, a strength of internet-panel data lies in their ability to quickly assess knowledge and behaviors in a community, identify predictors and relationships, and monitor changes over time. For example, because samples can be drawn and data can be collected more efficiently (ie, quicker and cheaper) in internet panels than in traditional surveys, internet panels could be used to rapidly assess a public health issue in a city or county and then quickly evaluate intervention efforts or messaging campaigns.

The objective of this study was to determine the feasibility of using an internet-panel survey to provide community-level data on a topic of emerging public health concern: perception of the harmfulness of e-cigarettes as compared with cigarette smoking. We compared our internet-panel survey results with data from the National Cancer Institute’s Health Information National Trends Survey (HINTS).

Methods

We conducted a pilot test of an internet-panel health survey in 3 US cities: Cleveland, Ohio; New York, New York; and Seattle, Washington. Because one strength of internet-panel data is the ability to quickly assess population perceptions of emerging topics and because previous evaluations^10,11 assessed the use of internet-panel data for generating prevalence estimates, we chose to examine the ability of an internet panel to provide data on an emerging topic (e-cigarettes) that could be used to develop public health messaging. We derived questions for the internet-panel survey primarily from HINTS. Both the internet-panel survey and HINTS used the same question to assess awareness and perceived harm from e-cigarettes as opposed to smoking cigarettes. Because this question was not asked in other health surveys, we compared our internet-panel data only with HINTS data.

Questionnaire development, web survey programming, and testing took place from September through December 2014. We obtained institutional review board approval in January 2015. We fielded the survey from January 27 through February 9, 2015. We then cleaned and weighted the data and produced a final analytic file in March 2015. The ICF International Institutional Review Board approved the study.

Data Sources

The internet-panel survey participants were recruited by Research Now (www.researchnow.com), which operates several national panels with >3 million members in the United States. We selected Research Now because of cost and the company’s willingness to collaborate in various sampling and survey experiments. For this feasibility study, all respondents were recruited from the e-Rewards panel, whose members are invited to participate by partner organizations, such as airlines, hotels, and retailers. Panel members were then sampled and invited to participate in the online survey. Respondents received credits for small redeemable rewards for participation.

Research Now conducts standard quality checks that remove respondents if they consistently provide poor quality or inconsistent data.⁸ Quality checks include assessing the time used to answer each question, overuse of nonresponse options (eg, “don’t know”), gibberish open-ended responses, and illogical or inconsistent responses. Research Now also uses electronic fingerprinting technology to eliminate duplicate responders and imposes limits on survey participation to avoid professional survey takers. We selected Cleveland, Seattle, and New York City as study sites because of their various locations, population sizes, and population composition.

We selected internet-panel samples based on the following demographic characteristics: residential ZIP code, age group, sex, race/ethnicity, and education. Respondents received a direct email invitation to the survey (rather than being invited via internet traffic or a routed sample) from Research Now. The initial survey invitation was followed by a reminder (no sooner than 36 hours after the initial invitation). Generally worded subject lines and survey invitation text were used to limit any potential bias. Research Now continually sampled panel members to produce a final study population that matched US Census demographic characteristics (age group, sex, race/ethnicity, and education) for each city, a standard process for internet-panel surveys, also known as quota sampling. The protocol targets the hardest-to-reach populations first and then sends additional invitations as quotas are attained. The goal for the pilot study was to reach 500 completed surveys per site. Participants were then weighted to their communities based on population data from the 2013 American Community Survey,¹² and weights were poststratified with raking to age, sex, race/ethnicity, marital status, and education level.

We compared results from the internet panel with results from HINTS 4, Cycle 4, which was fielded in 2014.¹³ HINTS is a mail survey conducted by the National Cancer Institute and is intended to be nationally representative of noninstitutionalized adults aged ≥18. Hispanic and non-Hispanic black populations were oversampled. The response rate for HINTS 4, Cycle 4, was 34%.¹³ Details on HINTS and data collection methods can be found elsewhere.⁶

Survey Questions Assessed

In addition to demographic questions, we asked questions about self-reported health, smoking status, awareness of e-cigarettes, and perceived harm from e-cigarettes as compared with smoking cigarettes. We dichotomized self-reported health into fair/poor and excellent/very good/good. We categorized smoking status into never, former, and current. Respondents who reported that they smoked >100 cigarettes in their lifetime were categorized as former or current smokers based on their response to a question asking if they “now smoke cigarettes every day, some days, or not at all.” If they answered “not at all,” they were categorized as former smokers. Otherwise, respondents were considered never smokers.

To assess awareness and perceived harm from e-cigarettes as opposed to smoking cigarettes, we used a single question, which was identical in both surveys: “New types of cigarettes are now available called electronic cigarettes (also known as e-cigarettes or personal vaporizers). These products deliver nicotine through a vapor. Compared to smoking cigarettes, would you say that electronic cigarettes are…much less harmful, less harmful, just as harmful, more harmful, much more harmful, I’ve never heard of electronic cigarettes?” The internet survey also included the response “not sure.” Respondents were considered aware of e-cigarettes if they selected any response except “I’ve never heard of” or “not sure.” We dichotomized perceived harm into (1) those who believed that e-cigarettes were less harmful than smoking cigarettes and (2) those who felt that e-cigarettes were as harmful or more harmful than smoking cigarettes. We excluded from analysis those who were unaware of e-cigarettes.

Statistical Analysis

We calculated weighted distributions of the study populations and estimates separately for HINTS and for each internet-panel site. Because the 3 internet-panel sites were in metropolitan areas, we limited the HINTS data to counties in areas designated as metropolitan based on the rural/urban designation variable.⁶ HINTS results accounted for sampling design in the analysis and used jackknife sampling to produce 95% confidence intervals (CIs). Because the internet-panel data were derived from a nonprobability sample, true CIs could not be calculated.⁸ We did not include alternative estimates of precision,¹⁴ because research is ongoing to determine the most appropriate and acceptable methods. We conducted all analyses using SAS version 9.4.¹⁵

We calculated prevalence estimates of perceived harm from e-cigarettes vs smoking cigarettes overall and by the following characteristics: sex, age group (18-49 or ≥50 years), education level among those aged ≥25 (less than a college degree or a college degree or more), race/ethnicity (non-Hispanic white, non-Hispanic black, Hispanic, or Asian American/Pacific Islander), self-rated health, and smoking status. Because the pilot study was not designed to produce estimates by race/ethnicity, there were small numbers (<25) of non-Hispanic black, Hispanic, or Asian American/Pacific Islander respondents in Cleveland and Seattle. To provide stable estimates, we tabulated results only when all cell sizes for a variable were ≥5.

Because we expected differences in prevalence estimates by internet-panel site, we made comparisons among multivariable adjusted odds ratios (ORs) of perceived harm from e-cigarettes under the assumption that patterns of ORs would be more similar across sites than patterns of prevalence estimates would be. Without the ability to make comparisons between internet-panel data and HINTS data by using statistical testing, we were primarily interested in determining whether associations for the internet-panel data and HINTS data were in the same direction and whether the point estimate for each internet-panel OR fell within the 95% CIs for the HINTS OR. We ran models separately for each site and included all demographic and health characteristics; however, because of small numbers of Hispanic and non-Hispanic black respondents in Seattle, the model included a dichotomous non-Hispanic white/other variable, and we did not tabulate the results for race/ethnicity.

Results

The percentage of completed surveys in each internet-panel site was as follows: 12.7% in Cleveland, 17.3% in Seattle, and 6.7% in New York City (Table 1). A larger weighted percentage of women than men responded to the survey in Cleveland (52.6% vs 47.4%) and New York City (53.2% vs 46.8%); in Seattle, the distribution was roughly even (Table 2). Similarly, whereas the greatest unweighted percentage of respondents in each internet-panel site was non-Hispanic white, the percentage of non-Hispanic white respondents in New York City (34.8%) was smaller than the percentage in Cleveland (73.4%) and Seattle (66.5%). The distribution by age group was roughly even between the 2 age groups in Cleveland, but the weighted percentage of those aged 18 to 49 was 62.9% in Seattle and 58.3% in New York City. Smoking status differed by internet-panel site: Seattle had the highest percentage of never smokers (71.8%), and Cleveland had the lowest percentage (55.5%).

Table 1.

Unweighted response rates for internet-panel surveys on the harm of electronic cigarettes, by pilot survey site, 2015^a

Variable	Cleveland, OH	Seattle, WA	New York, NY
Invitations, n	3987	2962	7766
Completed surveys, n	506	513	520
Response rate, %	12.7	17.3	6.7

^aThe internet-panel survey participants were recruited by Research Now (www.researchnow.com); respondents received credit for small redeemable rewards (eg, airline miles, hotel points) for participation.

Table 2.

Unweighted number of respondents and weighted^a distribution of demographic and health characteristics in internet-panel surveys on the harm of electronic cigarettes, by pilot survey site, 2015, and in HINTS 4, Cycle 4, 2014^b

	Unweighted No. (Weighted %)			HINTS (n = 3217)
Characteristic	Cleveland, OH (n = 506)	Seattle, WA (n = 513)	New York, NY (n = 520)	Unweighted No.	Weighted % (95% CI)
Sex
Male	184 (47.4)	175 (49.6)	237 (46.8)	1195	49.4 (47.8-51.0)
Female	322 (52.6)	338 (50.4)	283 (53.2)	1761	50.6 (48.9-52.2)
Missing data	0	0	0	261	NA
Race/ethnicity
Non-Hispanic white	419 (73.4)	366 (66.5)	251 (34.8)	1655	63.4 (62.2-64.5)
Non-Hispanic black	44 (18.8)	20 (5.3)	102 (21.6)	490	12.3 (11.6-12.9)
Hispanic	20 (4.2)	22 (7.8)	117 (27.2)	512	16.9 (16.1-17.7)
Asian American/Pacific Islander	19 (3.2)	98 (18.3)	43 (13.3)	119	5.3 (4.6-5.9)
Other	4 (0.4)	7 (2.1)	7 (3.1)	100	2.1 (1.8-2.6)
Missing data	0	0	0	341	NA
Age, y
18-49	252 (50.3)	300 (62.9)	265 (58.3)	1112	60.2 (59.2-61.1)
≥50	254 (49.7)	213 (37.1)	255 (41.7)	1946	39.8 (38.9-40.8)
Missing data	0	0	0	159	NA
Education
<College degree	305 (67.1)	123 (38.3)	256 (64.3)	1761	56.5 (54.3-58.7)
≥College degree	201 (32.9)	390 (61.7)	264 (35.7)	1326	43.5 (43.1-45.7)
Missing data	0	0	0	130	NA
Self-rated health
Fair/poor	56 (13.8)	43 (9.2)	68 (14.8)	542	12.5 (10.9-14.1)
Excellent/very good/good	450 (86.2)	470 (90.8)	451 (85.2)	2578	87.5 (85.9-89.1)
Missing data	0	0	1	97	NA
Smoking status
Never smoker	288 (55.5)	371 (71.8)	341 (66.0)	1930	62.7 (60.1-65.2)
Former smoker	158 (31.0)	114 (20.7)	118 (20.8)	820	23.2 (20.8-25.6)
Current smoker	60 (13.5)	28 (7.5)	61 (13.2)	425	14.2 (11.9-16.4)
Missing data	0	0	0	42	NA

Abbreviations: HINTS, Health Information National Trends Survey; NA, not applicable.

^aWeighted to their communities based on population data from the 2013 American Community Survey¹²; weights were poststratified with raking to age, sex, race/ethnicity, marital status, and education level.

^bHINTS data were limited to data from counties in areas designated as metropolitan based on the rural/urban designation variable.⁶

Among the 3 internet-panel sites and in HINTS, almost all (range, 93.2%-98.4%) respondents were aware of e-cigarettes, and between 35.9% and 43.0% believed that e-cigarettes were less harmful than smoking cigarettes (Table 3). The internet-panel population differed from the HINTS sample by race/ethnicity and education level. In Cleveland and New York City, non-Hispanic black respondents were more likely to consider e-cigarettes less harmful than cigarette smoking than non-Hispanic black respondents in HINTS (Cleveland, 51.1%; New York City, 42.3%; HINTS, 26.4%). Whereas non-Hispanic black respondents in Cleveland were more likely (OR = 1.5) than non-Hispanic white respondents to consider e-cigarettes less harmful than cigarette smoking, non-Hispanic black respondents in HINTS were less likely (OR = 0.3; 95% CI, 0.2-0.5) than non-Hispanic white respondents in HINTS to consider e-cigarettes less harmful than cigarette smoking. We also found consistent differences between internet-panel sites and HINTS data by education level. In HINTS, respondents with a college degree or more were more likely than those with less than a college degree to consider e-cigarettes less harmful than smoking cigarettes. However, in all 3 internet-panel sites, those with less than a college degree were more likely than those with a college degree or more to consider e-cigarettes less harmful than smoking cigarettes. Among those with less than a college degree, the percentages who considered e-cigarettes less harmful than smoking cigarettes in all 3 internet-panel sites (range, 40.4%-42.7%) were similar to the percentage in HINTS (39.8%). However, among those with a college degree or more, the percentages were >10 percentage points lower in each internet-panel site (range, 32.9%-35.4%) than in HINTS (47.4%).

Table 3.

Weighted unadjusted prevalence and multivariable^a adjusted odds of respondents who believe that smoking electronic cigarettes (e-cigarettes) is less harmful than smoking cigarettes, from an internet-panel pilot study in 3 US cities, 2015, and HINTS 4, Cycle 4, 2014^b

	Cleveland, OH		Seattle, WA		New York, NY		HINTS
Variable	%	aOR	%	aOR	%	aOR	%	aOR (95% CI)
Have overall awareness of e-cigarettesc	97.4	NA	93.2	NA	98.4	NA	94.3	NA
Believe e-cigarettes are less harmful than smoking cigarettesc	39.9	NA	35.9	NA	39.8	NA	43.0	NA
Sex
Female	38.1	1 [Reference]	33.8	1 [Reference]	32.7	1 [Reference]	37.4	1 [Reference]
Male	41.9	1.4	37.9	1.5	48.3	1.9	49.7	1.6 (1.2-2.1)
Race/ethnicity
Non-Hispanic white	38.1	1 [Reference]	41.2	1 [Reference]	45.5	1 [Reference]	49.8	1 [Reference]
Non-Hispanic black	51.1	1.5	—d	—d	42.3	0.7	26.4	0.3 (0.2-0.5)
Hispanic	29.6	0.5	—d	—d	32.6	0.4	27.9	0.3 (0.2-0.6)
Age, y
≥50	32.5	1 [Reference]	25.0	1 [Reference]	37.5	1 [Reference]	37.3	1 [Reference]
18-49	46.9	1.8	41.8	2.4	41.2	1.6	47.1	1.7 (1.3-2.2)
Education
≥College degree	34.8	1 [Reference]	32.9	1 [Reference]	35.4	1 [Reference]	47.4	1 [Reference]
<College degree	42.6	1.5	40.4	1.4	42.7	1.3	39.8	0.8 (0.6-0.11)
Self-reported health
Excellent/very good/good	40.1	1 [Reference]	36.1	1 [Reference]	39.1	1 [Reference]	42.7	1 [Reference]
Fair/poor	38.7	0.6	33.3	0.7	43.9	1.1	45.4	1.4 (1.0-2.1)
Smoking status
Never smoker	41.9	1 [Reference]	35.2	1 [Reference]	37.3	1 [Reference]	37.6	1 [Reference]
Former smoker	30.7	0.7	36.3	1.2	40.5	1.1	47.6	1.5 (1.1-2.1)
Current smoker	52.9	1.3	40.6	1.0	50.4	2.7	57.2	2.1 (1.4-3.4)

Abbreviations: aOR, adjusted odds ratio; HINTS, Health Information National Trends Survey; NA, not applicable.

^aModels included all demographic and health characteristics.

^bHINTS data were limited to data from counties in areas designated as metropolitan based on the rural/urban designation variable.

^cTo assess awareness and perceived harm from e-cigarettes compared with smoking cigarettes, we used a single question, which was identical in the internet-panel surveys and HINTS: “New types of cigarettes are now available called electronic cigarettes (also known as e-cigarettes or personal vaporizers). These products deliver nicotine through a vapor. Compared to smoking cigarettes, would you say that electronic cigarettes are…much less harmful, less harmful, just as harmful, more harmful, much more harmful, and I’ve never heard of electronic cigarettes.” Respondents were considered aware of e-cigarettes if they selected any response except “I’ve never heard of” or “Not sure.” Perceived harm was dichotomized into (1) those who believed that e-cigarettes were less harmful than smoking cigarettes and (2) those who felt that they were as harmful or more harmful than smoking cigarettes. Those who were unaware of e-cigarettes were excluded from analysis.

^dNot reported because there were <5 respondents in each category.

We found little difference in the unadjusted percentages for perceived harm by self-rated health among the 3 internet-panel sites and HINTS. However, after adjustment, we found slightly inverse associations between fair/poor health and perceived harm in Cleveland and Seattle and a positive association between fair/poor health and perceived harm in HINTS. Current smokers were consistently more likely than never smokers to consider e-cigarettes less harmful than smoking cigarettes. However, in Seattle, the OR comparing current smokers with never smokers (OR = 1.0) was outside the 95% CI for the OR comparing current smokers with never smokers in HINTS (OR = 2.1; 95% CI, 1.4-3.4). In HINTS, former smokers were more likely than never smokers to believe that e-cigarettes were less harmful than smoking cigarettes, but the OR for this comparison (OR = 1.5) was smaller than the OR comparing current smokers with never smokers (OR = 2.1). In New York City and Seattle, we found little difference in perceived harm between former smokers and never smokers; however, former smokers in Cleveland were less likely than never smokers in Cleveland to believe that e-cigarettes were less harmful than smoking cigarettes. In HINTS, respondents with a college degree or more were more likely than those with less than a college degree to consider e-cigarettes less harmful than smoking cigarettes.

Discussion

Our feasibility study is one of the first to demonstrate that internet panels can be used to quickly conduct a local health survey, thus avoiding the use of traditional population-based methods, such as telephone or mail surveys, which can be costly and time-consuming. After receiving institutional review board approval, we fielded the survey and produced a final analytic data set within 2 months. In comparison, the release of data from many state and national surveillance systems can take more than a year.

As expected, we observed some differences in the prevalence of smoking and beliefs about e-cigarette harm across the internet-panel sites; however, most patterns by respondent characteristics were consistent with patterns in national HINTS data. The largest differences between internet-panel data and HINTS data were by race/ethnicity and education. For example, respondents with a college degree or more in the internet panels were less likely than similarly educated adults recruited from the general population to consider e-cigarettes less harmful than smoking cigarettes. Large differences between subgroup estimates were shown in a previous comparison of population-based data and other internet-panel surveys and are one reason for caution when relying on nonprobability sampled data.^10,11 We could not determine what might account for the differences between our panel-generated estimates and the HINTS estimates, but these differences underscore the need to recognize the potential underlying differences between internet-panel respondents and population-based survey respondents. Because panel members are identified through their interaction with businesses, internet panels and population-based surveys may differ in their representativeness.

Previous studies demonstrated that internet-panel data can play an important role in health surveys and are useful for estimating the prevalence of behaviors and beliefs and their associations with selected characteristics. Recent studies used internet-panel data to evaluate access to school-based physical activity resources, estimate influenza vaccination rates, assess the characteristics associated with the consumption of caffeinated alcoholic beverages, examine the alcohol brands most often consumed by underage young people, and determine correlates of exceeding maximum acetaminophen dosage.^10,16–19 Although internet-panel data may not be ideal for providing precise prevalence estimates in a community,^8,9,11 may be less reliable among subgroups than for overall estimates,¹⁰ and are limited in coverage to those with internet access, the strength of the data may be in the ability to assess relationships and monitor changes over time with repeated fielding. Consistent with this strength, we focused this feasibility study on the potential of using internet-panel data to measure shifts in public perceptions of public health concerns—through a onetime survey, as demonstrated here, or through multiple administrations of a survey to assess changes over time.

Internet-panel data could be used to assess the general prevalence of e-cigarette use in a community, as well as the currently held beliefs and perceptions, and then determine the population characteristics associated with them. Importantly, if the data were used to design interventions or messaging campaigns, additional fielding of the survey over time could be conducted to quickly monitor the success (or lack thereof) in the community. As a relatively new data source, with more work and evaluations conducted with this type of data, more acceptable uses will be identified.

Limitations

One major difficulty with nonprobability internet-panel data is evaluating results against a true gold standard. For our pilot study, HINTS was a problematic gold standard because it is a mail-based survey intended to be nationally representative and it struggles with response rates. Furthermore, although HINTS was conducted only 1 year before our pilot survey, because of the rapid increase in awareness and use of e-cigarettes,⁴ temporal differences between our pilot survey estimates and HINTS estimates could exist. Another potential data source for comparison is the Behavioral Risk Factor Surveillance System (BRFSS). Although the state-based BRFSS has data on metropolitan areas that overlap with our pilot sites, it is a telephone-based survey that also has less-than-optimal response rates.²⁰ In addition, BRFSS does not have a question on perceptions of e-cigarettes similar to the question that we used; therefore, we did not use BRFSS data as comparison data in this feasibility study. Comparing estimates between internet-panel data and a gold standard population-based sample is difficult, because determining whether differences are due to underlying population differences, natural geographic variation in behaviors and attitudes, survey mode, or other biases may not be possible. Additional work in survey research is needed to develop methods and benchmarks to assess the quality of nonprobability internet-panel data.

Conclusion

In our pilot study assessing the feasibility of using internet-panel data to examine awareness of e-cigarettes and perceptions of harm, patterns found in internet-panel data from 3 US cities were similar to patterns found in national HINTS data. Because the cost of conducting an internet-panel survey should be within the resources of most state and local health departments and because data collection for internet panels is faster than data collection through traditional mail or telephone surveys, internet panels could be a viable option to quickly produce community-level data for locally targeted public health interventions and evaluations. However, efforts are needed to determine the most appropriate uses and limitations of internet-panel data and ascertain when differences in underlying populations might affect the validity of their use.