Evidentiary Support in Public Comments to the FDA's Center for Tobacco Products

Abstract

Electronic Nicotine Delivery Systems (ENDS) were introduced into the U.S. market in 2007, and until recently these devices were unregulated at the federal level. In 2014, the U.S Food and Drug Administration (FDA) published a Notice of Proposed Rulemaking asserting its intention to regulate ENDS and requesting public comments on numerous related issues, including potential limits on the sale of flavored ENDS. This paper analyzes key comments submitted to the FDA on the issue of flavor regulation in ENDS, and examines the weight and credibility of the evidence presented by both supporters and opponents of regulation. It also describes the final deeming rule, published in May 2016, and the FDA's response to the evidence submitted. This is the first study to examine public comments submitted to the FDA's Center for Tobacco Products, and it concludes that opponents of regulation were more likely to rely on sources that were not peer-reviewed and that were affected by conflicts of interest. In light of these findings, the FDA and the research community should develop processes to carefully and critically analyze public comments submitted to the FDA on issues of tobacco regulation.

Introduction

In 2009, Congress enacted the Family Smoking Prevention and Tobacco Control Act (Tobacco Control Act), giving the U.S. Food and Drug Administration (FDA) the authority to regulate “tobacco products.” The Tobacco Control Act provided the FDA immediate authority over certain products (“cigarettes, cigarette tobacco, roll-your-own tobacco, smokeless tobacco”), as well as the ability to extend its regulatory oversight to “any other tobacco products that the Agency by regulation deems to be subject to the law” (the deeming process) (Family Smoking Prevention and Tobacco Control Act, 21 U.S.C. § 301 (2009)).

Shortly before the Tobacco Control Act was enacted, the FDA attempted to restrict the import of Electronic Nicotine Delivery Systems (ENDS)¹ into the U.S. under the Federal Food, Drug, and Cosmetic Act, which gives the FDA authority to regulate drugs and drug-delivery devices (Federal Food, Drug, and Cosmetic Act, 21 U.S.C. 9 § 301 (1938)). This action was challenged in court, and before the case reached a decision, the Tobacco Control Act was signed into law (Smoking Everywhere, Inc. v. United States FDA, 680 F. Supp. 2d 62 (D.D.C. 2010)). Ultimately, a federal appellate court ruled that ENDS met the definition of a “tobacco product” under the Tobacco Control Act (so long as they were not marketed with therapeutic claims) and, therefore, could only be regulated pursuant to the FDA's new tobacco-related authorities (Sottera, Inc. v. FDA, 627 F.3d 891 (D.C. Cir. 2010)). Until May 2016, the FDA did not exercise its “deeming authority” with regard to ENDS. As a result, nearly ten years went by with ENDS sales almost entirely unregulated by the federal government.

In the absence of regulation, the ENDS market expanded rapidly, and major cigarette manufacturers made the strategic decision to begin producing and selling ENDS (Orellana-Barrios et al. 2015). In 2008, ENDS sales in the U.S. totaled only $20 million; in 2014, sales were $1.7 billion, and sales are projected to grow about 25% a year through 2018 (Wahba 2014). In 2010, only 3% of US adults had ever tried ENDS; by 2013, that figure was 13%. More rapidly increasing is youth use; current ENDS use (use in the last 30 days) among high school students tripled from 4.5% to 13.4% between 2013 and 2014 (Arrazola et al. 2015).

In April 2014 the FDA issued a Notice of Proposed Rulemaking (NPRM) that included a draft deeming rule. In the NPRM, the FDA proposed extending its authority to all ENDS, in addition to all other products meeting the statutory definition of a “tobacco product” (“any product made or derived from tobacco that is intended for human consumption” that is not a “drug,” “device,” or “combination product” as those terms are defined by the Food, Drug and Cosmetic Act). This triggered a 90-day public comment period (later extended for an additional 30 days), which closed in August 2014. In the NPRM, the FDA asked for comments on numerous topics that could potentially be incorporated into a final rule or into subsequent regulatory actions. For example, the FDA stated: “Once they are deemed, FDA may establish a product standard prohibiting flavors in those [newly-deemed] products. FDA requests information and data that would support establishing such a standard” (“Tobacco Products” Subject to the Federal Food, Drug, and Cosmetic Act, as Amended by the Family Smoking Prevention and Tobacco Control Act, 79 Fed. Reg. 23141 (Jun. 24, 2014)).

During the public comment period, FDA received more than 135,000 comments (Food and Drug Administration 2015). These comments included substantive comments from many different interested parties, including state and local governments, public health organizations, cigarette manufacturers, the ENDS industry, and tobacco retailers. However, the majority of the comments were brief messages from individual citizens discussing their own personal beliefs, opinions, and experiences. (79 Fed. Reg. 23141). Many of these comments were prompted by tobacco/ENDS companies or public health organizations that provided form submissions for individuals to complete (Bryer 2013). While the FDA is tasked with reviewing and analyzing all of these comments, comments that specifically addressed the FDA's requests and offered evidentiary support for any arguments or assertions are most likely to influence the FDA's subsequent actions.

In May 2016, when the final deeming rule was released, it extended the FDA's jurisdiction to ENDS and placed some restrictions on their sale. For example, it prohibited sales to minors and required the inclusion of warning labels on packaging and advertisements. However, the final rule did not include any restrictions on flavors for any of the newly deemed products. The FDA did suggest that it planned to issue a future rule restricting flavors in cigars, but no such rule was suggested for ENDS. (Deeming Tobacco Product to be Subject to the Federal Food, Drug, and Cosmetic Act, as Amended by the Family Smoking Prevention and Tobacco Control Act, 81 Fed. Reg. 90 (May 10, 2016)).

Although the deeming rule is now final, it is important to examine and analyze the process of rulemaking, including the submission and analysis of public comments. Previous research analyzing public comments submitted to other U.S. regulatory agencies—including other centers within the FDA—has found that regulated industries tend to be the most active commentators (at least on issues of low public salience), and that such industries, unsurprisingly, typically seek to reduce the stringency of proposed rules (Hwang et al. 2014; Yackee 2006; J. W. Yackee and Yackee 2006; Wagner, Barnes, and Peters 2010; West and Raso 2013). Because regulated businesses are able to focus more resources on the public comment period than public interest organizations or individual citizens, some have concluded that inviting the “public” into the regulatory process ironically “diminishes regulatory stringency and exacerbates the disproportionate influence of private interests” (Hwang, Avorn, and Kesselheim 2014).

Studies examining the submission of public comments by the tobacco industry in the United Kingdom have criticized the industry's selective and misleading use of evidence. For example, Ulucanlar et al. provided examples of tobacco companies “partially quoting the original source and omitting qualifying information” in order to “distort or even contradict the meanings in the original source” (Ulucanlar et al. 2014). Identifying such misleading uses of evidence can be extremely time-consuming for a regulatory agency, as it requires carefully crosschecking each citation against the original source. Hatchard et al. further noted that many sources cited by the tobacco industry were funded by the industry, and these sources were less likely to be published in peer-reviewed journals (Hatchard et al. 2014).

Despite its importance to the regulatory process, no research has yet systematically examined the content of public comments submitted to the FDA's Center for Tobacco Products. Building upon previous literature, this paper seeks to analyze a selection of the comments submitted to the FDA's Center for Tobacco Products regarding its deeming rule, in order to examine the source and reliability of the evidence being submitted to the FDA by interested parties.

Methods

The purpose of this study was to evaluate the types of evidence used by entities submitting comments to the FDA regarding the FDA's deeming rule. To provide a specific focus for the analysis, this project examined only comments relating to the FDA's potential regulation of flavors in ENDS. Comments were accessed at the regulations.gov website. Over 135,000 comments were received by the FDA on the issue of deeming (79 Fed. Reg. 23141). However, the sampling frame was limited to the 79,363 comments accessible on the regulations.gov website. In a cursory review of the comments, a large majority were from individuals who submitted simple statements, such as, “Save lives. Let the people vape.”(Anonymous 2014) Thus, in order to focus the review on comments that addressed the specific questions in the deeming proposal and included evidentiary support, our study excluded text-only comments (without any attachments). Text-only comments were limited to 5,000-characters, and it was assumed that major substantive that detailed comments would exceed this limit. There were 117,558 text-only comments eliminated, leaving 1,471 comments with attachments.

Next, the researchers excluded comments that came from “anonymous” (or any common misspellings of this word). From the subset of comments deemed eligible (n=1,004), the researchers systematically reviewed these comments to produce a targeted sample to represent different viewpoints of types of commenters identified from previous work. These categories of organizations/companies identified as likely to respond in detail to requests for comment included public health organizations, public health legal entities (both governmental and non-profit), the tobacco industry, the ENDS industry, and retailers/distributors (Yackee 2006; Wagner, Barnes, and Peters 2010).

The researchers identified commentators in each category that contributed detailed, substantive comments; this review resulted in comments from four leading public health organizations,² in addition to one comment that represented 24 public health organizations; two public health legal entities, including a combined comment by 18 state Attorneys General; three trade organizations representing distributors and retailers; four tobacco industry representatives; and five ENDS industry representatives. From among the substantive comments submitted, the researchers selected tobacco industry and ENDS industry comments for inclusion by seeking both market leaders (in terms of retail sales) (Maxwell 2015; C.S.D. 2014) and trade organizations that represent smaller manufacturers (see Table 1).

Table 1. Reviewed Comments by Type of Organization or Entity.

Public Health
24 Public Health Groups	American Public Health Association
American Legacy Foundation	Center for Tobacco Free Kids
American Heart Association
Public Health Law
Tobacco Control Legal Consortium	18 State Attorneys General
Tobacco Industry
Reynolds America, Inc. (RAI)	Lorillard
National Association of Tobacco Outlets	Small Manufacturers Association for the Reasonable Treatment of Tobacco
ENDS Industry
CB Distributors*	Electronic Cigarette Industry Trade Association
Logic	NJOY
NuMark
Retailers/Distributors
National Association of Convenience Stores	National Association of Tobacco Outlets (NATO)
Smoke Free Alternatives Trade Association**

^*CB Distributors are a distributor but also manufacture ENDS (21^st Century) and have the 5^th largest retail market share (C.S.D. 2014).

^**SFATA represents the “vaping industry” broadly from manufacturers to retailers/distributors.

After this systematic review of comments and commenters, the final study sample included 19 comments. The language of the comments was coded for whether the comments addressed the regulation of flavors in ENDS, with the researchers identifying (1) the theme or topic of each argument supporting or opposing regulation, and (2) the type of evidentiary support provided for each argument advanced.

All comments were independently coded by two researchers (NH and AF). An inter-rater reliability test was applied to measure the consistency in coding between the two researchers. The rate of divergence between the independent coders was .25, which indicated that a second round of coding should be performed. After clarifying coding standards, a second round of independent coding was done and the new rate of divergence was 0, which means all discrepancies were eliminated (Burris 2014).

In phase one of the content analysis, the researchers searched selected comments for substantive arguments either for or against the regulation of flavors in ENDS. Each argument was categorized into one or more of nine pre-identified topics (teenage use; adult progression from ENDS to cigarette use; initiation by adult non-tobacco users; use of flavors to assist with smoking cessation; flavors and illegal drugs; safety of the ingredients in flavors; accidental consumption of e-liquid by children attracted by flavors; the use of flavors to improve palatability for adults; and other).

Phase two of the content analysis involved identifying and categorizing the evidentiary support provided for each argument advanced. This phase involved determining whether the arguments asserted were supported with references to supporting studies or reports and, if so, characterizing such evidence as peer-reviewed journal articles, government documents, non-peer-reviewed/non-government materials, or journal articles currently under review. Additionally, evidence that was not peer-reviewed was divided into several categories: government press releases, organization reports, newspaper articles, letters to the editor, tobacco documents, internal data, online organization data, and other. Finally, evidentiary materials were examined for industry sponsorship or other conflicts of interest.

Results

Of the 19 comments reviewed, 14 discussed the issue of regulating flavors in ENDS. Of those, all of the public health and legal commenters endorsed either limitations or a complete ban on flavors in ENDS, while the ENDS industry, tobacco industry, and trade organization comments uniformly opposed such regulations. Some industry comments were, however, supportive of regulations that would require child safety mechanisms on flavored cartridges to avoid accidental consumption by children. Additionally, although not specifically related to flavors, one tobacco industry comment (RAI) suggested that the FDA should completely ban “open-systems” (systems that use replaceable cartridges) to reduce risks to children.

One comment (NATO) addressed flavors but did not offer any evidentiary support for its arguments. Of the 13 comments that addressed flavors and offered evidentiary support, there were 91 unique sources of evidentiary support offered (some sources supported more than one argument). Seventy-two of these unique sources were provided by those supportive of regulation and 19 were from those opposed to regulation. Unique evidentiary sources were characterized as peer-reviewed journal articles, government reports, sources that were not journal articles or government reports, and articles that were under peer-review at the time of commenting.

As illustrated in Table 2, commenters who supported regulation cited 33 peer-reviewed journal articles, 11 government reports, 7 newspaper articles, 7 reports from organizations, 5 tobacco documents, and 7 other types of sources. Commenters opposed to regulation referenced 11 peer-reviewed journal articles, 3 organization reports, 1 letter to the editor, 1 newspaper article, 1 internal data report, 1 online organization data report, and 1 journal article that was under review (see Table 3).

Table 2. Descriptive Number of Unique Evidentiary Sources by Type.

	Supportive of Regulation (n=7)	Opposed to Regulation (n=12)
Peer-Reviewed Journal Articles	33	11
Government Reports	11	-
Non-Journal/Government Sources	28	7
Under Review Journal Articles	-	1
Total	72	19

Table 3. Descriptive Number of Non-Journal Sources by Type.

	Supportive of Regulation (n=7)	Opposed to Regulation (n=12)
Government Press Releases	2	-
Organization Reports	7	3
Newspaper Articles	7	1
Letters to the Editor	-	1
Tobacco Documents	5	-
Internal Data	-	1
Online Organization Data	-	1
Other	7	-
Total	28	7

The topics addressed by the commenters are shown in Table 4. Commenters in favor of prohibiting flavors in ENDS focused primarily on teen use/initiation and safety issues (safety of flavors generally and accidental consumption by children attracted to the flavors). Commenters opposed to regulation also focused on teen use/initiation (or lack thereof), as well as the use of flavors in aiding cessation of combustible cigarettes.

Table 4. Descriptive Number of Evidentiary Sources by Topic.

	Supportive of Regulation (n=7)	Opposed to Regulation (n=12)
Teen Users	50	7
Initiation	8	2
Progression	4	-
Cessation	1	12
Use of Other Drugs	1	-
Safety	7	2
Accidental Consumption	12	-
Palatability	4	3
Other	4	-
Total	91	26

Additionally, each referenced source was examined for conflicts of interest. Conflicts of interests were defined to include (a) conflicts identified by the authors in the article notes, or (b) authorship by an author who had been paid by the tobacco or ENDS industry to conduct research in the past, even if a conflict of interest was not noted in the specific article cited. For those opposed to regulation, of the 19 unique sources, 7 sources did not present a known bias or conflict, 5 were industry sponsored or written, and a conflict of interested was identified for 7 others (see Table 5).

Table 5. Unique Sources from Those Opposed to Regulation (n=12) Identifying Type, Industry Funding, Conflicts of Interest, and Peer-Review.

Author/Organization (Date)	Source Type	Industry Sponsored	Conflicts of Interest	Peer-Reviewed
ASH UK (Jul 2014)	Organization Report	No	No	Unknown
ASH UK (Nov 2014)	Organization Report	No	No	Unknown
Barbeau et al. (Mar 2013)	Journal Article	No	No	Yes
Cohen et al. (Nov 2010)	Journal Article	No*	No*	Yes
Dawkins et al. (Mar 2013)	Journal Article	No	Yes	Yes
Etter (May 2010)	Journal Article	No	No**	Yes
Etter and Bullen (Jul 2011)	Journal Article	No	Yes	Yes
Farsalinos et al. (Dec 2013)	Journal Article	No	Yes	Yes
Farsalinos et al. (Apr 2014)	Journal Article	No	Yes	Yes
Farsalinos et al. (Apr 2014)	Letter to Editor	No	Yes	No
Houtsmuller et al. (Jun 2002)	Journal Article	Unknown	No**	Yes
Laugesen (Oct 2008)	Organization Report	Yes	Yes	No
Lorillard (blu) Data	Internal Data	Yes	Yes	No
Pepper et al. (Feb 2013)	Journal Article	No	No	Yes
Richtel (Jul 2014)	Newspaper Article	No	N/A	N/A
Shiffman (ahead of Publication)***	Journal Article ahead of Publication	Yes	Yes	Under review
Vardavas et al. (Jun 2014)	Journal Article	No	No	Yes
Zhu et al. (May 2014)	Journal Article	No	No	Yes
Vaping.com (Jul 2014)	Online Organization Data	Yes	Yes	No

^*Research Funded by Chewing Gum Industry

^**Discloses funding from or conflict relating to Nicotine Replacement Therapy Industry

^***Article under review at time of Comment Submission, however, has since been published (Shiffman et al. 2015)

The two topics with the most discussion and evidence by each group were the use of flavored ENDS by teens/youth and the use of ENDS in cessation from combustible smoking. Descriptions of the evidence from these two categories are detailed below.

Use of Flavored ENDS by Teens

From the comments surveyed, the most frequently addressed topic was the issue of teen users. Proponents of regulation offered numerous studies in support of the notion that teens are attracted to flavors in ENDS (King et al. 2013; J. E. Brown et al. 2014; King et al. 2014; Villanti et al. 2013; Choi et al. 2012; Olson 2014). For example, commenters pointed to studies showing that when teens and young adults (ages 18 – 25) are surveyed about ENDS use, 100% of those that report using ENDS also report using flavored ENDS (Benjamin 2014 citing Couch et al. 2014). In one study, this compares to only 65% of older adults (over the age of 25) (Olson 2014).

Because specific research into ENDS flavors was limited at the time of commenting, many proponents of regulation used scientific evidence derived from flavors in combustible cigarettes or other forms of tobacco to demonstrate an analogous concern with ENDS. For example, the comment from 18 State Attorneys General pointed to a 2004 study that found that in the past month, 22.8% of 17-year-old smokers used flavored cigarettes, compared to only 6.7% of smokers over the age of 25 (Madigan et al. 2014 citing Klein et al. 2008). (The Tobacco Control Act prohibits the sale of flavored cigarettes, with an exception for tobacco and menthol flavors). Proponents also pointed to the rise in flavored cigar use by youth, noting, for example, a survey of Maryland high school-aged youth that showed that 76% of youth who use cigars use flavored products, compared to only 2% of adult cigar smokers (N. Brown 2014 citing Maryland Department of Health and Mental Hygiene 2011).

In response, the opponents of regulation pointed to two main studies and surveys to suggest that teen users are not attracted to the flavors in ENDS and that adults are the main users of flavored ENDS. One source of evidentiary support for this assertion was a survey commissioned and paid for by an ENDS company (NJOY) (Weiss 2014) and the other was a journal article regarding the awareness and use of ENDS by 11 to 19 year old boys in 2013 by Pepper, et. al. (no known conflict of interest) (Pepper et al. 2013).

In the first study, NJOY commissioned Pinney Associates to conduct a study regarding flavor descriptors in ENDS. When testing product interest for several flavors, including fruity and candy-like flavors, the study found that adult smokers were significantly more attracted to flavors than teen non-smokers. Proponents responded by pointing out that there are significant problems with using this research to support the conclusion that flavored ENDS do not attract teens. Specifically, most of the adult smokers surveyed were current users or former users of ENDS, while the teens surveyed had never smoked cigarettes or used ENDS. As the comment submitted on behalf of 24 Public Health Groups states, “[i]t is not surprising that the adult smokers in the study, many of whom had already used e-cigarettes, reported higher interest in e[-]cigarettes as compared to teen never-smokers—what is being measured is interest in flavors in a group predisposed to use of a product versus interest in flavors in a group not predisposed to use the product at all” (24 Undersigned Public Health Groups 2014).

In the second study, opponents of regulation pointed out that in a survey of 228 males ages 11-19, willingness to try ENDS if offered by a best friend (18%) did not differ based on whether or not the product was flavored (Swauger 2014 citing Pepper et al. 2013). Proponents argued that because this study was based on a survey conducted in 2011, the data is outdated due to the fact that, at that time, ENDS use among teens was extremely rare. In Pepper et al.'s survey, only 2 participants (less than 1% of the sample) had ever tried ENDS. Moreover, proponents pointed out that the question asked participants about their willingness to use an “e-cigarette” or a “flavored e-cigarette” without specifying that the former category referred only to unflavored e-cigarettes (24 Undersigned Public Health Groups 2014).

Use of ENDS for Smoking Cessation

Another topic that was heavily addressed was the use of flavored ENDS to aid in cessation of smoking combustible cigarettes. Only one of those in favor of regulation addressed this issue with scientific evidence, but many other commenters called into question many of the evidentiary sources that those opposed to regulation were likely to rely upon.

Most comments opposing regulation referenced a study published in 2013 by Farsalinos et al., which concluded, “[flavors] play a major role in the overall experience of dedicated [ENDS] users and support the hypothesis that they are important contributors in reducing or eliminating smoking consumption”(Farsalinos et al. 2013). In addition to the concern that the first author, Farsolinos, has conducted industry-sponsored research in the past, proponents of regulation detailed other significant limitations to this study. First, it used a convenience sample of participants recruited from popular ENDS discussion forums, and the study itself noted that the participants were mainly current ENDS users who had a positive experience with ENDS. Additionally, as the comment from 24 Health Groups concluded, “this internet-based survey just asked participants to rate the importance of flavor variability in reducing or quitting smoking but did not actually assess if having access to a variety of flavors leads to reduced consumption or quitting or if those who said they had quit stayed quit over time”(24 Undersigned Public Health Groups 2014).

Opponents of regulation also pointed to a study commissioned by Vaping.com, which reported that two-thirds of ex-smokers surveyed answered that flavors (other than tobacco and menthol) were important in helping them quit traditional cigarettes. Proponents of regulation explained that, as with the Farsalinos study, this study used a non-representative sample and asked participants only for their opinions; it did not track whether flavored products did in fact contribute to smoking cessation (24 Undersigned Public Health Groups 2014).

Proponents of regulation cited a 2013 study by Brian King et al. that revealed that users of flavored products had lower quit intentions than those that use non-flavored products. (Blanke 2014 citing King et al. 2014). King et al. found that that prevalence of not intending to quit was higher among youth users of flavored little cigars (59.7%) than among those who did not use flavors (49.3%).

In sum, the case studies reviewed here suggest that proponents of the regulation of flavors in ENDS provided more peer-reviewed, credible sources of evidence to support their assertion that flavors in ENDS are harmful to youth. The sources provided on this topic from opponents of the regulation of flavors in ENDS lacked independent credibility and suffered from weak methods in choosing sample populations and relevant comparisons. While proponents of regulation were limited in the availability of evidence directly assessing whether or not flavored products facilitate smoking cessation, the evidence that was provided by opponents—asserting that flavored ENDS products assisted cessation—was sufficiently weak to question their assertions.

Discussion

Overall, commenters in our targeted sample who supported the regulation of flavors in ENDS were considerably more likely to use peer-reviewed sources of evidence to support their positions. Matching the finding of Hatchard et al., those opposing regulation were more likely to use non-peer-reviewed sources, and those sources that were peer-reviewed research were often sponsored and/or written by the industry or had a clear conflict of interest (Hatchard et al. 2014). This suggests that the FDA should not weigh all comments—or all arguments—equally. Instead, the FDA should pay attention to the provenance, relevance, and reliability of the evidence presented in public comments, and it should be wary of overreliance on studies affected by self-interest.

The FDA should be particularly wary of research funded or influenced by the tobacco industry, as the industry has a long history of manipulating scientific research to promote its products and avoid regulatory oversight. From the 1950s through the 1990s, the tobacco industry funded the Tobacco Industry Research Committee (later renamed the Council for Tobacco Research), an entity “controlled by the industry yet promoted as independent,” with the ultimate goal not of advancing science, but of “utiliz[ing] ‘science’ in the service of public relations”(Brandt 2009). It later developed the Council for Indoor Air Research, a grant-making entity “designed to provide counterarguments against … emerging [secondhand smoke] regulatory efforts,” and even launched a purportedly independent journal in which to publish industry-friendly research (Garne et al. 2005). To this date, research projects funded by the tobacco industry are carefully selected (often by industry attorneys) in order to promote corporate interests and minimize legal risks; not to advance scientific knowledge (Proctor 2012). In consideration of this history, the FDA Center for Tobacco Products should proceed with caution when weighing industry-sponsored research.

It should be noted that the majority of ENDS companies are not affiliated with “traditional” tobacco companies in any way. However, some of the major ENDS companies (such as NuMark, which was included in the study sample) are subsidiaries of larger tobacco companies, and it is expected that tobacco companies will increasingly dominate the ENDS market (Tobacco Control Research Group 2014). In addition, “[ENDS] manufacturers have shown little interest in conducting or supporting peer-reviewed research,” and have instead based public claims on research affected by significant conflicts of interest (Etter 2015).

One challenge in addressing the issue of self-interested research is the FDA's institutional culture, which, outside the context of tobacco regulation, often works closely with industry and views regulated companies as its “partners.” (Carpenter 2010; Light 2013). For example, the pharmaceutical industry works closely with the FDA's Center for Drug Evaluation and Research, and most clinical studies that inform drug approval decisions are sponsored by the industry (Carpenter 2010). This might not be viewed as a problematic conflict of interest, as both the regulated industry and the FDA arguably have a shared interest in the approval of drugs that will be safe and effective for consumers. (Jenson 2016) However, this is not the case in the tobacco context, where the public health interest in reducing tobacco use is in direct conflict with the industry's interests (Conference of Parties 2008).

Moreover, even in the drug approval context, numerous researchers have asserted that the close relationship between the FDA and the pharmaceutical industry sometimes leads the FDA to make decisions detrimental to public health (Hilts 2003; Carpenter 2010; Angell 2005; Abramson 2005; Healy 2006). For instance, Hilts and others have demonstrated that the industry has used sophisticated lobbying operations to lower the standards for new drug approvals and to pressure the FDA to approve new drugs more quickly. (Hilts 2003; Angell 2005; Abramson 2005) This has “rewarded the hidden business model of developing … drugs with minor benefits,” while increasing the risk that newly-approved drugs pose serious safety risks. (Light 2013)

As a new center within the FDA, with much of its employee base recruited from other parts of agency, the Center for Tobacco Products should by hyper-vigilant regarding the issues of conflict of interest, and should recognize that the challenges faced in regulating tobacco are distinct from the challenges in other FDA-regulated domains. Indeed, the World Health Organization has taken the position that tobacco regulators should “avoid the creation of any perception of a real or potential partnership” with the industry, given the industry's conflicting motives and its history of working to evade and subvert regulation (Conference of Parties 2008). While industry-funded research may be a necessary input into regulatory decisions in some cases, the FDA should consider developing procedures to minimize conflicts for interests. In 2011, the Institute of Medicine (since renamed the Heath and Medicine Division of the National Academies) proposed several such measures, including the creation of an independent Tobacco Research Governance Entity (TRGE) to oversee research on tobacco products and ensure its independence (Institute of Medicine 2012).

As noted above, FDA did not restrict flavored ENDS in the final deeming rule, instead referencing the need to balance the “concerns regarding flavored tobacco products' appeal to youth [against] the emerging evidence that some adults may potentially use certain flavored tobacco products to transition away from combusted products” (81 Fed. Reg. 90, 28997 (May 10, 2016)). While the FDA pointed to the weaknesses in the oft-cited Farsalinos study regarding adult use of flavored ENDS products for smoking cessation, it did not point to similar weaknesses in other evidence discussed above. This shows that while some examination of the evidentiary support has occurred, it does not appear to have been a systematic inquiry. Importantly, the final deeming rule, in discussing the information presented in public comments, made no mention of conflicts of interest in research. Thus, there is no evidence that the FDA analyzed or took into account conflicts or biases when evaluating the evidence presented evidence in regards to flavors (81 Fed. Reg. 90 (May 10, 2016)).

After the deeming rule was published, it was revealed that the FDA had initially proposed requiring flavored ENDS (and other newly-deemed products) to be withdrawn from the marketplace until they had undergone FDA review (Food & Drug Admin., FDA-2014-N-0189, TAB B 2014-850 Deeming Final Rule Redline Changes (2016)). A 17-page section of the draft rule, analyzing the evidence supporting such a rule, was deleted by the White House Office and Management and Budget (OMB) and was not included in the final rule. This deleted text, while referencing some of the evidence provided by opponents of the regulation of flavors, likewise did not reference actual or potential conflicts of interests. The OMB replaced the deleted text with a statement that “concerns regarding flavored tobacco products' appeal to youth [against] the emerging evidence that some adults may potentially use certain flavored tobacco products to transition away from combusted products” (81 Fed. Reg. 90, 28997 (May 10, 2016)). While it is unknown why the OMB chose to remove the proposed restrictions on ENDS, the FDA's failure to point out the conflicts of interest in the research on adult use of flavorings may have led to the OMB to put undue weight on this evidence.

Notably, more recent research appears to provide additional support for the proponents' arguments regarding ENDS flavors being attractive to youth and lacking in utility for cessation. For example, a 2014 study found that 43.8% of youth surveyed were attracted to experimenting with ENDS products due to flavors (Kong et al. 2014). Moreover, most studies conducted since the end of the comment period have suggested that ENDS – flavored or not – are not effective in assisting current smokers in quitting (Grana, Popova, and Ling 2014; Brose et al. 2015; Sutfin et al. 2015).

This research adds to the literature on industry involvement in regulatory processes. Prior research into regulatory rulemaking has highlighted the ability of regulated industries to use the public comment period to their advantage and has demonstrated the tobacco industry's willingness, at least in the context of U.K. regulatory processes, to make selective and misleading use of data to support its claims (Hatchard et al. 2014; Hwang, Avorn, and Kesselheim 2014; Hwang et al. 2014; Ulucanlar et al. 2014). While there is no way to prevent misleading evidence from being submitted during the public comment period, the FDA arguably has a legal duty to identify and screen out such evidence. Under the Administrative Procedure Act, the FDA cannot make decisions that are “arbitrary or capricious,” and under Tobacco Control Act, the FDA must regulate in ways that are “appropriate for the protection of public health” (Administrative Procedures Act, 5 U.S.C. § 706(2) (2006); 21 U.S.C. § 301). Basing regulatory decisions on evidence of questionable validity would run counter to both of these legal requirements. Furthermore, given the importance of tobacco use to public health, it is critically important that the FDA's notice and comment process leads to an objective assessment of the available evidence and is not systematically biased towards reducing the stringency of proposed regulations.

Currently, the FDA's process for analyzing submitted comments is a regulatory black box; information not available to the public. Although that information is unknown, this research suggests that the FDA, if not already doing so, should use a systematic and thoughtful approach to analyzing and evaluating the quality of comments and evidentiary sources submitted during comment periods. Such an approach should be tiered in such a way that evidence that is peer-reviewed and non-conflicted should be given the most weight, and evidence that is not peer-reviewed and conflicted should be disregarded or discounted when the FDA considers arguments made by commenters.

Given the challenge to the FDA in critically analyzing the large volume of comments submitted, the research community should also consider developing a process to quickly review and analyze comments submitted to the FDA by industry-affiliated entities. The FDA can assist with this process by making the comments easier to access and download into searchable databases. Also, this research can serve as an important reminder to the research community that when performing policy-relevant research, researchers should consider how their evidence can assist the FDA in making informed regulatory decisions, and, if not doing so already, they should actively engage in the public comment process.

Limitations

Only 19 comments were reviewed and coded, out of the thousands received by the FDA. Of note, however, the vast majority of public comments submitted to the FDA consisted of a brief statement of support or opposition and did not present any evidence or substantive assertions. By focusing on a targeted sample of organizations and companies, we sought to identify the major sources and types of evidentiary support presented to the FDA. A further limitation is that only those evidentiary sources cited in the results were examined for the strength of their methods or whether they supported the commenter's position. Like some of the journal articles discussed above, it is possible that other articles, addressing other issues raised by the FDA in its proposed deeming rule, may have had their findings exaggerated or misrepresented in public comments. Finally, while the implications for the research community apply broadly, the findings regarding the FDA regulatory process are limited to the Center for Tobacco Products and regulations under the Tobacco Control Act.

Conclusion

The FDA is tasked with examining public comments submitted before issuing final rules relating to the regulation of tobacco. When doing so, the FDA should consider not just the arguments that are made, but also the quality and strength of the evidence that is offered on all sides of the issue. The research community can assist in this process by conducting reviews to help the FDA evaluate the strength of evidence on each relevant issue. In addition, new methods are needed to more systematically review the evidence submitted in all public comments.

Biographies

Natalie Hemmerich, JD, is a policy post-doctoral researcher at the Center of Excellence in Regulatory Tobacco Science at Ohio State University. She researches and writes about policy in tobacco regulatory science. Her most recent publications include Emotion in the Law and the Lab: The Case of Graphic Cigarette Warnings (Tobacco Regulatory Science, 2016, co-authored with Ellen Peters, Abigail Evans, and Micah Berman) and Online E-cigarette Marketing Claims: A Systematic Content and Legal Analysis (Tobacco Regulatory Science, 2016, co-authored with Elizabeth Klein, Micah Berman, Cristen Carlson, Susandi Htut, and Michael Slater).

Elizabeth G. Klein, PhD, MPH is an Associate Professor in Health Behavior & Health Promotion at the Ohio State University College of Public Health. Dr. Klein has spent her public health career primarily focused on environmental and policy change to promote the prevention of tobacco use. She has published research regarding graphic warning labels, smoke-free policies, and other policy strategies to reduce tobacco use among vulnerable populations. Dr. Klein is currently serving as a co-Investigator on the Ohio State University's Center of Excellence in Regulatory Tobacco Science.

Micah Berman, JD, is an Assistant Professor of Public Health and Law at Ohio State University's College of Public Health and Moritz College of Law. He teaches courses on public health law, health care law, and tobacco regulation. Currently, he is the Principal Investigator on an NCI-funded study that seeks to integrate scientific risk assessment methodologies with legal analysis. He is co-authoring a Public Health Law textbook that will be co-published by Oxford University Press and the American Public Health Association.