Abstract
Policy Points.
Food and Drug Administration (FDA) advisory committee recommendations and the agency's final actions exhibit high rates of agreement, with cases of disagreement tending to reflect the proposed action type and degree of advisory committee consensus.
In the case of disagreements, the FDA tended to be less likely than its advisory committees to approve new products, approve new supplemental indications, or enact new safety changes.
These findings raise important issues regarding the factors that differentially shape decision making by advisory committees and the FDA as an agency, including institutional or reputational concerns.
Context
The Food and Drug Administration (FDA) convenes advisory committees to provide external scientific counsel on potential agency actions and to inform regulatory decision making. The degree to which advisory committees and their respective agency divisions disagree on recommendations has not been well characterized across product and action types.
Methods
We examined public documents from FDA advisory committee meetings and medical product databases for all FDA advisory committee meetings from 2008 through 2015. We classified the 376 voting meetings in that period by medical product, regulatory, and advisory committee meeting characteristics. We used multivariable logistic regression to determine the associations between these characteristics and discordance between the advisory committee's recommendations and the FDA's final actions.
Findings
Twenty‐two percent of the FDA's final actions were discordant with the advisory committee's recommendations. Of these, 75% resulted in the FDA making more restrictive decisions after favorable committee recommendations, and 25% resulted in the agency making less restrictive decisions after unfavorable committee recommendations. Discordance was associated with lower degrees of advisory committee consensus and was more likely for agency actions focused on medical product safety than for novel approvals or supplemental indications. Statements by public speakers, advisory committee conflicts of interest, and media coverage were not associated with discordance between the committee and the agency.
Conclusions
The FDA disagrees with the recommendation of its advisory committees a minority of the time, and in these cases it tends to be less likely to approve new products or supplemental indications and take safety actions. Deviations from recommendations thus offer an opportunity to understand the factors influencing decisions made by both the agency and its expert advisory groups.
Keywords: US Food and Drug Administration, advisory committees, drug approval, device approval, consumer product safety
The food and drug administration (fda) is responsible for the safety and effectiveness of medical products, including drugs, biologics, and medical devices. When making decisions, the FDA may convene advisory committees to consult on matters of scientific and regulatory importance, such as the evaluation of new medical products, the assessment of safety concerns, the establishment of new drug development programs, and the drafting of new guidance.1, 2, 3, 4 These committees are composed of individuals otherwise unaffiliated with the FDA who possess relevant clinical, research, statistical, or other expertise. They review evidence presented by FDA staff and product sponsors; hear comments offered by members of the public; and engage in public deliberation in response to guiding questions developed by the agency for their consideration. In many cases, the committees’ deliberations conclude with formal votes and recommendations for consideration and subsequent action by the FDA.5
Advisory committees are intended to improve the quality of FDA decision making by engaging diverse stakeholders—including industry, payers, and patients—and to offer an exchange of information between internal bureaucrats and external implementers.6, 7, 8, 9 However, the consequences of this engagement for the actual outcome of FDA decision making are complex. While previous work has developed frameworks exploring how the counsel of advisory committees may be understood within FDA's regulatory activities,9 it has not extended to analyzing the outcome of that counsel, nor has it examined the full breadth of the issues that the advisory committees are asked to resolve.
Existing analyses of advisory committee meeting outcomes have been limited to the outcomes of only some divisions’ advisory committee meetings and have found that rates of agreement between advisory committees and final agency decision making range from 60% to 87% overall.10, 11, 12 In addition, multiple studies have investigated individual factors that may contribute to committees being called, such as product complexity9 or media interest,2 or that may contribute to committee recommendations, such as the high prevalence of financial conflicts of interest among committee members13, 14, 15, 16 or public speakers,17, 18 but they have not always linked these to the agency's final decisions in order to understand the full impact of their effects.
Our objectives were thus to understand how the FDA interprets the recommendations of its advisory committees and, in particular, to explore potential contributing factors to cases in which the FDA as an agency disagreed with its advisory committees’ recommendations. Accordingly, we characterized voting FDA advisory committee meetings across all divisions and meeting types; examined associations with measures of industry influence, public engagement, and unmet product need; and described examples of cases in which the decisions conflicted.
Study Data and Methods
Data Sources
We searched the Advisory Committees Section of the FDA website to identify all advisory committee meetings from January 2008 through December 2015.6 These meetings were defined by topic area, so that a meeting covering a single topic over multiple days was considered a single meeting, whereas a meeting covering multiple products or multiple indications over a single day was considered multiple meetings. We focused on those advisory committee meetings evaluating the safety and/or efficacy of medical products for human use. We excluded advisory committee meetings convened for general scientific and policy guidance; for diagnostic and imaging agents; and for mandated evaluation of FDA recommendations, such as those required by the Best Pharmaceuticals for Children Act (BPCA), in which voting questions are phrased affirmatively.
We restricted our analysis to those meetings at which dichotomous votes were cast for a voting question relating to an FDA action to be taken on a single product or a class of products, consistent with prior studies.13, 16 We selected a single voting question for analysis for each meeting, prioritized by a previously defined hierarchy: (1) questions considering approval of a product, label change (supplemental indication or safety warning), or withdrawal of a product; (2) questions considering the expedited approval of a product; and (3) questions considering whether safety or efficacy alone had been established for a product.13, 16 For those meetings with more than one voting question of the same level, we selected the voting question using a randomized number generator.
Main Outcome Measures
In order to identify cases of discordance between an advisory committee's recommendation and an FDA action, we evaluated two outcomes of each advisory committee meeting: the advisory committee's recommendation and the FDA's action within 12 months of the meeting. We found agreement when both the advisory committee's recommendations and the FDA actions were classified as either favorable or unfavorable, and we found disagreement when one was classified as favorable and the other unfavorable.
To identify an advisory committee's recommendations, we used the committee meeting's minutes, sometimes known as “24‐Hour Summaries,” and transcripts. If the minutes were available, we identified the single voting question according to the hierarchy described previously from the section “Questions to the Committee” and recorded the number of votes for each option. If the minutes were not available, we identified the single voting question and the results from the committee transcript directly. Advisory committee recommendations were classified as favorable when the majority of voting members voted in favor of FDA actions that supported the market availability of a product, whether by means of an approval, a supplemental indication approval, or the lack of a safety warning or withdrawal. To determine the strength of an advisory committee's recommendations, we measured their degree of favorability, defined as the proportion of total advisory committee votes recommending market availability of the product. We also measured the degree of consensus, defined as the proportion of total advisory committee votes recommending the majority action, regardless of the recommendation's direction.
To identify the FDA's actions, we used the Drugs@FDA database, the FDA's Premarket Approval database, and the FDA's list of licensed biological products. As with advisory committee recommendations, actions that supported a product's market availability were defined as favorable. Thus, the FDA was determined to have taken favorable action after an advisory committee meeting if the outcome was an approval, supplemental indication, or failure to institute a safety warning or withdrawal for that medical product corresponding to the advisory committee's voting question within a year of the meeting. In contrast, the FDA was determined to have taken unfavorable action if the outcome was no subsequent approval or supplemental indication, or if a safety warning or withdrawal was instituted. In those cases in which no action was found, the FDA's press releases and drug safety announcements were used as a secondary source to identify any other actions.
Medical Product, Regulatory, and Advisory Committee Meeting Characteristics
We used the minutes and transcripts of advisory committee meetings to determine the meeting date, advisory committee type, and proposed action type (approval, supplemental indication, or safety). The advisory committee's type was defined by the FDA center or division within which each advisory committee was convened and to which it reported. To measure potential industry influence, we also collected information on conflicts of interest among the committee's voting members. Using the meeting's minutes, transcripts, and waivers, we tallied the number of committee members with conflicts in favor of the product sponsor, and the number of committee members with conflicts in favor of a competitor to the product sponsor, that is, two categories not mutually exclusive. We then categorized the committees based on whether they included voting members with sponsor conflicts only, competitor conflicts only, both kinds of conflicts, or no conflicts.
To measure the unmet need for a product, we considered the product's regulatory characteristics, such as an orphan designation or inclusion in a special regulatory program. An orphan designation is given to products that treat rare conditions, defined as affecting fewer than 200,000 (for drugs) or 8,000 (for devices) people in the United States, and is designed to incentivize the development of products for rare conditions that otherwise have no treatments.19 We used both the FDA Orphan Drug Designation database and the FDA Humanitarian Device Exemption database to identify whether a medical product had been given orphan status. Special regulatory programs, including fast track, priority review, accelerated approval, and breakthrough designation, are designed to direct resources to products that demonstrate major advances in treatment or otherwise address unmet needs for serious or life‐threatening diseases for which there are no or limited therapies20 and have been used in prior studies as a proxy for drug novelty.9, 21, 22 We used the Drugs@FDA database and the FDA Pre‐Market Approval (PMA) database as the primary data sources to identify whether a medical product was evaluated using a special regulatory program. When a special regulatory program could not be identified using these databases, we searched the FDA listings of approved drugs receiving accelerated approval, fast track, or breakthrough designation, as well as the advisory committee's meeting transcripts and briefing materials to find more designations. For those advisory committee meetings evaluating safety actions, we investigated only whether the indications affected by the safety action (eg, warning, risk evaluation and mitigation strategies [REMS], or withdrawal) received an orphan designation or were approved using a special regulatory program. For those safety actions involving multiple products (eg, a class of drugs), any one product receiving the designation/program was sufficient.
To measure public engagement with a product, we considered the participation of speakers in the public hearing section and media coverage before the committee meeting. We then abstracted the speaker's statements from the meeting's transcripts, tallied them, and classified them as favorable if they supported a product's market availability through approval, supplemental indication, or lack of safety warning or withdrawal. For each committee, we calculated the degree of speaker favorability as the proportion of speakers expressing favorable views, as well as the degree of speaker consensus as the proportion of speakers expressing the majority view, regardless of the direction of those views. To measure popular media coverage, we used the archives of the New York Times, Wall Street Journal, and Washington Post, indexed by LexisNexis Academic, to find news articles that were published within six months before the advisory committee meeting and that contained the brand name or generic name of the medical product being considered. The articles were screened for relevance, excluding those unrelated to FDA action (eg, obituaries), and tallied as a measure of media coverage.
Statistical Analysis
All analyses were performed using R version 3.5.1 (R Foundation for Statistical Computing). Descriptive statistics were used to characterize the sample of advisory committee meetings. Associations between variables and discordance between advisory committees and the FDA were assessed using multivariate logistic regression, accounting for all described medical product, regulatory, and advisory committee meeting characteristics. All statistical tests were two‐tailed and used a significance level of p<0.05.
Results
A total of 759 topic‐specific meetings were held by FDA advisory committees from January 2008 through December 2015 that were eligible for inclusion in our study. Of these, 446 (59%) were product‐specific meetings, of which 404 (91%) involved voting questions. After excluding meetings that discussed contrast agents or diagnostic tests (n = 26) and those for which no information was available (n = 2), we studied 376 meetings, representing meetings held by 27 advisory committees or panels (median eight meetings per committee, range 1‐37) discussing 298 unique products or product classes.
Of these 376 meetings, 237 (63%) involved drug products; 78 (21%) involved biologics; and 61 (16%) involved medical devices. The proposed action types were initial product approvals in 271 (72%) meetings, supplemental indications in 78 (21%) meetings, and safety actions in 27 (7%) meetings. The most common therapeutic areas were cardiovascular, antimicrobial, and oncologic treatments, which together represented 41% (n = 155) of all meetings. Voting members with conflicts of interest were present among 57 (15%) meetings, 22 (6% overall) involving sponsor conflicts, and 45 (12% overall) involving competitor conflicts, with the number of voting members with reported conflicts in any individual committee meeting ranging from 0 to a maximum of 3. Indications receiving orphan designation were the subject of 88 (23%) meetings, and products evaluated using a special regulatory program accounted for 153 (41%) meetings.
With regard to public engagement, 308 (82%) of the meetings included at least one speaker in the public hearing portion of the meeting (median three speakers, IQR 1‐8). The majority of these 308 meetings (n = 268, 71%) involved speakers expressing favorable opinions with respect to the product, with a median proportion of favorable speakers among these meetings of 90% (IQR 50%‐100%). At least one media mention in the six months before the committee meeting was found in 128 (34%) of the meetings (IQR 0‐1 mentions). Table 1 summarizes other characteristics of the meetings.
Table 1.
Characteristics of FDA Advisory Committee Meetings, 2008‐2015
| Medical Product, Regulatory, and Advisory Committee Meeting Characteristics | |
|---|---|
| Year, No. (%) | |
| 2008 | 41 (11) |
| 2009 | 65 (17) |
| 2010 | 46 (13) |
| 2011 | 47 (13) |
| 2012 | 56 (15) |
| 2013 | 46 (12) |
| 2014 | 43 (11) |
| 2015 | 32 (9) |
| Advisory Committee, No. (%) | |
| CDER | 291 (77) |
| − Antimicrobial | 47 (13) |
| − Oncologic | 37 (10) |
| − Endocrine and Metabolic | 34 (9) |
| − Cardiovascular and Renal | 30 (9) |
| − Pulmonary and Allergy | 27 (7) |
| − Other Drugs | 116 (31) |
| CBER | 24 (6) |
| CDRH | 61 (16) |
| Action Type, No. (%) | |
| Approval | 271 (72) |
| Supplemental Indication | 78 (21) |
| Safety | 27 (7) |
| Orphan Designation, No. (%) | 88 (23) |
| Special Regulatory Program, No. (%) | 153 (41) |
| Members with COI, No. (%) | |
| Any | 57 (15) |
| Product Sponsor | 22 (6) |
| Product Competitor | 45 (12) |
| Median No. Public Speakers (IQR) | 3 (1‐8) |
| Median Percentage of Favorable Public Speakers (IQR) a | 90 (50‐100) |
| Median No. Media Mentions (IQR) | 0 (0‐1) |
Among committees with public speakers (n = 308).
CDER = Center for Drug Evaluation and Research; CBER = Center for Biologics Evaluation and Research; CDRH = Center for Devices and Radiological Health; COI = conflicts of interest; IQR = interquartile range.
The advisory committees’ recommendations and the FDA's actions were discordant in 83 meetings (22%) (Table 2, κ = 0.519). In multivariable analysis, advisory committee–agency action discordance was associated with the type of action and the degree of advisory committee consensus. Discordance rates were 21% for products evaluated for approval, 18% for supplemental indication, and 48% for safety (supplemental indication vs original approval, OR 0.94, 95% CI 0.44‐1.99; safety vs original approval, OR 4.73, 95% CI 1.56‐14.3; cumulative p = 0.02). In addition, a greater percentage of advisory committee members voting for the majority opinion was associated with lower odds of discordance between advisory committee and agency action (OR 0.97, 95% CI 0.95‐0.98, p<0.001). No other characteristics were associated with discordance, including public speaker favorability or conflicts of interest among committee members (Table 3). The results were consistent when limited to the FDA's advisory committees convened by CDER (Supplement Table 4).
Table 2.
Rates of Favorable Advisory Committee Recommendations, Favorable FDA Actions, and Agreement, 2008‐2015
| FDA Action, | |||
|---|---|---|---|
| Advisory Committee | No. (%) | ||
| Recommendation, No. (%) | Favorable | Unfavorable | Overall a |
| Favorable | 204 (77%) | 62 (23%) | 266 (71%) |
| Unfavorable | 21 (19%) | 89 (81%) | 110 (29%) |
| Overall | 225 (60%) | 151 (40%) | |
κ = 0.519.
FDA = Food and Drug Administration.
Table 3.
Multivariable Association Between Medical Product, Regulatory, and Meeting Characteristics and Discordance Between Advisory Committee Recommendation and FDA Action, 2008‐2015
| Medical Product, | Likelihood of Discordance Between Advisory Committee Recommendation and FDA Action | |
|---|---|---|
| Regulatory, and Meeting Characteristics | Adj. OR (95% CI)a | p−value |
| Year | 0.20 | |
| 2008 | Ref | |
| 2009 | 0.57 (0.22‐1.48) | |
| 2010 | 0.66 (0.23‐1.84) | |
| 2011 | 0.35 (0.12‐0.99) | |
| 2012 | 0.55 (0.20‐1.52) | |
| 2013 | 0.27 (0.09‐0.86) | |
| 2014 | 0.28 (0.08‐0.92) | |
| 2015 | 0.24 (0.06‐0.96) | |
| Committee | 0.10 | |
| CDER | ||
| − Antimicrobial | 0.28 (0.10‐0.82) | |
| − Oncologic | 0.41 (0.13‐1.30) | |
| − Endocrine | 0.54 (0.18‐1.56) | |
| − Cardiovascular | 0.34 (0.10‐1.15) | |
| − Pulmonary | 0.44 (0.13‐1.51) | |
| − Other Drugs | Ref | |
| CBER | 0.98 (0.30‐3.22) | |
| CDRH | 1.06 (0.47‐2.37) | |
| Action Type | 0.02 | |
| Approval | Ref | |
| Supplemental Indication | 0.94 (0.44‐1.99) | |
| Safety | 4.73 (1.56‐14.3) | |
| Orphan Designation | 1.36 (0.65‐2.85) | 0.41 |
| Special Regulatory Program | 1.49 (0.79‐2.83) | 0.22 |
| No. Public Speakers | 0.98 (0.93‐1.04) | 0.56 |
| Public Speaker Consensus % | 0.93 | |
| Consensus (>50%) | 0.90 (0.42‐1.96) | |
| No Consensus (<50%) | 1.19 (0.16‐9.16) | |
| No Public Speakers | Ref | |
| No. Media Mentions | 0.93 (0.83‐1.04) | 0.15 |
| Conflicts of Interest | 0.24 | |
| Sponsor Only | 0.19 (0.02‐1.71) | |
| Competitor Only | 1.53 (0.64‐3.64) | |
| Both | 0.95 (0.12‐7.41) | |
| None | Ref | |
| Committee Consensus % | 0.97 (0.95‐0.98) | <0.001 |
OR = odds ratio; CI = confidence interval; FDA = Food and Drug Administration; CDER = Center for Drug Evaluation and Research; CBER = Center for Biologics Evaluation and Research; CDRH = Center for Devices and Radiological Health
Table 4.
Characteristics of Discordant Advisory Committee Recommendation and FDA Action Pairs, 2008‐2015
| Medical Product, Regulatory, and Meeting Characteristics | FDA More Restrictive (n = 62) | FDA Less Restrictive (n = 21) |
|---|---|---|
| Year, No. (%) | ||
| 2008 | 15 (24.2) | 2 (9.5) |
| 2009 | 13 (21.0) | 2 (9.5) |
| 2010 | 8 (12.9) | 4 (19.0) |
| 2011 | 8 (12.9) | 2 (9.5) |
| 2012 | 8 (12.9) | 4 (19.0) |
| 2013 | 5 (8.1) | 2 (9.5) |
| 2014 | 2 (3.2) | 4 (19.0) |
| 2015 | 3 (4.8) | 1 (4.8) |
| Committee, No. (%) | ||
| CDER | ||
| − Antimicrobial | 6 (9.7) | 0 (0) |
| − Oncologic | 1 (1.6) | 4 (19.0) |
| − Endocrine | 6 (9.7) | 0 (0) |
| − Cardiovascular | 3 (4.8) | 1 (4.8) |
| − Pulmonary | 2 (3.2) | 5 (23.8) |
| − Other Drugs | 22 (35.5) | 11 (52.4) |
| CBER | 6 (9.7) | 0 (0) |
| CDRH | 16 (25.8) | 0 (0) |
| Action Type, No. (%) | ||
| Approval | 48 (77.5) | 8 (38.1) |
| Supplemental Indication | 13 (21.0) | 1 (4.8) |
| Safety | 1 (1.6) | 12 (57.1) |
| Orphan Designation, No. (%) | 16 (25.8) | 4 (19.0) |
| Special Regulatory Program, No. (%) | 28 (45.2) | 6 (28.6) |
| Median No. Public Speakers (IQR) | 3 (1‐9.5) | 5 (1‐6) |
| Public Speaker Favorability, No. (%) | ||
| Favorable (≥50%) | 39 (62.9) | 10 (47.6) |
| Unfavorable (<50%) | 9 (14.5) | 9 (42.9) |
| No Public Speakers | 14 (22.6) | 2 (9.5) |
| Median No. Media Mentions (IQR) | 0 (0‐1) | 0 (0‐1) |
| Conflicts of Interest | ||
| Sponsor Only | 1 (1.6) | 0 (0) |
| Competitor Only | 11 (17.7) | 2 (9.5) |
| Both | 2 (3.2) | 0 (0) |
| None | 48 (77.4) | 0 (0) |
| Committee Consensus (%) | 80.6 (50.0‐100) | 73.9 (53.8‐100) |
FDA = Food and Drug Administration; CDER = Center for Drug Evaluation and Research; CBER = Center for Biologics Evaluation and Research; CDRH = Center for Devices and Radiological Health.
The 83 cases of discordance between advisory committees’ recommendations and the FDA's actions were further classified as either those in which the FDA took a more restrictive stance (n = 62; 75%) or a less restrictive stance (n = 21; 25%) than its advisory committees’ recommendations (Tables 4 and 5). Of those 62 decisions in which the FDA took a more restrictive stance—that is, issuing an unfavorable decision after a favorable recommendation by the advisory committee—48 (77%) concerned novel product approvals, 13 (21%) supplemental indications, and one (2%) a safety action. Of the 61 more restrictive actions regarding novel product approvals and supplemental indications, 33 (54%) were eventually approved, representing 29 novel products and four supplemental indications. Twenty‐nine (47%) decisions involved products with special regulatory designations, 16 (26%) with an orphan designation, and 28 (45%) with a special regulatory program. Twenty (32%) meetings received media coverage in the six months before the meeting, and 48 (77%) meetings featured public speakers. The median proportion of advisory committee members voting with the majority was 80.6% (range 50%‐100%).
Table 5.
Examples of Discordant Advisory Committee Recommendation and FDA Action Pairs, 2008‐2015
| Meeting | Product | Voting Question | Final Action Date |
|---|---|---|---|
| Less Restrictive FDA Action After Unfavorable Advisory Committee Recommendation | |||
| CDER Pulmonary‐Allergy Drugs (12/11/2008) | LABAs | Risk‐benefit analysis for treatment of asthma in adults | 6/25/2010 (561 days) |
| CDER Oncologic Drugs (7/20/2010) | Avastin (bevacizumab) | Withdrawal of indication for initial treatment of patients with metastatic breast cancer | 12/20/2011 (518 days) |
| CDER Reproductive Health Drugs (9/9/2011) | Bisphosphonates | Label modification for osteonecrosis of the jaw and atypical femur fractures that may be associated with long‐term use of bisphosphonates | NA |
| CDER Anesthetic and Analgesic Drug Products (12/7/2012) | Zohydro ER (hydrocodone bitartrate ER) | Approval for management of moderate to severe chronic pain when a continuous, around‐the‐clock opioid analgesic is needed for an extended period of time | 10/25/2013 (322 days) |
| CDER Oncologic Drugs (11/6/2014) | Farydak (panobinostat) | Approval for treatment of patients with multiple myeloma who have received at least one prior therapy, in combination with bortezomib and dexamethasone | 2/23/2015 (109 days) |
| More Restrictive FDA Action After Favorable Advisory Committee Recommendation | |||
| CDER Psychopharmacologic Drugs (7/10/2008) | Chronically administered antiepileptic drugs | Warning for suicidality | 12/16/2008 (159 days) |
| CDER Cardio‐Renal Drugs (3/19/2009) | Xarelto (rivaroxaban) | Approval for prophylaxis of deep vein thrombosis and pulmonary embolism in patients undergoing hip replacement surgery or knee replacement surgery | 7/1/2011 (834 days) |
| CDER Psychopharmacologic Drugs (4/8/2009) | Seroquel XR (quetiapine fumarate) | Supplemental indication for treatment of major depressive disorder as a monotherapy | NA |
| CDER Anti‐Infective Drugs (4/3/2012) | ciprofloxacin | Supplemental indication for treatment of pneumonic plague | 2/2/2015 (1035 days) |
| CDER Psychopharmacologic Drugs (3/21/2013) | Probuphine (buprenorphine HCl and ethylene vinyl acetate subdermal implant) | Approval for maintenance treatment of opioid dependence | 5/26/2016 (1162 days) |
Of those 21 decisions in which the FDA took a less restrictive stance—that is, issuing a favorable decision after an unfavorable advisory committee recommendation—12 (57%) concerned safety actions, eight (38%) novel product approvals, and one (5%) a supplemental indication. Eight (38%) decisions involved products with special regulatory designations, four (19%) with an orphan designation, and six (29%) with a special regulatory program. Eight (38%) meetings received media coverage in the six months before the meeting, and 19 (90%) meetings featured public speakers. The median proportion of advisory committee members voting with the majority was 73.9% (range 53.8%‐100%).
Discussion
We reviewed all FDA advisory committee meetings from January 2008 through December 2015 at which dichotomous votes were cast for a voting question relating to an FDA action to be taken on a single product or a class of products. Our study showed that 22% of meetings resulted in discordance between the advisory committee's recommendation and the FDA's eventual action, with the likelihood of disagreement associated with the type of proposed action and the degree of consensus expressed in the advisory committee's vote. In cases of discordance, the FDA was more likely to make decisions that were more restrictive than those of their advisory committees, and in most of them, the FDA was less likely to approve new products or new supplemental indications. These findings offer insights into how the FDA interprets the recommendations of its advisory committees and reflects them in their final decision making. In particular, instances of discordance between the two bodies offer an opportunity to explore the conditions in which the advisory committee system might fail to predict the FDA's ultimate decision.
Although advisory committee members and FDA agency members may be broadly aligned in their goals, issues like the agency's concerns about its reputation may result in different patterns of decision making.2 For example, the FDA has an incentive to avoid reversing earlier decisions insofar as those reversals might damage its credibility. This is suggested by the increased likelihood of discordance in meetings discussing proposed safety actions, in which the FDA tended to take a less restrictive stance in maintaining the product's availability for wider use even when the advisory committees tended to recommend restricting them. Furthermore, the FDA has the same incentive to reduce future regulatory reversals. In contrast to disagreement over decisions involving safety actions, disagreement over decisions involving novel products and supplemental indications usually resulted in more restrictive actions by the FDA than its committees’ decisions, with the FDA deciding either not to approve products at all or delaying their entrance to market. The latter cases often took advantage of multiple review cycles to request further information from sponsors, or else defined the parameters for appropriate use more carefully, in such a way as to mitigate the need for future regulatory intervention. In doing so, the FDA is able to take advantage of both a mechanism and a longer timeline not available to the advisory committee to reduce uncertainty about the product being evaluated. Overall, these tendencies are captured by the FDA's appearing to have a higher threshold than the advisory committees for enacting new approvals, supplements, or safety actions under conditions of uncertainty. Indeed, we found an increasing likelihood of discordance between the two bodies coupled with a decreasing degree of consensus among advisory committee members in a vote and the FDA being more or less restrictive based on the action type proposed.
These findings align with the framework developed by Moffitt for conceptualizing the work of expert advice to government, a framework that has mapped multiple contexts through which the FDA and other regulatory bodies engage their advisory committees.4 In what she calls “participatory bureaucracy,” the interdependence between the FDA and the participating publics both increases the gathering of relevant information from external sources and places constraints on the range of actions available to the FDA as it seeks to legitimize its actions and to distribute its responsibility for making decisions among multiple stakeholders. In “participatory oversight” and “public oversight” contexts, the committees may allow the FDA to gather information about the public's current or perceived future demand regarding an action and act accordingly. Our findings are consistent with each of these contexts, as we found overall high rates of agreement consistent with the mutually supportive function of participatory bureaucracy, with an occasional disagreement that may reflect the FDA's awareness of the oversight, both formal and informal, to which it is subject.
Three other hypothesized sources for discordance were the disparate influence of industry within the advisory committee as captured by the rates of conflicts of interest among advisory committee members; the influence of public demand as captured by measurements of public speakers and media interest; and the influence of unmet need as captured by product orphan status and the use of special regulatory programs. Ultimately, none of these were found to be significant predictors of disagreement between an advisory committee's recommendations and the FDA's actions. In the first case, we observed lower rates of conflicts of interest among committee members than previously documented.13, 15 This is likely in part a consequence of FDAAA, which set a cap on the number of advisory committee members permitted conflict‐of‐interest waivers as a proportion of a base percentage of the number of waivers issued in 2007. These lower rates persisted even after the cap was lifted after the FDA Safety and Innovation Act (FDASIA) in 2012,23 suggesting a recalibration of norms regarding conflicts of interest.
Meanwhile, the number of public speakers offering statements in committee meetings has been increasing. While representation of public speakers can be viewed as a barometer for public demand for a product, studies that find high rates of sponsorship by industry17, 18 have led to concerns about an imbalanced presentation of viewpoints at the public hearing portion unduly influencing the committee's deliberations. We were reassured to find that the public speakers’ statements and another metric of public engagement, media coverage, were not associated with discordance or with either the advisory committees’ recommendations or the FDA's actions. It is possible, though, that this study may not have fully captured the effect of recent trends toward an increasing number of public speakers or sponsorships, and thus further scrutiny is warranted.24
Finally, we hypothesized that advisory committees and the FDA may respond differently in cases in which the product fulfills an unmet need. For instance, the FDA may expect public oversight pressure from patient advocacy groups or Congress if products addressing unmet needs are not made available on the market, and thus it may be more likely to break from an advisory committee's recommendations to make the product available. However, we did not find any association between a product's orphan status or the use of a special regulatory program and discordance between an advisory committee's recommendations and the FDA's actions. Indeed, the FDA actually adopted a more restrictive stance than its advisory committees did in a majority of cases of disagreement over products with either of these two designations. These designations, however, capture a wide variety of products with varying types of need—for instance, few alternative therapies in the case of orphan diseases or serious or life‐threatening conditions in the case of fast track and accelerated approval—and we cannot exclude the presence of more targeted effects than could be captured with these proxies.
While our findings offer insights into the role of FDA advisory committees in influencing the agency's decision making and the boundaries of their influence, further research is needed to characterize the drivers of discordance and, more generally, to assess the value and integrity of the advisory committee system to the FDA and the stakeholders affected by the committees’ recommendations regarding new and existing products. Ultimately, the long‐term effects of this system can be measured in the availability and safety of medical products available on the US market.
Limitations
Our study has several limitations that deserve consideration. First, we could not take into account some important characteristics that may influence either advisory committee recommendations or FDA actions, including the quantity or quality of underlying evidence that might have supported either a committee's recommendation or the FDA's action. Second, we could not fully account for trends in the process of selecting products for evaluation by the advisory committee, including systematic changes in the quality of applications or systematic differences between the divisions' or centers' use of advisory committees. Third, we did not account for certain procedural factors that could influence voting patterns within the advisory committees, such as seating patterns, voting sequence, or chair leadership,25, 26, 27 nor did we account for conflicts of interest among public speakers, as these were not systematically disclosed or documented. Finally, we limited our analysis to the FDA's advisory committees at which dichotomous votes were cast for a voting question relating to FDA action to be taken on a single product or a class of products. While consistent with earlier research,13, 16 this approach to encoding the recommendation of the committee through a dichotomous vote captures only a portion of the information the committee provides to the FDA and may limit the generalizability of our study findings to other FDA advisory committees in which qualitative feedback plays an even greater role.
Conclusions
Among all voting FDA advisory committee meetings from January 2008 through December 2015, approximately one‐fifth showed disagreement between the advisory committees’ recommendations and the FDA's actions, with the disagreement associated with the type of action proposed (approval, supplemental indication, or safety action) and the degree of the advisory committee's consensus. Disagreement was more common in which the FDA was less likely than its advisory committees to enact new product approvals, supplemental indications, or safety actions, reflecting a more conservative approach to resolving ongoing regulatory uncertainty. These findings offer insights into factors that shape the FDA's interpretation of specific advisory committee recommendations carried out in its subsequent actions.
Supporting information
Supplement Table 1. Multivariable Association Between Medical Product, Regulatory, and Advisory Committee Meeting Characteristics and Favorable Advisory Committee Recommendation, 2008‐2015
Supplement Table 2. Multivariable Association Between Medical Product, Regulatory, and Advisory Committee Meeting Characteristics and Favorable FDA Action, 2008‐2015
Supplement Table 3. Medical Product, Regulatory, and Advisory Committee Meeting Characteristics 2008‐2015, Stratified by Center
Supplement Table 4. Multivariable Association Between Medical Product, Regulatory, and Advisory Committee Meeting Characteristics and Discordance Between Advisory Committee Recommendation and FDA Action, 2008‐2015, Stratified by Center
Funding/Support
None.
Conflict of Interest Disclosures: All authors completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. In the past 36 months, Dr. Ross received research support through Yale University from Johnson and Johnson to develop methods of clinical trial data sharing; from Medtronic, Inc., and the Food and Drug Administration (FDA) to develop methods for postmarket surveillance of medical devices (U01FD004585); from the Food and Drug Administration to establish the Yale–Mayo Clinic Center for Excellence in Regulatory Science and Innovation (CERSI) program (U01FD005938); from the Blue Cross Blue Shield Association to better understand medical technology evaluation; from the Centers of Medicare and Medicaid Services (CMS) to develop and maintain performance measures that are used for public reporting (HHSM‐500‐2013‐13018I); from the Agency for Healthcare Research and Quality (R01HS022882); from the National Heart, Lung and Blood Institute of the National Institutes of Health (NIH) (R01HS025164); and from the Laura and John Arnold Foundation to establish both the Good Pharma Scorecard at Bioethics International and the Collaboration for Research Integrity and Transparency (CRIT) at Yale. Dr. Schwartz received research support through Yale University from the National Library of Medicine of the National Institutes of Health (NIH) (G13LM012460). Ms. Zhang is receiving research support from the Laura and John Arnold Foundation and the Yale–Mayo Clinic CERSI program (U01FD005938).
References
- 1. Jasanoff S. The Fifth Branch: Science Advisers as Policymakers. Cambridge, MA: Harvard University Press; 1998. [Google Scholar]
- 2. Carpenter D. Reputation and Power: Organizational Image and Pharmaceutical Regulation at the FDA. Princeton, NJ: Princeton University Press; 2010. [Google Scholar]
- 3. Schwartz JL. Fifty years of expert‐advice—pharmaceutical regulation and the legacy of the drug efficacy study. N Engl J Med. 2016;375(21):2015‐2017. 10.1056/NEJMp1002530. [DOI] [PubMed] [Google Scholar]
- 4. Moffitt SL. Making Policy Public: Participatory Bureaucracy in American Democracy. Cambridge, UK: Cambridge University Press; 2014. [Google Scholar]
- 5. Ciociola AA, Karlstadt RG, Pambianco DJ, Woods KL, Ehrenpreis ED. The Food and Drug Administration advisory committees and panels: how they are applied to the drug regulatory process. Am J Gastroenterol. 2014;109(10):1508‐1512. 10.1038/ajg.2014.85. [DOI] [PubMed] [Google Scholar]
- 6. US Food and Drug Administration, Advisory Committees . https://www.fda.gov/AdvisoryCommittees/default.htm. Accessed October 5, 2018.
- 7. Lavertu S, Weimer DL. Federal advisory committees, policy expertise, and the approval of drugs and medical devices at the FDA. J Public Adm Res Theory. 2011;21(2):211‐237. 10.1093/jopart/muq046. [DOI] [Google Scholar]
- 8. Carpenter DP. The political economy of FDA drug review: processing, politics, and lessons for policy. Health Aff (Millwood). 2004;23(1):52‐63. 10.1377/hlthaff.23.1.52. [DOI] [PubMed] [Google Scholar]
- 9. Moffitt SL. Promoting agency reputation through public advice: advisory committee use in the FDA. J Polit. 2010;72(3):880‐893. 10.1017/S002238161000023X. [DOI] [Google Scholar]
- 10. Smith JF, Townsend SA, Singh N, Ma P. FDA advisory committee meeting outcomes. Nat Rev Drug Discovery. 2012;11(7):513‐514. 10.1038/nrd3747. [DOI] [PubMed] [Google Scholar]
- 11. Brass EP. Assessing the benefit‐risk for new drugs: are the FDA's Endocrinologic and Metabolic Drugs Advisory Committee and the Division of Metabolism and Endocrinology Products in sync? Diabetes Care. 2013;36(7):1823‐1826. 10.2337/dc13-0891. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Jackson J, Braccio N, Brevig H, Sengupta D. FDA advisory committee trends since FDASIA. Avalere Health. 2015:16 https://avalere-health-production.s3.amazonaws.com/uploads/pdfs/1448383851_112315_Avalere_AdCommFocusReport.pdf. Accessed June 6, 2017. [Google Scholar]
- 13. Lurie PG, Almeida CM, Stine N, Stine AR, Wolfe SM. Financial conflict of interest disclosure and voting patterns at Food and Drug Administration drug advisory committee meetings. JAMA. 2006;295(16):1921‐1928. 10.1159/000209492. [DOI] [PubMed] [Google Scholar]
- 14. Eastern Research Group . Measuring conflict of interest and expertise on FDA advisory committees. 2007. https://www.fda.gov/oc/advisory/ergcoireport.pdf. Accessed June 6, 2017.
- 15. Pham‐Kanter G. Revisiting financial conflicts of interest in FDA advisory committees. Milbank Q. 2014;92(3):446‐470. 10.1111/1468-0009.12073. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Tibau A, Ocana A, Anguera G, et al. Oncologic Drugs Advisory Committee recommendations and approval of cancer drugs by the US Food and Drug Administration. JAMA Oncol. 2016:1‐7. 10.1001/jamaoncol.2015.6479. [DOI] [PubMed] [Google Scholar]
- 17. Abola MV, Prasad V. Characteristics and conflicts of public speakers at meetings of the Oncologic Drugs Advisory Committee to the US Food and Drug Administration. JAMA Intern Med. 2016:1‐3. 10.1001/jamainternmed.2015.7805. [DOI] [PubMed] [Google Scholar]
- 18. McCoy MS, Pagán O, Donohoe G, Kanter GP, Litman RS. Conflicts of interest of public speakers at meetings of the Anesthetic and Analgesic Drug Products Advisory Committee. JAMA Intern Med. April 2018. 10.1001/jamainternmed.2018.1325. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. US Food and Drug Administration . Orphan drug designations and approvals. https://www.accessdata.fda.gov/scripts/opdlisting/oopd/index.cfm. Published 2018. Accessed October 31, 2018.
- 20. US Food and Drug Administration . Guidance for industry: expedited programs for serious conditions—drugs and biologics. 2014:1‐36. http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm358301.pdf. Accessed June 6, 2017.
- 21. Olson MK. Managing delegation in the FDA: reducing delay in new‐drug review. J Health Polit Policy Law. 2004;29(3):397‐430. 10.1215/03616878-29-3-397. [DOI] [PubMed] [Google Scholar]
- 22. Carpenter D, Fendrick AM. Accelerating approval times for new drugs in the US. Regul Aff J. 2004;15(6):411‐417. [Google Scholar]
- 23. Wood SF, Mador JK. Uncapping conflict of interest? Science. 2013;340(6137):1172‐1173. 10.1126/science.1231955. [DOI] [PubMed] [Google Scholar]
- 24. Schwartz JL. Real‐world evidence, public participation, and the FDA. Hastings Center Rep. 2017;47(6):7‐8. 10.1002/hast.779. [DOI] [PubMed] [Google Scholar]
- 25. Broniatowski DA, Magee CL. Does seating location impact voting behavior on Food and Drug Administration advisory committees? Am J Ther. 2013;20(5):502‐506. 10.1097/MJT.0b013e31821109d5. [DOI] [PubMed] [Google Scholar]
- 26. Urfalino P, Costa P. Secret‐public voting in FDA advisory committees. Secrecy and Publicity in Votes and Debates. 2015. https://hal.archives-ouvertes.fr/hal-01226341. Accessed July 11, 2018. [Google Scholar]
- 27. Gibbs JN. PMA advisory panels: the impact of FDA's change in policy on voting pattern. 2018:5 https://hpm.com/publications/pma-advisory-panels-the-impact-of-fdas-change-in-policy-on-voting-pattern/. Accessed June 6, 2017. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplement Table 1. Multivariable Association Between Medical Product, Regulatory, and Advisory Committee Meeting Characteristics and Favorable Advisory Committee Recommendation, 2008‐2015
Supplement Table 2. Multivariable Association Between Medical Product, Regulatory, and Advisory Committee Meeting Characteristics and Favorable FDA Action, 2008‐2015
Supplement Table 3. Medical Product, Regulatory, and Advisory Committee Meeting Characteristics 2008‐2015, Stratified by Center
Supplement Table 4. Multivariable Association Between Medical Product, Regulatory, and Advisory Committee Meeting Characteristics and Discordance Between Advisory Committee Recommendation and FDA Action, 2008‐2015, Stratified by Center