Optimizing therapeutic decision-making for off-label medicines use:a scoping review and consensus recommendations for improving practice and research

Abstract

Purpose:

Off-label medicines use is a common and sometimes necessary practice in many populations, with important clinical, ethical and financial consequences, including potential unintended harm or lack of effectiveness. No internationally recognized guidelines exist to aid decision-makers in applying research evidence to inform off-label medicines use. We aimed to critically evaluate current evidence informing decision-making for off-label use use and to develop consensus recommendations to improve future practice and research.

Methods:

We conducted a scoping review to summarize the literature on available off-label use guidance, including types, extent and scientific rigor of evidence incorporated. Findings informed the development of consensus recommendations by an international multidisciplinary Expert Panel using a modified Delphi process. Our target audience includes clinicians, patients and caregivers, researchers, regulators, sponsors, health technology assessment bodies, payers and policy makers.

Results:

We found 31 published guidance documents on therapeutic decision-making for off-label use. Of 20 guidances with general recommendations, only 35% detailed the types and quality of evidence needed and the processes for its evaluation to reach sound, ethical decisions about appropriate use. There was no globally recognized guidance. To optimize future therapeutic decision-making we recommend: 1) seeking rigorous scientific evidence; 2) utilizing diverse expertise in evidence evaluation and synthesis; 3) using rigorous processes to formulate recommendations for appropriate use; 4) linking off-label use with timely conduct of clinically meaningful research (including real-world evidence) to address knowledge gaps quickly; and 5) fostering partnerships between clinical decision-makers, researchers, regulators, policy makers, and sponsors to facilitate cohesive implementation and evaluation of these recommendations.

Conclusions:

We provide comprehensive consensus recommendations to optimize therapeutic decision-making for off-label medicines use and concurrently drive clinically relevant research. Successful implementation requires appropriate funding and infrastructure support to engage necessary stakeholders and foster relevant partnerships, representing significant challenges that policy makers must urgently address.

PLAIN LANGUAGE SUMMARY:

Off-label medicines use is for an indication, dose, route or age other than what has been approved by a medicines regulator. It is sometimes needed and appropriate but is associated with important clinical, safety, ethical, and financial consequences. We critically reviewed the published literature to understand the scientific rigor of available guidance to inform therapeutic decision-making for such use. There is currently no internationally recognized guidance for optimizing therapeutic decision-making for the off-label use of medicines. Of available guidances we identified, only one-third provide explicit frameworks to support rigorous therapeutic decision-making. To address this gap, our international multidisciplinary expert panel developed comprehensive consensus recommendations to optimize therapeutic decision-making for off-label use and to concurrently drive clinically relevant research. We recommend: 1) seeking rigorous scientific evidence; 2) using diverse expertise in evidence evaluation and synthesis; 3) using rigorous processes to formulate recommendations for appropriate use; 4) linking off-label use with timely conduct of clinically meaningful research (including real-world evidence) to address knowledge gaps quickly; and 5) fostering partnerships between clinical decision-makers, researchers, regulators, policy makers, and sponsors to facilitate cohesive implementation and evaluation of these recommendations. Successful implementation requires appropriate funding and infrastructure support, with significant challenges that policy makers must urgently address

INTRODUCTION

Off-label use of medicines is use for an indication, dose, route, or age other than what has been approved by a regulator and remains a critical issue in modern medicine.(1, 2) Rates of off-label prescribing vary considerably by population and setting, with up to 69–100% of pediatric and 72–95% of adult patients receiving off-label prescriptions.(1, 3, 4) Global variability in regulatory approvals, prescribing practices and methods for evaluating off-label medicines use is a significant barrier to understanding its true scope and impact. A medicine’s labelling status in one country may also not always reflect the latest available evidence internationally(5, 6). Off-label use is sometimes needed and appropriate, but adequate research evidence to support common off-label uses is often lacking, with 73% of such prescriptions in one large US study found to have little or no scientific support(7). There are important clinical, ethical, and financial consequences of off-label medicines use (8, 9), including from potential ineffective or harmful effects.(3, 10, 11) The COVID-19 pandemic highlighted the negative consequences of inappropriate non-evidence-based off-label use when certain treatments that were widely used off-label were later shown to be ineffective or harmful.(12, 13)

Despite efforts by national,(2, 14, 15) regional,(16) and global organizations like the World Health Organization,(17) there is currently no internationally accepted guidance for off-label medicine use. This contrasts with the extensive international guidances for clinical trials and medicine labeling. Specific guidance to address uncertainties in off-label prescribing would better support: 1) clinicians making prescribing decisions; 2) patients seeking more effective treatments with favorable risk:benefit characteristics; 3) guideline developers focusing on evidence-based prescribing recommendations; and 4) policy makers designing systems for safe and effective use of medicines.

To address this need, our international, multidisciplinary expert panel mapped the available literature to examine the range, nature, and characteristics of guidance for off-label prescribing, highlighting pediatrics as a use case. Our objectives were to: 1) describe the types, extent, and scientific rigor of evidence that informs therapeutic decision-making for off-label medicines use; 2) critically review available national, regional, and international guidance informing off-label prescribing practices; and 3) provide recommendations for optimal generation, evaluation, interpretation, and clinical application of research evidence regarding off-label use of medicines.

METHODS

This work involved two components: a scoping review and a consensus development process. We undertook a scoping review using the Arksey and O’Malley methodological framework.(18) The findings from the scoping review were then used to develop consensus recommendations for evaluation and clinical application of research evidence, which was accomplished using a modified Delphi process(19).

Scoping Review

Identifying Relevant Papers

We identified relevant papers from multiple sources, as recommended by Arksey and O’Malley. Sources included electronic databases, reference lists, and existing networks and relevant organizations. The electronic database search strategy, developed in collaboration with a research librarian (EG), surveyed PubMed, Embase and the Cochrane library for literature published from 2003 to May 2018 (Appendix A). After de-duplication, citations were uploaded into Rayyan(20) for review. We added key publications through 2020 from peer-reviewed and grey literature sources by: 1) searching reference lists of full-text papers meeting initial selection criteria; and 2) accessing information from professional organizations and networks.

Study Selection

Identified papers were screened by five teams of two independent reviewers in two phases: 1) title/abstract and 2) full text review (Figure 1). Disagreements between reviewers were resolved through discussion. Data were abstracted from papers meeting the eligibility criteria (Appendix B). Papers were categorized as A) general guidances regarding off-label medicines use or B) therapeutic area-specific guidances for off-label use.

Figure 1:

Flow diagram of study selection for scoping review

Data Collection

Elements abstracted from eligible papers included author names, title, journal, publication year, PubMed identifier, authors’ countries of origin, article type, organizations represented, target audience, patient population and topics covered, recommendation or guideline development process, evidence used, stakeholder involvement, guidelines for medical decision-making, and audience/expert type. REDCap (Research Electronic Data Capture) tools(21) hosted at the University of Colorado Denver facilitated data abstraction. The lead investigator (MG) reviewed all included papers and abstracted additional details to ensure capture of relevant information and to synthesize key concepts and themes, which were then reviewed by other team members for comments and refinement. Further discussion amongst the research team led to refinement of a conceptual framework.

Consensus Development: Modified Delphi Process

Recommendations were developed for a target audience of clinicians, patients and caregivers, researchers, regulators, sponsors, health technology assessment bodies, payers and policy makers. We used a modified Delphi process,(19) with our multidisciplinary team of 11 pharmacoepidemiologists serving as an expert panel. The panel had diverse expertise (clinical and methodological, with focus in pediatrics) (Appendix C), work experience (academia, clinical practice, pharmaceutical industry, medicines policy) and geographic perspectives (Australia, Canada, Europe, USA). Panelists included pediatricians and clinicians with specialized expertise in clinical pharmacology and therapeutics (MG, BC), neonatology (JS), critical care (AC), mental health (SDR), and rheumatology (DH, MG), therapeutic areas with high rates of off-label use. Six panel members (MG, SdR, JS, DH, BC, AK) drafted recommendations based on concepts and themes synthesized from guidances identified in the scoping review. Following review with written feedback, the full expert panel discussed draft recommendations at a virtual meeting (January 2022). After incorporating feedback, revised recommendations were reviewed by all panel members, who voted on each recommendation (3-point Likert, Agree-Neutral-Disagree) and provided relevant additional comments. The panel again met virtually to discuss the recommendations and address areas of disagreement (March-May 2022). Additional rounds of voting by the full panel were conducted until ≥80% agreement was reached on each recommendation.

RESULTS

Summary of the Scoping Review

Identification of Relevant Papers:

A total of 5,109 publication records were screened, 31 of which were included in the analysis (Figure 1). Of these, 21 (68% of total) publications refer to 20 unique guidances providing general considerations, frameworks, and recommendations to guide decision-making for off-label medicines use.(8, 14–17, 22–37) Ten papers (32%) provided off-label use recommendations for specific medicines or therapeutic categories but also described the types of evidence, expertise, and processes used in developing recommendations and thus were included.(38–47)

Thematic Concepts of General Guidances for Off-Label Medicine Use:

The majority of general guidances were described as guidelines, guiding principles, policies, or standards (Table 1). Target audiences were mixed, but most guidances (85%) focused on clinicians, prescribers, and health care professionals. Many (65%) also targeted other relevant stakeholders, including institutional committees, local or national policy makers, patients, caregivers, researchers, payers, or industry. Most publications originated from Europe, the UK, North America, or Australia; only two had a global scope.(17, 36) Notably, none were primarily developed by major medicines regulators, although these agencies occasionally collaborated on guidances developed by others.(14, 24, 34)

Table 1:

Characteristics of general guidances on off-label medicine use

First author, year, location	Type of guidance	Organization(s)	Medicine focus	Included population	Target audience
Policy or Position Statement
Ansani 2006, US	Policy Statement	Major Academic Medical Centre	All medicines	All ages	Institutional P&T Committee, hospital prescribers
Fox & Sammons 2013, UK	Policy Statement (update of previous statement originally produced in 2000)	Royal College of Paediatrics and Child Health & Neonatal & Paediatric Pharmacists Group, Joint Standing Committee on Medicines	All medicines	All children (ages not specified)	Health care professionals, health service managers, parents & caregivers; all who prescribe, dispense, administer or have responsibility for medicines for children
Neville 2014, US	Policy Statement	American Academy of Pediatrics, in liaison with other national professional societies/groups; Regulatory agency for medicines; and other government agencies	All medicines	All paediatric (0–18 years)	Prescribers, researchers, journals, institutions, payers
Schrier 2020, Europe	Policy Statement	European Academy of Paediatrics, European Society of Developmental Perinatal & Paediatric Pharmacology	All medicines	All paediatric	Health care professionals
Gazarian 2007, International	Position statement	World Health Organisation	All medicines	All children	Primarily decision-makers involved in developing WHO’s first Model Essential Medicines List for Children (EMLc); also intended to inform therapeutic decision-making in the paediatric population more generally at a global level
Sharma 2016, UK	Position Statement	British Association for Psychopharmacology	Psychotropics1	All paediatric (0–18 years)	Health care professionals, researchers, regulators, industry
Guidance, Guidelines, and Guiding Principles
Royal College of Psychiatrists Psychopharmacology Committee 2017, UK	Guidance (update of original guidance from 2007)	Royal College of Psychiatrists, British Association of Psychopharmacology	Psychotropics1	All ages, with limited consideration of “vulnerable” groups, e.g., children & adolescents	Not specified, but implied audience is prescribers of psychotropics
Centers for Medicare & Medicaid Services 2008, US	Guidance	US government agency	All medicines	All ages	Not specified, but applies to various groups: payers, prescribers, researchers, institutions, public
Council of Australian Therapeutic Advisory Groups 2013 Gazarian & Morris 2014 Australia	Guiding Principles	Council of Australian Therapeutic Advisory Groups	All medicines	All ages, including specific recommendations for paediatric age group	Health care professionals, consumers, DTCs
Larcher 2018, UK	Guiding Principles	Tertiary children’s hospitals	All medicines	All paediatric (0–18 years)	Not specified in document, but intended use of guidance by other children’s hospitals seems implied
Brierley & Larcher 2009, UK	Other (Ethical framework)	Other (individual health care professionals)	All medicines	All paediatric	Clinicians in paediatric practice
Gazarian 2006, Australia	Guidelines	Independent group of Clinical Pharmacologists, Directors of Pharmacy & other clinicians representing 50 hospital Drug & Therapeutic Commitees (DTCs) (New South Wales Therapeutic Advisory Group, NSW TAG)	All medicines	Adult (18 years and older) All pediatric (0–18 years)	Prescribers Policymakers, eg DTCs, Therapeutic guideline or Medicines information developers Payers
Other Statements or Recommendations
Gillick 2009, US	Other (Opinion)	Other (Individual health care professionals)	High-cost & high-risk medicines	Adult (18 years and older) All paediatric (0–18 years)	Focus on funders of medicines use, including institutional DTCs; government agencies and health insurance companies
National Institutes of Health (NIH), 2010, US	Other (Consensus statement)	NIH	Inhaled nitric oxide2	Premature infants ≤34 weeks	Not specified
Lenney 2013, UK	Other (Children’s medicines formulary)	Professional societies/groups; Government agencies	All medicines	All paediatric (0–18 years)	Not specified in document, but implied use by all who prescribe, dispense or administer medicines to infants & children
Parrish 2015, Canada	Other (Opinion, general recommendations)	Other (Individual health care professionals)	All medicines	All paediatric (0–18 years)	Not specified in document
British National Formulary for Children 2017, UK	Other (General and specific recommendations)	Joint publication of the British Medical Association, Royal Pharmaceutical Society, Royal College of Paediatrics & Child Health & Neonatal & Paediatric Pharmacists Group	All medicines	All paediatric	Prescribers, pharmacists, other health care professionals
Czaja 2017, US	Other (Opinion, general recommendations)	Comparative Effectiveness Research Through Collaborative Electronic Reporting Consortium	All medicines	All paediatric (0–18 years)	Paediatric clinicians, researchers & policy makers
Dooms 2017, Europe	Other (“Declaration on Good Off-Label Use Practice”)	Coalition of ad hoc group of European organisations involved in patient care, medical research & innovation	All medicines	Whole population, with consideration of issues in special populations, including paediatric	Prescribers, pharmacists, patients, public at large
Kelly 2017, International	Other (General recommendations)	International paediatric pharmacy advocacy group, including academic specialist paediatricians, pharmacists, pharmaco-epidemiologists	All medicines	All paediatric (0–18 years)	Health care professionals involved in selecting, prescribing, dispensing, preparing, administering, monitoring or advising on medicines for children

DTC, Drug & Therapeutic Committee; P&T, Pharmacy and Therapeutics (P&T) Committee

¹Overall recommendations are general in nature and not specific to single medication or group of medication

²Process of developing recommendations for use of a single medicine, with relevance or applicability to developing recommendations for use of other medicines or group of medicines

Guidance Development Processes and Expertise Involved

Sponsors of recommendations included national or regional professional and health care organizations as well as groups concerned with Health Technology Assessment (HTA) or appropriate use of medicines. None received pharmaceutical industry funding. Half of general guidances explicitly described their internal guidance development processes (Table 2).(8, 14, 15, 17, 22, 28, 29, 32, 34, 36, 37) One-third of guidances described systematic consensus development processes (e.g., Delphi, US NIH consensus development process) or invited input from external organizations.(8, 14, 15, 32, 34, 36, 37) Multidisciplinary committees, task forces, or clinical teams were the usual developers. The types of expertise, experience, and perspectives of those involved variably included clinicians, methodologists, patients, caregivers, and payers. All development groups included clinicians from medical and pharmacy backgrounds. Of seven guidances addressing off-label use in both adults and children, only three reported involvement of pediatric clinical experts in the development process.(8, 14, 16)

Table 2:

Processes and topics of general guidances on off-label medicine use

First author, year, location	Guidance development process explicitly described	Decision-making process for appropriate off-label use explicitly addressed within guidance recommendations	Other Topics covered
Ansani 2006, US	YES Multidisciplinary Task Force (physicians, pharmacists, nurses, risk management team, ethicists, attorneys, IRB) met over 9 months to develop new policy to address regulation of innovative off-label use of medicines	YES Standardized formulary Review, with 4 categories of decision, based on grading of evidence level. Uses systematic process advocated by US AHRQ to evaluate quality of supporting evidence Additional peer review by a senior expert group to adjudicate any decisions in conflict	Definition, prevalence, reasons for off-label and innovative use Discusses consequences, including lack of efficacy and concerns regarding safety, cost, ethical issues Need for detailed therapeutic guideline for proposed use Need to prospectively evaluate outcomes (safety, efficacy, cost) to minimize risks from continued use Ethical obligations for informed consent
Fox and Sammons 2013, UK	NO Policy updated by Joint Standing Committee on Medicines of two national professional bodies representing paediatricians and paediatric pharmacists No further description of range of expertise involved or process for development of recommendations	NO General statements about need for “sound information,” reference to “respectable, responsible body of professional opinion” to justify individual prescriber decisions Recommends BNFc as key source of "sound information" and guidance for health professionals and parents/caregivers	General reference to definition, reasons for unlicensed prescribing Recommends use of licensed medicines (where available) before any off-label or unlicensed uses Informed consent issues
Neville 2014, US	NO	NO General recommendation for individual physicians basing decision-making on “sound scientific evidence” and “expert medical judgement”	Definition of and reasons for off-label use Role of US FDA Ethical implications and informed consent Legal implications of off-label use Legislation to increase drug testing in children
Schrier 2020, Europe	NO Authors have expertise in paediatrics, paediatric oncology, neonatal intensive care, clinical medicine, health research, paediatric pharmacology, pharmacy and pharmaceutical science	YES Recommends 7 considerations when prescribing off-label medicines to paediatric patients: 1. All other options, including the use of medicines approved by regulatory authorities, are unavailable, not tolerated, less optimal, too expensive, not reimbursable or containing potentially harmful excipients 2. The prescriber is competent to prescribe off-label medicines in children 3. Off-label prescription of the medicine is appropriate to meet the needs of the individual patient within the available resources 4. Off-label prescription should be rational and clinically appropriate 5. The patient and parents/caregivers should be informed and involved 6. The patient should be monitored for efficacy and adverse events 7. The prescriber should consider if off-label prescribing should be part of a clinical trial	Cross-references to paediatric specific guidances and a range of general recommendations, regulations from European countries with good practice or professional guidelines for off-label use, including reimbursement decisions
Gazarian 2007, International	YES Guidance based on systematic evaluation of key concepts in existing guidance available from international medicines selection bodies and guideline developers Input from WHO Department of Medicines Policy and Standards, individual members of EMLc committee	YES Detailed description of types of research evidence to be sought, how evidence should be critically evaluated to determine clinical and comparative effectiveness and safety, and cost-effectiveness Categorizes off-label use into 3 groups based on assessments of the quality of available evidence, balance of benefits and risks, seriousness of health problem or magnitude of disease burden, and availability of alternative treatments	Off-label definition, prevalence, reasons for use and consequences Identifies key needs, including: More explicit definition of the people and processes for evaluation of appropriateness of proposed off-label uses Competent drug regulatory bodies to define approved vs. unapproved uses at a global level Authoritative medicines information resources to define well accepted off-label uses of older medicines Appropriate grading system and presentation of recommendations Linkage of identified evidence gaps in areas of clinical need to a global research agenda Broader consultation process to develop useful global policy
Sharma 2016, UK	NO	YES Proposes 6 general principles to guide decision-making: 1. Be familiar with the evidence base for the psychotropic agent 2. Incorporate overall evidence base (not just labeling status) and needs of individual child 3. Obtain a second opinion when evidence base is lacking or suggests potentially unfavourable benefit/risk profile 4. Inform patient and parent/caregivers about potential benefits and risks, document this discussion 5. Provide informational leaflets 6. “Start low and go slow” and actively monitor response	Definition of off-label use Reasons for off-label prescribing Safety and efficacy implications Patient information Informed consent Medico-legal considerations
Royal College of Psychiatrists Psychopharmacology Committee 2017, UK	NO Developed by members of the Royal College of Psychiatrists Psychopharmacology Committee, with representation from the British Association of Psychopharmacology (all psychiatrists)	YES Makes 10 recommendations: 1. Exclusion of available licenced alternatives 2. Satisfaction with evidence base 3. Obtaining advice from other prescribers ± specialist pharmacists 4. Consideration of balance of risks and benefits (especially in vulnerable populations) 5. Full informed consent 6. Explanation of unlicensed status of prescribed medicine 7. Documentation of informed consent 8. Close monitoring of therapy 9. Appropriate communication with other health care professionals involved in care 10. Withdrawal of therapy if lack of benefit or emergent risks outweigh benefits	Definition of licensed and unlicensed prescribing Types of unlicensed prescribing and possible motivations Potential benefits and risks Other key guidance on unlicensed prescribing
Centers for Medicare and Medicaid Services (CMS) 2008, US	YES Process for national coverage determinations: 1. CMS evaluation of current “state of the art” for proposed therapy (e.g., safe, effective, reasonable, and necessary for beneficiaries) 2. Independent guidance and expert advice from the MEDCAC, which judges quality and strength of evidence and the potential effects on the health of beneficiaries, after consideration of medical literature, technology assessments (e.g. by AHRQ), relevant regulatory documents (e.g. from FDA), public testimony, other relevant information Expertise of MEDCAC includes clinical and administrative medicine, biological and physical sciences, public health, patient advocacy, health informatics, health economics, medical ethics	YES Detailed guidance provided to MEDCAC members on how to evaluate scientific evidence, with particular emphasis on quality of evidence, relevance of findings to beneficiaries, magnitude and direction of risks and benefits Background documents and references (e.g. JAMA Users’ Guides to the Medical Literature) are provided for more detailed guidance on methods for interpreting clinical evidence	Considerations for inadequately studied, promising technologies: Directly supporting more research to generate the needed evidence Approving coverage limited to use within a research protocol (e.g. clinical trial) Providing flexibility to cover promising treatments or conditions that are too rare to support definitive study but have high potential for benefiting beneficiaries and evidence of overall safety
Gazarian 2006, Australia	YES Developed by a multi-disciplinary working party, including expertise in hospital medicine, pharmacy, paediatrics, general medicine, oncology, clinical pharmacology and therapeutics, clinical epidemiology, evidence-based medicine, health ethics, law Consensus development process involving wide organizational consultation, external consultation with government health department and advisory service for medications in pregnancy	YES Decision algorithm guiding clinicians in critical evaluation of research evidence on efficacy and safety of proposed off-label use, assessment of benefit-risk ratio for different clinical contexts Algorithm identifies 3 broad categories of appropriate off-label use:  1. Use justified by high quality evidence  2. Use in context of formal research proposal  3. Exceptional use justified by individual clinical circumstances (criteria pre-specified in algorithm) Highlights considerations for special populations (e.g. paediatrics) Cross-references to published guidelines for critical appraisal of therapeutic studies, for grading strength of evidence and judging applicability of research evidence to individual patient circumstances	Off-label definition and prevalence Reasons for off-label prescribing Consequences of off-label prescribing (clinical, safety, ethical) Types of evidence needed to evaluate full spectrum of safety (inclusion of observational studies) Appropriate patient consent (e.g. usual vs. written consent) Legal and ethical dimensions Benefits of a systematic approach to decision-making, including helping to “stimulate more clinically relevant medicines research”
Council of Australian Therapeutic Advisory Groups 2013 & Gazarian and Morris 2014; Australia	YES Multi-stage, systematic process of development involving a specialized project team with clinical pharmacology and pharmacy expertise, expert advisory group with range of expertise (therapeutics, quality use of medicines, evidence-based medicine, adult and paediatric clinical medicine and pharmacy, nursing, consumer issues Medico-legal advice and input from national medical indemnity organisations External consultation with wide range of key national organisations, including national medicines regulator Systematic consensus development process to develop final recommendations	YES Explicit process for decision-making about appropriateness of use and allocation to one of 5 categories of decision, based on grading of evidence and risk:benefit analysis for proposed use:  1. Routine use  2. Exceptional/individual use  3. Conditional use, with evidence development  4. Research or investigational use  5. Not recommended Systematic process to evaluate quality of supporting evidence, with reference to published guides on critical appraisal of therapeutic studies and grading of strength of evidence Detailed specification of types of research evidence to be sought for evaluating effectiveness and safety (including paediatric specific considerations) and consideration of comparative effectiveness and cost effectiveness Explicit specification of appropriate mix of expertise in decision-making body Includes specific recommendations for paediatric expertise to inform decisions for any paediatric uses	Discusses off-label definition, prevalence, reasons for use and consequences (efficacy, safety, financial, ethical, legal) Additional core guiding principles for off-label use include: Consider off-label use only if approved alternatives "are unavailable, exhausted, not tolerated or unsuitable” Shared decision-making and informed consent with patient/caregiver Consultation of DTC except for routine off-label use Ensure appropriate information is available Monitor clinical outcomes Consider liability and accountability
Larcher 2018, UK	YES Process for developing core guiding principles is not described but cross-references an earlier paper (Brierly and Larcher 2009, see below)	YES Focuses on clinical ethics framework, with 8 core principles (Brierley and Larcher 2009, see below) for decision-making by a multi-disciplinary Clinical Ethics Committee (CEC) CEC members are from clinical, spiritual, social care, legal and lay backgrounds, with input from academic philosophers	Definition of innovative therapy Reports on results of the use of this framework over a 5-year period Summary of FDA, EU and UK laws, regulations and responsible entities (e.g. NICE) for accessing innovative or compassionate use medicinal products
Brierley and Larcher 2009, UK	NO	YES Primary focus on comprehensive consideration of various ethical dimensions (next column) General statement regarding “reasonable scientific basis” and “realistic expectation that the child is likely to benefit”	Definition of innovative therapy vs research Ethical framework for evaluating proposed innovative therapy, with 8 key considerations: 1. Determination of clinical need 2. Realistic and reasonable scientific basis for proposed use 3. Realistic expectation of benefit, eg some evidence of benefit from other human population, laboratory or animal research 4. Consensus of team with relevant expertise to determine treatment benefit vs risks/burdens and best interests of individual children 5. Comprehensive process for well informed consent by family and child (if competent), with clear statement about innovative nature of treatment and provision of full information (including written) tailored to individual needs and circumstances 6. Ensuring lack of coercion, full consideration of all alternatives, freedom to withdraw at any time 7. Resource implications justified against likely success 8. Duty to report outcomes, including adverse outcomes
Gillick 2009, US	NO	NO Discussion of existing strategies for controlling use and costs of off-label medicines, cross-referencing to US policy processes, e.g. CMS coverage with evidence development (CMS 2008, see above) Overall recommendation is to limit rigorous review processes only to high-risk and high-cost off-label medicines	Off-label definition and prevalence Reasons for off-label uses Benefits and burdens of off-label prescribing High financial costs of off-label prescribing
NIH 2010, US	YES Uses established NIH process for developing consensus and state-of-the-science statements	YES Statement prepared by independent panel of health professionals and public representatives on the basis of: 1. Results of systematic literature review prepared under AHRQ contract, using recognized tools and processes for critical review of published evidence 2. Presentations by relevant investigators and related discussion by attendees at a 2-day public conference 3. Closed deliberations (not described in detail) by the convened panel after the public conference Panel & conference speaker areas of expertise includes pediatric medical specialists (including in neonatal & perinatal medicine); nurses; consumers; research scientists & ethicists
Lenney 2013, UK	YES As for BNFc, 2017 (see below)	NO	Historical development of BNFc Process for content development informed by specialized clinical expertise (general and sub-specialist paediatricians, paediatric pharmacists) Sources of information to inform decisions
Parrish 2015, Canada	NO	NO	General critique of AAP policy statement [Neville 2014] Off-label definition Drug-related morbidity and mortality in children Regulation: limitations of current drug labeling system for paediatric population Decision-making principles: innovative practice vs research, the role of PBRNs in generating knowledge Future directions: recommendations for creating a child-specific medicines use system, including pragmatic clinical trials and better information on paediatric outcomes
British National Formulary for Children (BNFc) 2017, UK	YES Multi-stage process bringing together medicines information from multiple sources, including product label, published literature, consensus guidelines, reference sources, peer review, statutory information (including regulations, compendia, safety warnings and professional statements), comments from users, industry and market research Various groups with a range of expertise involved in process and governance: 1. Editorial team (all pharmacists) 2. Paediatric Formulary Committee (expertise in paediatric pharmacy, medicine, nursing, with representation from paediatric caregivers, national drug regulator, health department) 3. Dental Advisory Group 4. Nurse Prescribers’ Advisory Group 5. Expert clinical advisors (including doctors, pharmacists, nurses, and dentists)	YES Since January 2016: systematic process of evidence evaluation based on levels of evidence (AGREE II) and grades of recommendation (SIGN) Limited set of graded recommendations with plans to revalidate content on rolling basis every 3–4 years (Prior to 2016: Unclear process, with focus on consideration of clinical need and advice from paediatric clinicians on evidence and experience of safety and efficacy)	Indication and dose Unlicensed uses Minimising harm (e.g. cautions, contraindications, drug interactions) Use in special paediatric populations (e.g. renal disease, nursing infants) Administration and monitoring (e.g. pharmacogenetic screening, drug level monitoring) Counseling children and caregivers Medicinal forms (e.g. formulation, dose, concentration)
Czaja 2017, US	NO	NO	Distinction between off-label vs off-evidence use Multi-dimensional framework proposed, incorporating risk:benefit profile and level of uncertainty about off-label use to inform practice, research and policy decisions Future directions for evidence generation with real-world evaluations of benefits and risks of off-label uses
Dooms 2017, Europe	NO Authors of declaration have expertise in pharmacy, psychiatry, pharmacology and paediatrics	YES Off-label use can occur only if 5 criteria are met: 1. Medical therapeutic need determined by a qualified health care professional 2. Absence of authorized treatment and licensed alternatives tolerated by the patient or repeated treatment failure 3. Documented review and critical appraisal of available scientific evidence favours off-label use to respond to the unmet medical need of individual patient 4. Patients (or their legal representative) must be given sufficient information about the medicines that are prescribed to allow them to make an informed decision 5. Presence of established reporting routes for outcomes and adverse events linked to off-label use	Definition of off-label use Reasons for off-label prescribing Patient information and informed consent Economic factors that may impact off-label decision making
Kelly 2017, International	YES Delphi process to develop set of standard items for inclusion in paediatric drug monographs, after review of major international paediatric formularies available (including WHO)	NO Off-label indications and off-label dosages included in final set of 116 items, but no information on how such information to be developed	Recommendations for future work to address: 1. Paediatric drug safety and effectiveness hierarchy to clearly differentiate between off-label and off-evidence prescribing 2. Patient and parent information

AHRQ, Agency for Healthcare Research and Quality; BNFc, British National Formulary for Children; CMS, Centers for Medicare and Medicaid Services; DTC, Drug and Therapeutic Committee; EMLc, Essential Medicines List for Children; FDA, Food and Drug Administration; JAMA, Journal of the American Medical Association; MEDCAC, Medicare Evidence Development and Coverage Advisory Committee; NICE, National Institute for Health and Care Excellence; NIH, National Institutes of Health PBRN, Practice-Based Research Network; WHO, World Health Organisation

Only some guidance development processes noted participants’ expertise in evidence evaluation (30%), e.g., clinical epidemiology, evidence-based medicine, critical appraisal; and in specialized therapeutics (15%), e.g., clinical pharmacology, quality use of medicines; and health economics (40%). Other relevant areas of contributing expertise included medical ethics (60%), medical-legal issues (40%), and patient/consumers (50%) (Table 2).

Recommendations for Decision-making about Off-Label Uses

All guidances made recommendations on the need for sound decisions about the appropriateness of off-label prescribing, with 65% providing specific recommendations or key principles to guide this crucial step (Table 2). However, only one-third (35%) detailed frameworks for optimal management of proposed off-label uses, including types of evidence needed for evaluating benefits and risks, types of expertise (e.g., methodological expertise in evidence evaluation, specialized pediatric clinical expertise), and processes for evaluating evidence and reaching clinically sound judgments. Other recommendations included consideration of comparative or cost-effectiveness analyses (25%)(8, 14, 15, 17, 22, 31) and prospective evaluation of safety, effectiveness, and cost.(8, 14–17, 22, 24, 25) One-third (35%) discussed processes for improving therapeutic decision-making by linking identified evidence gaps to formal research initiatives to generate real-world evidence (RWE).(8, 14, 15, 17, 25, 28, 33)

Ethical Considerations

Guidance recommendations diverged widely in addressing the ethics of off-label medicines use (Table 2). Ethical recommendations included shared decision-making processes, such as providing appropriate information for patients and caregivers and obtaining informed consent. Five guidances provided detailed ethical frameworks for evaluating the appropriateness of off-label treatment, including comprehensive informed consent processes.(8, 14, 22, 29, 30)

Thematic concepts of Off-Label Use Guidances Specific to a Therapeutic Area:

Of ten papers addressing specific medicines or therapeutic categories (Table 3), most focused on treatment efficacy or effectiveness. With one exception,(42) safety considerations or risk-benefit evaluations were incompletely addressed. Comparative effectiveness or cost-effectiveness were generally not considered except occasionally as important evidence gaps.(40, 42)

Table 3:

Characteristics of medicine-specific guidances on off-label medicine use

First author, year, location	Type of guidance	Organization(s)	Medicine focus	Included population	Target audience	Other topics covered
Smyth 2010, Europe	Evidence synthesis	Professional societies/groups (European Respiratory Society)	Group of medications (Respiratory)	Adults and children (ages not specified)	Patients, prescribers, pharmaceutical industry, medicines regulator, academic researchers	Off-label definition Unlicensed medications Current evidence base Future directions (identification of research priorities to address evidence gaps)
Maglione 2011, US	Comparative Effectiveness Review	Government agencies guiding decision-making (AHRQ)	Group of medications (Atypical antipsychotics)	Adults and some paediatric (any inclusion of age <18 years)	Prescribers, institutional DTC	Off-label definition Current evidence base
Aoki 2003, Canada	Position Paper	Professional societies/groups (Canadian Infectious Disease Society)	Group of medications (Antivirals)	Adults and some paediatric (any inclusion of age <18 years)	Prescribers	Unlicensed medications Appropriate use guidelines Current evidence base
Anderson 2007, Canada	Guidelines	Other (National Advisory Committee and Canadian Blood Services convened panel of clinical and practice guideline experts)	Single medication (Intravenous Immunoglobulin)	Adults and some paediatric (any inclusion of age <18 years)	Prescribers, institutional DTC	Unlicensed medications Appropriate use guidelines Decision-making principles Current evidence base
Mallarkey 2008, Australia	Guidelines	Non-government groups, paediatric-specific group (State-wide health service Therapeutic Advisory Group, with input from a Paediatric Therapeutics Program)	Single medication (Eptacog Alpha)	Adults and all paediatric (0–18 years)	Prescribers, institutional DTCs	Appropriate use guidelines Current evidence base (including collation of comprehensive safety data from observational studies and unpublished sources) Future directions
Indinnimeo 2009, Europe	Guidelines	Non-government groups, professional societies/groups, paediatric-specific group (Italian Paediatric Society Task Force)	Group of medications (Asthma medicines)	All paediatric (0–18 years)	Patients, Prescribers	Off-label definition Unlicensed medications Appropriate use guidelines Current evidence base
Keszler 2012, US	Guidelines	Other (author's opinions based on literature review with no mention of search strategy)	Single medication (Inhaled Nitric Oxide)	Neonatal/NICU (less than 30 days)	Prescribers	Off-label definition Appropriate use guidelines Current evidence base
Boguniewicz 2017, US	Consensus statement	Other (steering committee of experts in dermatology, allergy, and patient advocacy group representative)	Group of medications (Atopic dermatitis)	Adults and some paediatric (any inclusion of age <18 years)	Prescribers	Unlicensed medications Appropriate use guidelines Decision-making principles Current evidence base
Palleschi 2018, Europe	Consensus statement	Professional societies/groups (Italian Society of Urodynamic)	Single medication (Botulinum toxin)	Adults, children, and adolescents (ages not specified)	Prescribers	Appropriate use guidelines Decision-making principles Current evidence base
Mihatsch 2012, Europe	Evidence synthesis	Other (academic and clinical practice settings)	Group of medications (Probiotics)	Other (preterm infants)	Prescribers	Unlicensed medications Current evidence base

AHRQ, Agency for Healthcare Quality and Research; DTC, Drug & Therapeutic Committee; NICU, Neonatal Intensive Care Unit

Only four papers used systematic consensus development processes.(41, 45–47) Multidisciplinary panels always included clinicians, usually specializing in the therapeutic area under consideration. Fewer reported participants with other relevant expertise, such as evidence-based medicine, practice guideline development,(41) or patient advocacy.(45) Just two guidances concluded their evidence review processes with identification of important research priorities to address evidence gaps.(38, 42)

Summary of the Consensus Development Process (Modified Delphi):Recommendations to Optimize Therapeutic Decision-Making for Off Label Medicines Use

Based on key thematic concepts identified by the scoping review and insights from our expert panel, we developed draft recommendations addressing five interconnected themes. The first round of voting achieved ≥80% consensus on one of the five recommendations. After several rounds of discussion, text revisions, and voting by the expert panel, ≥80% consensus was reached for all five recommendations. The final consensus framework, with key recommendations, sub-themes, and important relationships between them, are described in Figure 2.

Figure 2:

Consensus recommendations for optimizing therapeutic decision-making for off-label medicines use and linkage to research

Three of our five recommendations focus on optimizing therapeutic decision-making for off-label medicines use by: 1) Seeking rigorous scientific evidence to support proposed use; 2) Utilizing expertise in evidence evaluation and synthesis to critically evaluate available evidence, informed by relevant clinical issues and patient and caregiver perspectives; and 3) Using rigorous processes to formulate sound recommendations.

Recommendation 4 is to link the process of therapeutic decision-making for off-label medicines use routinely to the prioritization and conduct of relevant research (interventional and observational) so that knowledge gaps can be addressed in a timely manner.

Recommendation 5 is to foster enhanced partnerships among regulators, clinicians, policy makers, researchers, and sponsors to achieve the dual objectives of optimizing therapeutic decision-making and driving timely, relevant research concurrently.

DISCUSSION

We identified 31 publications informing an enhanced understanding of the evidence and scientific rigor underpinning therapeutic decision-making for off-label medicines use. We found no globally recognized guidance for optimizing evidence-based decision-making in this context, representing a significant gap. Of 20 guidances with general recommendations, only one-third provided explicit frameworks or guidance about the types and quality of evidence needed and rigorous processes for evaluating this evidence to reach sound, ethical judgements about appropriate off-label use. Key aspects of these more rigorous frameworks informed our recommendations for optimal practice (Figure 2 and below), which focus on evidence rather than labelling status as the central driving principle. We discuss our recommendations in the context of strengths, limitations, and gaps in currently available guidances.

Recommendation 1: Seek rigorous scientific evidence

To improve therapeutic decision-making for any intended off-label use, we recommend systematic evaluation of the best available research evidence (including RWE) about a medicine’s clinical effectiveness and safety, rigorous assessment of overall benefits versus risks for specific patients and contexts, and assessments of comparative effectiveness and safety relative to non-intervention or other available treatments. These assessments should be informed by evidence from populations and settings where off-label use occurs. For example, safety profiles of medicines can differ between adult and pediatric populations as well as within pediatric age groups,(17) with important implications for risk:benefit assessments for these sub-groups. Cost-effectiveness considerations are relevant, but such evidence is usually lacking for off-label uses.

We found few guidances that explicitly described the types of evidence needed (e.g., from randomized controlled trials, observational studies, RWE) or procedures for evaluation using recognized tools for appraising or grading evidence.(8, 14, 15, 17, 22, 28, 32, 34) Very few guidances described the specific types of evidence needed for comprehensive safety evaluations.(8, 14, 17) Only one pediatric-focused guidance provided descriptions of specific types of evidence needed to optimally evaluate appropriateness of off-label use in this population.(17) Our recommendations align with several of these key guidances which provide more details about the types of evidence to be sought and optimal approaches for evaluation to inform rigorous risk:benefit assessments.

Key national(23, 24, 26, 27) or regional(16, 25) guidances lacked explicit descriptions and recommendations about the quality and evaluation of evidence necessary for determining appropriateness of off-label use, instead relying on broad statements about prescriber competencies, judgements, or opinions.

Recommendation 2: Utilize expertise in evidence evaluation and synthesis

Input of relevant expertise is critical to inform judgements about the types of evidence needed, and to evaluate the validity, clinical meaningfulness, and applicability of evidence.(17) We recommend these complex tasks should be informed by expertise from multiple relevant domains: 1) methodological expertise: e.g., evidence-based medicine, clinical epidemiology, pharmacoepidemiology, clinical pharmacology, pharmacoeconomics; 2) clinical and therapeutics expertise: e.g., clinicians providing care to target populations (e.g., pediatricians, geriatricians, oncologists), and clinicians with broader therapeutics expertise, such as clinical pharmacology and clinical pharmacy, including specialized expertise in relevant sub-populations (e.g., pediatrics, geriatrics); and 3) diverse patient and caregiver perspectives.

Pharmacoepidemiology and pharmacovigilance expertise, methodologies, and tools are important for judicious decisions about the types of evidence needed and optimal evidence evaluations to inform rigorous risk:benefit assessments. Appropriate knowledge and skills are particularly important for critically appraising a range of data sources, methodologic approaches, and evidence produced, including RWE. Relevant clinical expertise must also inform these judgements concurrently. Similarly, patients and caregivers can provide key insights about which outcomes matter to them and how they balance risks and benefits. Appropriate processes must be in place to manage potential conflicts of interest and ensure independence of experts involved in these evaluations.

When the quality of available evidence is limited (common in pediatric and certain other populations), access to an independent, well-balanced mix of expertise and perspectives is likely to positively influence the rigor and soundness of overall evaluations, including decisions about any trade-offs between evidence and unmet clinical needs (see Recommendation 3).

Only one-fifth of general guidances provided explicit recommendations about the types of expertise to include in groups tasked with evaluating evidence and developing recommendations for off-label uses.(14, 28, 32, 34) This reflects a significant gap in most currently available guidances for therapeutic decision-making.

Recommendation 3: Use rigorous processes to formulate recommendations

Optimal therapeutic decision-making requires rigorous processes for integrating evidence with important clinical considerations. We recommend key elements of such processes include establishing evidence thresholds for appropriate off-label use; justifying any use with limited evidence based on pre-specified criteria (e.g., serious or rare condition, potential benefits of off-label use outweigh potential risks, and alternative treatments exhausted or not available);(8, 14) balancing clinical need with risks of off-label use; and acknowledging need for shared decision-making with patients and caregivers. Such systematic approaches, informed by appropriate expertise (Recommendation 2), allow careful assessment of potential trade-offs between evidence requirements and unmet clinical needs to develop sound recommendations about acceptable off-label use.

Only certain guidances provided explicit descriptions of systematic decision-making frameworks, algorithms, or other rigorous processes to integrate evidence with clinical considerations and expert judgements (see Table 2). We recommend adopting a similar approach. For example, one guidance(14, 15) described four categories of appropriate use (with instructive example scenarios), depending on the quality of available evidence and relative benefits and risks for different clinical situations: 1) routine use justified, supported by high-quality evidence; 2) exceptional use, in individual patients according to pre-specified criteria; 3) conditional use, in groups of patients, according to pre-specified criteria, with evidence development; and 4) research or investigational use, via research protocols. These categories were accompanied by recommendations for shared decision-making with patients or caregivers and appropriate informed consent processes in situations with potential greater risk or uncertainty. However, recommendations addressing processes for shared decision-making and informed consent procedures diverged widely among available guidances.

Recommendation 4: Link off-label medicines use to research

Evidence thresholds to justify off-label use (see Recommendation 3) would not only help reduce inappropriate use but could facilitate addressing identified knowledge gaps by creating stronger demand for needed evidence from clinical decision-makers. This demand could help complement the current major regulatory drivers for medicines research in populations such as pediatrics. Despite strong regulatory and research measures existing for many years in the US and EU,(3, 48–50) the prevalence of pediatric off-label prescribing remains high.(51, 52) Additional measures are urgently needed to provide more direct linkage between clinical use and evidentiary challenges and the prioritization of research to address them.(53) Therefore, we recommend more timely conduct of relevant medicines research (interventional and observational studies) to be prioritized based on identified knowledge gaps and the likely extent and potential risks of off-label use.

To inform future decision-making with more timely evidence, we also recommend new or existing unsupported off-label uses be routinely linked to prospective evaluation of outcomes (e.g., effectiveness, safety, comparative effectiveness/safety, cost-effectiveness). Recently the U.S. FDA has released guidance statements for the use of RWE to support regulatory approval(54) and the European Medicine Regulatory Network has also outlined an ambitious vision for the use of RWE across the spectrum of medicines regulation by 2025.(55) Routine linkage will catalyze generation of clinically meaningful RWE to address benefits, risks, and therapeutic alternatives of off-label use. Newly generated RWE can support ongoing re-evaluation of risk:benefit so that future use is continually informed with better evidence. While this model resembles regulatory pathways such as “conditional approval with evidence development,” such pathways do not usually promote evidence on comparative effectiveness/safety or costs(56), which are important considerations for optimizing therapeutic decision-making and should be better addressed in future. A number of included guidances identified recommendations for evaluating outcomes following off-label use, some anchoring this to approval by institutional committees.(14, 16, 22, 28–30).

Recommendation 5: Foster partnerships among regulators, clinicians, policy makers, researchers and sponsors

Major medicines regulators generally play a limited role in guiding individual prescribers’ decisions about off-label uses, deferring these matters to professional judgement.(57) Historically, these bodies have regulated sponsors’ promotion of off-label uses(9, 57) or intervened after significant safety concerns emerged.(58) Regulators and sponsors also monitor off-label uses through post-marketing studies, although the suboptimal effectiveness of these measures to date needs greater attention(59). Risk minimization measures may be initiated to address problems identified and improve adherence to existing evidence-based prescribing guidelines.

We recommend regulators and sponsors consider adopting a more proactive role by collaborating with relevant stakeholders (eg clinicians, researchers, policy makers) to develop guidance supporting a more explicit approach to therapeutic decision-making for off-label uses. Such guidance would enable various decision-makers (e.g., prescribers, HTA bodies, developers of prescribing guidelines/information, funders/payers) to be more consistent in decision-making and proactive oversight of off-label use, rather than responding only after problems arise. The overall aim would be to eliminate inappropriate off-label uses before they become established. This would also ensure that the incentive for the conduct of needed research is not potentially diminished by widespread off-label use.

Furthermore, we recommend enhanced collaborative partnerships between those who conduct research to generate needed evidence and those who use evidence to inform regulatory or therapeutic decisions. Strong partnerships among regulators, academic researchers, clinical decision-makers, and the pharmaceutical industry are vital for rapidly enabling and conducting needed research, especially in generating high-quality RWE (see Recommendation 4). This effort should also engage a range of relevant expertise (see Recommendation 2). Such collaborations are critical to optimizing therapeutic uses by quickly closing important evidence gaps, and enabling future improvements in labelling, eventually resulting in less off-label, more evidence-based use.

Strengths and Limitations

We used rigorous methods for our scoping review and consensus recommendations. Use of a multinational, multidisciplinary expert panel and formal consensus methods for developing recommendations strengthens their validity and generalizability. Our search strategy may not have identified all relevant publications, especially those from the grey literature, those that did not involve pediatric populations, and those published after the conduct of our literature search(60, 61). However, our panel’s broad range of expertise and experience enabled us to build on key findings of prior guidances. Our Delphi process did not include a consumer representative or patient advocate but we do recommend including these important stakeholders in the development of future guidance for therapeutic off-label medicines use. While we focused on the pediatric population, the issues identified and recommendations provided are generalizable to other populations where off-label use occurs.

Conclusions

Off-label use of medicines is common and sometimes necessary and appropriate, but internationally accepted guidance has been lacking to support judicious off-label use and informed decision-making. We provide comprehensive consensus recommendations to optimize therapeutic decision-making for off-label medicines use and drive clinically relevant research in an integrated manner. We recommend: 1) seeking rigorous scientific evidence; 2) utilizing diverse expertise in evidence evaluation and synthesis; 3) using rigorous processes to formulate recommendations for appropriate off-label use; 4) linking off-label use with timely conduct of clinically meaningful research to address knowledge gaps quickly; and 5) fostering partnerships between clinical decision-makers, researchers, regulators, policy makers, and sponsors to facilitate cohesive implementation and evaluation of these recommendations. Implementation of these recommendations will lead to more informed decision-making, more thoughtful and evidence-based use of medicines off-label, and better patient outcomes while helping drive needed, clinically relevant research. Successful implementation requires appropriate funding and infrastructure support to engage necessary stakeholders and foster relevant partnerships, which represent significant challenges that policy makers must urgently address.

Supplementary Material

Appendix A

Appendix A: Search strategy for scoping review

Appendix B

Appendix B: Study selection criteria for scoping review

Appendix C

Appendix C: Consensus development Expert Panel areas of expertise

KEY POINTS:

• Off-label medicines use is a critical issue in modern medicine, with important clinical, safety, ethical, and financial consequences for individuals and health systems.

• There is currently no internationally recognized guidance for optimizing therapeutic decision-making when using medicines off-label.

• Currently lacking are explicit frameworks based on rigorous evidence to support therapeutic decision-making.

• To address this gap, our international, multidisciplinary expert panel developed comprehensive consensus recommendations to optimize therapeutic decision-making for off-label use and concurrently drive timely, clinically relevant medicines research.

• Our guidance has important implications for clinicians, patients, researchers, regulators, the pharmaceutical industry, health technology assessment bodies, payers, and policy makers.