Unwarranted claims of drug efficacy in pharmaceutical sales visits: are drugs approved on the basis of surrogate outcomes promoted appropriately?

Abstract

Aims

This study compares physicians' recall of the claims of benefits on cardiovascular disease and diabetes made by pharmaceutical sales representatives for drugs approved on the basis of a surrogate outcome, i.e., an off‐label claim, compared with those approved on the basis of a serious morbidity or mortality (clinical) outcome.

Methods

Physicians in primary care practices in Montreal, Vancouver, Sacramento and Toulouse, who saw sales representatives as part of their usual practice and served a non‐referral population, were contacted in blocks of 25 from a randomized list of all physicians practising in the relevant metropolitan area. We compared how frequently physicians reported that sales reps made claims of serious morbidity or mortality (clinically meaningful) benefits for drugs approved on the basis of surrogate outcomes vs. drugs approved on the basis of clinical outcomes.

Results

There were 448 promotions for 58 unique brand name cardiovascular and diabetes drugs. Claims of clinically meaningful benefit were reported in 156 (45%) of the 347 promotions for surrogate outcome drugs, constituting unwarranted efficacy claims, i.e., off‐label promotion. Claims of clinical benefit were reported in 72 of the 101 promotions (71%) for drugs approved on the basis of clinical outcomes, adjusted OR = 0.3 (95% CI 0.2, 0.6), P < 0.001.

Conclusions

Claims of efficacy made in sales visit promotions for drugs approved only on the basis of surrogate outcomes extended beyond the regulator‐approved efficacy information for the product in almost half of promotions. Unapproved claims of drug efficacy constitute a form of off‐label promotion and merit greater attention from regulators.

What is Already Known about this Subject

• Drug regulatory authorities approve medicines on the basis of both surrogate and clinical outcomes.

• Sales representatives frequently visit doctors' offices to promote their companies' products.

• Sales representatives engage in off‐label promotion.

What this Study Adds

• When drugs are approved on the basis of clinical outcomes, sales representatives mention clinical benefits in 71% of promotions for medicines treating cardiovascular disease and diabetes.

• When drugs are approved on the basis of surrogate outcomes, sales representatives mention clinical benefits in 45% of promotions for medicines treating cardiovascular disease and diabetes, i.e., engage in off‐label promotion.

Introduction

When considering a new drug for market approval, regulatory agencies examine the evidence of the drug's benefits and harms provided by the company and only approve indications for which sufficient evidence has been provided 1, 2. Doctors may still prescribe drugs for other indications, termed off‐label prescribing 3, but companies are prohibited from promoting their products for non‐approved indications. Bans on off‐label promotion are intended to protect patient health as off‐label uses may pose greater risks than those evaluated and approved by a regulatory agency when there is little or no science to support the off‐label use 4.

Besides promotion for unapproved indications, off‐label promotion may also occur through efficacy claims that are inconsistent with approved information. Randomized clinical trials evaluate drug effects on specific outcomes. These can be clinical outcomes that measure how a patient feels, functions or survives, such as stroke incidence or health‐related quality of life 5, or they can be surrogate outcomes substituting for clinical outcomes, i.e., a biomarker expected to predict clinical effects based on epidemiologic, therapeutic, pathophysiologic, or other scientific evidence 6. The US Food and Drug Administration (FDA) will accept a surrogate for a traditional approval if it is known to predict clinical benefit (a validated surrogate) or a surrogate that is reasonably likely to predict a drug's intended clinical benefit for an accelerated approval, i.e., an approval using trials that are permitted to end before the occurrence of a hard clinical outcome 7. Similarly, Health Canada allows the use of surrogates as primary endpoints where the surrogate is reasonably likely or well known to predict a clinical outcome 8. The use of surrogate outcomes is widespread in pre‐marketing studies. In Canada, 114 out of 238 new drug approvals were based on surrogate outcomes between 2005 and 2014 9. In the US, between 2005 and 2012, 91 out of 201 indications (45%) for 188 new drugs were approved on the basis of trials that used surrogate outcomes as the primary outcome measure 10.

However, despite their frequent use, surrogate outcomes do not always predict important clinical outcomes 5, 11, 12, 13, 14. A correlation between a surrogate and clinical outcome is not sufficient to rule out the risk of negative or false positive conclusions 15. A valid surrogate outcome should be on the causal pathway of the disease, and all changes to the surrogate should also predict changes to the clinical outcome of interest 16. Surrogate outcomes can fail to accurately predict clinical effects on a number of grounds, the most important being that an intervention can affect both a surrogate outcome and a clinical outcome on the basis of two different mechanisms of action. Although the two effects would be correlated, as they resulted from the same intervention, it would be incorrect to assume that another intervention affecting the same surrogate measure would also have an effect on the clinical outcome 17.

Clinical trials using surrogate outcomes can also overinflate treatment benefits. Surrogate outcome‐only trials have more than twice the odds of leading to positive conclusions about treatment (62% or 52/84 trials) compared to trials using clinical outcomes (37% or 37/101 trials) across a wide range of drug families 18.

If a company claims a morbidity or mortality benefit for a drug approved only on the basis of surrogate outcomes, this constitutes one form of off‐label promotion. This study compares physicians' memories of the claims of benefits on cardiovascular disease and diabetes made by pharmaceutical sales representatives (sales reps) for drugs approved on the basis of a surrogate outcome vs. those approved on the basis of a serious morbidity or mortality (clinical) outcome.

Methods

We conducted a pre‐planned subgroup analysis of a prospective cohort study of pharmaceutical sales reps visits to primary care physicians in Montreal, Vancouver, Sacramento and Toulouse 19. Ethics approval for the original cohort study was obtained from the University of British Columbia behavioural ethics committee, the Ethics Committee of the Centre de recherche du Centre hospitalier de l'Université de Montréal (CR‐CHUM), University of California at Davis, and the Union Régionale des Professionnels de Santé – Médecins Libéraux – Midi Pyrénées.

Participants and study setting

Between 2009 and 2010, primary care physicians from Montreal, Vancouver, Sacramento and Toulouse were contacted in blocks of 25 from a randomized list of all physicians practising in the relevant metropolitan area.

Inclusion criteria were physicians who met with sales reps in their regular practice, worked at least 20 clinical hours per week, and served a greater than 50% primary care (non‐referral) patient population. Members of advocacy groups with a focus on drug promotion, such as Healthy Skepticism or No Free Lunch were excluded, as were current drug company employees.

Intervention

Physicians saw sales reps as per their usual practice and recorded information about the next eight consecutive drug promotions, using a questionnaire (Supporting Information Data S1) adapted from similar instruments used in observational studies in France 20, and Australia and Malaysia 21, and pilot‐tested in a sample of 15 physicians and 41 promotions in Victoria, British Columbia.

Outcomes

Claims of drug efficacy in sales rep promotions

We compared how frequently physicians reported that sales reps made claims of serious morbidity or mortality (clinically meaningful) benefit for drugs approved on the basis of surrogate outcomes vs. drugs approved on the basis of clinical outcomes, questionnaire item 7 (Supporting Information Data S1). We dichotomized the outcomes by consulting the approved product information in the relevant national physician prescribing compendium (Compendium of Pharmaceuticals and Specialties in Canada and Le Guide Vidal in France), or online drug database (Drugs@FDA archives in the US) available at the time of the promotion (2009–2010). In cases where drugs were approved for multiple indications, the drug was classified as being approved based on a clinical outcome if that was the basis for at least one of the indications even if approval for the other indications was based on surrogate outcomes. A promotion with a reported claim of a serious morbidity or mortality benefit was coded as such regardless of all other responses given. Promotions with no recalled claims of drug benefits, and open text responses were excluded from analysis.

The unit of analysis was a drug‐specific promotion for agents used in the treatment of diabetes, hypertension or hypercholesterolaemia for which the physician reported at least one efficacy claim was made. These disease categories included both drugs approved on the basis of surrogate outcomes and drugs approved based on clinical outcomes. Limiting our analysis to these particular classes of drugs also allowed us to avoid potential confounding of promotional information by differences in therapeutic goals. Supporting Information Data S2 provides the full list of promoted drugs, their summarized indications and how often they were promoted to physicians.

A set of physician, practice and sales visit characteristics were also measured. (See Supporting Information Data S3 for a full list and definition of variables.) Based on previous literature and our research question, we retained the following seven variables in the analysis: if the drug was previously prescribed, if it had been previously promoted to the physician, if the visit was in a group or one‐on‐one setting, the frequency of sales visits received, the physician's affiliation with an academic health science centre and the number of physicians in the same practice as the participant.

Intent to Prescribe

As a secondary objective, we also examined the physicians' intent to prescribe a surrogate outcome drug (questionnaire item 24) after a claimed effect on serious morbidity or mortality was made vs. a promotion where no such claim was made. Intent to prescribe was rated on a four‐point scale in which physicians rated themselves as ‘somewhat likely’, ‘very likely’, ‘somewhat unlikely’ or ‘very unlikely’ to prescribe. We combined these responses into two categories: ‘somewhat or very likely to prescribe’ and ‘somewhat or very unlikely to prescribe’.

Statistical methods

We used generalized estimating equations (GEE) to carry out binary logistic regression while adjusting the odds ratio (OR) and confidence intervals (CI) to account for multiple responses from the same physician. SPSS version 21 (IBM, Armonk, NY) was used for all analyses.

There were missing values in 27 (>0.8%) cases (see Supporting Information Data S4 for the list of variables with missing values.) Since the sample size was limited, we maximized available data using multiple imputation runs on SPSS to allow valid inference. All analyses were carried out using five sets of imputed data.

Results

The sample included 448 promotions for 58 unique brand name drugs used to treat cardiovascular disease (hypertension and hypercholesterolaemia) and diabetes. These drugs were promoted to 196 physicians (92 in two Canadian sites, 57 in France and 47 in the US, one site each) (Figure 1). Tables 1 and 2 summarize the characteristics of the physicians and promotions included in the sample, respectively.

Figure 1.

Physician recruitment

Adapted from Mintzes et al. 19. * ‘other’ includes maternity leave, sick leave, deceased, and unspecified. ** Physicians who withdrew before filling in any questionnaires

Table 1.

Physician and practice characteristics

Characteristic	Country
	Canada n a = 92 (%)	France n = 57 (%)	United States n = 47 (%)	Overall n = 196 (%)
Median promotions per physician (interquartile range)	2 (1.3)	2 (1.3)	2 (1.3)	2 (1.3)
Male physician	56 (60.9)	45 (78.9)	34 (72.3)	135 (68.9)
Mean year of physician graduation ± SD	1984 ± 10	1984 ± 10	1989 ± 9	1985 ± 10
Physician received sales visits at least twice weekly	47 (51.1)	50 (87.7)	40 (85.1)	137 (69.9)
Solo practice	20 (21.7)	20 (57.1)	11 (23.4)	51 (26.0)
Mean patients seen per week ± SD	131 ± 53	119 ± 40	98 ± 48	120 ± 50

^a n = the number of physicians receiving promotions per country

Table 2.

Characteristics of cardiovascular and diabetes drug promotions

Characteristic	Country
	Canada n a = 211 (%)	France n = 133 (%)	United States n = 104 (%)	Overall n = 448 (%)
Promotions of drug approved on the basis of surrogate outcomes	157 (74.4)	105 (78.9)	85 (81.7)	347 (77.4)
Number of unique drugs promoted	19	28	22	47
Physician previously prescribed drug	155 (73.5)	71 (53.4)	71 (68.3)	297 (66.3)
Visit ≤ 5 min in duration	87 (41.2)	37 (27.8)	49 (47.1)	173 (38.6)
One‐to‐one sales visit	148 (78.3)	130 (97.7)	75 (72.1)	371 (82.8)
First promotion received for drug	53 (25.1)	35 (26.3)	19 (18.3)	107 (23.9)

^a n = the number of promotions for all cardiovascular or diabetes drugs

Physicians received between one and seven promotions (median of two promotions) for diabetes or cardiovascular drugs. The majority of promotions (347 or 77%) were for drugs approved on the basis of surrogate outcomes: 157 (74%) in Canada, 105 (79%) in France, and 85 (82%) in the US. The majority (83%) of promotions occurred during one‐on‐one visits to physicians. In 76% of promotions, the drug had been promoted previously to the same physician, and in 67% the physician had previously prescribed the drug. Nearly two‐thirds (61%) of promotions were longer than 5 min in duration.

Table 3 compares the frequency of claims of clinically meaningful benefit recalled by physicians for drugs approved on the basis of the two different types of outcomes. Both pooled and country‐specific results are presented. In total, claims of clinically meaningful benefit were reported in 156 (45%) of the 347 promotions for surrogate outcome drugs, constituting unwarranted efficacy claims, i.e., off‐label promotion. These claims were also reported in 72 of the 101 promotions (71%) for drugs approved on the basis of clinical outcomes. Thus, claims of serious morbidity or mortality benefit were reported significantly less often in promotions for surrogate outcome than clinical outcome drugs; adjusted OR = 0.3 (95% CI 0.2, 0.6), P < 0.001.

Table 3.

Frequency of reported claims of serious morbidity or mortality benefit in promotions for surrogate vs. clinical outcome drugs

Country	Frequency of reported claims of serious morbidity or mortality benefit
	Promotions of surrogate outcome drugs a , n/N (%)	Promotions of clinical outcome drugs b , n/N (%)	Surrogate vs. clinical outcome promotions, adjusted OR (95% CI)	P
Canada	72/157 (45.9)	37/54 (68.5)	0.5 (0.2, 1.0)	0.07
France	62/105 (59.0)	24/28 (85.7)	0.1 (0, 0.8)	0.03
United States	22/85 (25.9)	11/19 (57.9)	0.3 (0.1, 1.1)	0.07
Overall	156/347 (45.0)	72/101 (71.3)	0.3 (0.2, 0.6)	<0.001

^aPromotions for drugs approved on the basis of surrogate outcomes; n = the number of promotions with a claimed clinical benefit; N = the total number of promotions for drugs approved on the basis of surrogate outcomes in region.

^bPromotions for drugs approved on the basis of serious morbidity or mortality outcomes; N = the total number of promotions for drugs approved on the basis of serious morbidity or mortality in region.

In Canada, claims of serious morbidity or mortality benefit were recalled for 72 (46%) of promotions for surrogate outcome drugs compared to 37 (68%) promotions for clinical outcome drugs; adjusted OR = 0.5 (95% CI 0.2, 1.0), P = 0.07. Comparable figures in France were 62 (59%) and 24 (86%), respectively; adjusted OR = 0.1 (95% CI 0.03, 0.8), P = 0.03 and in the US were 22 (26%) and 11 (58%), respectively; adjusted OR = 0.3 (95% CI 0.1, 1.1), P = 0.07.

In 342 of the 347 promotions for surrogate outcome drugs (98.6%), physicians had answered the question on intent to prescribe after the sales visit. Physicians were nearly two times more likely to report being somewhat or very likely to prescribe a surrogate outcome drug if a claim of serious morbidity or mortality benefit was reported during the promotion, as compared with other promotions for the same drug; adjusted OR = 1.9 (95% CI 1.1, 3.9), P = 0.01.

Discussion

This study has examined an aspect of off‐label promotion that has been subject to little investigation: unwarranted claims of a serious morbidity or mortality benefit for drugs that have been approved only on the basis of surrogate outcomes. Reassuringly, we found that physicians were less likely to report that sales reps made claims of clinically meaningful benefit for drugs approved only on the basis of surrogate outcomes than for those approved for clinical outcomes (45.0% vs. 71.3%). However, it is concerning that such claims were made in nearly half of promotions for surrogate outcome drugs.

There were a number of differences in our results among the three countries. The recruitment rate in the US site (57/73 = 78.1%) was substantially better than in the two Canadian sites combined (129/378 = 34.1%) or the French site (69/253 = 27.3%). The better US recruitment may be a reflection of how the doctors were initially identified. In the two Canadian sites the lists came from regulatory colleges, in France from the physicians' association, but in the US, physicians at two large practice groups did not see sales reps and therefore we used a list of doctors in independent physician associations. There were more visits to physicians in Canada and the US that were less than 5 min (41.2% and 47.1%, respectively) than in France (27.8%). This difference may be due to the eight‐page French Sales Visit Charter, adopted in 2004. Under this Charter, sales reps must provide physicians with approved product information including adverse events, precautions and contraindications 1, and this requirement may mean that sales reps have to spend longer with doctors. The greater percentage of one‐to‐one sales visits in France was probably due to the fact that 57.1% of French physicians were in solo practice compared to 21.7% in Canada and 23.4% in the US. The nonsignificant difference in the US in the proportion of surrogate outcome drugs promoted for serious morbidity or mortality benefit compared to the proportion of clinical outcome drugs is probably due to the low number of total promotions for clinical outcome drugs.

Pharmaceutical sales rep visits are the industry's largest promotional spending category 22. In most countries, regulations governing their visits require, at a minimum, that information provided during the visit be consistent with approved product information, and not be false or misleading 23. The unapproved claims of drug efficacy we observed in our study would be considered non‐compliant with regulations in all three countries. Despite the regulation on off‐label promotion, sales reps have been shown to provide off‐label drug information and influence physician prescribing in studies in multiple countries 24. In a study of sales visit promotions of antipsychotic medications in the US Veteran Affairs Department, over 90% of physicians recalled at least one claim made by the sales rep that was inconsistent with the FDA's approved drug package insert 25. In France, physicians in a 15‐year observational study of sales visits consistently recalled unapproved indications being highlighted in nearly one third of sales visits 20. In a three‐country study, primary care physicians (n = 255; 1692 promotions) recalled mention of unapproved indications in 13% of promotions in Canada, 16% in France and 10% in the US 19.

Inflated claims of drug efficacy have also been identified in medical journal advertisements in the US 26, 27, Sweden 28, Spain 29, Australia and Malaysia 21. In an analysis of FDA warning letters to pharmaceutical companies for non‐compliant promotions between 2000 to 2006, misleading or overstated efficacy claims constituted one of the top three (of 22) categories of violations 30.

In our study, when physicians heard off‐label promotions they were almost twice as likely to report being somewhat or very likely to prescribe the drug compared to when they did not hear an off‐label claim of beneficial effects on serious morbidity or mortality. This finding is troubling, as it suggests that physicians may rely on unwarranted efficacy claims to guide treatment of cardiovascular disease and diabetes, two serious chronic conditions. It also reinforces previous work on the influence on prescribing of industry‐provided information: a systematic review by Spurling and colleagues found that three measures of prescribing – frequency, cost and quality – either deteriorated or, at best, remained unchanged when doctors received information directly from companies 24.

Our study should not be interpreted to imply that all off‐label prescribing is dangerous or ineffective and, in fact, when the scientific evidence is strong, off‐label prescribing is just as safe as on‐label prescribing 4. As evidence accumulates, some drugs approved on the basis of surrogate endpoints are found to have clinical advantages, and changes in prescribing are appropriate. Initially the statins were only shown to reduce lipid levels but later on clinical trials established that they were useful in secondary prevention in people with cardiovascular disease. There are now clinical trials of drugs in some of the newer classes of oral hypoglycaemia that have shown promising results on cardiovascular disease 31. Some of the products that were promoted to physicians in our study might have accumulated enough evidence to justify off‐label use, although we did not specifically examine this possibility.

However, based on past experience, physicians also need to exercise caution in prescribing off‐label. A key aim of diabetes treatment is to prevent serious complications of diabetes, including cardiovascular disease. Rosiglitazone, a diabetes drug approved on the basis of surrogate outcomes, was found to increase, not decrease, cardiovascular risks 32. Additionally, some drugs approved on the basis of surrogate outcomes have been withdrawn from the market due to serious risks, such as the statin cerivastatin, which caused rhabdomyolysis 33, and troglitazone, which caused hepatotoxicity 34. We did not distinguish between validated surrogates, i.e., ones that are known to predict clinical benefit, and those that are only reasonably likely to predict clinical benefit. Promotion of a clinical benefit for drugs with a validated surrogate, closely linked to clinical benefit, would have been better justified than in cases of less well validated surrogates.

Fines in the billions of dollars have proved ineffective in the US in controlling off‐label promotion. Between 1991 and 2015 companies were fined over $11 billion for illegal, primarily off‐label, promotion 35 and some companies were fined multiple times. In 2012 Johnson & Johnson paid $181 million in a multistate settlement for off‐label promotion of risperidone (Risperdal) and another $2 billion the following year in a federal settlement for the same problem 35. In 2004 Pfizer pleaded guilty to off‐label promotion and paid out $430 million and in 2009 paid another $2.9 billion 36. France is taking another approach to try to curb off‐label use through the use of a mechanism termed a Temporary Recommendation for Use (TRU). These are granted for a maximum of 3 years, during which time the manufacturer is expected to provide data to expand the indications for its drug. The law also makes manufacturers responsible for controlling off‐label prescribing through monitoring whether prescriptions adhere to marketing authorizations and reporting unusual activities to regulators 37. While the French system is too new to have been evaluated, some authors have questioned how effective TRUs will prove to be 38.

Limitations

This study was carried out in a primary care setting and results cannot necessarily be generalized to sales visits received by specialist physicians, or to areas outside of the four urban areas where the physicians in our study practised. We also relied on physician reports of claims made by sales reps. Physicians could have made incorrect inferences about the efficacy messages conveyed during promotion. However, the messages that physicians take away from the sales visit ultimately form the basis of prescribing decisions. Moreover, the pragmatic nature of this study allowed for sales visits to take place naturally and did not distinguish between spontaneously volunteered information and information provided upon further prompting.

Second, this was a secondary analysis of previously collected data, limiting data availability to answer research questions in some cases. For instance, item 7 of the original cohort study questionnaire allowed us to investigate the category of efficacy claim (i.e., a claim of drug efficacy on serious morbidity or mortality), but not the exact efficacy claims recalled in each promotion. Finally, the number of observations in this sample, particularly for French and US promotions, was limited.

Conclusion

Although physicians recalled sales reps claims of efficacy more often for drugs approved on the basis of a serious morbidity or mortality outcome vs. those approved on the basis of a surrogate outcome, claims of efficacy for the latter group of drugs were frequent, occurring in nearly half of promotions. These claims extended beyond the regulator‐approved efficacy information for the product, which could have serious health consequences if a drug without proven clinical benefit is used as a result. Unapproved claims of drug efficacy constitute a form of off‐label promotion, contravene national regulations in most countries, and therefore merit greater attention from regulators through both more stringent control over off‐label promotion and limiting the use of surrogate outcomes for approving new drugs.

Competing Interests

In 2015–2016 J.L. received payment from non‐profit organizations for acting as consultant on a project to investigate indication‐based prescribing and a project that looked into which drugs should be provided free of charge by primary care providers. He received payment from a for‐profit organization for sitting on a panel that discussed extending drug insurance in Canada. He is on the Foundation Board for Health Action International.

B.M. was a paid expert witness on behalf of plaintiffs in two Canadian legal cases in 2014–2016: an application for a class action on cardiovascular risks of testosterone, and a class action suit on hormone replacement therapy and breast cancer.

A.H. is the recipient of a Career Investigator award from the Canadian Institutes for Health Research for innovation in decision‐making for patients, practitioners and policy makers. She is a senior drug policy advisor for all levels of government.

R.H. reports no conflicts of interest.

This research is based on a larger study funded by the Institute of Health Services and Policy Research, Canadian Institutes of Health Research, and by Michael Smith Health Research Foundation. The funders had no involvement in the design or conduct of the study; in the collection, management, analysis, or interpretation of the data; or in preparation, review, or approval of the manuscript or decision to submit for publication.

Supporting information

Data S1 Physician questionnaire

Data S2 Generic name, indications and frequency of promoted drugs, by outcome (clinical and surrogate) and country

Data S3 Variable definition and selection

Data S4 Missing data

Habibi, R. , Lexchin, J. , Mintzes, B. , and Holbrook, A. (2017) Unwarranted claims of drug efficacy in pharmaceutical sales visits: are drugs approved on the basis of surrogate outcomes promoted appropriately?. Br J Clin Pharmacol, 83: 2549–2556. doi: 10.1111/bcp.13360.