Breadth, Depth and Agreement among Provincial Formularies in Canada

Abstract

Background:

Previous studies have concluded that there is significant variation in drug coverage across Canadian provinces because conventional measures of inter-rater reliability for formulary listings are low. We sought to investigate whether conventional methods are appropriate for formulary concordance measurement by testing the hypotheses that (a) conventionally measured variations in provincial formularies are driven by disagreement over large numbers of drugs that represent very small segments of the market and (b) patterns in coverage levels and agreement across therapeutic categories might provide evidence of “potentially legitimate” variation in provincial formularies.

Methods:

We studied December 2006 formulary listings for general pharmacare programs in all but the smallest Canadian province. We characterized formularies in terms of the simple percentage of all available drugs that were listed on them and by a similar percentage that weighted each drug by its total national retail sales during 2006. We measured agreement among formularies using conventional inter-rater reliability scores (Kappa statistics) and a simple coverage-agreement measure.

Results:

Provincial formularies studied here listed between 55% and 73% of the 796 drugs analyzed. When formulary listings were weighted by national retail sales, the measure of formulary coverage exceeded 86% in all provinces studied. Conventional inter-rater reliability scores (Kappa statistics) indicate that coverage agreement among most provincial formularies was low to moderate; however, drugs that were listed on all nine provincial formularies studied accounted for 77% of total retail spending in Canada. When analyzed by therapeutic category, the extent of coverage offered was relatively consistent across provinces in all but three leading categories: anti-migraine drugs, anti-dementia drugs and sedatives.

Conclusion:

While variations in coverage for specific drug classes and drug products remain important areas for investigation and policy consideration, Canada is currently operating with a significant “implicit national formulary” by way of the fact that provincial formularies independently yet mutually list most of the top-selling medicines in the marketplace.

Variations in the number and types of drugs covered by provincial formularies are a continuing concern in Canada. Because provincial drug benefit plans (hereafter referred to as pharmacare programs) evolved through independent efforts to address region-specific health needs and political priorities, each has its own formulary identifying which drugs will be subsidized for its beneficiaries. While all provinces, with the exception of Quebec, now participate in a centrally coordinated Common Drug Review (CDR) – to which manufacturers must submit clinical and economic data in order to have a new drug considered for coverage under participating pharmacare programs – the recommendations of the CDR are just that: recommendations (McMahon et al. 2006; CADTH 2007). Final decisions concerning which drugs are listed on formularies rest with the provinces, making continued provincial variation in drug coverage possible, perhaps even likely (Morgan et al. 2006).

Several studies have investigated drug coverage under provincial pharmacare programs using conventional analytic techniques for measuring the extent to which independent raters (drug plans) classify the same subjects (drugs) into given categories (“covered” or “not covered”). Three often-cited studies conclude that there is relatively little agreement among provincial formularies based, at least in part, on results derived from conventional agreement statistics (Anis et al. 2001; Gregoire et al. 2001; MacDonald and Potvin 2004). Such findings – and the discussions aroused by them – have prompted federal and provincial governments to consider a national formulary (F/P/T Ministerial Task Force 2006).

Our objective with this research project was to re-investigate the notion that provincial formularies vary widely. We were motivated by a realization that the clinically focussed inter-rater reliability statistics that have been used in this area of policy research are based on two key underlying assumptions: (1) that all decisions are equal and (2) that there are no legitimate grounds for disagreement among independent decision-makers. We were therefore specifically interested in testing the hypotheses that (a) conventionally measured variations in provincial formularies are driven by disagreement over large numbers of drugs that represent very small segments of the market and (b) patterns in coverage levels and agreement across therapeutic categories might provide evidence of “potentially legitimate” variation in provincial formularies. To explore the second hypothesis further, we also tested to see whether coverage agreement is relatively high for drugs reviewed through the CDR process.

Previous Studies

Several previous studies have investigated the extent to which provincial formularies vary (Anis et al. 2001; Gregoire et al. 2001; MacDonald and Potvin 2004; CIHI 2005). Anis and colleagues (2001) did so by studying whether the 58 drugs brought to BC PharmaCare for consideration in 1996 and 1997 were covered by pharmacare programs in other provinces as of 1998. Owing to concerns that some provinces have special formularies for specific health conditions, they excluded drugs expected to be part of specialized plans in one or more provinces (e.g., drugs for cancer, HIV/AIDS and cystic fibrosis). These authors used Kappa statistics to summarize interprovincial coverage agreement. They found Kappa statistics in the range of –0.11 to 0.64 and concluded that there was relatively little agreement among provincial formularies.

Gregoire and colleagues (2001) assessed formulary agreement by studying provincial coverage as of January 1999 for 108 new types of drug that had been licensed for sale by Health Canada from 1991 through 1998. They analyzed provincial formulary listings for all these new drugs, acknowledging that they did not have coverage information for any of the specialized healthcare programs that might cover certain medicines in some provinces. With this caveat, these authors found that some provinces (Quebec, British Columbia, Manitoba and Saskatchewan) listed more than 70% of the new drugs, whereas others (New Brunswick, Newfoundland and Labrador, Ontario and Prince Edward Island) listed fewer than 50%. When comparing formularies, they found Kappa statistics in the range of 0.12 to 0.63 and concluded that there was relatively little formulary agreement across provinces.

MacDonald and Potvin (2004) studied formulary agreement by assessing listings of all products, new and old, on any one of the April 2003 pharmacare formularies from six provinces: British Columbia, Alberta, Manitoba, Ontario, Quebec and Nova Scotia. Rather than assess coverage at the individual product level, these authors argued that it is more clinically relevant to assess coverage status for chemical subgroups that are often used for the same indication. They did not have coverage information from specialized healthcare programs that might cover certain types of medicines in some provinces, and acknowledged that this may have affected their findings. With that caveat, MacDonald and Potvin found that only 41% of the 481 chemical subgroups analyzed were listed on formularies from all six provinces studied. As with previous researchers in this field, these authors used Kappa statistics to summarize agreement among formularies. They found Kappa statistics ranging from 0.23 to 0.45 and concluded that there is relatively little agreement among provincial formularies.

The Canadian Institute of Health Information (CIHI 2005) assessed listings for all products, new and old, on any one of the 2005 pharmacare formularies from nine provinces (all but Quebec). CIHI included medicines that may be covered under condition-specific healthcare programs in some provinces but did not have access to coverage data from such programs. CIHI found that three provinces (Newfoundland and Labrador, British Columbia and Manitoba) covered more than 70% of all analyzed drugs and that two provinces (Prince Edward Island and Ontario) covered roughly 50%. They found that 31% of the analyzed drugs were covered in all nine provinces studied and that 41% of the underlying drug classes (as per MacDonald and Potvin's method) were covered in all nine provinces. CIHI did not report Kappa agreement statistics nor draw any conclusions about the extent of variation indicated by results.

Measuring Formulary Agreement

Three previous studies have concluded that provincial formularies vary significantly, based in part on the finding that Kappa statistics summarizing formulary agreement generally fall below 0.6 (Anis et al. 2001; Gregoire et al. 2001; MacDonald and Potvin 2004). These conclusions are consistent with a conventional distinction that Kappa values above 0.6 indicate “full agreement” and that lower values indicate moderate agreement or worse (Landis and Koch 1977). This conventional interpretation stems from applications of Kappa statistics as measures of inter-rater reliability in scientific and clinical domains (Landis and Koch 1977; McGinn et al. 2004). We believe that the interpretation of Kappa statistics as measures of decision-making concordance ought to differ in policy contexts.

Kappa statistics describe the extent to which two independent raters of given items agree on how the items should be classified with a frequency beyond what would be expected by chance alone. For example, when each of two raters independently classifies 90% of items in a given way (e.g., each says “yes” 90% of the time), a Kappa value measures the extent of agreement between the two raters beyond the 82% agreement that would be expected by chance alone. (Here is the math: if the decisions are independent, the probability of “yes–yes” agreement by chance alone would be 0.9 × 0.9 = 0.81; the probability of “no–no” agreement by chance alone would be 0.1 × 0.1 = 0.01; and the sum of these two types of chance agreement is 0.82.) If the two raters in this example agreed “yes–yes” to 86% of the items, agreed “no–no” to 6% of the items and disagreed on the remaining 8% of items, the Kappa agreement statistic for their appraisals would be 0.55. This Kappa = 0.55 means that the total rate at which they agreed (92%) spanned 55% of the difference between the rate of agreement expected by chance alone (82%) and perfect agreement (100%).

A clinical interpretation of a 0.55 Kappa statistic would be that there was only moderate agreement between the decisions of the two raters. But in clinical contexts, raters often aim to differentiate infrequent but identifiable “true cases” through appraisal of diagnostic tests or other sources of information. When all such “true cases” have equal importance, the Kappa statistic's heavy weighting of rare cases of disagreement is imperative. For example, if the two raters described above were diagnosing cancer cases based on the same set of test results, there would be cause for serious concern about their 8% rate of disagreement.

Kappa statistics may cast too much negative attention on potentially defensible disagreements in the context of situations where, within a set of judgments to be made, some decisions are more “important” than others. They also may pass undue judgment on disagreements that occur in cases where there is no objective “truth” that separates items that should be appraised positively versus negatively.

There are therefore (at least) two reasons to reconsider conventional measures of provincial formulary agreement. First, not all drug coverage decisions are equal. Decisions about access to reasonable and effective treatment options for serious conditions (at the patient level and at the population level) are surely more “weighty” policy decisions than decisions about expanding choices without altering effectiveness or decisions to list drugs for relatively minor afflictions (e.g., male pattern baldness). Second, there may be some legitimate differences in decisions taken by different drug plans. Potentially legitimate sources of coverage variation include such things as regional differences in healthcare needs, priorities, prices and resources (Birch and Gafni 2004). To the extent that provincial formulary listings might have such potentially legitimate sources of variation, perfect concordance would not necessarily be ideal. For these reasons, we believe that a variety of measures are needed to convey information about coverage offered under drug benefit formularies.

Data

We obtained public formulary data reflecting coverage of drug products as of December 2006 for nine provinces: British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Quebec, New Brunswick, Nova Scotia and Newfoundland and Labrador. Data for all these provinces but Quebec were extracted in May 2007 from the National Prescription Drug Utilization Information System (the NPDUIS database). We supplemented these data with information from Manitoba Pharmacare's “Part 3” list (a special authority schedule not included in the NPDUIS database), and obtained data for Quebec directly from the Conseil du médicament. Formulary data for Prince Edward Island were not available in an analyzable format.

For most provinces, we analyzed formulary listings pertaining to the pharmacare program covering social assistance recipients and residents over age 65. In provinces that administer income-based pharmacare (i.e., British Columbia, Saskatchewan and Manitoba) or social insurance–style pharmacare (i.e., Quebec), formulary listings were extracted from the plan under which seniors are most likely to be covered.

To assess coverage of drugs for which the CDR had issued guidance, we obtained all coverage recommendations posted by the CDR up to December 2006.

We obtained estimates of the total retail sales for each drug in the Canadian marketplace during 2006 from IMS Health Canada Inc. These sales data came from the Canadian CompuScript Audit, a database containing data collected from over 5,000 retail pharmacies stratified by province. The data from the sample of pharmacies is projected to construct total regional sales estimates, which are validated against IMS Health databases containing factory gate and wholesale sales figures. The retail sales figures we used include pharmacists' fees and reflect the combination of private and public spending on each medicine.

We excluded from our analyses drugs with less than $1,000 in Canadian retail sales because such sales volumes suggest that those drugs may not be available in all provinces (regardless of coverage decisions by provincial pharmacare plans). To avoid biases acknowledged in previous studies, we also excluded drugs that might be covered under disease-specific programs in one or more provinces. The drugs excluded on this basis were those primarily indicated to treat cancer, cystic fibrosis, HIV/AIDS, tuberculosis, multiple sclerosis, Gaucher's disease, pulmonary hypertension and thalassemia, as well as erythropoiesis-stimulating agents and human growth hormone. No evaluation of relative clinical or economic value was conducted or should be inferred from these exclusions.

Methods

We defined drug coverage and therapeutic categories using groupings of products as per the World Health Organization's Anatomical Therapeutic Chemical (ATC) drug classification codes (WHO 2004). The fifth level of this system groups drug products together according to active chemical substances (e.g., N06AB03 = fluoxetine), inclusive of all strengths and brands of a given drug. We used groupings of products at this level of aggregation to define “drugs” for the purpose of our study.

We deemed a drug to be “listed” on a provincial formulary if at least one version of it (e.g., a generic) was covered in some way. If at least one version of a drug received unrestricted coverage under a provincial formulary, we deemed the listing to be “unrestricted.” Finally, we deemed coverage for a drug to be “restricted” if public subsidy was available only under certain conditions: for example, only after failure of an alternative treatment or only if a special authority request was submitted by the prescribing physician.

We analyzed formulary listings for drugs across all therapeutic categories and for drugs within the 22 largest therapeutic categories in terms of national retail expenditure. The therapeutic categories used in this analysis were based on ATC groupings at the therapeutic/pharmacological level (e.g., N06A = antidepressants). We applied minor modifications to such ATC groupings in order to ensure that the drug classes in our categories generally had a common primary indication. For example, our hypertension grouping included drug classes from more than one of the ATC systems' therapeutic groupings: specifically, our hypertension therapeutic class included specific beta-blocking agents (C07A and C07C), calcium channel blockers (C08C and C08D), agents acting on the renin–angiotensin system (C09A to C09D), diuretics (C03A to C03E) and antihypertensive agents (C02A, C02D, C02H and C02L).

We gauged the breadth of a formulary by the simple, unweighted percentage of all the drugs analyzed (in a given therapeutic category) that were on the formulary. This approach measures the crude variety of drugs (within the therapeutic category) covered by a provincial formulary.

We gauged the depth of a formulary by applying national expenditure weights to the breadth measure. That is, we gave each coverage decision a weight equal to the share of national expenditure on prescription drugs accounted for by the drug in question. It is important to note that no provincial drug plan actually finances all prescription drug expenditures within its jurisdiction (indeed, none finances even 50% of total prescription drug spending within province). Our expenditure weighting methods simply provided a proxy for the relative significance of coverage decisions at a system level. While it is not always true that the financial and clinical values of medicines are correlated, the expenditure-weighting behind our depth measure gauges the relative importance of drugs covered in the sense that coverage for many drugs with minimal national sales volume receives less weight than coverage of fewer drugs with greater national sales. We chose to use national retail expenditures for this purpose to avoid the potential that province-specific sales of products would be influenced by coverage on the provincial formulary.

As with measures of province-specific formulary coverage, we calculated measures of interprovincial formulary agreement across all drugs and within the leading 22 therapeutic categories. We computed standard Kappa statistics and a relatively simple nine-province agreement statistic, both computed with and without expenditure weighting. Our simple agreement statistic reports, as a percentage of all analyzed drugs, how many drugs were independently listed on all nine provincial formularies (“yes–yes” agreement across all nine provinces). It does not account for the type of interprovincial agreement that occurs in cases where a drug is listed on none of the formularies studied (“no–no” agreement across all nine provinces). As such, our simple agreement statistic is biased towards understating the total level of agreement across formularies studied.

Results

According to IMS Health data, 1,069 unique types of prescription drug – representing 6,140 different products by dose and brand – were sold in Canadian retail pharmacies in 2006. We excluded 97 of these drugs from our study because they had less than $1,000 in national sales. We also excluded 176 drugs used to treat conditions that receive disease-specific coverage in one or more provinces. Nationally, $2.3 billion was spent in 2006 on the drugs excluded from our study. We provide coverage statistics for the 176 specialized drugs excluded from this study in an appendix of detailed results.

After exclusions, our analysis focused on 796 drugs that accounted for $16.5 billion in national retail expenditures in 2006 (88% of all retail expenditure on prescription drugs that year). Table 1 lists expenditure-weighted and unweighted percentages of these drugs that were listed on each of the nine provincial formularies studied. Provinces provided unrestricted coverage for between 45% (Ontario) and 63% (Alberta) of all 796 drugs evaluated in this study. Weighted by national expenditure, unrestricted listings ranged from 61% (British Columbia) to 86% (Quebec). The rightmost column of Table 1 shows that approximately 30% of the 796 evaluated drugs received unrestricted coverage on all nine formularies studied. With expenditure weights, the drugs receiving unrestricted coverage on all nine formularies represented 49% of the $16.5 billion spent on all evaluated medicines during 2006.

TABLE 1.

Weighted and unweighted percentage of drugs listed on nine provincial formularies in Canada, excluding special program drugs, 2006

	BC	AB	SK	MB	ON	QC	NB	NS	NL	All 9
Unrestricted: unweighted (796 drugs)	60	63	49	59	45	62	52	58	62	30
Unrestricted: weighted ($16.5 billion)	61	84	66	71	68	86	62	66	64	49
Restricted: unweighted (796 drugs)	10	7	15	9	10	6	10	11	11	<1
Restricted: weighted ($16.5 billion)	25	8	27	21	19	7	28	25	28	<1
Total listings: unweighted (796 drugs)	69	69	64	68	55	67	62	70	73	41
Total listings: weighted ($16.5 billion)	86	92	93	92	87	93	90	91	92	77

Notes:

Percentages may not add up due to rounding.

Weights are based on 2006 national retail sales data from IMS Health. “Listed” combines restricted and unrestricted coverage. “Special program drugs” include drugs primarily indicated to treat cancer, cystic fibrosis, HIV/AIDS, tuberculosis, multiple sclerosis, Gaucher's disease, pulmonary hypertension and thalassemia, as well as erythropoiesis-stimulating agents and human growth hormone.

Provinces studied: British Columbia (BC), Alberta (AB), Saskatchewan (SK), Manitoba (MB), Ontario (ON), Quebec (QC), New Brunswick (NB), Nova Scotia (NS) and Newfoundland and Labrador (NL).

Authors' calculations based on national retail sales data from IMS Health, Canada Inc. and provincial formulary listings from the Canadian Institute for Health Information and Conseil du médicament.

All provinces applied restrictions on some drug listings, though the percentage of evaluated drugs to which restrictions were applied varied from 6% (in Quebec) to 15% (in Saskatchewan). Restrictions were generally placed on drugs with relatively high national sales volumes. Therefore, when expenditure weights are applied to decisions, five provincial formularies (those of British Columbia, Saskatchewan, New Brunswick, Nova Scotia and Newfoundland and Labrador) placed restrictions on listings for drugs accounting for 25% or more of the expenditure-weighted market. Despite use of restricted listings by all individual provinces, less than 1% of the drugs studied received restricted listings on all nine formularies.

When we combined restricted and unrestricted listings together, the percentage of the 796 analyzed drugs that were listed on provincial formularies ranged from 55% (in Ontario) to 73% (in Newfoundland and Labrador). Weighted by national expenditures, the drugs listed on each formulary accounted for at least 86% of the market. Furthermore, 41% of the 796 drugs analyzed were listed on all nine formularies studied. With expenditure weighting, the drugs listed on all nine formularies accounted for 77% of the $16.5 billion spent during 2006 on drugs analyzed in this study.

Extent of formulary agreement

Table 2 provides the ranges of bilateral (province-to-province) measures of formulary agreement that we found when assessing coverage for all 796 drugs evaluated in this study. We computed two types of such measures: Kappa statistics and rates of agreement to cover. (We provide complete sets of bilateral agreement statistics in an appendix of detailed results.)

TABLE 2.

Summary of weighted and unweighted coverage-agreement measures for all pairs of the nine provincial formularies studied, excluding special program drugs, 2006

	Rate of agreement to cover (%)			Kappa statistics
	Min	Med	Max	Min	Med	Max
Unrestricted listings of 796 drugs	37	47	54	0.46	0.60	0.74
Unrestricted listings, weighted by $16.5 billion	54	60	78	0.32	0.55	0.81
Restricted listings of 796 drugs	2	4	8	0.15	0.36	0.70
Restricted listings, weighted by $16.5 billion	2	13	22	0.11	0.46	0.78
Total listings of 796 drugs	49	59	65	0.49	0.64	0.72
Total listings, weighted by $16.5 billion	80	89	92	0.42	0.65	0.87

Notes: Weights are based on 2006 national retail sales data from IMS Health. “Listed” combines restricted and unrestricted coverage. “Special program drugs” include drugs primarily indicated to treat cancer, cystic fibrosis, HIV/AIDS, tuberculosis, multiple sclerosis, Gaucher's disease, pulmonary hypertension and thalassemia, as well as erythropoiesis-stimulating agents and human growth hormone.

Provinces studied: British Columbia (BC), Alberta (AB), Saskatchewan (SK), Manitoba (MB), Ontario (ON), Quebec (QC), New Brunswick (NB), Nova Scotia (NS) and Newfoundland and Labrador (NL).

Authors' calculations based on national retail sales data from IMS Health, Canada Inc. and provincial formulary listings from the Canadian Institute for Health Information and Conseil du médicament.

The simple rates of bilateral agreement to provide unrestricted coverage for the 796 drugs evaluated in this study ranged from 37% to 54% across all 36 possible pairs of the provinces studied. With expenditure weighting, these rates of agreement increased to between 54% and 78%. The Kappa statistics summarizing bilateral agreements on unrestricted coverage ranged from 0.46 to 0.70 for unweighted measures and from 0.32 to 0.81 for decisions weighted by national expenditure.

Reflecting the relatively infrequent use of coverage restrictions, rates of bilateral agreement on restricted formulary listings were low. The simple measure of agreement to restrict coverage ranged from 2% to 8% of the 796 drugs analyzed, and from 2% to 22% with expenditure weights. Kappa statistics summarizing bilateral agreement on restricted coverage ranged from 0.15 to 0.70 for unweighted measures and from 0.11 to 0.78 for decisions weighted by national expenditure.

When we combined restricted and unrestricted listings into a single “listed” status, rates of bilateral formulary agreement increased markedly. Rates of bilateral agreement to list (with or without restrictions) each of the 796 drugs analyzed ranged from 49% to 65% across pairs of provinces studied. Weighted by national expenditures, these rates of bilateral agreement represented between 80% and 92% of national sales on all analyzed drugs during 2006. The Kappa statistics summarizing agreement on formulary listings ranged from 0.49 to 0.72 for unweighted measures and from 0.42 to 0.87 for expenditure-weighted measures.

Coverage by therapeutic category

The leading 22 therapeutic categories accounted for $13.8 billion in Canadian retail expenditure in 2006 (84% of spending on all drugs included in this study). Table 3 and Table 4, respectively, summarize unweighted and weighted shares of drugs from each of these categories that were listed under each provincial formulary. Without expenditure weights (Table 3), the share of category-specific drugs listed on each provincial formulary was relatively high in the largest of these therapeutic categories: for example, antihypertensives, statins, acid-reducing drugs, antidepressants, respiratory drugs, antipsychotics, NSAIDs and anxiolytics. When we applied expenditure weights (Table 4), the measured level of coverage increased. With expenditure weights each of the nine provincial formularies studied here listed drugs accounting for at least 75% of national expenditure on drugs within 16 of the 22 leading therapeutic categories.

TABLE 3.

Unweighted percentage of drugs listed (with or without restrictions) on nine provincial formularies, by therapeutic category, excluding special program drugs, 2006

Category (total number of related drugs)	BC	AB	SK	MB	ON	QC	NB	NS	NL
Antihypertensives (67 drugs)	91	91	94	96	88	88	93	93	93
Statins (8 drugs)	75	75	88	75	88	75	88	100	88
Acid related (12 drugs)	100	92	100	92	92	100	75	92	67
Antidepressants (21 drugs)	86	86	81	86	81	86	86	86	86
Respiratory (26 drugs)	73	77	77	77	69	77	69	77	88
Antibiotics (47 drugs)	74	68	62	70	60	83	70	79	83
Antipsychotics (22 drugs)	100	100	95	91	77	100	86	86	95
Diabetes – oral (13 drugs)	54	62	69	69	31	69	54	54	62
Opioids (11 drugs)	82	73	73	64	55	73	45	73	82
NSAIDs (21 drugs)	76	81	76	86	62	76	62	81	81
Anti-epileptics (15 drugs)	87	87	93	93	73	93	87	93	93
Bisphosphonates (9 drugs)	67	78	67	78	78	78	56	67	44
Hormonal contraceptives (13 drugs)	77	77	77	77	69	77	69	77	77
Sex hormones – other (22 drugs)	55	55	55	50	45	59	36	55	73
Diabetes – Insulin (11 drugs)	82	73	64	73	55	73	64	64	64
BPH (5 drugs)	60	100	100	100	100	100	80	60	80
Antimigraine (10 drugs)	90	90	80	80	30	80	80	90	90
Anti-dementia (3 drugs)	0	100	100	100	100	100	100	100	0
Anxiolytics (10 drugs)	90	100	100	100	80	90	100	100	100
Erectile dysfunction (5 drugs)	20	0	0	0	0	0	0	0	20
Thyroid (3 drugs)	100	100	67	100	67	67	100	100	100
Sedatives (12 drugs)	83	67	42	58	42	50	50	42	83

Notes: “Special program drugs” include drugs primarily indicated to treat cancer, cystic fibrosis, HIV/AIDS, tuberculosis, multiple sclerosis, Gaucher's disease, pulmonary hypertension and thalassemia, as well as erythropoiesis-stimulating agents and human growth hormone.

Provinces studied: British Columbia (BC), Alberta (AB), Saskatchewan (SK), Manitoba (MB), Ontario (ON), New Brunswick (NB), Nova Scotia (NS) and Newfoundland and Labrador (NL); data on formulary listings for Quebec (QC).

Authors' calculations based on national retail sales data from IMS Health, Canada Inc. and provincial formulary listings from the Canadian Institute for Health Information and Conseil du médicament.

TABLE 4.

Expenditure-weighted percentage of drugs listed (with or without restrictions) on nine provincial formularies, by therapeutic category, excluding special program drugs, 2006

Category (total spending on related drugs)	BC	AB	SK	MB	ON	QC	NB	NS	NL
Antihypertensives ($3.1 billion)	84	99	100	100	99	99	100	99	100
Statins ($1.9 billion)	100	100	100	100	100	100	100	100	100
Acid related ($1.4 billion)	100	85	100	85	85	100	84	85	84
Antidepressants ($1.1 billion)	96	96	96	96	96	96	96	96	96
Respiratory ($0.9 billion)	82	100	100	100	99	100	95	100	100
Antibiotics ($0.7 billion)	73	87	86	86	81	87	81	87	81
Antipsychotics ($0.7 billion)	100	100	100	100	92	100	99	93	99
Diabetes – oral ($0.5 billion)	84	86	84	87	78	90	81	87	86
Opioids ($0.5 billion)	97	97	95	95	95	95	94	97	97
NSAIDS ($0.4 billion)	97	89	97	99	96	96	96	100	100
Anti-epileptics ($0.4 billion)	89	89	91	91	85	91	78	91	91
Bisphosphonates ($0.3 billion)	99	100	99	100	99	100	98	99	97
Hormonal contraceptives ($0.3 billion)	81	81	81	81	76	81	77	81	81
Sex hormones – other ($0.3 billion)	66	81	80	80	70	89	55	70	91
Diabetes – Insulin ($0.2 billion)	84	84	82	83	83	84	82	82	82
BPH ($0.2 billion)	45	100	100	100	100	100	90	45	90
Antimigraine ($0.2 billion)	96	96	95	92	2	95	76	96	96
Anti-dementia ($0.2 billion)	0	100	100	100	100	100	100	100	0
Anxiolytics ($0.2 billion)	94	100	100	100	89	99	100	100	100
Erectile dysfunction ($0.1 billion)	0	0	0	0	0	0	0	0	0
Thyroid ($0.1 billion)	100	100	98	100	98	100	100	100	100
Sedatives ($0.1 billion)	96	97	23	96	23	24	96	90	96

Notes: Weights are based on 2006 national retail sales data from IMS Health. “Listed” combines restricted and unrestricted coverage. “Special program drugs” include drugs primarily indicated to treat cancer, cystic fibrosis, HIV/AIDS, tuberculosis, multiple sclerosis, Gaucher's disease, pulmonary hypertension and thalassemia, as well as erythropoiesis-stimulating agents and human growth hormone.

Provinces studied: British Columbia (BC), Alberta (AB), Saskatchewan (SK), Manitoba (MB), Ontario (ON), Quebec (QC), New Brunswick (NB), Nova Scotia (NS) and Newfoundland and Labrador (NL);

Authors' calculations based on national retail sales data from IMS Health, Canada Inc. and provincial formulary listings from the Canadian Institute for Health Information and Conseil du médicament.

Table 5 summarizes the rates of nine provincial coverage agreements within these 22 leading therapeutic categories. The rate at which all nine provinces provided unrestricted coverage for the same drugs within the leading therapeutic categories ranged from 0% of drugs in the anti-dementia and erectile dysfunction categories to 81% of drugs in the antidepressants category. Weighting by national expenditure did not systematically alter the measure of nine-province agreement to provide unrestricted coverage in the leading therapeutic categories: in 13 categories the expenditure weights increased the measure of agreement, and in nine categories weighting decreased the measure of agreement.

TABLE 5.

Weighted and unweighted percentage of drugs listed on all nine provincial formularies studied, by therapeutic category, excluding special program drugs, 2006

	Unrestricted listings		Restricted listings		Total listings
Category (related drugs and expenditure)	Unweighted	Weighted	Unweighted	Weighted	Unweighted	Weighted
Antihypertensives (67 drugs, $3.1)	58	60	0	0	79	82
Statins (8 drugs, $1.9)	75	100	0	0	75	100
Acid related (12 drugs, $1.4)	33	9	0	0	67	84
Antidepressants (21 drugs, $1.1)	81	96	0	0	81	96
Respiratory (26 drugs, $0.9)	35	40	0	0	54	78
Antibiotics (47 drugs, $0.7)	21	19	2	1	38	67
Antipsychotics (22 drugs, $0.7)	68	49	0	0	73	92
Diabetes – oral (13 drugs, $0.5)	8	33	0	0	31	78
Opioids (11 drugs, $0.5)	27	40	0	0	45	94
NSAIDs (21 drugs, $0.4)	29	31	0	0	52	85
Anti-epileptics (15 drugs, $0.4)	47	37	0	0	67	72
Bisphosphonates (9 drugs, $0.3)	11	12	0	0	44	97
Hormonal contraceptives (13 drugs, $0.3)	62	72	0	0	62	72
Sex hormones – other (22 drugs, $0.3)	18	43	0	0	32	54
Diabetes – Insulin (11 drugs, $0.2)	27	56	0	0	45	81
BPH (5 drugs, $0.2)	20	15	0	0	60	45
Antimigraine (10 drugs, $0.2)	10	1	0	0	20	1
Anti-dementia (3 drugs, $0.2)	0	0	0	0	0	0
Anxiolytics (10 drugs, $0.2)	60	85	0	0	70	89
Erectile dysfunction (5 drugs, $0.1)	0	0	0	0	0	0
Thyroid (3 drugs, $0.1)	33	98	0	0	33	98
Sedatives (12 drugs, $0.1)	17	16	0	0	17	16

Notes: Weights are based on 2006 national retail sales data from IMS Health. “Listed” combines restricted and unrestricted coverage. “Special program drugs” include drugs primarily indicated to treat cancer, cystic fibrosis, HIV/AIDS, tuberculosis, multiple sclerosis, Gaucher's disease, pulmonary hypertension and thalassemia, as well as erythropoiesis-stimulating agents and human growth hormone.

Provinces studied: British Columbia (BC), Alberta (AB), Saskatchewan (SK), Manitoba (MB), Ontario (ON), Quebec (QC), New Brunswick (NB), Nova Scotia (NS) and Newfoundland and Labrador (NL).

Authors' calculations based on national retail sales data from IMS Health, Canada Inc. and provincial formulary listings from the Canadian Institute for Health Information and Conseil du médicament.

Simultaneous uses of restricted listings across all nine provinces were very rare: only in the class of antibiotics were any drugs listed with restrictions on all formularies studied. Expenditure weighting therefore had little effect on this measure of overall agreement to provide restricted coverage.

When we combined restricted and unrestricted coverage into a single “listed” category, the rates of all-formulary agreement increased for most of the leading therapeutic categories. Nevertheless, there were no categories in which all drugs were listed by all provinces. Indeed, in only three therapeutic categories were 75% or more of drugs listed on all nine formularies. Those categories were antihypertensives, statins and antidepressants. When weighted by expenditure, however, the drugs listed on all nine formularies accounted for at least 75% of national expenditure within 13 of the 22 leading therapeutic categories.

Drugs listed on all nine provincial formularies accounted for less than 50% of relevant markets (with and without expenditure weights) in four of the 22 categories: anti-migraine drugs, anti-dementia drugs, drugs for erectile dysfunction and sedatives. In the case of drugs for erectile dysfunction, none of the provinces studied listed the leading drug products as benefits under their general pharmacare formularies.

Coverage for drugs reviewed by the Common Drug Review

By the end of 2006, the CDR's Canadian Expert Drug Advisory Committee (CEDAC) had made coverage recommendations regarding 55 drugs. Many of these drugs were for treatment of conditions for which one or more provinces ran specialized healthcare programs. Table 6 lists rates of province-specific and all-province coverage for the 29 drugs with CEDAC recommendations that are not likely to be covered under those special programs. The Table is stratified into three types of CEDAC recommendations: an “unconditional” recommendation to cover; a “conditional” recommendation, which specifies certain criteria that should be met or states that the drug should be covered in a similar manner to other drugs in class; and a “do not cover” recommendation.

TABLE 6.

Weighted and unweighted percentage of drugs listed on nine provincial formularies in Canada, drugs reviewed by the Common Drug Review (CDR), excluding special program drugs, 2006

CDR recommended: Unconditional	BC	AB	SK	MB	ON	QC	NB	NS	NL	All
Unrestricted: unweighted (3 drugs)	33	33	33	33	0	33	67	67	100	0
Unrestricted: weighted ($17.5 million)	99	99	99	99	0	99	99	99	100	0
Restricted: unweighted (3 drugs)	0	0	0	0	0	0	0	0	0	0
Restricted: weighted ($17.5 million)	0	0	0	0	0	0	0	0	0	0
Total listing: unweighted (3 drugs)	33	33	33	33	0	33	67	67	100	0
Total listing: weighted ($17.5 million)	99	99	99	99	0	99	99	99	100	0
CDR recommended:Conditional	BC	AB	SK	MB	ON	QC	NB	NS	NL	All
Unrestricted: unweighted (10 drugs)	20	30	40	20	20	50	20	30	10	0
Unrestricted: weighted ($419.3 million)	11	94	13	10	1	98	10	11	10	0
Restricted: unweighted (10 drugs)	40	30	40	30	50	30	50	60	40	0
Restricted: weighted ($419.3 million)	87	4	86	86	97	1	88	88	88	0
Total listing: unweighted (10 drugs)	60	60	80	50	70	80	70	90	50	40
Total listing: weighted ($419.3 million)	99	99	99	96	97	99	99	99	98	96
CDR recommended: Do not cover	BC	AB	SK	MB	ON	QC	NB	NS	NL	All
Unrestricted: unweighted (16 drugs)	0	0	0	0	6	0	0	0	0	0
Unrestricted: weighted ($143.7 million)	0	0	0	0	1	0	0	0	0	0
Restricted: unweighted (16 drugs)	0	0	0	0	0	0	0	0	0	0
Restricted: weighted ($143.7 million)	0	0	0	0	0	0	0	0	0	0
Total listing: unweighted (16 drugs)	0	0	0	0	6	0	0	0	0	0
Total listing: weighted ($143.7 million)	0	0	0	0	1	0	0	0	0	0

Notes:

Percentages may not add up due to rounding.

Weights are based on 2006 national retail sales data from IMS Health. “Listed” combines restricted and unrestricted coverage. “Special program drugs” include drugs primarily indicated to treat cancer, cystic fibrosis, HIV/AIDS, tuberculosis, multiple sclerosis, Gaucher's disease, pulmonary hypertension and thalassemia, as well as erythropoiesis-stimulating agents and human growth hormone.

Provinces studied: British Columbia (BC), Alberta (AB), Saskatchewan (SK), Manitoba (MB), Ontario (ON), Quebec (QC), New Brunswick (NB), Nova Scotia (NS) and Newfoundland and Labrador (NL).

Authors' calculations based on national retail sales data from IMS Health, Canada Inc. and provincial formulary listings from the Canadian Institute for Health Information and Conseil du médicament.

By December 2006, all formularies studied but that of Ontario provided unrestricted coverage for at least one of the three drugs that had received an unconditional recommendation for coverage from CEDAC. With expenditure weights, the product covered on these eight formularies accounted for virtually all of the $17.5 million spent on all three drugs given an unconditional recommendation by CEDAC.

Provinces used a mix of restricted and unrestricted coverage for drugs that had received a conditional recommendation from CEDAC. While none of the 10 drugs that received a conditional CEDAC recommendation were given the same type of coverage under the nine formularies studied here, four of them were listed (with or without restrictions) on all nine formularies. When weighted by national expenditure, those four drugs accounted for 96% of all retail spending on all 10 drugs that had received a conditional recommendation from CEDAC.

Finally, only the Ontario formulary listed any of the 16 drugs that had received a “do not cover” recommendation from CEDAC. The Ontario formulary listed one such product, representing an expenditure-weighted share of just 1% of national retail expenditure on the 16 drugs that had received a “do not cover” recommendation from CEDAC.

Discussion

At first glance, drug coverage under general pharmacare formularies documented in this study appears to be somewhat low and variable across provinces, a finding that would corroborate results of previous studies (Anis et al. 2001; Gregoire et al. 2001; MacDonald and Potvin 2004). Two key findings would support this initial appraisal. First, individual provinces listed only two-thirds of the prescription drugs we studied. Even within therapeutic categories, there were few formularies that provided full coverage for all drugs. Second, many drugs listed on some provincial formularies were not listed on others. Kappa statistics, a standard measure of inter-rater reliability that has often been used to study formulary agreement, indicate low to moderate concordance between most pairs of provincial formularies studied here.

Initial conclusions based on standard measures of formulary coverage and agreement may be misleading because they rest on two key underlying assumptions: (1) that all coverage decisions are equal, and (2) that there are no legitimate sources of coverage variation. We believe that not all drug coverage decisions are equal and that there may be some legitimate sources of coverage variation in Canada.

When weighted by national expenditures, the drugs listed on the formulary of any province studied here accounted for an expenditure-weighted share of at least 86% of the 2006 Canadian market for all analyzed medicines. The drugs listed on all nine formularies studied accounted for an expenditure-weighted share of 77% of the market. These expenditure-weighted results suggest that general pharmacare programs in Canadian provinces offer coverage that may be more consistent than is commonly assumed.

When coverage was assessed within leading therapeutic categories, our results begin to suggest some of the sources of observed variation. Individual provinces failed to list (with or without restrictions) at least 25% of the drugs available in nearly half (sometimes more) of the 22 leading therapeutic categories (see Table 3). In only two of the 22 leading therapeutic categories (antihypertensives and statins) were 75% or more of the available drugs listed on all nine formularies studied (see Table 5). But those statistics pertain only to the specific sets of products listed by each province. When formulary listings were weighted by national expenditures, all provinces individually listed drugs accounting for expenditure-weighted shares of 75% or more of category-specific markets in 16 of the 22 leading therapeutic categories, including the 13 largest categories by national expenditure (see Table 4). This finding implies that coverage variations observed in leading therapeutic categories are related to interprovincial differences in the selection of therapeutic options around the margins of a core group of universally listed products representing a significant majority of sales in the relevant therapeutic submarkets. This result supports what CIHI (2005) alluded to as a “common core of medications available to those with coverage” in their unweighted analysis of provincial formularies. Variations around such a core are not likely to be as important from a perspective of national harmonization as variations within the core.

Other category-specific results suggest further sources of observed variation. For example, the limited and variable formulary listings for drugs to treat dementia and erectile dysfunction may have resulted from differences in the provinces' interpretation of clinical data, their health system priorities or both. In the case of drugs for dementia, many provinces appear to have decided that evidence available as of 2006 warranted coverage for these medications only under specific circumstances (e.g., seven of nine provinces studied list these medicines only with restrictions). In British Columbia, one of the provinces that did not list the leading dementia treatments, the government has explicitly stated that its experts deemed the evidence available as of 2006 insufficient to warrant public coverage under its pharmacare program (BC 2007). In the case of erectile dysfunction drugs, provinces appear to have determined that the newer drugs in this category are not a priority for general pharmacare programs. Analyzing coverage at the therapeutic category level therefore provides some indication that, for at least some decisions, variation in coverage decisions may stem from potentially legitimate differences in the interpretation of clinical data and economic costs against regional health system priorities and budget constraints.

Finally, the analysis of coverage for drugs assessed by the CDR provides some insight into the benefits of clear evidence of an objective “truth” that would separate cases that should be appraised positively versus negatively. The CDR creates a common evidence base for provincial formularies and appears to have begun to create harmony in “general” pharmacare listings. The 16 non-specialized drugs that had been reviewed and rejected by the CDR's CEDAC were almost universally rejected by the provinces studied here (see Table 6). These CEDAC recommendations and decisions by CDR-participating provinces appear to be corroborated by Quebec's independent assessment and rejection of the same 16 drugs. By December 2006, most provinces studied here had listed the major drugs (in expenditure terms) for which CEDAC had given conditional or unconditional recommendations for coverage. Further analysis of the particular cases for which coverage had not been granted by December 2006 is warranted, but the results presented in Table 6 suggest that there is relatively good agreement in listing decisions for drugs with a reasonably objective and independent signal of what those decisions should be.

Limitations

An important limitation of our study is the lack of information regarding drug coverage under public programs that provide for specific healthcare needs (e.g., cancer, HIV, multiple sclerosis, etc.). As with other researcher groups that have studied drugs listed under provincial pharmacare programs in Canada, we did not have the time and resources necessary to collect drug-specific coverage information from the many condition-specific healthcare programs offered by the nine provinces studied here. We chose to exclude such drugs from our analysis because one cannot know to what extent the general pharmacare listings for such drugs reflect actual differences in coverage or variations in the extent to which provinces list coverage for such drugs under their general pharmacare formularies. Our analysis therefore pertains only to coverage breadth, depth and agreement among the general pharmacare programs in provinces studied. Interested readers may find the general pharmacare coverage statistics for the specialized drugs that were excluded from the analysis above in an appendix of detailed results. We strongly urge caution when interpreting the appended findings and recommend that future research be specifically focussed on and tailored to the analysis of such programs, as has been done for coverage under provincial cancer programs by Menon and colleagues (2005).

Another limitation of this study stems from our methods for illustrating formulary depth – the relative financial “importance” of drugs listed. Expenditure weights are an imperfect measure of relative importance in terms of population health and health-related equity considerations. The ideal measure would be one that provided a sound scientific estimate of the value of a drug (or choices of related drugs) to the health of Canadian patients and populations. International lists, such as the WHO Essential Medicines List, also provide some indication of relative importance of available drugs; however, those lists are developed with consideration given to populations with very different health needs, health systems and financial constraints compared to Canada. We believe that, as increasing medicines are reviewed through the CDR, CEDAC recommendations may serve as a basis for assessing relative importance for Canadian patients and populations; results in Table 6 indicate the promise of such an analysis. In the interim, however, using expenditure weights is a first step towards presenting a more complete picture of provincial formulary coverage and variations.

Finally, this study is limited by definitions of therapeutic categories. The starting point for our determination of therapeutic categories was the WHO ATC classification system. Drugs within some classes, such as anti-epileptics or antihypertensives, have multiple indications. For analytical purposes, we had to group such drugs as best as possible into logically consistent (and mutually exclusive) categories.

Conclusion

Addressing formulary variations requires clarity about their nature and extent and an understanding of how harmonization might be achieved. Our research indicates that previously reported findings present a valid but incomplete portrait of drug coverage offered by general pharmacare plans. While it is true that there is coverage disagreement across provinces – particularly as measured by conventional indices such as the Kappa statistic – our findings suggest that there is a high level of agreement on coverage of high-volume medicines within and across leading therapeutic categories. While variations in coverage for specific drug classes and drug products remain important areas for investigation and policy consideration, our research shows that Canada is currently operating with a significant “implicit national formulary” by way of the fact that provincial formularies independently yet mutually list most of the top-selling medicines in the marketplace. We believe that this implicit national formulary can be used to help resolve areas of formulary disagreement. Even with ideal clinical data, one source of legitimate variation in drug coverage stems from differing resource constraints (Gafni and Birch 2003; Birch and Gafni 2004; Morgan et al. 2006). Some provinces may simply be unable to afford covering all the drugs they would like to cover.

Savings within select segments of the drug budget are one way to alleviate constraints on others. Thus, a route to national standards for drug coverage (for general pharmacare programs, at least) might be found by recognizing the potential for a national formulary to secure savings in the already-common segments of provincial formularies. As is routinely done with hospital formularies, community-based formularies might be used to consolidate purchasing power within and across provinces. It has been estimated that the savings from such formulary-based price negotiations in leading drug categories could be on the order of 50% of Canadian drug costs (Morgan et al. 2007). Thus, consider joining provincial formularies for, say, the 13 leading therapeutic categories with the express purpose of creating a national drug-purchasing strategy for such medicines. Those 13 classes represent over $12 billion in annual spending in Canada; all of the nine provinces studied already cover the drugs, accounting for most of that expenditure. If 50% or even 20% could be saved through a national formulary for such drugs, the $1 billion to $3 billion in funds freed from public drug budgets (about half the total potential savings) could go a long way towards harmonized coverage in therapeutic areas where significant regional variations exist.