Public reporting of antipsychotic prescribing in nursing homes: population-based interrupted time series analyses

Abstract

Background

Although sometimes appropriate, antipsychotic medications are associated with increased risk of significant adverse events. In 2014, a series of newspaper articles describing high prescribing rates in nursing homes in Ontario, Canada, garnered substantial interest. Subsequently, an online public reporting initiative with home-level data was launched. We examined the impact of these public reporting interventions on antipsychotic prescribing in nursing homes.

Methods

Time series analysis of all nursing home residents in Ontario, Canada, between 1 October 2013 and 31 March 2016. The primary outcome was the proportion of residents prescribed antipsychotics each month. Balance measures were prescriptions for common alternative sedating agents (benzodiazepines and/or trazodone). We used segmented regression to assess the effects on prescription trends of the newspaper articles and the online home-level public reporting initiative.

Results

We included 120 009 nursing home resident admissions across 636 nursing homes. Following the newspaper articles, the proportion of residents prescribed an antipsychotic decreased by 1.28% (95% CI 1.08% to 1.48%) and continued to decrease at a rate of 0.2% per month (95% CI 0.16% to 0.24%). The online public reporting initiative did not alter this trend. Over 3 years, there was a net absolute reduction in antipsychotic prescribing of 6.0% (95% CI 5.1% to 6.9%). Trends for benzodiazepine prescribing did not change as substantially during the period of observation. Trazodone use has been gradually increasing, but its use did not change abruptly at the time of the mass media report or the public reporting initiative.

Interpretation

The rapid impact of mass media on prescribing suggests both an opportunity to use this approach to invoke change and a warning to ensure that such reporting occurs responsibly.

Background

Antipsychotic medications are commonly prescribed to residents in nursing homes,¹ despite research indicating an association with serious adverse events including falls, stroke and mortality.^2–5 Guidelines recommend use of non-pharmacological options as first line for management of neuropsychiatric symptoms of dementia and provide algorithms to pursue deprescribing of antipsychotics when possible.^{6 7} High prescribing rates for potentially inappropriate medications in general⁸ and antipsychotic medications in particular⁹ have been a frequent focus of quality improvement initiatives in nursing homes. These initiatives are not reliably effective, and evaluations often highlight variation across homes in both prescribing rates and responsiveness to quality improvement interventions.^10–13

Public reporting interventions have emerged as a common approach to spur changes in clinical practice;¹⁴ such interventions can have substantial (intended and unintended) effects.¹⁵ For example, one study identified an increase in the use of antipsychotic medications in nursing homes after the initiation of public reporting regarding physical restraints.¹⁶ When public reporting is associated with mass media coverage,¹⁵ the health of entire populations may be affected, as appeared to be the case with media coverage on use of statins¹⁷ in the UK. In other instances though, public reporting interventions have had little or no impact.^{18 19} Best practices in public reporting interventions are emerging, but much remains uncertain.²⁰

Recently, a series of newspaper articles^21–23 criticised the nursing home sector in the province of Ontario for relatively high rates of antipsychotic prescribing. This was accompanied by heightened involvement of politicians in this issue,²⁴ and 1 year later, a national organisation initiated an online public reporting tool with home-level data.²⁵ We sought to assess the impact of these public reporting interventions on antipsychotic prescribing in nursing homes in the province of Ontario.

Methods

Study design and setting

We conducted an interrupted time series study on population-based data from Ontario, Canada, using administrative databases linked using encoded identifiers and analysed at the Institute for Clinical Evaluative Sciences (ICES). The majority of the cost of care in nursing homes is covered by the provincial health system and access is centrally managed. Prescription medications from a broad formulary, including the medications considered in this study (see the Outcomes section), are covered through the Ontario Drug Benefit (ODB) programme.

Data sources

The databases at ICES include information on all hospital and nursing home admissions in the province, all visits to emergency departments, physician billing claims, vital statistics, as well as prescription data for those covered under the provincial health insurance programme.^26–30 The nursing home sector in Ontario is heavily regulated to meet specific standards related to staffing and services. Ontario provides all nursing home residents with universal access to medically necessary physician services, ambulatory and hospital care.

Cohort definition

The Continuing Care Reporting System Resident Assessment Instrument (CCRS-RAI) provides detailed clinical information on all residents of Ontario admitted to a nursing home³¹ and was used to identify nursing home residents aged 66–104 who were admitted for at least 30 days and who could be linked to a most responsible physician (ie, prescriber) between 1 October 2013 and 31 March 2016. Residents entered the cohort on their second calendar month in a nursing home to allow for a complete transition of care. This was done to exclude those with brief admissions, to mitigate any prescriptions prescribed prior to admission and to allow for an assignment of a most responsible physician, defined using physician claims data from the month prior. Assigning a most responsible physician is necessary to explore secondary questions regarding prescriber characteristics that may be associated with change. Residents left the cohort when they were either discharged from the nursing home or died (with date of death assessed using the Registered Persons Database). If temporarily hospitalised during the study period, a resident would leave the cohort and then re-enter once they had returned to a nursing home for a time period that spanned two calendar months. Clinical characteristics of the cohort were described using data from the CCRS-RAI, with the exception of diagnoses of psychosis or dementia. Similar to previous work, to define history of psychosis, we looked back 5 years to identify admissions or physician billings for psychosis; to define dementia, we used a similar approach but also identified any dementia-specific medications in the ODB.¹¹

Public reporting interventions

In April 2014, the newspaper with the largest circulation in the province (and the country) printed an article that highlighted the relatively high rates of prescribing of antipsychotics in Ontario nursing homes and the associated risks.²¹ The government responded with a press release noting that an educational programme for nursing homes was already in development.³² This was followed by three additional features in the newspaper on the topic over the following month.^22–24 The general tone of these articles was not nuanced: ‘Ontario nursing homes are drugging helpless seniors at an alarming rate with powerful antipsychotic drugs, despite warnings that the medications can kill elderly patients suffering from dementia.’²¹ In June 2015, the Canadian Institute for Health Information (CIHI) released an online public reporting tool that described ‘potentially inappropriate use of antipsychotics’, as well as physical restraint use for each home.²⁵ The CIHI antipsychotic prescription indicator excludes end-of-life patients and those with a history of schizophrenia, Huntington’s chorea and/or delusions or hallucinations. Comparative data for the local region, the province and the country were also released, using data that were approximately 1 year old. CIHI is an independent, not-for-profit organisation funded by the federal and provincial governments to provide information on Canada’s health systems and the health of Canadians. The CIHI reports are free and currently available at: yourhealthsystem.ca.²⁵

Outcomes

We determined the proportion of residents with a prescription dispensed for any antipsychotic medication in a given month. As two balancing measures, we also determined the proportions of residents with a prescription to a benzodiazepine and/or trazodone. Balancing measures are outcomes identified specifically to determine whether changes designed to improve one set of outcomes (eg, prescribing of antipsychotics) are causing new problems in other outcomes (eg, alternative sedating agents). Among residents for whom antipsychotics were used for sedation, it is plausible that alternative sedating agents such as benzodiazepines or trazodone might be used instead of antipsychotics. The relationship in Ontario between prevalence of prescribing for antipsychotics, benzodiazepine and trazodone in the elderly has been previously shown.³³ To support causal inference, we also assessed a non-equivalent control, the proportion of residents with a prescription for a statin. Non-equivalent controls (also called negative controls) are outcomes that are not expected to change as a result of an intervention, but expected to respond to similar internal validity threats as the target outcome. Statins would be unlikely to have been affected by the public reporting interventions described above (because they were not featured in the reports), but may have shown a change if other factors were responsible for influencing prescribing patterns in nursing homes at the same time as the publication (ie, other initiatives to reduce polypharmacy). The ODB database includes complete dispensing information on these medication outcomes for all Ontarians living in nursing homes, and has been previously validated³⁰ for research purposes. We did not consider clinical diagnoses because we wanted to observe population-level rates.

Analysis

We used segmented autoregression analysis of the monthly proportions to determine whether there was a change in antipsychotic prescribing coinciding with the publication of the newspaper article, accompanied by no change in the equivalent controls. The segmented regression model included coefficients representing the preintervention level and slope, and interruptions in the level and slope corresponding with the timing of the article and separate interruptions in the level and slope corresponding with the timing of the public report. We hypothesised that there would be a 4–6 week delay from the time of publication of the article on 15 April 2014, before its effect can be fully observed since non-urgent medication changes may take a few weeks to arrange in this setting. In Ontario nursing homes, routine drug dispensing from the pharmacy typically occurs every 1–2 weeks and some patients are not reassessed weekly by the prescriber. To account for this delay, the variable was determined, a priori, to be coded 0 until April 2014; 0.5 in May 2014; and 1 from June 2014 onwards. A similar assumption was made for the public report, that is, coded 0 until June 2015; 0.5 in July 2015; and 1 from August 2015 onwards. We used Durbin-Watson statistics to test for the presence of autocorrelation between values of the series as a function of the time lag. We introduced lags of up to 6 months into the model and used backward elimination of the higher lags that were not significant. We examined goodness of fit by comparing observed versus predicted proportions using visual inspection. Proportions with any exposure at 30 months (ie, March 2016) were compared with forecasts that used the prepublication information only, that is, the predicted prescribing proportions had the article not been published.

To test our hypothesis that the change occurred in response to the newspaper article (ie, at the theoretical change point of June 2014), we fit separate models corresponding to interruptions in the 5 months before and 5 months after June 2014. For each model, we used goodness-of-fit statistics (minimum Bayesian information criterion (BIC), Akaike information criterion (AIC) and R²) to identify the interrupted month that explains the most variability in the data. The optimal change point suggested by the data was then compared with the theoretical change point. A similar approach was used to compare the theoretical change point (August 2015) for the CIHI public report versus the optimal change point suggested by the data.

For the antipsychotic prescription outcome, we explored whether changes in prescribing were greater in certain subgroups deemed a priori as plausible and potentially important effect modifiers. Specifically, we assessed two patient characteristics (sex and history of psychosis (defined as those with hospital admissions, emergency room department visits and/or physician billings with associated codes in the prior 5 years)), two physician characteristics (sex (using information from the ICES Physician Database)) and number of residents (greater or lesser than the median), and one home characteristic (public or private-for-profit ownership type (using data available at ICES that provides information regarding Ontario healthcare institutions)). Subgroup analyses were conducted as stratified analyses by fitting separate segmented autoregression models to each subgroup.

All analyses were conducted using SAS V.9.4.

Results

Table 1 describes characteristics of the residents, the physicians responsible for their care and their nursing homes. A total of 102 807 unique nursing home residents were identified during the study period, representing 120 009 resident episodes of care in 636 nursing homes in Ontario between 1 October 2013 and 31 March 2016, each followed for a median duration of 16 months (IQR: 7–28). These residents were associated with 1960 unique physicians (median of 33 residents per physician (IQR: 6–84)). Residents were primarily female (69.6%), averaging 85.5 years (SD: 7.5). Physicians were primarily male (80.1%), averaging 57.7 years (SD: 10.9). Nursing homes were primarily in urban settings (86.3%), with an average of 161 beds (SD: 80.8); 58.0% had for-profit ownership. See online supplementary appendix 1 for study flow diagram.

Table 1.

Characteristics of nursing homes, physicians and residents included in the analysis using data from the first month for each resident, Ontario, Canada (October 2013 to March 2016)

Resident-level characteristics	Total n=102 807
Female	71 598 (69.6%)
Age (mean±SD)	85.54±7.54
Charlson comorbidity score (mean±SD)	1.19±1.65
Psychosis, n (proportion)	5050 (4.9%)
Dementia, n (proportion)	88 946 (86.5%)
Neither psychosis nor dementia, n (proportion)	13 338 (13.0%)
Level of function (activities of daily living) (mean±SD)	15.79±7.37
Pain score (mean±SD)	0.49±0.73
Depression rating score (mean±SD)	1.90±2.26
Aggressive behaviour scale (mean±SD)	1.31±2.12
Physician characteristics (n=1960)
Residents with female physician, n (proportion)	20 436 (19.9%)
Age (mean±SD)	57.70±10.86
Years in practice (mean±SD)	31.38±11.60
Nursing home characteristics (n=636)
Home size (beds) (mean±SD)	161.42±80.80
Residents in rural home, n (proportion)	14 071 (13.7%)
Residents in home with for-profit ownership, n (proportion)	59 629 (58.0%)

Trends in antipsychotic prescribing

Figure 1A depicts the proportion of residents receiving prescriptions for antipsychotics, statins, benzodiazepines and trazodone each month between October 2013 and March 2016. In the 6 months prior to the newspaper articles, the proportion of residents receiving antipsychotics was stable at approximately 32.5%. In April 2014 (time of the newspaper articles), there is an inflection point in the time trend, but there is no similar marker in May 2015 (release of the online, home-level reports). Figure 1B compares the observed versus predicted trends in antipsychotic prescribing. Compared with the secular trend before publication of the newspaper article, there was a net absolute reduction in antipsychotic prescribing of 6.0% (95% CI 5.1% to 6.9%) by March 2016.

Figure 1.

(A) Proportion of residents receiving prescriptions for antipsychotics, statins, benzodiazepines and trazodone each month, Ontario, Canada (October 2013 to March 2016). Newspaper publication date was April 2014; CIHI report publication date was June 2015. (B) Observed versus predicted antipsychotic prescribing trends showing secular trend in prescribing, Ontario, Canada (October 2013 to March 2016). CIHI, Canadian Institute for Health Information.

Association of changes in prescribing with the public reporting interventions

The goodness-of-fit criteria for the candidate interruption months before and after the newspaper articles and the online, home-level reports are presented in online supplementary appendix 2. The optimal change point suggested by the data agrees with the hypothesised change point, with AIC and BIC reaching a clear minimum, and R² reaching a clear maximum in June 2014, six weeks after publication of the article. In contrast, there is little evidence for an interruption corresponding to the hypothesised change point after publication of the online, home-level reports.

Table 2 presents the results from the segmented regression analysis. For antipsychotics, we found a secular trend of 0.01% fewer antipsychotic prescriptions per month before the publication of the newspaper articles (p=0.49), followed by a statistically significant immediate drop of 1.28% (95% CI 1.08% to 1.48%) in prescriptions (p<0.0001) and an additional reduction of 0.2% fewer prescriptions per month (95% CI 0.16% to 0.24%) over and above the secular trend (p<0.0001).

Table 2.

Results of segmented regression analysis of prescribing patterns

	Estimate (%)	LCL	UCL	P values
Antipsychotic prescribing
Intercept	32.62	32.45	32.80	<0.0001
Monthly change before	−0.01	−0.05	0.03	0.4923
Immediate change after newspaper article	−1.28	−1.48	−1.08	<0.0001
Additional monthly change after newspaper article	−0.20	−0.24	−0.16	<0.0001
Immediate change after CIHI report	−0.51	−0.74	−0.29	<0.0001
Additional monthly change after CIHI report	0.06	0.02	0.09	0.002
Benzodiazepine prescribing
Intercept	14.16	13.91	14.40	<0.0001
Monthly change before	−0.06	−0.11	−0.004	0.0376
Immediate change after newspaper article	−0.45	−0.71	−0.19	0.0018
Additional monthly change after newspaper article	−0.01	−0.07	0.04	0.5989
Immediate change after CIHI report	0.10	−0.22	0.43	0.5196
Additional monthly change after CIHI report	0.03	−0.02	0.07	0.2507
Trazodone prescribing
Intercept	22.65	22.50	22.81	<0.0001
Monthly change before	0.10	0.06	0.13	<0.0001
Immediate change after newspaper article	−0.08	−0.26	0.09	0.3364
Additional monthly change after newspaper article	0.01	−0.03	0.05	0.5981
Immediate change after CIHI report	0.13	−0.07	0.32	0.1971
Additional monthly change after CIHI report	0.0003	−0.03	0.03	0.9848
Statins prescribing
Intercept	19.86	19.69	20.04	<0.0001
Monthly change before	−0.06	−0.09	−0.02	0.0048
Immediate change after newspaper article	0.02	−0.17	0.22	0.7913
Additional monthly change after newspaper article	−0.08	−0.12	−0.03	0.001
Immediate change after CIHI report	−0.22	−0.43	−0.01	0.0444
Additional monthly change after CIHI report	0.06	0.02	0.09	0.0051

CIHI, Canadian Institute for Health Information; LCL, lower control limit; UCL, upper control limit.

The statistically significant change in antipsychotic prescribing was not accompanied by any statistically significant changes in trazodone prescribing: there was a statistically significant monthly increase in prescribing before the article (p<0.0001), but no interruption in either intercept (p=0.34) or slope (p=0.6) corresponding to the timing of the article. For benzodiazepines, there was a monthly decline in prescribing even before the article was published (p=0.04), a statistically significant immediate drop of 0.45% coinciding with the article (p=0.002) but no statistically significant interruption in slope (p=0.60). For the non-equivalent control, statins, there was a small, but statistically significant monthly decline of 0.06% even before the publication of the article, no significant drop immediately after publication (p=0.79) and a small statistically significant change in slope of −0.08%, p=0.001.

The effect of the online, home-level reports on antipsychotic prescribing was less substantial, with an immediate drop of 0.51% fewer antipsychotic prescriptions (95% CI 0.29% to 0.74%, p<0.0001) but an attenuation of the already existing rate of decline in monthly prescriptions (0.06% more prescriptions per month, p=0.002). The online, home-level reports had no effect on the balancing measures and the non-equivalent control (statin prescribing) showed similar changes to those observed for antipsychotic prescribing (an immediate change of 0.22%, p=0.04 and an attenuation of the already existing rate of decline 0.06% more prescriptions per month, p=0.005). This finding, along with the poor fit for the change point in the model (online supplementary appendix 2), indicates that it is unlikely that changes observed in antipsychotic prescribing are attributable to the online, home-level reports.

Subgroup analyses

Table 3 presents the results of the segmented regressions stratified by two patient characteristics (sex, history of psychosis), two physician characteristics (sex, number of residents) and one home characteristic (ownership type). Greater, immediate changes were seen among those residents with no history of psychosis as compared with those with a history of psychosis. Resident sex did not seem to meaningfully moderate the effects observed, although a greater proportion of men were prescribed antipsychotics overall. Minor differences in response to the initiatives by physician sex were observed, with male physicians having a slightly greater immediate reduction in antipsychotic prescribing after the newspaper reports and female physicians a slightly greater reduction after the online, home-level reports. The stratification by number of residents cared for by physician indicated that those with small nursing home practices may have been less responsive, but underlying variation in rates among those with small practices makes this uncertain. Finally, greater changes were observed in for-profit, privately owned nursing homes compared with public, not-for-profit nursing homes, though the privately owned homes had higher prescribing rates throughout. Online supplementary appendix 3 includes a series of figures that demonstrates the trends in antipsychotic prescribing stratified by each of the subgroups.

Table 3.

Results of segmented regression analysis of antipsychotic prescribing patterns within defined subgroups

	Estimate (%)	LCL	UCL	P values
Patient characteristics
History of psychosis
Intercept	72.130	71.648	72.613	<0.0001
Monthly change before	0.069	−0.037	0.174	0.1887
Immediate change after newspaper article	−0.674	−1.217	−0.131	0.0167
Additional monthly change after newspaper article	−0.192	−0.303	−0.081	0.0015
Immediate change after CIHI report	−0.862	−1.467	−0.257	0.0069
Additional monthly change after CIHI report	0.057	−0.037	0.150	0.2219
No history of psychosis
Intercept	30.675	30.496	30.855	<0.0001
Monthly change before	−0.020	−0.059	0.020	0.3095
Immediate change after newspaper article	−1.298	−1.500	−1.096	<0.0001
Additional monthly change after newspaper article	−0.202	−0.243	−0.161	<0.0001
Immediate change after CIHI report	−0.458	−0.683	−0.233	0.0003
Additional monthly change after CIHI report	0.060	0.026	0.095	0.0015
Female resident
Intercept	31.515	31.314	31.715	<0.0001
Monthly change before	−0.014	−0.058	0.030	0.5176
Immediate change after newspaper article	−1.197	−1.423	−0.971	<0.0001
Additional monthly change after newspaper article	−0.213	−0.259	−0.166	<0.0001
Immediate change after CIHI report	−0.509	−0.761	−0.258	0.0003
Additional monthly change after CIHI report	0.067	0.028	0.106	0.0016
Male resident
Intercept	35.582	35.315	35.850	<0.0001
Monthly change before	−0.015	−0.074	0.043	0.5906
Immediate change after newspaper article	−1.502	−1.802	−1.201	<0.0001
Additional monthly change after newspaper article	−0.182	−0.244	−0.121	<0.0001
Immediate change after CIHI report	−0.524	−0.859	−0.189	0.0035
Additional monthly change after CIHI report	0.037	−0.015	0.089	0.1533
Physician characteristics
Female physician
Intercept	32.690	32.438	32.941	<0.0001
Monthly change before	−0.141	−0.196	−0.086	<0.0001
Immediate change after newspaper article	−0.852	−1.135	−0.569	<0.0001
Additional monthly change after newspaper article	−0.107	−0.165	−0.049	0.0008
Immediate change after CIHI report	−0.918	−1.233	−0.602	<0.0001
Additional monthly change after CIHI report	0.090	0.041	0.138	0.0009
Male physician
Intercept	32.619	32.445	32.793	<0.0001
Monthly change before	0.013	−0.026	0.052	0.5073
Immediate change after newspaper article	−1.377	−1.563	−1.191	<0.0001
Additional monthly change after newspaper article	−0.217	−0.255	−0.179	<0.0001
Immediate change after CIHI report	−0.373	−0.582	−0.164	0.0012
Additional monthly change after CIHI report	0.043	0.013	0.073	0.0073
Small panel
Intercept	34.875	33.797	35.953	<0.0001
Monthly change before	−0.291	−0.536	−0.046	0.0219
Immediate change after newspaper article	−0.452	−1.528	0.625	0.3935
Additional monthly change after newspaper article	0.082	−0.155	0.320	0.4808
Immediate change after CIHI report	−0.634	−1.894	0.627	0.308
Additional monthly change after CIHI report	0.139	−0.044	0.322	0.1285
Large panel
Intercept	32.578	32.423	32.732	<0.0001
Monthly change before	−0.016	−0.050	0.019	0.3565
Immediate change after newspaper article	−1.251	−1.418	−1.085	<0.0001
Additional monthly change after newspaper article	−0.201	−0.235	−0.167	<0.0001
Immediate change after CIHI report	−0.562	−0.748	−0.377	<0.0001
Additional monthly change after CIHI report	0.052	0.026	0.079	0.0005
Nursing home characteristic
Public LTC facility
Intercept	30.580	30.371	30.789	<0.0001
Monthly change before	0.048	0.003	0.094	0.0385
Immediate change after newspaper article	−0.770	−1.006	−0.535	<0.0001
Additional monthly change after newspaper article	−0.232	−0.281	−0.184	<0.0001
Immediate change after CIHI report	−0.422	−0.685	−0.160	0.0027
Additional monthly change after CIHI report	0.009	−0.032	0.050	0.6526
Private LTC facility
Intercept	34.206	34.024	34.388	<0.0001
Monthly change before	−0.065	−0.106	−0.024	0.0032
Immediate change after newspaper article	−1.609	−1.810	−1.408	<0.0001
Additional monthly change after newspaper article	−0.182	−0.222	−0.141	<0.0001
Immediate change after CIHI report	−0.566	−0.789	−0.343	<0.0001
Additional monthly change after CIHI report	0.102	0.070	0.134	<0.0001

CIHI, Canadian Institute for Health Information; LCL, lower control limit; LTC, long-term care; UCL, upper control limit.

Discussion

Key findings

The newspaper articles were associated with a significant decrease in prescribing, within 6 weeks of publication. Prescribing proportions decreased at an accelerated rate in the months and years thereafter. These changes correspond to approximately 3400 fewer nursing home residents in the cohort being exposed to the risks of harms from antipsychotics over this time. Trends for benzodiazepine prescribing did not change as substantially during the period of observation. Trazodone use has been gradually increasing, but its use did not change abruptly at the time of the mass media report or the public reporting initiative. In general, the subsequent online, home-level public reports did not seem to amplify prescribing changes beyond those already occurring; the trend for decreasing antipsychotic use continued to gradually decline in a nearly linear fashion. Finally, the observed trends of statin prescribing did not feature sudden changes at the time of the newspaper report and mirrored the trends observed for antipsychotic prescribing around the time of the CIHI reports.

Findings in the context of previous research

This study confirms the findings of previous work showing that media coverage can be associated with prescribing changes, especially when safety issues are identified.¹⁵ Sometimes the changes observed are seemingly mediated through the prescribers’ response to media reports,³⁴ as in our study. In other instances, it would appear that patient responses play a key role in the changes observed.¹⁷

The stratified analyses offer interesting insights about where prescribing changes occurred in response to the public reporting interventions. A range of facility characteristics seems to play a role in antipsychotic prescribing³⁵; our study is not the first to suggest that for-profit homes may have greater room for improvement.³⁶ It is also not surprising that greater prescribing changes were seen for those residents with no history of psychosis, as this is where quality improvement initiatives (appropriately) place their focus.³⁷ The observation that a greater proportion of male residents receive antipsychotics than female residents is concerning given that men seem to be at higher risk of adverse events.³⁸ On the other hand, male nursing home residents are known to have a higher incidence of aggressive behaviours,³⁹ which may offer an appropriate clinical explanation for this observation.

Evaluations of other types of public reporting initiatives have shown mixed effects.¹⁸ The Centers for Medicare and Medicaid Services (CMS) initiated public reporting for antipsychotic use in nursing homes in 2012 with a small effect on prescribing.⁴⁰ We may have observed less of an effect because the online, home-level reports evaluated in our study were not engaged with to the same extent, or because there may have been less room for further improvement efforts, or because the putative mechanism of action of public reporting interventions was stymied in this context. Public reporting may lead to improvements in care through enabling competition (when lower performing providers have the capacity to change) or collaboration (when lower performing providers can learn from others). It is plausible that residents and/or caregivers advocated for changes to prescriptions after seeing the newspaper article. However, patient choice seems an unlikely mechanism of change in this context¹⁹ because patients are unlikely to access the reports and because there is, unfortunately, often limited choice in available nursing homes (or prescribers) for these vulnerable patients in Ontario. Furthermore, nursing homes could have accessed their own prescribing prior to this initiative, raising questions about the relative advantage of the new online reporting system. For providers (organisations or individual clinicians) to respond as desired to public reporting initiatives, they must receive the relevant information, accept it and be able to act on it.⁴¹ The online, home-level reports produced by CIHI provide data that allow providers to understand where they stand in relation to others, but may have achieved greater effects if they included clear targets and action plans⁴² for safely achieving those targets, including supports to collaborate with high-performing facilities.⁴¹

Measurement is essential for quality improvement, but it can lead to unintended consequences, even by well-meaning actors.⁴³ Each quality indicator is usually an imperfect proxy for the desired outcome and it is important to avoid letting the ‘tail wag the dog’.⁴⁴ If the goal of measurement is to provide information that can help improve care, then it is important that providers receiving the information have the capacity to use it to make changes that are best for the patient rather than best for improving the score. Furthermore, the indicator used in the online, home-level reports may be subject to gaming^{45 46} by altering patient comorbidities to exclude them from the indicator^{47 48} or by use of alternative medications. Time spent managing the quality indicator in nursing homes may be better spent managing the residents. Mass media reports may identify important areas for change, but when they lack nuance, such reports may unduly paint well-meaning health professionals in a poor light. Prior work indicates that antipsychotic prescribing may be more associated with home-level factors than individual prescriber factors.⁴⁹ Consulting specialist prescribing patterns,⁵⁰ nursing home staffing levels³⁵ and communication skills⁵¹ may each represent targets for quality improvement interventions. In an unknown proportion of nursing home residents, the potential harms of antipsychotics may be warranted to reduce risk of harms for that resident or others around them, and sudden cessation of these prescriptions may not be desirable.⁵ Therefore, while mass media public reporting interventions such as the newspaper articles described herein may be effective, they may act as blunt instruments and should be pursued cautiously by journalists and health service researchers alike.

Strengths and limitations

This study has a number of strengths including a large sample size with population-wide data for an entire jurisdiction, enabling the precise estimation of effects. Inclusion of balance and non-equivalent control measures as well as careful interrogation of the observed versus hypothesised change point improve causal inference in the quasiexperimental design. There were not a large number of preintervention measures, but the secular trend estimated from our data (a 0.1% reduction per month) is similar to the publicly reported trend from 2010 to 2013 corresponding to a monthly change of 0.1%.⁵² In table 4, we describe the methodological features of this study in relation to Ramsay et al’s quality criteria for interrupted time series designs.⁵³

Table 4.

Consistency of study design with key quality criteria for interrupted time series

1. Intervention occurred independently of other changes over time.	The statistical analyses indicate that a change occurred just after the publication of the newspaper articles—this is reflected in the stepwise change seen at the time, and the absence of changes observed in the control series. It is possible that the postintervention trend change may have been facilitated by additional interventions. However, these additional interventions may themselves have been motivated by the newspaper articles.
2. Intervention was unlikely to affect data collection.	Data collection for the outcomes was not related to the intervention. Data were obtained from a population-level pharmaceutical dispensing database (the Ontario Drug Benefit database) which includes complete prescription data for all Ontario residents of nursing homes.
3. The primary outcome was assessed blindly or was measured objectively.	The prescribing outcomes are based on objective prescription dispensing data collected from the Ontario Drug Benefit database.
4. The primary outcome was reliable or was measured objectively.	The Ontario Drug Benefit database has been previously validated for research purposes; it is populated based on objective data from dispensing pharmacies. All medication outcomes of interest are captured in this database for the population of interest.
5. The composition of the data set at each time point covered at least 80% of the total number of participants in the study.	The data are complete at the population level for the jurisdiction of Ontario. Within nursing homes in Ontario, there are legislative requirements to complete the Resident Assessment Instrument, including documentation of nursing home date of admission and discharge and this was used to create the cohort for this study. Transparent and objective exclusion criteria on the cohort were applied to ensure an ability to assign a prescriber to each resident. As described in online supplementary appendix 1, after exclusion criteria were applied, a total of 153 263 resident episodes were reduced to 120 009 resident episodes and all of the eligible episodes at each time point were used in the analysis.
6. The shape of the intervention effect was prespecified.	Within Ontario nursing homes, medication dispensing occurs typically on a 1–2 week basis in nursing homes but residents may not be reassessed by physicians weekly unless there is a change in their status; monthly reassessments are more common. We prespecified that changes would take 4–6 weeks to occur, allowing enough time for reassessment and implementation of changes. We then tested our prespecified change point using goodness-of-model-fit criteria.
7. A rationale for the number and spacing of data points was described.	Our 30-month study period with monthly intervals was considered adequate to conduct the analysis given the fact that these data were assessed at the population level (ie, include all nursing home residents in Ontario) with minimal variation in the month-to-month prescribing data, and covered more than 120 000 nursing home resident care episodes across 636 nursing homes. Choosing weekly time intervals would have provided many more data points, but this would have been an artificial attempt at increasing the number of observations at the expense of adding unnecessary random variation, and would not have altered our conclusions. Conversely, quarterly time intervals may not have adequately captured turnover of residents.
8. The study was analysed appropriately using time series techniques.	We used segmented regression and accounted for autocorrelation.

Some important limitations with our analysis merit detailed discussion. First, the quasiexperimental design cannot account fully for other concurrent interventions. The attention on this issue in the sector likely generated a range of responses in the system, which may have supported ongoing improvements, but we do not know of an event that occurred exactly in the spring of 2014 that would explain the sudden change in prescribing. Other factors influencing use of antipsychotics over this time period include the Health Canada warnings for the atypical antipsychotics (such as a 2015 notice advising a restriction on the use of risperidone in dementia)^{54 55} and other provincial and national quality improvement initiatives (such as collaborative initiated in 2014 of 56 nursing homes, including 12 in Ontario, to reduce antipsychotic use, led by the Canadian Foundation for Healthcare Improvement).⁵⁶ There may also be spillover effects from initiatives conducted in the USA.⁵⁷ Secular trends in all the outcome measures may also be affected by increasing awareness of the risks of polypharmacy in this population and initiatives such as Choosing Wisely. Confidence in causal attribution may also have been increased by use of an ‘external’ control, that is, antipsychotic prescribing data from a province or jurisdiction outside Ontario but such data, measured in the same fashion as our outcome, could not be readily obtained to facilitate the comparison. Second, we cannot be certain whether the decreased prescribing of antipsychotics was always clinically appropriate. For example, we were unable to include a balancing measure of the rate of incidents of resident violence. Additionally, we did not assess clinical indications for the medications in question and we did not assess for changes in clinical outcomes, but the association between antipsychotics and adverse events is well established. Measures of trazodone and benzodiazepine offer reassurance that the decreased prescribing of antipsychotics was not completely replaced by other commonly prescribed sedating agents, but it is plausible that other alternatives were used instead that could not be accurately measured in the existing databases, such as antihistamines. Recent analyses of CMS initiatives to address antipsychotic prescribing indicate that prescribers may be preferentially selecting mood stabilisers.⁵⁸ We also did not assess physical restraint use, but note other reports indicating that physical restraints decreased throughout the observation period.⁵⁹ Future research might examine whether residents in whom antipsychotics are stopped have a greater likelihood of initiation of alternative treatments. Third, we did not capture reductions in antipsychotic dosage or frequency which may have occurred following the public reporting initiatives. Prescribers are usually advised to reduce antipsychotics with a gradual tapering regimen rather than sudden discontinuation and adverse effects may be dose dependent.⁵ Finally, we do not know the extent of exposure to either of the public reporting interventions. It is possible that among those who accessed their own data and actively reflected on them, meaningful improvements were achieved.

Conclusion

The observation that a mass media report in the form of a newspaper article seemed to result in a statistically and clinically important change in prescribing—and that a subsequent reporting initiative did not—raises interesting questions regarding different forms of public reporting. Research is needed to understand when and how to harness the potential benefits of public reporting via mass media, and how to mitigate unintended consequences. Responsible partnerships between researchers and journalists could play a role in addressing certain quality of care gaps.