Should English healthcare providers be penalised for failing to collect patient-reported outcome measures? A retrospective analysis

Summary

Objective

The best practice tariff for hip and knee replacement in the English National Health Service (NHS) rewards providers based on improvements in patient-reported outcome measures (PROMs) collected before and after surgery. Providers only receive a bonus if at least 50% of their patients complete the preoperative questionnaire. We determined how many providers failed to meet this threshold prior to the policy introduction and assessed longitudinal stability of participation rates.

Design

Retrospective observational study using data from Hospital Episode Statistics and the national PROM programme from April 2009 to March 2012. We calculated participation rates based on either (a) all PROM records or (b) only those that could be linked to inpatient records; constructed confidence intervals around rates to account for sampling variation; applied precision weighting to allow for volume; and applied risk adjustment.

Setting

NHS hospitals and private providers in England.

Participants

NHS patients undergoing elective unilateral hip and knee replacement surgery.

Main outcome measures

Number of providers with participation rates statistically significantly below 50%.

Results

Crude rates identified many providers that failed to achieve the 50% threshold but there were substantially fewer after adjusting for uncertainty and precision. While important, risk adjustment required restricting the analysis to linked data. Year-on-year correlation between provider participation rates was moderate.

Conclusions

Participation rates have improved over time and only a small number of providers now fall below the threshold, but administering preoperative questionnaires remains problematic in some providers. We recommend that participation rates are based on linked data and take into account sampling variation.

Background

Publication of comparative information on patient health outcomes after surgery in order to facilitate quality improvement is becoming increasingly common.^1,2 Some health systems have experimented with the use of such information in the design of reimbursement schemes; an approach known as pay-for-performance intended to sharpen the incentives to provide high quality care.^3,4

The ‘best practice tariff’ (BPT) for primary hip and knee replacement in the English NHS is an example of a pay-for-performance scheme, emphasising the Department of Health’s ambition to establish patients’ views and self-reported outcomes as a central component of hospital quality assessment and regulation.^5,6 Since April 2009, all providers of NHS-funded care have been required to collect patient-reported outcome measures (PROMs) for all patients undergoing unilateral hip and knee replacement (and also for varicose vein surgery and groin hernia repair). PROMs are structured questionnaires that allow patients to report their health status (or health-related quality of life) before and six months after surgery. By comparing these responses, changes in health can be identified and used to better understand differences in the systematic effect that individual hospitals have on improving their patients’ health.⁷

The new BPT, implemented in April 2014, links bonus payments to the requirement that providers do not perform statistically significantly worse than the national average with respect to risk-adjusted improvements in patients’ health status.⁸ The size of this bonus payment amounts to approximately 10% (550 GBP) of the nominal reimbursement price. However, in order to qualify for this bonus, providers must also satisfy a 50% participation rate criterion. The rationale for linking participation rates to the bonus is that the larger and more representative the evidence base, the greater the ability to identify systematic differences between providers.

The participation rate is calculated as a ratio of the number of completed preoperative PROM questionnaires relative to the number of eligible patients. Preoperative data may be missing because providers fail to ask patients to complete the questionnaire or patients decline to participate.

Providers can appeal if deemed not to qualify for the bonus. There may be three grounds for appeal over the participation rate. First, participation rates are subject to sampling uncertainty. For example, a participation rate of 49% may result from chance variation and not be statistically different from the 50% threshold. This can be assessed by constructing confidence intervals around the crude rates. Second, participation rates for providers with low volumes will be estimated imprecisely and random variation can lead to exceptionally high or low observed participation rates. For example, in the extreme case of only two eligible patients, the observed participation rates can only be 0, 50 or 100%, even though 80% (say) of patients participate nationally.⁹ The BPT guidance does not impose a minimum volume threshold for assessing participation rates. Precision-weighted estimates allow for low volumes, yielding participation rates more likely to reflect the true underlying rate.^9–12 Third, patients may differ systematically across providers and these differences may impact on the willingness of patients to complete the PROMs survey.¹³ This can be accounted for by risk adjustment of the participation rates.

A further issue arises from the way in which participation rates are calculated. Only PROMs questionnaires that can be linked to the patient’s electronic medical record are included in the analysis of health outcomes. Arguably, participation rates should also be calculated using only linked data. But the BPT guidance does not distinguish between PROM responses that can be linked and those that cannot. The consequences of this merit exploration.

The aim of this paper was to explore empirically these issues and their implications for the English PROMs programme based on pre-policy data. We constructed two participation rates: (a) a BPT rate based on all PROM records, and (b) a linked rate based on only those PROM records that could be linked to inpatient records. For these two rates, we determined the number of providers that fell short of the 50% participation rate in three financial years (April 2009 to March 2012) and studied the consistency of participation rates over time. We constructed confidence intervals around crude rates, employed hierarchical modelling techniques to adjust for differences in the precision of the provider estimates and explored the impact of risk adjustment. We draw conclusions about the likely impact of the BPT participation rate threshold introduced from April 2014, discuss the most appropriate definition of the participation rate and provide recommendations about the statistical treatment of participation rates.

Methods

Data

Data on all elective admissions for patients, aged 18 or over, who underwent NHS-funded, unilateral hip or knee replacement between April 2009 and March 2012 were extracted from the Hospital Episode Statistics (HES). Identification was based on the primary procedure codes recorded in the first HES episode of the inpatient spell (OPCS 4.5; see guidance⁸ for a full list of relevant procedure codes). These data were used to construct the denominator of the participation rate. HES was also the primary source of information on patient characteristics used for risk adjustment.¹³

To construct the numerator we used the number of non-duplicate complete preoperative PROM questionnaires collected as part of the national PROM survey. All NHS and private providers of NHS-funded care are required to administer a preoperative PROMs questionnaire to all patients deemed fit for hip or knee replacement. Consenting patients complete the paper-based questionnaire shortly before the surgery, i.e. during the last outpatient appointment preceding the surgery or on the day of admission. Responsibility for data collection lies with the provider of care.

For the BPT, patients’ health status is assessed using the Oxford Hip Score (OHS) or Oxford Knee Score (OKS).^14,15 The OHS/OKS questionnaires consist of 12 items pertaining to functioning and pain. For the PROM response to be considered ‘complete’ no more than two of the 12 OHS/OKS items must be missing.

Calculation of participation rates

The BPT guidance calculates the participation rate as ‘the number of pre-operative PROMs questionnaires completed, relative to the number of eligible HES spells’.⁸ This includes all preoperative PROM questionnaires completed, irrespective of whether these could be linked to HES. The BPT participation rate is given by

$$\mathrm{BPT}\mathrm{participation}\mathrm{rate}\mathrm{\hspace{1em}\hspace{1em}\hspace{1em}}=\frac{\mathrm{PROM}\mathrm{records}(\mathrm{linked}+\mathrm{unlinked})}{\mathrm{Eligible}\mathrm{HES}\mathrm{records}}$$

BPT participation rates are calculated for the provider recorded in the PROM records. Due to subcontracting, the provider recorded in the PROM record may not be the same as the one recorded in the HES record. Consequently, for providers with small volumes of eligible patients, rates can exceed 100%.

An alternative is to calculate ‘linked participation rates’ which rely solely on those PROM records that can be linked to HES, i.e.

$$\mathrm{Linked}\mathrm{\hspace{1em}\hspace{1em}\hspace{1em}}\mathrm{participation}\mathrm{\hspace{1em}\hspace{1em}\hspace{1em}}\mathrm{rate}=\frac{\mathrm{PROM}\mathrm{\hspace{1em}\hspace{1em}\hspace{1em}}\mathrm{records}\mathrm{\hspace{1em}\hspace{1em}\hspace{1em}}(\mathrm{linked})}{\mathrm{Eligible}\mathrm{\hspace{1em}\hspace{1em}\hspace{1em}}\mathrm{HES}\mathrm{\hspace{1em}\hspace{1em}\hspace{1em}}\mathrm{records}}$$

Linkage was achieved through a matching variable provided by the Health & Social Care Information Centre. Responses were assigned to the provider recorded in HES so that participation rates could be adjusted for patient factors routinely recorded, and thus observed, for both responders and non-responders.

BPT participation rates are greater or equal to linked participation rates due to their more inclusive definition of the numerator.

Statistical analysis

The analysis aimed to determine how many providers do not achieve the 50% participation rate threshold. The BPT guidance does not mandate adjustment for sampling uncertainty, patient characteristics or the precision of the rates. We compared crude rates with three more refined measures. First, we calculated confidence intervals, based on Wilson’s approach, around crude rates to account for sampling uncertainty.^16,17 Second, we estimated hierarchical logistic models and obtained precision-weighted estimates of the participation rates with associated Bayesian credible intervals (see Appendix 1). This is common in performance assessments and (a) recognises that the probability of missing data may be more similar within than across providers, and (b) ‘borrows strength’ from other providers to estimate more reliable participation rates for providers with low volumes of activity.^9–12 Precision-weighted estimates were calculated as weighted averages of the crude participation rate and the population mean, where weights reflect the precision of the crude rate (i.e. sample size of the provider). Rates for small providers are shrunken towards the mean, whereas for large providers they are largely unaffected. Third, we calculated risk-adjusted, precision-weighted linked participation rates. BPT participation rates could not be adjusted for patient factors because such information is unavailable for non-linked data.

For each provider, we tested whether the true underlying participation rate was below 50% using one-sided hypothesis tests at the 95% confidence level. Data were analysed separately by financial year (1 April to 31 March of the following year) to examine changes over time. We calculated the correlation of provider participation rates across years using Spearman’s rank correlation coefficient and displayed this using scatter plots. Providers with >100% participation rate were excluded from the statistical analysis to facilitate modelling and visualisation.

Results

The scale of the problem

Figure 1 shows the participation rates at national level for hip and knee replacement surgery. The proportion of complete preoperative PROM questionnaires increased from 65% in 2009/10 to 75% in 2011/12 for hip replacement procedures. The proportion of knee replacement patients with complete preoperative PROM questionnaire was slightly higher (70% in 2009/10 to 83% in 2011/12). When only linked episodes were considered, these rates were substantially lower.

Figure 1.

National participation rates for hip and knee replacement surgery in financial years 2009/10 to 2011/12.

Item non-response did not explain the observed patterns. Of those who responded to the preoperative PROM questionnaire, only approximately 1.0% of hip replacement patients and 1.1% of knee replacement answered fewer than 11 of the 12 questionnaire items.

Proportion of providers not meeting the participation rate threshold

Table 1 shows the number of providers that did not meet the 50% threshold in the financial years 2009/10 to 2011/12 under the four different measures. Five main findings emerged and these are drawn out for the hip replacement results. First, as expected, the number of providers that did not meet threshold was much lower when assessed using BPT rates instead of linked rates. Second, in line with the increase in the national participation rates (Figure 1), the number of providers failing to meet the threshold had been declining over time, from 74 (29.1%) providers in 2009/10 to 46 (15.4%) in 2011/12, taking the crude BPT rates. Third, the confidence intervals overlapped the threshold for a substantial number of providers deemed to fall below it. In 2011/12, after accounting for sampling variation, the number of providers below the threshold fell from 46 (15.4%) to 34 (11.4%) (reductions were even greater in 2009/10) (Figure 2). Fourth, precision weighting identified a handful of low volume providers which failed to meet the 50% threshold because their crude rates were imprecisely estimated (Figure 3). Fifth, risk adjustment had an effect on linked participation rates and, depending on the provider’s case-mix, increased or decreased the number of providers identified as failing to meet the threshold (estimated coefficients are in Appendix 1). Similar results were obtained for the knee replacement data.

Table 1.

Number (%) of providers identified as failing the 50% participation rate threshold.

Financial year	Total number of providers	Crude – no confidence intervals		Crude – with confidence intervals		Precision weighted*		Precision weighted + risk adjusted
		N	%	N	%	N	%	N	%
						Hip replacement
BPT participation rates†
2009/10	254	74	29.1	50	19.6	48	18.8
2010/11	290	49	16.8	39	13.4	36	12.4
2011/12	297	46	15.4	34	11.4	30	10.1
Linked participation rates
2009/10	254	121	47.6	76	29.9	72	28.3	78	30.7
2010/11	290	90	31.0	59	20.3	55	18.9	54	18.6
2011/12	297	82	27.6	56	18.8	50	16.8	54	18.1
						Knee replacement
BPT participation rates†
2009/10	245	61	24.9	40	16.3	40	16.3
2010/11	287	50	17.4	36	12.5	35	12.1
2011/12	299	36	12.0	27	9.0	22	7.3
Linked participation rates
2009/10	245	122	49.7	83	33.8	81	33.0	81	33.0
2010/11	287	102	35.5	69	24.0	63	21.9	67	23.3
2011/12	299	89	29.7	56	18.7	50	16.7	50	16.7

Note: The total number of providers may differ across BPT and linked datasets because of issues of subcontracting.

^*Precision weighting was based on statistical analysis of either (a) all providers (when analysing linked participation rates) or (b) only those providers with no higher than 100% participation rates (when analysing BPT participation rates). The percentage of providers failing the participation rate threshold is calculated as the proportion of identified providers over the total number of providers (i.e. in the case of BPT rates including providers with >100% rates).

^†The numbers of providers used in the assessments of BPT participation rates are as follows: hip replacement – 202 (FY 2009/10), 237 (FY 2010/11), 246 (FY 2011/12); knee replacement – 189 (FY 2009/10), 205 (FY 2010/11), 218 (FY 2011/12).

Figure 2.

Uncertainty around crude BPT provider participation rates in financial years 2009/10 to 2011/12 (hip replacement; only providers with <50% participation rate). Note: Providers with participation rates statistically lower than 50% are highlighted dark grey, whereas providers with statistically not significantly lower participation rates are highlighted light grey.

Figure 3.

Effect of precision-weighting in low- and high-volume hospitals (hip replacement, financial year 2011/12). Note: Dots represent providers. Light grey dots represent crude BPT participation rates; dark grey dots represent precision-weighted BPT participation rates.

Participation rates over time

Longitudinal comparison suggested that BPT participation rates were moderately correlated over time. Correlation coefficients were 0.68 (2009/10 to 2010/11), 0.54 (2010/11 to 2011/12) and 0.45 (2009/10 to 2011/12) for hip replacement and 0.62 (2009/10 to 2010/11), 0.64 (2010/11 to 2011/12) and 0.50 (2009/10 to 2011/12) for knee replacement. The longitudinal correlations were similar for linked participation rates. Providers falling below the 50% threshold in one year were likely to achieve low participation rates in the following year. Figure 4 illustrates this for the hip replacement data in years 2010/11 and 2011/12.

Figure 4.

Correlation of precision-weighted BPT participation rates in 2010/11 and 2011/12 (hip replacement). Note: The solid line shows the estimated slope of regression of BPT participation rates in 2010/11 on rates in 2011/12, i.e. the estimated year-on-year correlation. The dashed line is the 45° line, i.e. perfect correlation. Dots represent providers. Those providers with participation rates statistically significantly lower than 50% in both years are highlighted black.

Discussion

Main findings

As with other patient self-reported surveys, PROMs are prone to different forms of missing data. This has the potential to bias assessments of comparative hospital performance due to small numbers and potentially unrepresentative case-mix. Foreseeing this in designing the BPT for hip and knee replacement, Monitor and NHS England have mandated that provider participation rates must be 50% or more in order to be considered for a bonus.⁸

Participation rates, like any other performance metric, are likely to be influenced by random variation and this should be taken into account when assessing whether providers meet the 50% threshold. This can be achieved through one-sided confidence intervals and precision weighting as routinely applied in many performance measurement schemes.^9–12 Our results emphasise the importance of such refinements. Accounting for sampling uncertainty is particularly important, substantially reducing the number of providers deemed to have failed to meet the threshold. Precision weighting reduces the number further by correcting rates for providers with low volumes. An alternative is to implement a minimum volume threshold similar to that used in the BPT (i.e. at least 30 cases) when analysing performance in relation to health improvements.

We found that participation rates had been improving over the three financial years from 2009/10 to 2011/12, but that complete preoperative PROMs were still missing for more than 15% of patients, and that linked records are not available for approximately 40% of patients. In line with this, after allowing for sampling variation and precision weighting, the number of providers identified as failing the BPT participation threshold declined over time from approximately 20% in the financial year 2009/10 to 9% in 2011/12 for hip replacement surgery, and 16% (2009/10) to 6% (2011/12) for knee replacement surgery. These improvements in participation rates seem to have applied approximately uniformly across all providers as suggested by high inter-temporal correlation in provider rates. Providers with low participation rates in one year had low participation rates in subsequent years.

A previous study has identified a range of patient characteristics associated with non-response.¹³ Our results suggest that risk adjustment may matter for some providers. However, while the case for risk adjustment is generally undisputed in the context of health outcome measurement, it is less clear when analysing participation rates. By adjusting for case-mix one assumes implicitly that non-response is due to patients declining to participate, not providers failing to engage sufficiently with harder to reach patient groups. In contrast, unadjusted rates assign responsibility exclusively to the provider and do not allow for differences in patients’ willingness to participate associated with observed characteristics. Ultimately, it is a policy choice about where to assign responsibility.

Policy implications

We argue that participation rates should be based on only those data that can be linked to the patient’s administrative record. There are three reasons for this. First, it ensures consistency with the data used in the assessment of health outcomes. For example, the BPT outcome assessment excludes all patients undergoing revision surgery, but these patients can only be identified from procedure codes recorded in HES. Second, it allows risk adjustment of the participation rates if deemed appropriate. Third, if the aim of the BPT participation rate threshold is to ensure that outcome assessment is based on as much data as possible, then linkage is crucial and should be incentivised as well. To a large extent, linkage failure is due to incorrect values in the fields used for linkage (e.g. the patient’s NHS number) in the PROM survey. Responsibility for data quality rests with the provider, and the risk of failing to meet the participation threshold will incentivise improvements in coding behaviour. Note that the issue of subcontracting, which affects approximately 1% of all linked hip and knee replacement records, is also a coding issue and is therefore not directly relevant to the argument for or against use of linked data in calculating participation rates.

The large variation in participation rates and their stability over time implies substantial scope for providers to improve participation rates and that it is appropriate to condition bonus payments on participation rates. However, the declining number of providers that would have been affected by this rule raises questions about whether to maintain the threshold at the 50% level. If participation rates kept increasing after the end of our study period, the incentives set by the BPT to improve participation further may be rather weak.

Weaknesses

Our study has limitations. First, linkage of PROM and HES records was based on the linkage algorithm developed by the Health and Social Care Information Centre and improvements to this algorithm are ongoing.⁸ Second, the number of eligible patients was based on HES records, which may be subject to miscoding and measurement error. However, the impact on the estimated participation rates is likely to be small.⁸ Third, we conducted analysis of BPT participation rates using data on only those providers with rates between 0 and 100%. This had a small effect on the precision-weighted estimates, which were now calculated on the basis of a smaller inter-provider variance and a lower mean across all providers; arguably both are more realistic. Assessments based on crude rates with confidence intervals were unaltered by including subcontracted providers with participation rates above 100%. An alternative modelling strategy would have been to estimate hierarchical Poisson models with the numerator as dependent variable and the denominator as exposure term. Finally, we analysed data that predate the introduction of the BPT. If the BPT leads to changes in data collection then future participation rates may diverge from the reported historic trends.

Conclusions

The implementation of the BPT is inconsistent in the treatment of statistical uncertainty and risk factors, both of which are accounted for in the assessment of outcomes, but not in the assessment of participation rates. Providers deemed not to have met the 50% threshold for participation may have grounds to appeal the penalty.

We believe that penalties for poor participation rates are appropriate but these must be calculated correctly and should aim at increasing the number of data points available for outcome assessments. Furthermore, participation rates should be calculated on the basis of completed and linked questionnaires, as these are the relevant responses in the outcome assessment and providers can help ensure linkage through better coding. Given increasing participation rates in the first three years of the PROM programme, there may be scope for raising the threshold above 50% in order to maintain pressure on providers to continually improve data collection.

Appendix 1: Hierarchical logistic modelling

The hierarchical logistic model is specified as follows: Let $R_{\mathrm{ij}}$ be the indicator for the preoperative PROM response, where $R_{\mathrm{ij}}=1$ if the response for patient i in provider j is observed and complete, and 0 otherwise. The probability of complete data can then be described as a function of observed patient characteristics ($X_{\mathrm{ij}}$) and a random provider effect $u_j\hspace{1em}~N(0,{\sigma }_u^2)$ which measures systematic deviations from the case-mix adjusted mean probability of response, ${\alpha }_0$. The observed response is related to these factors through the logit function, so that

$$\mathrm{Pr}(R_{\mathrm{ij}}=1|X_{\mathrm{ij}},u_j)=\frac{\text{exp}({\alpha }_0+X_{\mathrm{ij}}\text{'}\beta +u_j)}{1+\text{exp}({\alpha }_0+X_{\mathrm{ij}}\text{'}\beta +u_j)}$$

which can be estimated using maximum likelihood.^18–20 Based on the estimated coefficients and Empirical Bayes estimates of u_j, we predict precision-weighted provider participation rates and calculate one-sided confidence intervals. These predictions are carried out at the means of all patient characteristics. To obtain precision-weighted participation rates that are not adjusted for case-mix, we re-estimate the model excluding the term $X_{\mathrm{ij}}\text{'}\beta$.

Descriptive statistics and estimated odds ratios – hip replacement surgery.

	FY 2009/10			FY 2010/11			FY 2011/12
Variable	Summary	OR	95% CI	Summary	OR	95% CI	Summary	OR	95% CI
Patient age (n, %)
18–45 years	2868 (4.3)	0.96	(0.88–1.04)	2911 (4.1)	0.90	(0.82–0.98)	2989 (4.1)	0.90	(0.82–0.97)
46–55 years	6125 (9.2)	0.99	(0.93–1.06)	6573 (9.3)	1.04	(0.97–1.10)	6792 (9.4)	0.97	(0.92–1.03)
56–65 years	15,708 (23.5)	1.09	(1.04–1.14)	16,793 (23.7)	1.10	(1.05–1.15)	17,186 (23.8)	1.06	(1.01–1.11)
66–75 years	23,599 (35.3)	1.00	–	24,827 (35.1)	1.00	–	25,425 (35.2)	1.00	–
76–85 years	15,981 (23.9)	0.83	(0.80–0.87)	17,068 (24.1)	0.77	(0.74–0.81)	17,265 (23.9)	0.86	(0.82–0.90)
86 years or older	2566 (3.8)	0.68	(0.62–0.75)	2634 (3.7)	0.62	(0.57–0.68)	2631 (3.6)	0.69	(0.63–0.75)
Gender (n, %)
Female	39,531 (59.1)	1.00	–	42,081 (59.4)	1.00	–	43,049 (59.6)	1.00	–
Male	27,252 (40.8)	1.04	(1.00–1.07)	28,654 (40.5)	1.03	(0.99–1.06)	29,155 (40.3)	1.05	(1.02–1.09)
Missing	64 (0.1)	0.17	(0.03–0.83)	71 (0.1)	0.43	(0.20–0.91)	84 (0.1)	0.90	(0.27–3.03)
Ethnicity (n, %)
White	59,240 (88.6)	1.00	–	62,406 (88.1)	1.00	–	64,228 (88.9)	1.00	–
Mixed	113 (0.2)	0.82	(0.55–1.23)	132 (0.2)	0.83	(0.57–1.21)	118 (0.2)	0.72	(0.48–1.07)
Asian	303 (0.5)	0.82	(0.64–1.05)	325 (0.5)	0.60	(0.47–0.76)	332 (0.5)	0.57	(0.45–0.72)
Black	460 (0.7)	0.83	(0.68–1.02)	444 (0.6)	0.90	(0.73–1.11)	404 (0.6)	0.76	(0.61–0.94)
Other or missing	6731 (10.1)	0.91	(0.85–0.96)	7499 (10.6)	0.92	(0.86–0.97)	7206 (10.0)	0.88	(0.83–0.94)
Income deprivation (n, %)
1st quintile – least deprived	13,977 (20.9)	1.00	–	15,887 (22.4)	1.00	–	16,201 (22.4)	1.00	–
2nd quintile	14,773 (22.1)	1.00	(0.95–1.05)	16,082 (22.7)	0.93	(0.88–0.97)	16,219 (22.4)	0.97	(0.93–1.02)
3rd quintile	12,618 (18.9)	1.00	(0.94–1.05)	13,117 (18.5)	0.91	(0.86–0.96)	13,427 (18.6)	0.95	(0.91–1.01)
4th quintile	11,739 (17.6)	0.94	(0.89–1.00)	12,017 (17.0)	0.93	(0.88–0.98)	12,449 (17.2)	0.95	(0.90–1.00)
5th quintile – most deprived	12,880 (19.3)	0.91	(0.86–0.96)	12,901 (18.2)	0.86	(0.82–0.91)	13,103 (18.1)	0.92	(0.87–0.97)
Missing	860 (1.3)	0.67	(0.57–0.80)	802 (1.1)	0.69	(0.58–0.82)	889 (1.2)	0.72	(0.61–0.85)
Healthcare Resource Group (HRG) (n, %)
HB12C	51,894 (77.6)	1.00	–	48,455 (68.4)	1.00	–	44,365 (61.4)	1.00	–
HB11C	1546 (2.3)	0.79	(0.71–0.88)	7933 (11.2)	0.90	(0.85–0.95)	8051 (11.1)	0.93	(0.88–0.99)
HB12B	2939 (4.4)	0.96	(0.89–1.04)	3104 (4.4)	0.89	(0.82–0.96)	7501 (10.4)	0.91	(0.86–0.96)
HR05Z	3070 (4.6)	0.89	(0.79–1.01)	3207 (4.5)	1.01	(0.90–1.14)	3282 (4.5)	0.87	(0.78–0.97)
HB12A	2742 (4.1)	0.86	(0.79–0.94)	2631 (3.7)	0.83	(0.76–0.91)	2747 (3.8)	0.86	(0.79–0.93)
Any other HRG	4656 (7.0)	0.66	(0.59–0.73)	5476 (7.7)	0.80	(0.73–0.88)	6342 (8.8)	0.81	(0.75–0.88)
Number of hospital admissions in last 365 days (mean, SD)	0.58 (2.1)	0.98	(0.97–1.00)	0.58 (2.3)	0.99	(0.99–1.00)	0.57 (2.0)	0.98	(0.97–0.99)
Number of co-morbidities (Elixhauser) (mean, SD)	0.38 (0.9)	0.92	(0.90–0.94)	0.42 (0.9)	0.91	(0.89–0.93)	0.45 (1.0)	0.91	(0.89–0.93)
Waiting time (n, %)
Less than three months	39,005 (58.3)	1.01	(0.98–1.05)	39,840 (56.3)	0.99	(0.96–1.03)	39,263 (54.3)	0.93	(0.90–0.97)
Three months or more	27,842 (41.7)	1.00	–	30,966 (43.7)	1.00	–	33,025 (45.7)	1.00	–
Revision surgery (n, %)
No	59,662 (89.3)	0.60	(0.55–0.67)	63,476 (89.6)	0.51	(0.47–0.56)	64,865 (89.7)	0.57	(0.52–0.62)
Yes	7185 (10.7)	1.00	–	7330 (10.4)	1.00	–	7423 (10.3)	1.00	–
Number of observations		66,847			70,806			72,288

CI: confidence intervals; FY: financial year; OR: odds ratio; SD: standard deviation.

Note: Dependent variable is an indicator of whether the preoperative questionnaire was completed (=1) or not (=0). OR>1 indicates a higher probability of completing the preoperative questionnaire.

Descriptive statistics and estimated odds ratios – knee replacement surgery.

	FY 2009/10			FY 2010/11			FY 2011/12
Variable	Summary	OR	95% CI	Summary	OR	95% CI	Summary	OR	95% CI
Patient age (n, %)
18–45 years	850 (1.2)	0.90	(0.78–1.05)	809 (1.1)	0.88	(0.75–1.03)	761 (1.0)	0.97	(0.83–1.13)
46–55 years	4880 (6.8)	1.13	(1.06–1.21)	5081 (6.8)	1.09	(1.02–1.17)	5239 (6.9)	1.06	(0.99–1.13)
56–65 years	18,276 (25.3)	1.09	(1.04–1.13)	19,027 (25.3)	1.11	(1.07–1.16)	19,611 (25.7)	1.06	(1.01–1.10)
66–75 years	27,697 (38.4)	1.00	–	29,134 (38.8)	1.00	–	29,690 (38.8)	1.00	–
76–85 years	18,318 (25.4)	0.85	(0.82–0.88)	18,805 (25.0)	0.82	(0.79–0.85)	18,918 (24.7)	0.82	(0.79–0.86)
86 years or older	2156 (3.0)	0.73	(0.67–0.81)	2302 (3.1)	0.65	(0.60–0.72)	2227 (2.9)	0.66	(0.60–0.72)
Gender (n, %)
Female	40,919 (56.7)	1.00	–	42,955 (57.2)	1.00	–	43,565 (57.0)	1.00	–
Male	31,177 (43.2)	1.03	(1.00–1.07)	32,110 (42.7)	1.04	(1.01–1.07)	32,738 (42.8)	1.03	(1.00–1.07)
Missing	81 (0.1)	0.07	(0.01–0.34)	93 (0.1)	0.69	(0.35–1.36)	143 (0.2)	0.36	(0.14–0.93)
Ethnicity (n, %)
White	62,001 (85.9)	1.00	–	64,253 (85.5)	1.00	–	65,738 (86.0)	1.00	–
Mixed	174 (0.2)	0.84	(0.61–1.16)	195 (0.3)	0.64	(0.47–0.87)	192 (0.3)	0.86	(0.63–1.18)
Asian	2267 (3.1)	0.63	(0.58–0.70)	2403 (3.2)	0.56	(0.51–0.61)	2526 (3.3)	0.63	(0.57–0.69)
Black	920 (1.3)	0.68	(0.59–0.79)	863 (1.1)	0.76	(0.65–0.88)	866 (1.1)	0.81	(0.69–0.94)
Other or missing	6815 (9.4)	0.90	(0.85–0.96)	7444 (9.9)	0.89	(0.84–0.95)	7124 (9.3)	0.90	(0.85–0.96)
Income deprivation (n, %)
1st quintile – least deprived	18,648 (25.8)	1.00	–	20,451 (27.2)	1.00	–	21,015 (27.5)	1.00	–
2nd quintile	9427 (13.1)	0.96	(0.91–1.01)	10,031 (13.3)	1.01	(0.96–1.07)	10,277 (13.4)	1.03	(0.98–1.09)
3rd quintile	13,193 (18.3)	0.97	(0.92–1.02)	13,747 (18.3)	0.95	(0.91–1.00)	13,891 (18.2)	0.95	(0.90–0.99)
4th quintile	15,727 (21.8)	0.92	(0.88–0.97)	16,256 (21.6)	0.95	(0.90–0.99)	16,451 (21.5)	0.96	(0.92–1.01)
5th quintile – most deprived	14,391 (19.9)	0.88	(0.84–0.93)	13,944 (18.6)	0.84	(0.80–0.88)	14,077 (18.4)	0.87	(0.83–0.91)
Missing	791 (1.1)	0.70	(0.59–0.84)	729 (1.0)	0.70	(0.58–0.85)	735 (1.0)	0.70	(0.58–0.85)
Healthcare Resource Group (HRG) (n, %)
HB12C	59,964 (83.1)	1.00	–	59,903 (79.7)	1.00	–	53,388 (69.8)	1.00	–
HB11C	3559 (4.9)	1.05	(0.97–1.13)	4230 (5.6)	0.87	(0.81–0.93)	12,124 (15.9)	0.95	(0.91–1.00)
HB12B	3669 (5.1)	0.93	(0.86–1.00)	3741 (5.0)	0.85	(0.79–0.91)	3901 (5.1)	0.98	(0.92–1.06)
HR05Z	405 (0.6)	0.87	(0.69–1.09)	1970 (2.6)	0.71	(0.63–0.80)	2795 (3.7)	0.72	(0.65–0.79)
HB12A	2593 (3.6)	0.48	(0.44–0.53)	1360 (1.8)	0.27	(0.23–0.32)	551 (0.7)	0.27	(0.22–0.34)
Any other HRG	1987 (2.8)	0.67	(0.60–0.75)	3954 (5.3)	0.77	(0.69–0.87)	3687 (4.8)	0.65	(0.57–0.73)
Number of hospital admissions in last 365 days (mean, SD)	0.65 (2.2)	0.99	(0.99–1.00)	0.64 (2.2)	0.98	(0.97–0.99)	0.63 (2.0)	0.99	(0.98–0.99)
Number of co-morbidities (Elixhauser) (mean, SD)	0.44 (0.9)	0.94	(0.93–0.96)	0.49 (1.0)	0.93	(0.91–0.95)	0.52 (1.0)	0.94	(0.93–0.96)
Waiting time (n, %)
Less than three months	40,976 (56.8)	0.99	(0.95–1.02)	40,256 (53.6)	0.99	(0.95–1.02)	39,113 (51.2)	0.92	(0.88–0.95)
Three months or more	31,201 (43.2)	1.00	–	34,902 (46.4)	1.00	–	37,333 (48.8)	1.00	–
Revision surgery (n, %)
No	66,387 (92.0)	0.43	(0.40–0.47)	69,305 (92.2)	0.48	(0.44–0.53)	71,198 (93.1)	0.60	(0.54–0.67)
Yes	5790 (8.0)	1.00	–	5853 (7.8)	1.00	–	5248 (6.9)	1.00	–
Number of observations		72,177			75,158			76,446

CI: confidence intervals; FY: financial year; OR: odds ratio; SD: standard deviation.

Note: Dependent variable is an indicator of whether the preoperative questionnaire was completed (=1) or not (=0). OR>1 indicates a higher probability of completing the preoperative questionnaire.

Declarations

Competing interests

None declared

Funding

NG, CB and AS are funded by a grant from the Department of Health to the Policy Research Unit in the Economics of Health and Social Care Systems (Ref 103/0001). MG is funded by the UK Medical Research Council Early Career Fellowship in Economics of Health (Ref MR/K02177X/1).

Ethical approval

Not required for analysis of anonymised secondary data.

Guarantor

AS

Contributorship

NG and AS conceived the work and designed the study with MG. All authors drafted the article, helped interpret the results, reviewed the article critically for important intellectual content and approved the final version.

Acknowledgements

The authors would like to thank Rafael Goriwoda, Katharina Hauck, Andrew Hutchings, Jo Partington, Frederick Wheeler and members of the PROM advisory group, as well as participants of the Health Economic Study Group meeting, Sheffield (January 2014) for their useful comments and suggestions. The views expressed are those of the authors and may not reflect those of the funders.

Provenance

Not commissioned; peer-reviewed by Khalid Ismail