Release of individual surgeon data to the public: patients’ and surgeons’ views

Abstract

Background

Limited surgeon-specific outcomes data are currently released to the public. Existing schemes generally result from the recommendations of public enquiries, addressing breaches to patient safety and malpractice. We found limited evidence in the literature about patients’ or orthopaedic surgeons’ wishes regarding the release of such data to the public.

Methods

We surveyed 80 joint replacement patients and 41 orthopaedic surgeons regarding their wishes concerning collection and release of individual surgeon data to the public.

Results

Of 80 patients, 30% (24/80) were aware of data on the NHS-My Choices website, 16% (13/80) had reviewed data prior to operation and 95% (76/80) wanted data concerning surgeons’ experience, length of stay and complications including revisions. Patients expected more current monitoring of data than occurs. Of 41 surgeons, 20% (8/41) thought national joint registry (NJR) derived data accurately reflected their NHS work. Surgeons did not think this data improved patient outcomes (34%, 14/41), and that it reduced innovation (61%, 25/41) and training (75%, 31/41) and increased risk of adverse behaviour (61%, 25/41). Surgeons wanted a minimal data set accurately presented and risk adjusted.

Conclusion

In the future, it is likely that more individual surgeon data will be released to the public. There needs to be an agreed, accurate minimum dataset collected, reviewed in local clinical governance meetings and published with explanatory notes regarding the interfering variables and what conclusions can be drawn regarding the ability of the surgeon. This process needs to be overseen by an independent body trusted by the public.

Introduction

Currently, there are limited surgeon-specific outcomes data released to the public. Existing schemes to release such data generally result from the recommendations of public enquiries, addressing breaches to patient safety and malpractice.

For instance, the Kennedy inquiry (prompted by concerns regarding the performance of cardiac surgeons) led to the first release of surgeon-specific data in 2004.¹ Following this, the freedom of information act in 2005 and the Francis report in 2012 (prompted by poor performance of the Mid Staffordshire NHS foundation Trust) led to the NHS England initiative ‘Everybody counts’ in 2012.^2–4 This included recommendations for greater transparency in healthcare. As part of this initiative, the NHS ‘My Choices’ website was developed in 2013 and made consultant surgeon outcomes data available across ten surgical specialties.⁵

The data on the ‘My Choices’ website regarding individual orthopaedic surgeons are sourced from the national joint registry (NJR).⁶ The NJR was established in response to the poor performance of the 3M capital hip prosthesis.⁷ Currently the NJR collects data regarding arthroplasty of the hips, knees, ankles shoulders and elbows (Box 1).⁶

Box 1. Orthopaedic surgeon data currently available to the public.

• Publicly available data for knee replacements

•  Number of primary knee replacement operations in 12 months

•  Risk adjusted 90-day mortality rate after knee replacement

•  Patients treated at higher risk of medical problems

•  Patients treated who were less than 60 years old

•  Patients treated who were male

•  Patients treated who were severely obese

•  Patients treated with conditions other than osteoarthritis

• Publicly available data for hip replacements

•  Number of hip replacement revisions carried out in 12 months

•  Number of primary hip replacement operations carried out in 12 months

•  Risk adjusted mortality rate after hip replacement

•  Use of ODEP-rated hip cup implants in 12 months

•  Use of ODEP-rated hip stem implants in 12 months

•  Patients treated at higher risk of medical problems

•  Patients treated who were less than 60 years old

•  Patients treated who were male

•  Patients treated who were severely obese

•  Patients treated with conditions other than osteoarthritis

Further individual surgeon data release is likely in the future.⁸ For instance, the recent Paterson Inquiry Report recommended release of some individual surgeon data to the public for all surgeons, particularly number of procedures performed.⁹

Neither patients nor the relevant surgical communities have set the agenda for data release schemes and their wishes are unknown. We reviewed the literature regarding the release of individual surgeon outcomes data. We were unable to find any relevant publications concerning patients’ wishes. Regarding surgeons’ wishes, we identified four publications: none concerned orthopaedic surgery.^10–13

The aims of this study were to determine the wishes of both orthopaedic patients and their surgeons regarding release of individual surgeon outcomes data. We surveyed both groups asking:

• 1.What outcomes data they thought were currently available to the public.
• 2.What outcomes data they wished to see available.

Methods

We developed two surveys using the online survey platform Google Docs (Google, California, USA): one for patients and one for orthopaedic surgeons. The surveys evaluated two key domains in relation to published outcomes data: (1) awareness of the currently available data; (2) and wishes regarding release of surgeon data to the public. In addition, we asked surgeons their opinions on the relationship of published data on training and innovation in orthopaedic surgery. Following the UK policy framework for health and social care research tool, this project was classified as service evaluation and did not require formal ethical approval.¹⁴

Patient questionnaire

• Questions asked: see Tables 1 and 2.

• Duration of study:

  • 4 May 2020–6 June 2020.
• Inclusion criteria:

  • We included 170 patients who had a hip (THR) or knee (TKR) replacement at the Sussex Orthopaedic Treatment Centre (SOTC) (Lewes Road, Haywards Heath, West Sussex RH16 4EY, UK) between October and December 2019. We included only THR and TKR as surgeons’ results for these procedures are available to the public on My Choices and the NJR (Box 1).
• Exclusion criteria:

  • Patients with confirmed dementia (n = 4).
• Study methodology:

  • Patients were contacted by telephone. One of the authors (HH) gave a scripted explanation of the study and offered an email or telephone survey. Of 166 eligible patients, it was possible to contact 101 and 79% (80/101) completed the questionnaire.

Table 1 .

What outcomes data orthopaedic patients thought were currently available to the public

Question	Answered yes (N = 80)
Awareness
Were you aware of NHS-my choices?	24/80 (30%)
Did you access the online information about your individual orthopaedic surgeon before your operation?	13/80 (16%)
What do you think is available? (A free text question. Answers categorised into the following broad areas)
The career outline of the surgeon	15/80 (19%)
How long the surgeon has been operating	8/80 (10%)
How many operations performed	5/80 (6%)
Complications	16/80 (20%)
Patient feedback	10/80 (13%)
Technical details	3/80 (4%)
Nothing	2/80 (3%)
I don’t know	43/80 (54%)
Who do you think monitors and acts on this information?
The surgeons themselves	36/80 (45%)
The orthopaedic department	50/80 (62%)
The trust	56/80 (70%)
National government body	42/80 (53)
An independent body	33/80 (41%)
Who do you think monitors and acts on this information?
Answered I do not know to one of these questions	26/80 (32%)
Engagement
If you were aware of this data, would you have looked it up? (This includes everyone that would have and everyone that did)	34/80 (42.5%)

Table 2 .

What outcomes data orthopaedic patients wished to see available

Question	Answered yes (N = 80)
What patients would like to have available
How many joint replacements they have done in the last year	59/80 (74%)
The average length of stay in hospital for a patient after an elective joint replacement	54/80 (68%)
How long the surgeon has been operating for	54/80 (68%)
How many patients had to return for another operation because of a complication	51/80 (64%)
When the surgeon last did an operation	51/80 (64%)
Feedback from previous patients – their reported experience and outcome after the operation	50/80 (63%)
How many patients had to come back to hospital with a complication after their operation	47/80 (59%)
How many joint revisions (modifications to previous operations) they have done in the last year	43/80 (54%)
How many patients they operate on that are high risk owing to comorbidities	36/80 (45%)
The mortality rate	36/80 (45%)
How many of their patients are age > 60	32/80 (40%)
Patients with conditions other than osteoarthritis	26/80 (33%)
The type of materials used in the joint replacements (implants) and the testing of those implants	15/80 (19%)
The genders of their patients	8/80 (10%)
I do not think any data should be available	4/80 (5%)
Why should data be available?
To check the standard of the surgeon	70/80 (88%)
Accountability	56/80 (70%)
For surgeons to compare their work with colleagues	39/80 (49%)
Identify malpractice	60/80 (75%)
Who should monitor data?
Individual surgeons	40/80 (50%)
Department	64/80 (80%)
Trust	65/80 (81%)
Nationally	60/80 (75%)
Independent body	51/80 (64%)

Patient demographics: Of 80 respondents 50 were female, 73 white British, 69 had internet access. Equal numbers (40 each) had TKR and THR.

Surgeon questionnaire

Questions asked: see Tables 3 and 4.

• Duration of study:

  • 28 May 2020–21 June 20.
• Inclusion criteria:

  • Orthopaedic consultants in Kent, Surrey, Sussex, Wessex, London and Severn deaneries who undertook THR and TKR.
• Study methodology:

  • We contacted 41 surgeons by email and asked them to complete the survey online: 41 did so.

Table 3 .

What outcomes data orthopaedic surgeons thought were currently available to the public

Questions	Answered strongly agree or agree (N = 41)
Awareness
‘I am aware of my published data. (The information on ‘NHS-My Choices’ and ‘iwantgreatcare.org’) ’	31/41 (76%)
Function
‘Published surgeon-specific data are beneficial for’
Improving patient outcomes	14/41 (34%)
Improving surgeon’s performance	20/41 (49%)
Improving trust performance	21/41 (51%)
Monitoring trusts	24/41 (59%)
Identifying malpractice	11/41 (27%)
Accuracy
‘Data from the NJR allows accurate reflection of…’
NHS work	8/41 (20%)
Private work	13/41 (32%)
Innovation
‘Published data have negatively affected innovation’	25/41 (61%)
Training
‘Published data have negatively affected training’	31/41 (76%)
Risk adverse behaviour
‘Published data have led to risk adverse behaviour in…’
Myself	25/41 (61%)
Others	34/41 (83%)

Table 4 .

What outcomes data orthopaedic surgeons wished to see available

Question	Answered yes (N = 41)
What should be collected
‘Surgeon-specific data should be…’
Collected by the trust	32/41 (78%)
Used for local monitoring of individual surgeons	38/41 (93%)
Used for national monitoring of individual surgeons	26/41 (63%)
Used for national monitoring of trust performance	31/41 (76%)
What should be published?
How many joint revisions they have done in the last year	32/41 (78%)
How many joint replacements they have done in the last year	31/41 (76%)
Proportion of high-risk patients with significant comorbidity	30/41 (73%)
Risk adjusted mortality rate	27/41 (66%)
How long the surgeon has been operating for as a consultant	27/41 (66%)
Use of ODEP implants	25/41 (61%)
Career outline of the orthopaedic surgeon	25/41 (61%)
Patients’ reported outcomes (PROMS)	22/41 (54%)
How many patients had to return to theatre for a revision or a complication	22/41 (54%)
Proportion of patients older than 60 years	21/41 (51%)
Patients with conditions other than osteoarthritis	20/41 (49%)
The average length of stay for a patient for an elective joint replacement	17/41 (42%)
Ratios of patients’ gender	16/41 (39%)
When they last did an operation joint replacement	16/41 (39%)
How many patients were re-admitted with a postoperative complication?	15/41 (37%)
Surgical details and techniques of the orthopaedic surgeon	13/41 (32%)

Surgeon demographics: Of the 41 respondents 39 were male, 25 performed both THR and TKR, 8 each performed only THR or TKR.

Statistical analysis

Statistical analysis was conducted using chi-square testing to determine any significant relationship between gender, ethnicity and age and the response to each question. Otherwise descriptive analysis was used to discuss the results of the surveys.

Results

Patients: what outcomes data orthopaedic patients thought were currently available to the public (Table 1)

In our cohort of 80 patients, 16% (13/80) had chosen to access online information regarding their surgeon prior to their procedure. In general, the public awareness of published data was poor, with only 30% (24/80) having heard of ‘NHS-My Choices’. Patients were asked what information they thought may be publicly available, and 54% (43/80) answered that they did not know. We found that patients thought data were already collected and monitored at a surgeon, department, trust and national level, though 32% (26/80) answered ‘I do not know’ to one of these questions (Table 1). Although the younger age groups were more likely to look up data, this was not statistically significant in our cohort (p-value = 0.11)

Patients: what outcomes data orthopaedic patients wished to see available (Table 2, Figure 1)

Figure 1 .

Surgeons’ and patients’ wishes regarding data to be collected

In our cohort, 95% (76/80) of patients would welcome publicly available data. For each of the 14 outcomes, the response rates were high, particularly details that indicated the experience of the surgeon. Patients were keen to see a considerable amount of simple data. Patients wanted simple assessments of practice and were not keen to see more nuanced data that allowed risk adjustment (ie for comorbidity, age). Patients thought this data should be used to oversee surgeons’ practice at many levels, though they did not think this should be the responsibility of surgeons.

Surgeons: what outcomes data orthopaedic surgeons thought were currently available to the public (Table 3)

Of the 41 orthopaedic surgeons surveyed, 76% (31/41) reported that they were aware of their own published data. Overall 59% (24/41) of surgeons thought these data were beneficial for monitoring trusts and 51% (21/41) thought this led to improving trust performance, while 49% (20/41) thought this led to improving surgeon performance. Few surgeons (34%, 14/41) thought that published data led to improving patient outcomes. Surgeons did not think the data currently published would identify malpractice (27%, 11/41). Surgeons thought data negatively affected innovation (61%, 25/41) and training (76%, 31/41) and led to risk adverse behaviour (colleagues: 83%, 34/41; themselves: 61%, 25/41).

What outcomes data orthopaedic surgeons wished to see available (Table 4)

Surgeons wanted a minimal data set fairly presented and risk adjusted. Work volume aside, few of the surgeon’s data choices correlated with patient wishes. Surgeons ranked data importance differently to patients (ie length of stay) (Figure 1). They wished to see data used at many levels to monitor group rather than individual practice. Any assessment of individual surgeon’s practice should be undertaken in a local context (ie at clinical governance meetings). They were wary of others collecting data about their practice and acting on it outside the trust.

Discussion

The aim of this paper was to determine the wishes of orthopaedic surgeons and their patients regarding the release of individual surgeon data to the public. We found that the two groups had different aspirations for the type of data to be released.

Patients

Patients had a limited understanding of, and interest in, the data currently available to the public.

Individual surgeon’s data have been available since 2013, but patients had poor awareness of the ‘NHS-My Choices’ resource and only 16% (13/80) of patients accessed the online data. Although 95% (76/80) of patients would welcome published data, only 50% said that they would have used the ‘NHS-My Choices’ website had they known about it.

The reasons for the low interest in published data remain obscure, though nine patients (11%) volunteered feedback suggesting published outcomes would scare them away from having an operation they deemed necessary. Limited engagement in published data has also been noted for cardiothoracic data.¹⁵

Going forward, patients wanted simple to interpret data to allow them to make decisions regarding their surgeon (however unrealistic this might be). Some of the data patients wanted available would be difficult to accurately collect and present in a contemporaneous fashion (ie when the surgeon last performed the operation, length of stay, complications).

Patients might prefer qualitative outcomes data: 63% (50/80) of patients in our survey welcomed published feedback from previous patients. Some online websites do offer independent qualitative feedback.¹⁶ Surgeons may prompt patients to enter this data and therefore it introduces an element of selection bias that should be avoided in the NJR audit data. A large-volume surgeon with correspondingly high numbers of positive feedback may find this improves patient confidence and engagement with the surgeon.

Surgeons

Surgeons wished to see a different dataset available to the public. This data would be collected by their trust. Surgeons were aware of the difficulty in collecting accurate data and of the limited use of the information currently available. Surgeons wanted an opportunity to review and explain data in clinical governance sessions prior to its release.

Previous surveys of surgeon’s perceptions of public reporting showed similar results to our survey with concerns regarding the validity and interpretation of results and a preference towards national monitoring of trust performance rather than individual performance.^10,12,13,17

Only 27% (11/41) of surgeons thought that the currently published data would help identify malpractice, but 49% (20/41) thought it would improve individual surgeon performance.

Previous studies of surgeons in other specialties found evidence of risk adverse behaviour and ‘gaming’ with as many as 25% admitting to avoiding operating on high-risk patients, while in our survey 61% (25/41) admitted to their own risk averse behaviour.^18–20

What data to collect and how to present it to the public?

Agreement needs to be reached concerning the data to be collected and released to the public, and supervision of the process.

Currently only surgeons performing THR and TKR have extensive data released to the public. Orthopaedic surgeons not performing TKR and THR can currently spend their career without scrutiny of their results. The publication of comprehensive data is not a new idea, but gradually this concept is being accepted by The Royal College and The Independent Healthcare Providers Network (IHPN).^21,22

Any data presented to the public in future will need to be:

• 1. Of adequate quality:

Quality (completeness×accuracy) varies with the type of data collected. Data quality is generally high for the nature of the operation but low for complications.²³ Existing surgeon data collection schemes have used operations to prompt collection. For the NJR this is the primary and revision operations.⁶

Currently the NJR misses 5 to 8% of primary operations with a higher rate of missing revisions (9 to 12%).^24,25 There is also significant underreporting of complications. Afzal et al report that in their centre over a 11-year study period, reporting of adverse soft tissue reaction to particulate debris for THR was underreported by 11%.²⁴

• 2. Of sufficient numbers to draw conclusions:

The reliability of data increases with larger numbers. To identify individual outliers, significant data numbers are required. There is little evidence that current outcome measures can reliably distinguish actual differences in surgeon quality (signal) from measurement error (noise) as the numbers of operations performed are too small.^26,27 It is unlikely that an orthopaedic surgeon doing hip and knee replacements will accumulate enough data annually to make their statistics reliable.

• 3. Presented appropriately:

The data must be statistically valid and presented in a way the public understand with explanatory notes. We found that public understanding of this data was limited with few understanding the concept of risk adjustment.

If data for every UK surgeon are going to be collected, there needs to be a robust, automated mechanism to collect and process these data. The existing clinical governance mechanism could be used to analyse and explain the large amounts of data produced.

Presented data need to take account of the fact that most surgeons are average, and that surgeon data cannot easily be used to identify good or bad surgeons.

In the example of Ian Paterson, convicted of 17 counts of wounding with intent and gross misconduct in 2017, he could not have practised as he did if he had been part of a robust clinical governance mechanism.⁹ In his case, there was a failure of clinical governance. One could argue that releasing data concerning a surgeon’s participation in the clinical governance mechanism is equally important to releasing data about his or her practice.

• 4. Acted on:

If data considered at clinical governance level did suggest a problem with a surgeons’ practice, there will need to be agreed mechanisms to intervene. Currently NJR data concerning TKR and THR are analysed twice a year to identify potential adverse performance. The surgical performance committee of the NJR reviews this data, and where there are concerns about adverse performance, the medical director of the hospital is informed.²⁵

What effect will publishing this data have?

Currently there is no evidence that NJR published outcomes data regarding THR and TKR have affected patient outcomes. The same is not true of all nationally collected data: improvements have been shown in the 30-day postoperative mortality rate from 4.3% in 2000 to 2.6% following the publication of paediatric cardiothoracic data in England.²⁸

Failings of this study

This was a single centre survey and will be limited by catchment bias. Similarly, the surgeons were from the regions of South East and West Thames. A telephone and email survey may have led to responder bias with patients more likely to access online data, also more likely to pick up the phone or answer an online survey. Patients with more polarising views may additionally be more likely to respond.

Conclusion

Both patients and orthopaedic surgeons seem to agree that some individual surgeon data should be collected and released to the public in an understandable format. We suggest that the clinical governance mechanism could add value to this data.

Conflict of Interest

B Rogers is the Editor-in-Chief of the Annals. He was not involved in the peer review process of this paper, on which he is listed as an author.