The role of case mix in the relation of volume and outcome in phacoemulsification

Abstract

Background/aim: The authors previously demonstrated a decrease in complication rate with an increase in volume of cases performed by a surgeon. All studies of volume and outcome are potentially hampered by the issue of case mix, in that some lower volume surgeons may in fact do fewer cases because they have more complex patients. This study was designed to assess the influence of case mix on the volume-outcome relation in phacoemulsification surgery that had previously been demonstrated.

Methods: This study took place wholly in Sunderland Eye Infirmary. 667 cases from between 1996 and 2001 were randomly selected from the operative lists of the six surgeons involved in a previous study. The case complexity was assessed using a potential difficulty score (PDS) devised from preoperative data predictive of potential surgical difficulty. The PDS was validated by a retrospective analysis of a sample of 100 cases.

Results: 528 complete sets of notes were retrieved. The overall PDS scores ranged from 1 to 6. There was a difference between the proportions of patients with each PDS value (p = 0.015) in the two groups, which suggested that the low volume surgeons were doing potentially more difficult cases. The median PDS for each volume group were the same ( = 1.0). Retrospective validation analysis of the PDS score revealed higher mean and median values in complicated cases compared to uncomplicated cases.

Conclusion: This follow up study re-emphasises the importance of case mix adjustment in comparative assessment of healthcare quality. These results may explain in part the trend previously demonstrated of lower complication rates for higher volume surgeons.

In our previous study investigating the relation of volume and outcome in phacoemulsification,¹ the results indicated strong evidence of a decrease in complication rate with increase in the number of performed cases. Although not statistically significant, it was also noted that low volume surgeons (defined in our study as surgeons performing fewer than 400 cataracts per year) had a higher average complication rate than the high volume ones (defined as surgeons performing more than 400 cataracts per year).

In many studies looking at volume-outcome correlations, imbalances in patient case mix among individuals and/or institutions can seriously confound the relation.² Our previous study did not address the issue of case mix and this current study was designed to ascertain if some or all of the differences noted between high and low volume surgeons results were related to case mix.

METHODS

We compared the ocular and general co-morbidity in patients undergoing cataract surgery under six consultants in a single specialised eye unit. The preoperative risk determining factors for case difficulty were assessed. The study was wholly conducted at Sunderland Eye Infirmary, which is a single specialty ophthalmic hospital geographically separated from the main general hospital. The medical record department is contained on site with separate ophthalmic notes and a computerised patient database.

The six consultants included were the same consultants included in our previous study evaluating the relation between volume and outcome in cataract surgery.¹

We randomly selected one month per year for every consultant. Once a particular month had been selected (for a particular consultant), it was excluded from the randomisation process for the remaining years. This was to gain a reasonably representative spread throughout the year. The study period was from the beginning of 1996 to the end of 2001.

Within each of the selected months, all patients who underwent phacoemulsification were identified; 20 cases were then randomly selected from these operating lists. (If a surgeon has performed fewer than 20 in a month, all the cases were included in that month.)

The case notes were obtained and the data were extracted from the preoperative notes. We did not include intraoperative or postoperative data, as we thought the information could be biased by the transcriber, following the occurrence of an intraoperative complication. Intraoperative findings may also be related to the surgeons’ performance during the operation rather than the difficulty of the operation (for example is a long phacoemulsification time related to the hardness of the cataract or to the surgeon’s technique?). For these two reasons the potential difficulty of each case was assessed using the preoperative clinical findings (as assessed from the patients notes).

This data was extracted by an experienced ophthalmic trainee and double checked by a consultant ophthalmologist who was not one of the consultants included in the study.

To assess the potential complexity of each case, a scoring system was designed based on:

1. A questionnaire sent out to all the ophthalmic consultants in the unit (n = 14). This asked them to rank the preoperative features of an eye or patient that they thought most likely to predict intraoperative complications.

2. Previous work done by Willerscheidt et al³ and Najjar and Awwad,⁴ who looked at standard questionnaire assessment of cataract surgery risk score. We also looked at other studies that compared certain risk factors with intraoperative complications during cataract surgery.^5–7

From these two sources we devised a “potential difficulty score ” (PDS). Table 1 summarises the important factors included in this scoring system. The factors can be divided into four main categories:

Table 1.

 Potential difficulty score

Co-morbidity	Score
General condition
Unable to lie flat
(spinal deformity, asthma, heart failure)	1
Severe anxiety	1
Head tremors	1
Ocular history
Previous angle closure glaucoma	1
History of complications in fellow eye	1
Previous vitrectomy plus or minus silicone oil	1
Ocular co-morbidity
Corneal cloudiness	1
Shallow anterior chamber	1
Poor pupillary dilatation and/or posterior synaechiae	1
Pseudoexfoliation	1
Weak zonules	1
High hypermetropia (axial length <20)	1
High myopia (axial length >27)	1
Cataract density
Nuclear density grade 1–2	1
Nuclear density grade 3	2
Mature brunescent cataract	3

1. General co-morbidity—for example, chest or spine diseases that could prevent optimal positioning of the patient.

2. Significant ocular history—for example, previous attack of angle closure glaucoma or vitreoretinal surgery,⁶ problems with the first eye.

3. Ocular co-morbidity—for example, corneal opacity, unstable lens. Pseudoexfoliation has previously been thought to be an independent risk factor even in the absence of obvious weak zonules or a small pupil.^4,5 Similarly post-angle closure eyes were thought to be a potential risk even without short axial lengths or shallow anterior chambers.

4. Cataract density as recorded at the last preoperative visit before the operation.⁷

The overall score was calculated for each case. Total scores could range from 1 in simple cases to a maximum possible score of 13 points in difficult cases (although for a patient to have a maximum score would be highly unlikely).

The predictive effect of the PDS was then evaluated by a retrospective validation study. We randomly selected 50 complicated and 50 non-complicated cases of patients who underwent phacoemulsification surgery. The cases were equally distributed between high and low volume surgeons. The complications reported were rupture posterior capsule plus or minus anterior vitrectomy, dropped nucleus, anterior chamber intraocular lens implantation, and conversion to extracapsular cataract extraction.

The case notes were retrieved. The preoperative medical records were photocopied and randomly presented to an ophthalmologist who was masked about the intraoperative events and the final surgical outcome.

The preoperative data were scored using the PDS and the overall score was recorded for each case. The data were collected and analysed. The mean and median values were compared between the complicated and non-complicated cases.

RESULTS

A total of 667 phacoemulsification procedures were identified during the selected months for the six included consultants; 533 sets were retrieved (80%). The retrieval rate was similar in high and low volume surgeon groups.

In all, 442 notes were retrieved at the first request and 91 sets with the second. The retrieval proportion was similar for high and low volume surgeons for the two requests. There was no statistically significant difference between the median and range of scores in the high or low volume surgeons between the first and second retrievals.

A total of 134 sets of notes could not be retrieved. Failure of retrieval was mainly secondary to destruction of notes. Basic demographic details, admission history, and surgical procedures for the cases were checked through the hospital computerised database. It was noted that these patients tended to be older in age with many having died (which explains the destruction of the notes). However, similar numbers had died in low and high volume groups and there did not appear to be an excess number of procedures—for example, retinal detachment surgery, that may have suggested a disproportionate number of difficult cases in one group or the other. In other words there was no evidence that the irretrievable notes would have changed our findings.

Five sets of notes were excluded because of incomplete preoperative data; therefore, a total of 295 cases were included within the high volume surgeons’ group verses 233 cases in the low volume group. There was an equal proportion of first and second eye in each group.

We used the χ² test to assess whether the proportion of patients with each PDS differed between high and low volume consultants. In addition, we used the rank sum test to assess whether patients operated on by high volume consultants tended to have higher PDS scores than patients operated on by low volume consultants.

When the data were analysed, the total complication score ranged from 1 to 6. The spread of scores between the high and low volume is shown in table 2. There was evidence that the high volume surgeons had patients with lower PDS scores and low volume had patients with higher scores. There was a difference between the proportions of patients with each PDS values (p = 0.015) in the two groups—for example, the proportion of patients with score of 1 is slightly higher in the high volume group than in the low volume (68% v 59%), while the proportion of patients with a score of 3 is greater in the low volume than the higher volume (13% v 7%) as shown in table 2. This was supported by the rank sum test (p = 0.022). However, the difference observed between the two groups was not great; thus the median PDS for each volume group was the same ( = 1).

Table 2.

 Spread of potential difficulty score between the high and low volume surgeon groups

Scores	High volume group	Low volume group	Total
1	201 (68%)	137 (59%)	338 (64%)
2	61 (21%)	56 (24%)	117 (22%)
3	20 (7%)	32 (13%)	52 (10%)
4	12 (4%)	6 (3%)	18 (3.5%)
5	0 (0%)	2 (0.8%)	2 (0.4%)
6	1 (0.3%)	0 (0%)	1 (0.1%)

There was a slight difference between the spread of the individual identified risks between the individual consultants for high versus low volume groups.

In the retrospective validation study, the mean value of the PDS was substantially higher in complicated cases, measuring 2.3 with a median value of 2 compared to a mean value of 1.4 with a median of 1 in uncomplicated cases.

DISCUSSION

In an era of continued health service modernisation, measuring the quality of care has become a major concern for funders and providers.⁸ Performance indicators are commonly used as measures of quality and efficiency within and between healthcare systems.

A number of high volume individuals and institutions have been shown to have better outcomes across a wide range of procedures and conditions.² Halm et al systematically reviewed volume versus outcome studies in the medical literature and found that 71% of all studies of hospital volume and 69% of studies of physician volume reported a statistically significant association between higher volume and better health outcome. This association was found to be in several disparate conditions, including pancreatic and oesophageal cancer, abdominal aortic aneurysms, paediatric cardiac problems, and in the treatment of AIDS.

The majority of the studies that compare health care performances are retrospective. The use of this type of data has the advantage of being readily available and provides information about large numbers of patients with different conditions. There are, however, potential pitfalls that can hamper the analysis and conclusions of these studies.⁵ One of the most important factors that needs to be taken into consideration to ensure reliable interpretation of the results is the issue of case mix. It is essential to try to ensure that data are derived from similar patient groups with similar clinical characteristics—that is, comparing like with like. Case mix is generally established by estimating the relative frequency of various types of patients seen by the provider in question during a given time.

Our previous study suggested that high volume surgeons tended to have lower complication rates when compared to low volume surgeons in the same unit.¹ There are many reasons why this may occur but one of the most important to try to exclude is that some or all of these differences are explained by variations in difficulty between cases. If high volume surgeons were picking potentially easier cases to operate on (and are therefore more likely to complete their large lists), they would appear to have lower complication rates. Similarly, if low volume surgeons were doing potentially more difficult cases, they would require more time to do them and therefore have smaller lists. There is also a converse situation to consider. It has been suggested that higher volume surgeons do more potentially complicated cases (presumably because of a perception that they have more experience of them) despite the large numbers on their lists.

Our results do suggest that there is a difference in case mix between the high and low volume surgeons. Although these differences are not great in that there was a similar median PDS in both groups and the spread of individual risk factors was also even, they do suggest that in the sample chosen, the low volume surgeons were operating on potentially more difficult cases. This result may therefore explain, at least in part, the trend we previously demonstrated of lower complication rates for higher volume surgeons.

These results are consistent with a number of previous studies that have shown that subtle differences in disease severity or co-morbidity may partly explain the associations between volume and outcome. Halm et al noted, in their review, that studies performing sophisticated risk adjustment using clinical data were less likely to report a positive effect of hospital volume on outcome than were studies that did not adjust for risk factors.² Similarly, a review of seven published studies demonstrating reduced mortality rates with increased volume of coronary artery bypass graft surgery found a decrease in the relation as the degree of risk adjustment increased.⁹ In a comparative study evaluating cataract surgery outcomes, Willerscheidt and coworkers noted a variation in visual acuity results by surgeon and surgical volume, yet the relation was not significant when adjusted for the patient mix factor.³

Once again, care must be taken when interpreting volume outcome studies—especially at the level of individual surgeon. As a patient, choosing a high volume hospital or surgeon does not necessarily mean complications are less likely.

In an attempt to evaluate the predictive power of our newly designed scoring system, retrospective analysis of a sample of 100 cases revealed a difference in the average scoring of complicated cases verses non-complicated cases with higher PDS score associated with higher complication rates. This suggests that the PDS can appropriately predict the extent of possible surgical difficulty in that these cases are more likely to have complications. The results of this validation help to justify its comparative use in evaluating case mix.

We have not assessed other variables that could contribute to the final outcome such as surgeon’s experience or theatre environment. A prospective validation study is thus currently under way to assess the impact of other variables on the final surgical outcome.

There are certain potential biases in our study. The data extracted from the preoperative notes were variable, as the pre-assessment notes were recorded by ophthalmologists of different grades and experience. There were also differences in documented clinical details within the notes, although this bias was reduced by the fact that this would be equally true for both high and low volume surgeons patients. The operating surgeons themselves only saw a proportion of the patients preoperatively so their potential individual bias in clinical assessment and notes writing in the study was reduced.

Because of our retrospective validation and the fact that the PDS scores were comparative, we suggest our scoring system itself did not introduce significant bias into the results.

In conclusion, our study re-emphasises the importance of case mix adjustment in comparative assessment of healthcare quality. It is important not to overestimate the magnitude of the relation between higher volume and better outcome unless case mix adjustments have been made. For these results to be truly valid, the study needs to be repeated in different units with different patient populations.

Abbreviations

• PDS, potential difficulty score