Measuring the quality of care provided to women with pelvic organ prolapse

Abstract

OBJECTIVE

Health care providers are increasingly being evaluated by the quality of care they provide. Our aim was to assess the feasibility of recently developed quality indicators (QIs) for pelvic organ prolapse (POP) and identify possible deficits in care.

STUDY DESIGN

A panel ranked 14 QIs based on the RAND appropriateness method assessing screening and diagnosis, pessary management, and surgery for POP. Retrospective chart abstraction was performed after identifying patients with a diagnosis of POP evaluated within a hospital-based multispecialty group using International Classification of Diseases, ninth edition, diagnosis codes.

RESULTS

Of 283 patients identified, 98% of those with a new complaint of vaginal bulge had a pelvic examination. The POP was described but not staged in 6% and not documented at all in 25.1%. Among those managed with pessaries, 98% had vaginal examinations at least every 6 months. Forty-nine percent of the patients who had surgery had complete preoperative POP staging. Only 20% of women undergoing apical surgery had documentation of counseling regarding different surgical options, and of the women who underwent a hysterectomy for POP, only 48% had a concomitant vault suspension. Although 71% had documentation about the risk of postoperative stress incontinence, only 14.5% had documented counseling regarding risks of mesh. Only 37% of patients implanted with mesh for POP had documented follow-up at 1 year. An intraoperative cystoscopy was performed in 86% undergoing cystocele repair or apical surgery.

CONCLUSION

The quality of care for women with POP can be feasibly measured with QIs. Processes of care were deficient in many areas, and our findings can serve as a basis for quality improvement interventions.

Quality indicators and measures are used by the US government to collect data on hospitals to measure compliance with evidence-based hospital practices.¹ The theory is that by collecting these measures, which are publicly available, hospitals will be inclined to improve their clinical outcomes and develop their own clinical guidelines to minimize complication rates.² This has been demonstrated to be a validated theory in the intensive care unit, with those units being monitored by an intensive care specialist having better clinical outcomes.³ It is hypothesized that these validated quality measures can then be further used by employers and insurance payers to develop policy.²

McGlynn et al⁴ used 439 quality indicators (QIs) to measure the quality of medical care for US adults and found that patients received only 54.9% of recommended care. They concluded that the deficits identified in adherence to recommended processes for basic care pose “serious threats to the health of the American public.”⁴ Although a great deal of research on quality of care has been conducted in other areas of medicine and surgery, there is still a paucity of data regarding assessment and quality outcomes pertaining to treatment of women with pelvic floor disorders (PFDs).

Extrapolating data from the US Census Bureau and the 2005 National Health and Nutrition Examination Survey, the number of women in the United States with at least 1 PFD will increase from 28.1 million in 2010 to 43.8 million in 2050.⁵ The lifetime risk of undergoing surgery for pelvic organ prolapse (POP) by the age of 80 years is 11%.^6,7 Of these, 29% will require reoperation.⁶

It has been estimated that 1 in 9 women will undergo a hysterectomy in her lifetime, and up to 11.6% of these women will require surgery for symptomatic vaginal vault prolapse.^6,8 Despite the large number of women affected by POP, relatively few measures of quality exist.

The primary objective of this study was to assess the feasibility of identification of a recently developed set of POP QIs in the electronic medical record (EMR). Our second objective was to evaluate whether these measures could be used to identify and quantify possible deficits in overall care provided to patients.

Materials and Methods

Approval for this study was obtained from the Cedars-Sinai Medical Center Institutional Review Board. As part of the Evaluating the Quality of Urinary Incontinence and Prolapse Treatment project, we previously developed QIs for both urinary incontinence and POP.^9,10 These QIs were modeled after those described in the Assessing the Care of Vulnerable Elders project using the if-then-because format.¹¹ For examples, if a woman has symptoms of prolapse, then she should be offered a pessary because pessaries are an effective, low-risk, nonsurgical means to improve symptoms.

These QIs, unlike in the Assessing the Care of Vulnerable Elders project, addressed care for women of all ages, not just vulnerable elders. As previously described, an expert panel of 9 physicians, including 3 urologists with expertise in female urology, 3 internists with expertise in quality-of-care research, and 3 urogynecologists, ranked 14 of 21 potential QIs for POP based on the RAND appropriateness method.¹⁰ The QIs addressed screening, diagnosis, and management of POP (Appendix; Supplementary Table 1).¹⁰ Subjects were identified based on International Classification of Diseases, ninth edition (ICD-9), code for POP (codes 618.0–618.9, Supplementary Table 2) who were treated within a multispecialty group based at Cedars-Sinai Medical Center, a tertiary level nonprofit hospital (Los Angeles, CA) between April 1, 2010, and July 31, 2011.

Eligible study subjects needed to have a complaint of POP and qualify for at least 1 QI. Patients who were identified with a diagnosis of POP by ICD-9 codes but were asymptomatic were excluded from the study because they did not receive additional care or surgery. The time frame chosen was the one that occurred after the incorporation of a new EMR.

In addition to primary care providers, 3 fellowship-trained female pelvic medicine and reconstructive surgery (FPMRS) specialists (2 female urologists and 1 urogynecologist), 6 general gynecologists, and 2 urologists who perform prolapse surgery provided the care to the patients in the cohort. Care was assessed at the patient level, meaning that if a generalist provider did not perform an indicated physical examination or specified QI but the patient was referred to specialist physician who did, then the patient received the appropriate care. Primary care physicians and gynecologists did not lose scoring points if they did not complete a QI. Gynecologists were included because they were adequately trained in assessing POP, managed pessaries, performed POP surgery, and performed clinical follow-ups. This was a retrospective chart abstraction of the EMR performed by trained nurses with experience in chart abstraction and quality assessment. Documentation of counseling was identified through office notes, surgical consents, and operative reports. Office notes and surgical consents were completed by the primary surgeon. All operative reports were reviewed by the primary surgeon if dictated by a resident.

Abstracted data were recorded into a chart abstraction tool and scoring sheet. This abstraction tool was reviewed by a senior nurse consultant at the RAND Corp (C.R.) and tested with a small sample of medical records. Abstractors considered all parts of the patient’s records when assessing whether a patient was eligible for and received the indicated care over a 6 month period of time.¹² This allowed adequate time for compliance with each QI.

Because multiple providers performed the pelvic examinations, staging systems were not standardized (ie, both Baden-Walker and Pelvic Organ Prolapse Quantification systems could be used). Partial credit was given for documentation of specific components of the pelvic examination, the anterior, posterior, and apical areas. This was calculated based on the number of evaluated compartments divided by the maximum number of compartments, which was 3. Physicians with expertise in FPMRS reviewed the records that required a more detailed clinical assessment to assess compliance with the QIs. Patient data were identified with an identification number to protect confidentiality. The database was designed and managed by the Cedars-Sinai Medical Center’s Biostatistics and Bioinformatics Core using OpenClinica, an Oracle-based, open source, web-based platform for clinical data management. Statistics were calculated using SAS version 9.2 (SAS Institute, Cary, NC). A QI was considered feasible if it could be identified even once in the EMR.

Quality of care provided was measured by constructing aggregate scores, as described by McGlynn et al.⁴ We specified the combination of variables needed to determine whether a patient was eligible for the process denoted by each indicator and whether the patient then received the appropriate evaluation and treatment. Each indicator was scored at 1 of 3 levels, that of the patient, the patient-provider, or the episode of care.⁴

For patient-level quality indicators, a pass score was given if at least 1 of the patient’s providers delivered the indicated care. For patient-provider level indicators, every dyad was scored differently. That is, the number of times a patient was counted in the denominator was dependent on the number of providers who saw the patient and could have performed the specified process. For indicators scored at the episode level, such as a follow-up visit after a pessary is placed, the number of eligible events depended on the number of episodes that occurred. To produce aggregate scores, we divided all encounters in which recommended care was given by the number of times patients were eligible for specific indicators. The results were described in proportions, ranging from zero to 100%.

Results

Ten percent of records were reabstracted to evaluate the interrater agreement, which was 97%.^12,13 There were 288 cases identified by ICD-9 codes, of which 5 were omitted because the patients were either treated outside the study period or had their surgery performed at another facility and records could not be obtained. An additional 24 were not eligible for any QIs because they were asymptomatic and had no further management. These 24 were included in the analysis only for stage of prolapse because they had a documented pelvic examination. Therefore, there were 259 cases that met inclusion criteria and qualified for at least 1 QI. The mean aggregate score for the 259 were 67.8% ± SD 35.2, meaning that overall patients received 68% of the indicated care. Of those, 110 (42%) scored 100%, but 36 (14%) scored zero percent.

Initial diagnosis

Ninety-eight percent of patients with a new complaint of vaginal bulge (83 of 85) had a documented pelvic examination by 1 or more providers (Supplementary Table 1). POP staging, either by Pelvic Organ Prolapse Quantification or Baden-Walker systems, was not documented in 71 of 283 (25.1%). POP was described but not staged in 17 of 283 (6%). POP was documented as stage I in 18 of 282 (6.4%), stage II in 84 of 283 (29.7%), stage III in 57 of 283 (20.1%), and stage IV in 36 of 283 (12.7%, Figure 1). There was no documentation at any visit prior to surgery of a preoperative prolapse assessment for specific compartment defects in 51 of 283 (18%), 138 of 283 (48.8%), and 105 of 283 (37.1%) for anterior, posterior, and apical prolapse, respectively (Figures 2–4).

FIGURE 1.

Prolapse staging for all providers

Alas. Quality of care provided for pelvic prolapse. Am J Obstet Gynecol 2015.

FIGURE 2.

Documentation of anterior prolapse

Alas. Quality of care provided for pelvic prolapse. Am J Obstet Gynecol 2015.

FIGURE 4.

Documentation of apical prolapse

Alas. Quality of care provided for pelvic prolapse. Am J Obstet Gynecol 2015.

Treatment with a pessary

All patients who complained of symptomatic prolapse and desired treatment were considered eligible for a pessary. For patients eligible for pessary treatment, documentation of a pessary discussion was present in 43% (85 of 198, Supplementary Table 1). Of those who were managed with a pessary, 98% (96 of 98) had a vaginal examination every 6 months.

Surgical management

Patients were considered asymptomatic if they were referred by another physician for POP but, however, had no elicited complaint at the initial visit. For patients with asymptomatic POP stage I or less, 83% (5 of 6) were appropriately managed without surgery. Among surgeons, POP staging documentation for women who underwent surgery was low overall.

Complete staging of all compartments preoperatively was 49% (43 of 88). However, 11% (10 of 88) had either 0 or 1 compartment staged, and 28% (25 of 88) had 2 of the 3 recommended compartments staged. Women who underwent apical surgery had documented counseling about different surgical approaches 20% of the time (8 of 41), but 71% (42 of 59) were counseled about the risk of postoperative stress incontinence. Sixty-nine patients were implanted with mesh for prolapse, but only 10 patients (14.5%) had documentation of preoperative counseling regarding mesh complications.

Ten of 21 hysterectomies performed for prolapse (48%) had a concomitant vault suspension. All 13 patients who had a sacrocolpopexy had documented counseling regarding continence procedures. Intraoperative cystoscopy was performed in 86% of women undergoing cystotcele and/or apical surgery (51 of 59).

To be eligible for QI number 11 (Supplementary Table 1) and be considered a candidate for colpocleisis, a woman needed to be older than 65 years and no longer be interested in sexual activity. Only 1 patient qualified for colpocleisis, and she was offered the proper surgical intervention based on the QI. Within 3 months of having surgery, 89% of patients (78 of 88) had a postoperative follow-up pelvic examination. However, only 37% of those who received mesh augmentation (23 of 63) had a pelvic examination and follow-up at 1 year.

Comment

In 2007, the National Institutes of Health stated research in PFDs needs to address management, treatment, and outcomes for urinary incontinence, POP, and surgical complications.¹⁴ These deficits in research were further emphasized by the American Urogynecologic Society in 2011 in which it was determined that quality-of-care research is needed to not only determine where care is inadequate but also to identify areas of overuse, in which care is of no proven benefit yet costly to the patient.¹⁵ We were able to demonstrate that QIs for POP can be used as a feasible means to abstract data from the EMR and can be used to measure quality of care. However, care in many areas was insufficient in that only 67.8% of patients received the minimum of care or the recommended evaluation, treatment, and follow-up based on the QIs for which they qualified.

Due to recent publicity related to mesh after the updated Food and Drug Administration (FDA) warning on July 13, 2011, attention has been focused on quality improvement in the surgical management of prolapse.^16,17 However, nonsurgical management of prolapse has also been largely unregulated.

The gold standard for nonsurgical treatment of symptomatic POP is the pessary. Despite being first-line therapy for many women, there remains a dearth of guidelines on its management.¹⁸ The quality indicator recommending pessary checks every 6 months based on expert opinion was under scrutiny among the expert panel members. However, it was ultimately accepted, given the vast range of patient compliance and concern for those women who might sustain a sudden decline in physical or mental capability and no longer be able to care for their pessary.

Even though pessaries are considered low risk, side effects can occur and include vaginal discharge, vaginal bleeding, and vaginal erosions. Serious complications such as vesicovaginal and rectovaginal fistulae, pessary impaction, vaginal fibrosis, and enterocele rupture with vaginal evisceration are rare.¹⁹ Complications likely arise because of neglect and can potentially be avoided with adequate follow-up.

A pelvic examination was recommended for all patients who complained of a new vaginal bulge because the examination would guide treatment. Although the majority of patients did receive an examination, the documentation regarding the presence or absence of anterior, posterior, and apical prolapse was lacking in 18%, 48.8%, and 37.1%, respectively. In addition, prior to undergoing surgery, in which preoperative staging is crucial, only 49% of patients had documentation of all compartments by their surgeon.

Studies have shown that high failure rates of isolated anterior or posterior repairs are likely secondary to unaddressed apical prolapse, and it is important that all components of prolapse are addressed.^20,21 In our study, 37.1% did not have a documented apical prolapse examination. To improve clinical practice, the apex must be addressed.

In addition to the lack of physical examination and POP staging, there was poor compliance with documentation of patient counseling. First, there is level A evidence to support pessary use, and the American College of Obstetrics and Gynecology recommends all women with symptomatic POP be offered conservative management as a first-line treatment approach.²² However, only 43% of the records had documentation that this was discussed with the patient. Given that the pessary is considered the gold standard for nonsurgical management of symptomatic POP, all women should be aware of this minimally invasive nonsurgical option.

Second, 80% of patients who had surgery for an apical defect did not have documentation that counseling about the various surgical approaches was provided. It is important to address alternative approaches because each technique has a unique success and failure rate as well as convalescence and possible complications. The sacrocolpopexy is considered the gold standard for surgical management of vaginal vault prolapse, and prospective randomized trials have demonstrated its superiority over vaginal vault suspensions.^23–25 However, the vaginal approach is found to have fewer complications and allows patients an earlier return to daily activities.^24,25

Finally, only 14.5% of patients who had transvaginal mesh placed had documentation of preoperative counseling about the risk of future mesh complications. The compliance with this QI was alarming low and, given the recent FDA mesh warnings, puts physicians at potential legal risk.^16,17 Given that mesh complications can be serious, it is imperative that patients are informed of potential risks so they have appropriate expectations and can make an informed decision about their surgical treatment.

Although there is lack of level 1 data comparing hysterectomy alone vs hysterectomy with vaginal vault suspension (VVS) for uterine POP, epidemiological studies have shown that women who undergo a hysterectomy for POP have a higher risk of recurrent prolapse than women who have a hysterectomy for other indications.²⁶ In fact, Dallenbach et al²⁷ showed that women who underwent hysterectomy alone for prolapse had a 4.7 times higher risk of subsequent prolapse repair than women who had a hysterectomy for other gynecological problems, emphasizing the importance of a VVS at the time of hysterectomy. Based on this evidence, a QI was developed that all hysterectomies for symptomatic POP be done with a concomitant VVS to reduce future POP recurrence (QI number 8, Supplementary Table 1). However, only 52% of patients with POP had a VVS during the time of their hysterectomy.

We evaluated 2 QIs that addressed counseling about the option of a concomitant procedure to prevent the development of de novo stress urinary incontinence (SUI) after prolapse surgery. QI number 9 (Supplementary Table 1) recommended counseling regarding the development of post-operative SUI after anterior or apical surgery and the placement of a mid-urethral sling based on the Outcomes Following Vaginal Prolapse Repair and Mid Urethral Sling trial.²⁸ Compliance with counseling on this QI was 71%.

QI number 10 (Supplementary Table 1) recommended counseling about concomitant continence procedures at the time of colpopexy, regardless of the presence of demonstrable SUI during preoperative testing. This is based on findings of an increased risk of postoperative incontinence after colpopexy, based on the Colpopexy and Urinary Reduction Efforts trial.²⁹ For this QI we found 100% compliance.

Additionally, there was only 1 woman older than 65 years who was no longer interested in sexual activity and therefore was eligible for a colpocleisis. Therefore, we were able to determine only the feasibility of this QI but cannot comment on patterns of use for colpocleisis.

We found varying degrees of compliance with QIs addressing follow-up. Although 89% of patients who had surgery underwent a postoperative pelvic examination within 3 months, 100% compliance would ensure that postoperative complications are identified. However, the lack of follow-up (either at 3 months or 1 year) in 67% of the patients with mesh augmentation is worrisome, given that mesh complications can present in a delayed fashion.^16,17 Compliance with long-term follow-up could have been low secondary to poor patient follow-up or patient following up with another physician. However, it should also be the responsibility of the surgeon to counsel patients regarding the importance of long-term follow-up.

We recognize several limitations to our study. First, this is a retrospective chart abstraction that required data mining and data entry, which can lead to interpretation bias and errors. However, this was minimized by the use of 2 skilled nurse abstractors with an interrater agreement of 97%. Second, although the RAND appropriateness method is a validated means to develop QIs, the final QIs that were ranked might have been different had another set of panelists participated.

In addition, QIs need to be updated as new evidence becomes available. Also, although it appears many physicians did not comply with the recommended QIs, it is possible the QIs were met, but documentation on the part of the provider was poor.

Lastly, this was a single institutional study that included generalists as well as fellowship-trained surgeons in female pelvic medicine and reconstructive surgery, and the findings may not be generalizable to other practice settings. However, given that the group in this study included fellowship-trained female pelvic medicine specialists, it is more likely the care we measured was better, not worse, than that provided in many communities.

We acknowledge that including primary care providers and gynecologists who have not been trained in FPMRS raises some questions about study design and analysis. However, the purpose of the study was to assess overall care given to patients by all providers, not specifically looking at the care provided by each type of provider. In addition, the gynecologists at our institution were usually the physicians performing the initial patient evaluation as well as providing pessary care and performing vaginal surgery. It is likely that at most institutions and hospitals, there are gynecologists providing similar care to their patients, so it is important to develop QIs that can be applied to the care they are providing. Given that QIs will most likely be used in the future by hospitals, companies, and payers to evaluate the quality of health care physicians are providing in relation to patient outcomes, it is imperative that we develop QIs for all those providing care. In our future studies, we plan to further investigate patient care at the level of provider and specifically evaluate the care provided by those who are fellowship trained vs those who are not. Further areas of study could also include a larger prospective multicenter review of the QIs using a standardized POP classification system.

Other comments

The quality of care for women with POP can be feasibly measured with QIs. The processes of care were deficient in many areas, and our findings can serve as a basis for quality improvement interventions.

Clinical impact

There are several items of clinical impact from this study. First, pelvic organ prolapse quality indicators can be used as a measure to assess patient care in the electronic medical record. Second, QIs will most likely be used in the future by hospitals, companies, and payers to evaluate the quality of health care physicians are providing in relation to patient outcomes. It is imperative that we develop quality indicators for all physicians providing care. And finally, future studies are needed to assess the quality of care provided to patients, based on physician specialty and fellowship training.

FIGURE 3.

Documentation of posterior prolapse

Alas. Quality of care provided for pelvic prolapse. Am J Obstet Gynecol 2015.