Quality Assurance Through Quality Improvement and Professional Development in the National Breast and Cervical Cancer Early Detection Program

Abstract

Quality assurance (QA) is the process of providing evidence that the outcome meets the established standards. Quality improvement (QI), by contrast, is the act of methodically developing ways to meet acceptable quality standards and evaluating current processes to improve overall performance. In the case of the National Breast and Cervical Cancer Early Detection Program (NBCCEDP), the desired outcome is the delivery of quality health care services to program clients. The NBCCEDP provides professional development to ensure that participating providers have current knowledge of evidence-based clinical standards regarding breast and cervical cancer screening and diagnosis and are monitoring women with abnormal screening results for timely follow-up. To assess the quality of clinical care provided to NBCCEDP clients, performance data are collected by NBCCEDP grantees and compared against predetermined Centers for Disease Control and Prevention (CDC) benchmarks known as Data Quality Indicator Guides. In this article, the authors describe 1) the development and use of indicators for QI in the NBCCEDP and 2) the professional development activities implemented to improve clinical outcomes. QA identifies problems, whereas QI systematically corrects them. The quality of service delivery and improved patient outcomes among NBCCEDP grantees has enhanced significantly because of continuous monitoring of performance and professional development. By using QA, NBCCEDP grantees can maximize the quality of patient screening, diagnostic services, and follow-up. Examples of grantee activities to maintain quality of care are also described in this report.

INTRODUCTION

Since the mid-1990s, the Institute of Medicine (IOM) has addressed the quality of healthcare in the United States. The IOM established a committee called the National Roundtable on Health Care Quality, which was charged to identify issues and provide recommendations related to improving the quality of health care across the nation.¹ In 1990, the IOM defined quality care as the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.² On the basis of that definition, they identified 3 major problems with health care: 1) excessive, unnecessary, or inappropriate care; 2) under-use of needed, effective, and appropriate care; and 3) errors and negligence in care. From that report, it was concluded that measuring the quality of health care is needed to identify the problems and provide solutions that improve health and prevent harm. Quality assurance (QA) is the process of providing evidence that the outcome meets established standards. Quality improvement (QI), by contrast, is the act of methodically developing ways to meet acceptable quality standards and evaluating current processes to improve overall performance.^3,4 QA identifies problems, and QI systematically corrects them. Consequently, many health agencies began to develop quality measures and to implement continuous QI efforts.

When drafting the public law that authorized the National Breast and Cervical Cancer Early Detection Program (NBCCEDP), the US Congress wanted to ensure that quality standards were maintained for all clinical services provided to women enrolled in the program. The Breast and Cervical Cancer Mortality Prevention Act (Public Law 101-354) states that all NBCCEDP grantees shall have access to mechanisms through which they “can monitor the quality of screening procedures for breast and cervical cancer, including the interpretation of such procedures.⁵” Therefore, since the inception of the NBCCEDP, QA and QI were incorporated as integral components of program performance. Before developing and implementing QI activities, 4 quality principles were implemented.⁶

The program must:

1. Identify and meet the needs of patients, their families, and providers;

2. Understand the systems and processes encountered by patients moving through the medical system as they receive care and the processes by which clinical decisions are made;

3. Assure that appropriate stakeholders are committed and vested in quality outcomes; and

4. Collect data to identify problems/barriers and measure program performance.

From these principles, key performance standards and QI efforts were established. QI involves assessing the level of program performance, implementing improvements based on that assessment, and monitoring the improvements made. QI allows flexibility for grantees to conduct specific activities that aim to:

• Identify and resolve root causes of quality problems;

• Identify process steps for meeting quality standards;

• Improve efficiency of services provided;

• Identify and meet the needs of clients; and

• Ensure cost-effective program performance.

The law also mandates that professional development services be included as an integral part of the program. It describes this function as activities “to improve the education, training, and skills of health professionals (including allied health professionals) in the detection and control of breast and cervical cancer.”⁵ Funds must be available to ensure that the care provided to program clients is evidence based using the most current information. Educational efforts are directed toward all patient care providers participating in the Breast and Cervical Cancer Control Program to ensure quality of the health care services. This includes mammography technicians, radiologists, nurse practitioners, registered nurses, case managers, clinical nurse midwives, physician assistants, pathologists, laboratory technicians, obstetricians/gynecologists, internists, family practitioners, and adolescent medicine physicians as well as providers’ non-clinical staff who are responsible for duties such as patient enrollment, scheduling, and referrals to specialists.⁷

In this article, we discuss mechanisms of QA through both QI and professional development in the NBCCEDP and demonstrate how these mechanisms are evaluated to determine overall success. We also provide 4 grantee examples of activities implemented to achieve QI.

Using Performance Standards for QI

Because the NBCCEDP grantees are mandated by public law to plan and implement QA activities that evaluate health services provided to clients, a set of Minimal Data Elements (MDEs) were developed to be used by grantees to ensure the collection of data components for measuring the quality of clinical care.⁸ The data collected include screening examination dates and results, information on whether or not further diagnostic testing is planned, what type of diagnostic testing is required, final diagnoses, and initiation of treatment if cancer is diagnosed. These data are reported on all women to the NBCCEDP grantees by the providers at the point of screening. Data are then consolidated and submitted biannually to the Centers for Disease Control and Prevention (CDC). Each grantee’s aggregated MDE data are compared against a set of performance indicators developed by the Centers for Disease Control and Prevention (CDC) known as data-quality indicator guides (DQIGs) (Table 1). The DQIGs assist grantees with monitoring completion of the screening process, the provision of appropriate clinical services, and the timeliness of diagnosis and treatment. A subset of 11 DQIGs (identified by the NBCCEDP as core performance indicators) is used to monitor the quality of key breast and cervical cancer clinical care services—timeliness of final diagnosis after an abnormal screening result and completeness of care. A grantee’s performance is based on meeting the predetermined performance standard for each indicator. The linkage of data quality and quality of clinical care delivered are intertwined in a 2-way relation that helps to ensure that women enrolled in the NBCCEDP are receiving quality and timely follow-up care.

TABLE 1.

National Breast and Cervical Cancer Early Detection Program Core Performance Indicators and Standards

Core Performance Indicator	Performance Standard, %
Breast cancer
Abnormal screening results with complete follow-upa	≥90
Time from abnormal screening results to a final diagnosis >60 d	≤ 25
Treatment started for breast cancera	≥90
Time from diagnosis of breast cancer to start of treatment >60 d	≤20
Cervical cancer
Abnormal screening results with complete follow-upa	≥90
Time from abnormal screening results to a final diagnosis >90 d	≤ 25
Treatment started for diagnosis of HSIL, CIN2, CIN3, CIS, invasive carcinomaa	≥90
Time from diagnosis of HSIL, CIN2, CIN3, or CIS to treatment >90 d	≤20
Time from final diagnosis of invasive carcinoma to start of treatment >60 d	≤20

Abbreviations: CIN, cervical intraepithelial neoplasia; CIS, carcinoma in situ; HSIL, high-grade squamous intraepithelial neoplasia.

^aThis indicator is not measured until about 9 months postscreening for both breast and cervical cancer.

The CDC provides DQIG feedback reports to grantees with a summary indicating whether their performance meets program performance standards. This report is used by the grantees, as well as the CDC, to identify key areas of quality concern for intervention. In response, many grantees initiate internal QI monitoring activities to target the key performance areas identified for improvement. These activities have resulted in performance improvement in areas such as the percentage of clients with abnormal screening results who have complete diagnostic follow-up and the percentage with a cancer diagnosis that have initiated treatment.⁹ To reward excellence and efficiency for program performance, the CDC developed a performance-based funding process. This process, which was first implemented in 2004, involves the CDC rating each grantee’s performance based on the core performance indicators. Programs considered “high performers,” are eligible for modest budget increases if funds are available. In contrast, “low performers,” may receive no budget increase or may have budget reductions. Linking program performance to funding awarded to grantees by the CDC has resulted in a steady improvement in the number of grantees meeting the DQIG performance standards. For more details, see the article by DeGroff et al in this supplement.¹⁰

Grantees incorporated QI strategies as they developed, implemented, and evaluated procedures, practice guidelines, treatment protocols, and expected health care outcomes for all clients. Quality has improved measurably among the CDC’s 68 grantees as measured by meeting the performance standards of the DQIGs.¹⁰ Before 2004, aggregate data for all grantees indicated that only 7 of 10 core performance standards were met. However, after implementing the performance-based funding process, grantees are meeting all 10 standards. An example is depicted in Figure 1, illustrating the change over time in the percentage of breast and cervical cancer abnormal screening results that have had complete diagnostic follow-up. Therefore, performance-based funding seems to have provided a stimulus for implementing QI in all aspects of program performance.

Figure 1.

Completeness indicators for the data-quality indicator guides are measured as the percentage of abnormal breast and cervical cancer screening results that had complete diagnostic follow-up: the National Breast and Cervical Cancer Early Detection Program standard is ≥90% of abnormal screens. PY indicates program year.

Grantees use the performance standards for both prospective and retrospective program evaluation.¹¹ Prospective use includes tracking the current screening status of a woman to promote regular rescreening and prompts to providers about results still pending. This real-time use of data allows case managers to intervene (if needed) to facilitate timely and complete care. Retrospective use allows grantees to assess what has already occurred and to identify overall program trends. Reviewing retrospective data provides opportunities to improve practice patterns of individual providers, evaluate the types of clinical services performed for the patient, and assess whether an improvement made was the result of a QI activity.

Using the DQIG feedback reports, the CDC has also been able to assess the feasibility and usefulness of these performance standards. For example, the standard for the time from an abnormal cervical screening result to diagnosis was initially 60 days. Despite numerous interventions by grantees, the 60-day standard had been difficult to meet for various reasons, such as limited availability of colposcopy providers that resulted in increased wait times for colposcopy appointments, the rescheduling of appointments, and delays in the referral of women with abnormal Papanicolaou (Pap) tests from external sources into the NBCCEDP for diagnostic evaluation. In 2009, the CDC noted that this performance standard was established based on the best clinical judgment and standards of care at the time it was developed and that, in the absence of new scientific data suggesting otherwise, the standard could be revised. Therefore, the timeliness standard was modified from 60 days to 90 days for the time from abnormal Pap test results to diagnosis. This change demonstrates how monitoring the quality of clinical care was used to revise the performance standard for a quality indicator.

Each grantee implements QA/QI activities to meet the unique needs of their program and health care delivery system. Some grantees manage all of their QI activities centrally, whereas other grantees decentralize their QA activities and have contractors monitor and intervene on quality concerns at a regional level. All grantees prioritize their QA activities by using DQIG data to identify problems and monitor trends in performance. It has been most effective in evaluating both what was done (adherence to benchmarks, standards, protocols, etc) and how it was done (process of delivering services).

Professional Development/Education

Providers participating in the NBCCEDP are expected to provide patient care consistent with professional clinical recommendations and NBCCEDP policies. However, guidelines are only useful to the extent that they are adopted and routinely used by providers. Over the years, the focus of professional development has evolved from developing and implementing clinical program guidelines and policies and conducting training (breast and cervical cancer epidemiology, clinical updates, cancer screening, and diagnostic testing) to placing greater emphasis on effectively disseminating this information. In recent years, the focus of the CDC has been on evidence-based interventions that increase breast and cervical cancer screening, as recommended by the Guide to Community Preventive Services,¹² and developing the capacity to use evidenced-based techniques, such as academic detailing,¹³ to deliver education and training to program providers. By using research-proven methods, grantees are better able to improve clinical practice and increase screening rates. Compliance with the national program’s clinical and performance policies are required in a grantee’s provider contracts.

To assess the actual practice of professional development in the NBCCEDP, in October 2007, Dr. Cam Escoffery and colleagues at the Emory University Rollins School of Public Health conducted an inventory of the NBCCEDP interventions. The purpose of that study was to provide an overview of professional development activities being implemented by grantees and to assess the extent to which these activities were based on evidence-based strategies.¹⁴

In total, 252 professional development activities were reported. Among the grantees, 59% reported using conferences such as workshops, lectures, and grand rounds; 49% reported using educational materials such as tool kits, manuals, Web sites, and training videos or webcasts; 23% used educational outreach/academic detailing; 12% used audit and feedback reports; 10% used local opinion leaders; and 7% used organizational activities (Table 2). Some grantees (29%) reported using multi-component activities. The objectives for the professional development activities reported by the grantees included: to increase provider knowledge (72%), change provider performance (51%), change provider attitudes (46%), and increase the number of women recruited into the program (44%). When deciding which topics to address for professional development activities, the top 3 reasons reported were CDC’s policy change (93%), questions from providers (88%), and findings from the DQIG reports (80%).

TABLE 2.

Reported Professional Development Activities Used by National Breast and Cervical Cancer Early Detection Program Grantees

Types of Activities	No. of Grantees (%)a
Conferences	148 (59)
Educational materials	124 (49.4)
Educational outreach/academic detailing	57 (22.7)
Audit and feedback	29 (11.6)
Local opinion leader	26 (10.4)
Organizational interventions	17 (6.8)
Provider incentives	10 (4)
Provider reminders	9 (3.6)
Multicomponent activities	74 (29.4)

^aActivities are not mutually exclusive; therefore, percentages will not total 100%.

Programs reported that they perceived the most effective professional development activities to be conferences (34%), trainings (24%), educational materials (22%), meetings (12%), speakers (7%), updates (7%), and technical assistance (7%). Whereas it has been demonstrated that conferences and workshops are some of the least effective methods of changing provider behavior,¹⁵ it has been demonstrated that audit and feedback,¹⁶ provider reminders,¹⁷ and academic detailing^13,18,19 are more effective strategies. These methods offer unique opportunities for providers to deliver the best care for their patients in a complex and rapidly changing medical world.

State Grantee Activities

Michigan—Improving clinical and data quality

The Michigan Breast and Cervical Cancer Control Program (MBCCCP) operates in 20 county/regional health departments and at 1 cancer institute. Breast and cervical screening and diagnostic services are provided to approximately 24,000 women each year by more than 700 participating health care providers. The 21 agencies operating the program are responsible for contracting with providers in their service delivery area to provide the MBCCCP clinical services to women enrolled in the program. The MDE data are entered by the local agencies into a state-wide, Web-based system, the Michigan Breast and Cervical Cancer Control Information System.

In 1999, the MBCCCP was failing to meet several DQIG core performance standards. Staff at the Michigan Department of Community Health observed that records in the database did not accurately reflect the quality of clinical services being provided to MBCCCP clients. Documentation of clinical services in the statewide data-base was inconsistent and often incomplete compared with the medical record. The MBCCCP clinical and data staff collaborated with the Michigan Peer Review Organization to develop processes that addressed clinical protocols and MDE reporting, and they assisted local agencies in understanding the interaction between delivery of care and proper recording of data that accurately reflected that care. Regional workshops, called clinical problem-solving sessions, were developed for both clinicians and data staff that demonstrated the linkage between clinical care delivery and DQIGs. These sessions gave local and state staff the opportunity to work together in identifying potential solutions to problems and interventions that could be implemented to improve their agency’s processes. In addition to these workshops, Michigan instituted telephone and in-person consultations by state staff and developed clinical data reports for local staff to highlight areas in which performance standards were not met.

Data and clinical staff joined forces and began to understand the complexities and learn the language of each other’s domain. Staff began to work as a team to determine whether the quality issues were data driven, clinically driven, or both. This team approach allowed Michigan to build a strong foundation for QI. Michigan began its team approach by clarifying terminology into standard language and creating clinical and data algorithms to follow the medical standards of care. Both clinical staff and data staff used these algorithms to translate clinical care delivery to data requirements with an emphasis on the accurate documentation of care. All of the measures taken as part of the QA/QI program in Michigan have resulted in a reduction in data errors, increased adherence to the medical guidelines, and reduction in the average cost per woman for direct clinical services. Michigan continues to strive toward excellence in data quality, clinical care delivery, and responsible stewardship of program funds. Over the past several years, Michigan has consistently met all of the DQIG core performance standards.

Louisiana—Improving quality care

The Louisiana Breast and Cervical Health Program (LBCHP) provides breast and cervical cancer screening and diagnostic services through public and private healthcare providers. LBCHP provides screening services to about 13,000 women each year in the state of Louisiana. In 2006, the LBCHP determined that it had failed to meet the performance standards for the cervical cancer screening timeliness indicator of obtaining a final diagnosis after an abnormal screening Pap test result. At that time, the indicator standard was <60 days from the date of abnormal screening. After a review of their MDE data, they were able to identify all cases that had not met the standard. The providers who performed the Pap test for each of the cases were requested to review their cases and provide details on why there was a delay in obtaining a final diagnosis. It was noted that 1 LBCHP clinic disproportionately contributed to the number of untimely diagnoses. Internal review indicated that the delays to final diagnosis were caused by repeated missed appointments. In many of these cases, the client missed up to 6 scheduled appointments. The LBCHP services coordinator then contacted these women to determine the reasons. They observed that women either did not comprehend the need for follow-up or were afraid.

The LBCHP staff began to work closely with the provider clinics to monitor all high-grade, abnormal Pap test results. For each scheduled follow-up appointment, the client received a reminder call about her appointment 1 or 2 days before the appointment. The client was asked whether she had any barriers to attending the appointment (eg, not being able to get time off work, not having transportation, not being able to get child care, etc). The provider also made an extra effort to explain the Pap test finding and why showing up for the appointment and completing the diagnostic evaluation was so important. The day before the appointment, the client was called again to remind her of her appointment and to ask whether she would have any barriers or needed help making her appointment. If a barrier was identified, then the provider contacted the LBCHP services coordinator for assistance. LBCHP has been able to provide assistance such as gas cards to clients when providers identified this as a need.

Since implementation of this intervention, LBCHP has been able to consistently meet the standard for this cervical cancer screening indicator. LBCHP continues to monitor each abnormal Pap test result. The providers continue to make reminder calls to the clients and identify whether there are any barriers to making the scheduled appointments. LBCHP has had no further problems meeting their goal. The average length of time from abnormal screening result to final diagnosis in the LBCHP is now 55 days.

South Carolina—Office detailing

South Carolina’s program, the Best Chance Network (BCN), provides services through contracts with a statewide network of over 200 providers with at least 1 in each of the state’s 46 counties. The American Cancer Society has managed the service coordination, professional development, and outreach and recruitment components of the BCN since 1995 through a contract with South Carolina Department of Health and Environmental Control. Professional development is an integral component of the BCN and provides a continuum of training opportunities that were developed to assure optimal service delivery. Regional Service Coordinators provide hands-on guidance and on-site professional education to provider practices, often going beyond clinical instruction to assisting the practices to better serve the patient and improve office operations.

The BCN Professional Development Manager designs and revises educational materials to reflect the needs identified by the program’s data, thus assuring consistency with the quality-of-care guidelines from the NBCCEDP policy. Fashioned after office detailing (or academic detailing), ¹⁵ presentations are given by the Regional Service Coordinators to providers at their individual office settings with dates and times that accommodate providers’ schedules. Currently, there are 2 professional development programs available to BCN providers, and both offer continuing education (CE) credits. Orientation to the Best Chance Network Program (1.5 CE credits) is a mandatory presentation for all new providers and their staff. Each new provider is given the Best Chance Network Provider Manual, a comprehensive guidebook that covers eligibility requirements, clinical guidelines, program procedures, forms completion, scheduling, screening and follow-up, and billing and reimbursement procedures. The second program offered by BCN is Professional Education for Best Chance Network Providers. This educational program (1 CE credit) includes up-to-date statistical information on breast and cervical cancer, recommended screening guidelines, follow-up of abnormal breast and cervical findings, and the importance of cultural sensitivity.

From 1991 to 1995, a centralized model of education delivery was used but was identified as ineffective, with fewer than 100 providers and staff attending the centralized seminars. Since 1996, professional development has been provided through on-site offerings in provider offices. Approximately 458 providers received on-site training in the first year, and greater than 8359 individuals have attended on-site professional development programs since its implementation. This on-site strategy has proven to be a cost-effective and efficient way to educate and update providers.

Kentucky: Web-based learning program

The state of Kentucky has 120 counties and is geographically diverse. The Kentucky Department for Public Health is made up of 41 independent local county health departments and 15 district health departments. To enhance the learning process for clinicians, the Kentucky Women’s Cancer Screening Program (KWCSP) contracted with the Kentucky Cancer Program at the University of Louisville to develop Web-based learning modules for its providers. These modules educate providers on evidence-based clinical screening practices used by the KWCSP. The modules are also used to train nurse case managers in local Health Departments who conduct day-to-day quality-assurance activities, contact patients with results, and schedule appropriate follow-up to the test results. The modules promote continuity of case management and clinical service delivery across the state.

The Kentucky Cancer Program has also developed 3 modules for the use of health care providers in the community. One module introduces physicians and their staff to the KWCSP and the Breast and Cervical Cancer Treatment Program, and 2 modules provide information on how to identify and effectively outreach to women who have never or rarely been screened for breast and cervical cancer.

The platform used for the Web-based modules is the Kentucky Training-Finder Real-Time Affiliate Integrated Network (TRAIN) program. These modules are also available for use by private health care providers in Kentucky to increase their use of the KWCSP and the Breast and Cervical Cancer Treatment Program. Continuing medical education credit is provided for Kentucky physicians through a collaborative agreement between the Kentucky Cancer Program and the Kentucky Medical Association. The Kentucky Cancer Program promotes the TRAIN modules to physicians, nurse practitioners, physician assistants, and registered nurses through presentations at state-wide professional meetings, journal articles, Web postings, direct mailings, etc.

Conclusion

Improving quality in the delivery of health care services is a complicated endeavor that continues to evolve. A growing body of evidence indicates that certain QI processes are associated with better clinical outcomes. Measures taken to ensure that clients receive timely and appropriate screening and diagnostic services and timely treatment services are critical components in reducing cancer mortality.⁹ In the current health care environment, the importance of integrating both QI measures and professional development into cancer screening programs is recommended. Thus, as the health care system undergoes a reconfiguration, NBCCEDP grantees will be challenged to implement improvement processes that balance the effective and efficient delivery of quality care while experiencing shrinking financial resources.

CDC grantees have established program operations to meet their unique needs and, for the past 20 years, have implemented various QA approaches and techniques. The most program successes have been validated through the evaluation of both what is done (adherence to benchmarks, standards, protocols, etc) and how it is done (process of delivering services). To provide QI, NBCCEDP grantees must: 1) identify quality issues using data, 2) determine the causes of the issues (data driven or clinically driven), and 3) develop effective program monitoring and improvement processes to rectify those issues. In addition, to ensure that the most appropriate, high-quality care is being provided, professional development/education activities should be designed to educate providers on the science that supports the clinical guidelines and protocols. It is imperative that programs separate data issues from clinical care issues to help ensure that women enrolled in the NBCCEDP are receiving the best care possible. The CDC will continue to support and encourage the use of QI and professional development activities by grantees as part of their QA.