Measuring Advanced Practice Provider Productivity at the National Comprehensive Cancer Network's Member Institutions

Abstract

Introduction:

The utilization of advanced practice providers (APPs) in oncology has been growing over the last decade; however, there is no standard method for assessing an APP's contributions to oncology care.

Methods:

The NCCN Best Practices Committee (BPC) created an APP Workgroup to develop recommendations to support the roles of APPs at NCCN Member Institutions. The Workgroup conducted surveys to understand how NCCN centers measure productivity. This article will review the survey results and provide recommendations for measuring APP productivity.

Results:

Although 54% of responding centers indicated they utilize relative value units (RVU) targets for independent APP visits, 88% of APPs are either unsure or do not believe RVUs are an effective measurement of overall productivity. Relative value units do not reflect non-billable hours, and APPs perform a significant number of non-billable tasks that are important to oncology practices. Sixty-six percent of APPs believe that measuring disease-based team productivity is a more reasonable assessment of APP productivity than measuring productivity at the individual level.

Conclusion:

Our recommendation for cancer centers is to focus on the value that APPs provide to overall care delivery. Advanced practice provider productivity metrics should consider not only the number of patients seen by APPs, but also the high quality and thorough care delivered that contributes to the overall care of the patient and practice. Advanced practice providers can help improve access to care, deliver improved outcomes, and increase patient and provider satisfaction. Reducing the focus on RVUs, accounting for important non-RVU–generating activities, and incorporating quality and team metrics will provide a better overall picture of APP productivity.

Advanced practice providers (APPs) specializing in oncology are critical team members in cancer centers across the US. Many cancer patients will see an APP team member, which enables oncologists to see more patients, thereby increasing access to care. As the US population ages and fewer medical oncologists are available to care for an ever-larger number of patients, it is expected that the role of the APP will continue to expand in the future (Bruinooge et al., 2018). Despite this, there is currently no standard method for assessing an APP's contributions to oncology care or measuring APP productivity.

The ability to measure productivity in a meaningful way is a challenge for all health-care organizations (Sheiner & Malinovskaya, 2016). Productivity has numerous definitions, including “the effectiveness of productive effort, especially in industry, as measured in terms of the rate of output per unit of input” (Lexico.com, n.d.). A common business maxim states, “If you can't measure it, you can't improve it.” With the advent of clinical analytics, this maxim has become increasingly relevant in health care as organizations strive to improve and succeed.

The work relative value unit (RVU) has long been used in the US to assess productivity in health care and calculate reimbursement. Work RVUs account for the provider's work when performing a procedure or service. Variables factored into this value include the skills, physical effort, mental effort and judgement, stress related to patient risk, and the amount of time required to perform the service or procedure. RVUs are the basic component of the resource-based relative value scale, which is a methodology used by the Centers for Medicare and Medicaid Services and private payers to determine payment (American Academy of Professional Coders, n.d.). Some studies have suggested that this model does not appropriately reflect providers' time and effort spent on patient care (Pickard, 2014; Satiani, 2012; Schwartz et al., 2014).

In order to address these concerns, the National Comprehensive Cancer Network (NCCN) Best Practices Committee created an APP Workgroup to develop recommendations and best practices to support the roles of APPs at NCCN Member Institutions. The group is comprised of APP leaders from NCCN's Member Institutions. As part of the work effort, the group conducted surveys to better understand the current state of APP program structure, staffing models, and productivity metrics and measurements. This article will review the survey results and provide recommendations for measuring APP productivity at cancer centers.

METHODS

The Best Practices Committee conducted the first APP survey in 2017, before the inception of the APP Workgroup. Members were interested in learning more about the structure of centers' APP programs. Questions focused on the number of independent vs. shared APP visits, number of half-day clinics, RVU targets, clinical support provided to APPs, time spent credentialing APPs, and duration of training. The survey was drafted by members of the Best Practices Committee and was distributed to two cancer centers to pilot for content accuracy. Following the pilot, the survey was distributed via a web-based survey tool (SurveyMonkey) to the entire Committee (27 Member Institutions in 2017) for completion. Survey responses were categorized by specialty, which included Medical Oncology, Surgical Oncology, Radiation Oncology, and Stem Cell/Bone Marrow Transplant (BMT). The data were analyzed and results were sent to the Committee for review and reference.

Following the initial APP survey, the APP Workgroup was developed to further assess the 2017 survey results, collect additional information, and make recommendations for optimizing the role of APPs in oncology. The APP Workgroup designed two additional survey tools. Both surveys were piloted by two cancer centers to ensure content accuracy. After the surveys were piloted, they were administered through SurveyMonkey, and the data were analyzed by NCCN. One survey, the APP Structure and Productivity survey, was conducted in March 2019, and distributed to each center's APP lead or director of APPs for completion. The survey requested data related to the structure of inpatient and outpatient APP programs, professional development opportunities offered to APPs, and metrics utilized to measure APP productivity. The other survey, the APP Productivity and Professional Development survey, was also conducted in March 2019. This survey was anonymous, included subjective questions, and was sent to NCCN Best Practices Committee members with the request to distribute the survey to multiple APPs throughout the 28 NCCN Member Institutions. The survey requested APPs' thoughts related to productivity metrics and professional development opportunities. Survey results were maintained and analyzed in a deidentified database to ensure participant confidentiality. Preliminary results for both APP Workgroup surveys were presented to Workgroup members, as well as distributed to the Best Practices Committee for review and reference (Figure 1).

Figure 1.

Timeline of NCCN APP Workgroup and surveys.

RESULTS

Note: Some survey respondents did not answer all questions; therefore, the total number of responses varied per question for each of the three surveys.

Survey 1: NCCN Best Practices 2017 APP Survey

Seventy-one respondents from 20 NCCN Member Institutions completed the survey. Fifteen centers submitted multiple responses reflecting various specialties throughout the cancer center, including Medical Oncology (27%), Hematology (17%), Surgical Oncology (14%), BMT (14%), Radiation Oncology (6%), Pediatric Oncology (4%), Neuro-Oncology (4%), and others (14%). Of 61 respondents, 54% indicated that they do have RVU targets for APP independent visits (only independent visits can be billed by the APP). Eighty-six percent of respondents commented that the Medical Group Management Association (MGMA) is utilized as the benchmark for APP RVUs, and a couple centers reported that some divisions and departments created RVU measurement criteria internally. Annual RVU targets for APPs ranged from 1,263 to 3,000 RVUs per APP. Twenty-seven percent of 71 respondents reported that APPs participate in incentive plans. A majority of the 27% (12 respondents) indicated that incentives are based on performance and productivity metrics. Other centers commented that incentives were based on hospital-wide incentives, publications, departmental goals, and community service.

Survey 2: APP Structure and Productivity Survey

Twenty-three of 28 NCCN Member Institutions (82%) responded to this survey. Fifteen centers (65%) provided other metrics (apart from RVUs) utilized to assess APP productivity in the inpatient and outpatient settings, which included measures such as: the number of notes and orders, patient census, and length of stay (60%). Fifty-three percent indicated that the number of procedures were measured, followed by number of treatment plans (33%), number of phone calls (27%), amount of oversight (27%), and the number of prescriptions (27%). Forty-one percent of 22 centers indicated they measure team productivity goals in the inpatient setting and 40% (of 20 responding centers) track team productivity benchmarks in the outpatient setting. Team metrics tracked at centers included team RVUs (18%), number of visits (11%), and discharge times (11%). Other metrics tracked included patient satisfaction scores (7%), length of stay (7%), and patient volume (7%). Four centers indicated that the team is comprised of the physician and APP. Three centers indicated that the physician, APP, and additional staff, such as nurse coordinator, operations administrator, operations manager, and social worker, were included in the team.

Survey 3: APP Productivity and Professional Development Survey

The survey anonymously captured responses from 492 APPs at 23 NCCN Member Institutions. Sixty-eight percent of respondents were nurse practitioners and 32% were physician assistants. Fifty-two percent do not believe RVUs are an effective method to measure APP productivity, followed by 36% who remain unsure, and 12% who reported that measuring RVUs is an effective metric to assess productivity. Of 276 respondents who responded to this open-ended question, APPs indicated that the top two roles and responsibilities not accounted for in APP productivity metrics include phone calls and emails (28%) and care coordination (including outside of patient visits; 13%). Other responsibilities that were reported as unaccounted for included research, documentation, clinic preparation, prior authorizations, teaching and mentoring, peer-to-peer authorizations and activities, orders, and patient and family education.

Four hundred forty-four respondents suggested metrics that could be utilized to track APP productivity, including the amount of oversight responsibility (infusion, sick visit unit, urgent care unit, etc.) throughout the day (79%), number of phone calls (72%), number of procedures (61%), number of treatment plans (58%), and number of prescriptions ordered (58%). Sixty-six percent of 454 respondents believe their center should be measuring disease team–based productivity, and 142 respondents suggested measuring team productivity as follows: patient acuity (33%), team metrics should be determined by the department or specialty (15%), patient volume (14%), team-based RVUs (9%), and the number of procedures (6%).

DISCUSSION

Takeaway 1: RVUs Are Not Effective As the Sole Measure of APP Productivity

Although 54% of responding centers indicated they have RVU targets for independent APP visits (Survey 1), 88% of APPs are either unsure or do not believe RVUs are an effective measure (Survey 3; Figure 2). Reasons for this include that RVUs do not reflect non-billable hours, and APPs perform a significant number of non-billable tasks throughout the day that are important to oncology practices. Most APPs spend a vast amount of time answering patient phone calls and emails, providing care coordination, and assisting with obtaining prior authorizations. It was also noted that RVUs do not account for the complexities of caring for high acuity cancer patients who suffer from multiple comorbidities (Survey 3). Further clouding the RVU issue is the use of incident-to, shared visit billing, and global visits for surgical cases. These methodologies “hide” the work of the APP by billing only under the physician's name (Pickard, 2014).

Figure 2.

Survey question on RVUs as an indicator of APP productivity (N = 489).

Major themes presented in the comments from Survey 3 included the notion that using RVUs to measure APP productivity may negatively impact the MD-APP relationship if there is a real or perceived competition for RVUs. It has been suggested that when APPs and physicians have separate APP productivity targets, they may compete for RVUs, which creates a disincentive for collaboration. This may cause many downstream effects, including poor utilization of the APP resource, as well as poor job satisfaction and retention challenges. It is important that organizations understand the complexities of the collaboration needed between the physician and the APP in the complex, high acuity matrix of oncology (Pickard, 2014).

Takeaway 2: Team Goals May Be an Effective Measure to Assess APP Productivity

Surveys 2 and 3 indicated that less than half of responding centers measure team productivity in the inpatient and/or outpatient settings. It was also apparent that most responding APPs believe that measuring disease-based team productivity is a more reasonable assessment of APP productivity than measuring productivity at the individual level (Figure 3). Metrics that could measure team productivity include patient volume, discharge times, patient satisfaction scores, length of stay, team RVUs, and number of medication errors. Measuring team-based metrics would also eliminate RVU competition between the physician and APP if team RVUs were one of the metrics assessed.

Figure 3.

Survey question on measuring disease team–based productivity (N = 454).

The NCCN APP Workgroup recommends an all-encompassing approach to measuring APP productivity that recognizes both billable and non-billable activities. This would provide a broader and more realistic base of assessing productivity, as well as educate administrators about the role and scope of practice of the APP.

Takeaway 3: Other Metrics May Provide Better Insight Into APP Productivity

Survey respondents from Survey 3 suggested several other metrics that could be utilized to assess APP productivity, such as amount of oversight responsibility, number of phone calls and emails, number of procedures, treatment plans, and number of prescriptions ordered. Creating additional productivity metrics such as these may better assist centers in understanding how APPs utilize their time. Table 1 provides a compilation of APP responsibilities as reported in Survey 1 and a 2014 study by Todd Pickard, MMSc, PA-C, from The University of Texas MD Anderson Cancer Center (Pickard, 2014). These tasks are often cited as activities that APPs perform but may not be captured in a traditional RVU model.

Table 1. APP Activities That Bring Value But May Not Generate RVUs.

1.	Global visits for preoperative and postoperative care
2.	Clarification of orders for pharmacy and hospital staff
3.	Peer-to-peer review (for insurance)
4.	Hospital admission from the outpatient setting
5.	Triage/Rapid Response or Code Team duties
6.	Infusion coverage
7.	Teaching new clinical staff or trainees
8.	Committee work
9.	Administrative projects
10.	Hospital rounds/notes/discharge summary
11.	Clinical research activities (protocol review, site initiation visits, etc.)
12.	Telephone triaging
13.	Chemotherapy teaching
14.	Coordination of care
15.	Symptom management via telephone
16.	Family and Medical Leave Act, disability, insurance paperwork
17.	Over-the-counter drug counseling
18.	Radiation treatment teaching
19.	Prior authorizations

The duties in Table 1 are most often cited as being qualitative and difficult to accurately measure. The amount of time for each task can vary greatly, depending on the complexity and the needs of the patient or task. It is important to note that many of these tasks are not billable but are critical for patient care. For efficiencies, it is key for organizations to look at their care models to determine if everyone is working to the top of their ability and license. The ripple effect and cost to the institution can be enormous when clinical staff are not used to their fullest potential. Fortunately, many organizations (American Medical Association and Medicare) have worked to accurately identify and capture this work. In particular, there are new codes for hospice and palliative care counseling, shared decision-making, and telephone/telehealth billing. One challenge facing many organizations is accurately capturing this data and billing for it.

Cancer centers should assess what work APPs are doing that registered nurses (RNs) or administrative staff could be performing as members of disease-based teams. Tasks 12 to 19 in Table 1 are sometimes assigned to APPs but could, in many cases, be completed by RNs, clinical research coordinators, or administrative staff at cancer centers. Reviewing the responsibilities in Table 1 and determining what tasks are able to be reassigned to other members of the clinical team can free APPs up for more billable work and reduce the overall costs of care delivery. There is variation among centers with regards to support staff models. Center staffing models and space may determine how available and accessible support staff are to take on certain responsibilities from APPs. Creating collaborative relationships with key nursing, pharmacy, and operational leaders is critical to accomplishing this imperative. Institutions are constantly assessing staff models and ideally, this work should be included in those efforts with an aim to achieve a model that maximizes efficiency and allows staff to work at the top of their license.

A recent study from the University of Pennsylvania also found that the metrics utilized to assess APP productivity should encompass the many dimensions of APP responsibilities. Productivity categories were broken up into four areas: financial impact, professional development, patient satisfaction, and quality indicators. The study noted that important quality indicators could be utilized to measure APP productivity, including documentation and reconciliation of medication lists, pain assessment plans and documentation, smoking cessation plans, and closure of patient encounter in the medical record within 7 days of visit (Gilbert & Sherry, 2016).

Finally, value-based care is an important aspect to consider when assessing APP utilization. Under value-based care agreements, providers are rewarded for helping patients improve their health and reducing unnecessary treatments and hospitals stays. Value-based care differs from a fee-for-service, in which providers are paid based on the amount of health-care services they deliver. The “value” in value-based health care is derived from measuring health outcomes against the cost of delivering the outcomes (NEJM Catalyst, 2020). Currently in oncology, this can be difficult to measure based on disease states and the outcomes associated with each patient, but in the future, this may be a feasible option for measuring APP productivity.

CONCLUSION

Our recommendation for cancer centers is to focus on the value that APPs provide to care delivery more holistically. Advanced practice provider productivity metrics should consider not only the number of patients that APPs see, but also the high-quality care they provide that contributes to the health of their patients and the health of the practice. If utilized effectively, APPs can help improve access to care, deliver improved outcomes, and increase patient and provider satisfaction. Reducing the focus on RVUs, accounting for important non-RVU generating activities (e.g., care coordination), and incorporating quality and team metrics will provide a better overall picture of APP productivity.

Recommendations

The first concept that needs to be agreed upon is what behavior should be incentivized or what the value proposition is for the organization. In the oncology setting, volume is, by most standards, not the most important goal. Instead, value, based on cost, quality, and patient outcomes, is vital and encompasses much more than the generation of RVUs. The transactional nature of RVUs, when used as a sole measure, can impact the well-being of the workforce and lead to moral distress and burnout (Sheppard & Duncan, 2020). A comprehensive and inclusive definition of oncology patient care is essential and is what APPs strive to achieve in everyday practice. Utilizing terms such as nonproductive time carries a negative connotation and the inference that the APP is not adequately contributing. Comprehensive patient management is a better description when discussing the work of APPs. Utilizing a bundled approach that includes all aspects of the care provided by APPs is an ideal tactic that includes many of the concepts provided in this article.

Limitations

The results from this study reflect only a small sample of large academic cancer centers and do not address APP productivity measures utilized at community cancer centers. The methods used in this survey mimic the standard protocol that the Best Practices Committee utilizes to conduct its surveys. However, it should be noted that the identified Best Practices Committee members who received the survey request forwarded it to an individual at their institution who was thought to be appropriately qualified to complete the query. Therefore, there may be variability in the specific knowledge of those who completed each survey. Finally, for the anonymous, subjective survey, one Member Institution had a large number of APP responses compared with other centers, which may have skewed results on some of the questions addressed. Only certain questions from the survey delineated between nurse practitioners and physician assistants; thus, the authors are unable to make broad associations regarding differences between nurse practitioners and physician assistants in oncology practice. Finally, not all respondents completed every question, and some questions were not relevant to all institutions.