Abstract
In medicine, neuro-oncology practice falls outside the scope of established practice requirements for the specialties of neurology, medical oncology, and neurosurgery, justifying the prerequisite of specialized training to practice neuro-oncology. Neuro-oncology advanced practitioners (AP) also require specialization beyond the scope of population-based generalist training and education. This quality improvement project’s primary purpose was to develop a professional practice model (PPM) for APs employed at an academic medical center (AMC) ambulatory neuro-oncology practice. Using the focus, analyze, develop, execute, and evaluate (FADE) quality improvement methodology, the authors (1) reviewed literature and relevant professional organizations to identify possible professional competencies for neuro-oncology APs; (2) analyzed data to develop evidence-based practice domains; (3) used purposive sampling to recruit an interprofessional team of neuro-oncology experts; and (4) conducted a Delphi study with an interprofessional team of experts to gain consensus on practice domains and professional competencies. Twenty-three participants (n = 23) were recruited for the Delphi study, which was executed via electronic transmission using the Web-based software Qualtrics. After two rounds of the Delphi survey, the expert team reached consensus on six domains of practice, with 50 corresponding competency statements. Through interprofessional collaboration and consensus, this quality improvement project successfully created a PPM for an AMC neuro-oncology AP team. The PPM supports neuro-oncology APs by validating the unique set of skills that combines several specialties. The PPM provided the framework to standardize orientation and training, evaluate performance, and support the professional development of an AMC neuro-oncology AP team.
Neuro-oncology is a subspecialty that involves the neurological, medical, surgical, and oncologic management of primary central and peripheral nervous system neoplasms. Neuro-oncology practice manages the neurologic complications that result directly from the disease and provides symptom management related to the modalities used to treat the disease (American Academy of Neurology, 2013). In medicine, neuro-oncology practice falls outside the scope of established practice requirements for the specialties of neurology, medical oncology, and neurosurgery, justifying the prerequisite of specialized training in order to practice neuro-oncology (American Academy of Neurology, 2013).
There is a growing workforce of advanced practitioners (APs) in specialty care, especially in oncology, and the demand for their services will continue to grow (Coombs, 2015). Newly insured patients, an aging population, and long-term needs of survivors (American Society of Clinical Oncology, 2015) are driving the demand for cancer care. Advanced practitioners contribute significantly to quality oncology care through collaborative physician partnerships. There are a number of different models in which APs and physicians deliver care. These collaborations have demonstrated improved patient care, increased clinical productivity, improved access for new patients, urgent care patient management, survivorship care, and coverage of the academic physician (Coniglio, 2013; Shulman, 2013). However, there is a dearth in literature regarding standardized orientation practices of an AP working in specialty and subspecialty care.
Practice in neuro-oncology also requires AP specialization beyond the scope of generalist training and education currently provided by AP programs. Unfortunately, no postgraduate fellowships for APs provide the education and training necessary for neuro-oncology practice, and no standard professional competency framework exists.
BACKGROUND
This quality improvement project involves a neuro-oncology program in a hospital-based subspecialty clinic nested within a designated National Cancer Institute Comprehensive Cancer Center, an entity of a southern academic medical center. The mission of the neuro-oncology program is to improve the outcomes of patients with primary central nervous system tumors through laboratory, translational, and clinical research while providing supportive, quality care to patients and their families. Care delivery occurs through an interprofessional patient-centered care model.
Within the neuro-oncology program, the clinical care team includes physicians, APs, registered nurses (RNs), a clinical pharmacist practitioner, social workers, a psychiatrist, a neuro-psychologist, and a child life specialist. Each clinical team comprises two or three physicians, four APs, and two RNs. Advanced practitioners and RNs support the panels of patients assigned to the respective physicians on their team.
Over the previous 6 years, the neuro-oncology program experienced substantial turnover at all levels. Reasons for attrition are multifactorial, but the turnover illuminated the inherent challenges of efficiently educating and training new staff, specifically APs. Ten APs were hired, eight of whom were new graduates. Of the new graduates, three resigned prior to completing 2 years of service. Adequately educating and training new providers while continuing to provide high-quality care has been challenging for the senior staff. Prior to this project, no standard orientation model existed. Additionally, as far back as 2012, it was documented as an action item in the meeting minutes of a clinical operations meeting (which includes physicians, APs, neuro-oncology fellows, and the administrative director) that as part of the collaborative practice between physicians and APs, core competencies for APs would be developed. Prior to this project, no work had been started.
The financial burden on this organization from staff turnover has been substantial. A rough estimate of the direct and indirect cost of AP turnover at this academic institution is 1.5 times the incumbent’s salary. At the start of this project, the minimum starting salary for APs was $67,000. Using only the minimum starting salary, the cost per person lost was approximately $100,000, which means turnover cost the program at the very least more than a million dollars over the past 6 years (Duke Human Resources, 2015). A market adjustment for fiscal year 2017 brought the starting salaries for APs up to $81,010 annually, which will increase the financial burden to the institution if AP turnover remains high.
Literature has demonstrated that an orientation for newly graduated APs without clear expectations increases AP turnover and overall job dissatisfaction (Sargent & Olmedo, 2013). Several studies have validated that structured orientation programs and professional development programs substantially improved retention rates (Opperman, Liebig, Bowling, Johnson, & Harper, 2016). Pursuing strategies for AP retention is a practical financial goal for health-care organizations.
OBJECTIVES AND AIMS
The purpose of this quality improvement project was to develop an innovative neuro-oncology professional practice model for the AP employed with the academic medical center neuro-oncology program. The project aims were to (1) use data from the literature review and professional organizations to develop evidence-based competencies for neuro-oncology advanced practice; (2) use the Delphi technique with an interprofessional team of experts to verify and gain consensus on professional competencies; and (3) use the newly created professional competencies along with information gleaned from a strengths, weaknesses, opportunities, and threats (SWOT) analysis to develop an orientation model for APs in the neuro-oncology program (Figure 1).
Figure 1.
SWOT analysis. AP = advanced practitioner.
METHODOLOGY
Project Design
The project design utilized the quality improvement process FADE (focus, analyze, develop, execute, and evaluate) as the guiding framework for organization and implementation (US Department of Health and Human Services, 2015). The focus of the project was to identify professional competencies for APs in neuro-oncology by utilizing all relevant literature and resources from several professional organizations, including but not limited to, the American Academy of Neurology, the American Association of Colleges of Nursing, the National Organization of Nurse Practitioner Faculties, the American Association of Neuroscience Nurses, the Oncology Nursing Society, and the American Academy of Physician Assistants. The authors categorized competencies by practice domains.
Setting
The project was implemented in a subspecialty oncology practice within a designated National Cancer Institute Comprehensive Cancer Center. Findings from an organizational SWOT analysis performed by the authors elucidated the challenges related to organizational structure, culture, and climate that further validated the need for this quality improvement project (Figure 1). A SWOT analysis looks at internal and external factors that may affect an organization in either positive or negative ways (Moran, 2014).
Participants
Experts in this project had specific knowledge and experience in neuro-oncology practice. Inclusion criteria for the panel of experts for this project included current or former physicians and APs practicing both inpatient and outpatient neuro-oncology at Duke within the past 5 years. The authors deployed a purposive sampling technique to identify participants. The literature does not clearly describe the minimally sufficient number of participants for a Delphi study. Many Delphi studies have used 15 to 20 participants; however, 10 to 15 participants may be adequate if the group is homogenous (Hasson, Keeney, & McKenna, 2000; Hsu & Sandford, 2007). Other groups have found that seven is a suitable minimal panel size (Day & Boveva, 2005).
The expert panel for this project was heterogeneous because individuals were from several different professions; however, a homogenous characteristic was that all potential participants are experienced clinicians in neuro-oncology. For this project, the maximum number of expected participants was 23. Allowing for at least a 40% drop in response rate after each survey round, the results would be considered meaningful as long as the sample size remained greater than seven (Day & Boveva, 2005). There were participants from several southern and northern United States academic medical centers.
Ethical Approval
In addition to receiving institutional support, this project met the Duke University Health System Internal Review Board (IRB) criteria for a declaration of exemption from further IRB review as it did not meet the current descriptions for human subject research.
Methods
Using the Delphi technique, the authors conducted structured reiterative surveys with an interprofessional panel of experienced clinicians in adult neuro-oncology. The Delphi technique is different from traditional survey methods, whose goal is generalization. Rather, the validity of the method is rooted in the repeated cycles of surveys that over time demonstrate convergence of expert opinion on a given topic (Cole, Donohoe, & Stellefson, 2013). The authors conducted a minimum of two Delphi rounds, with the expectation of completing a third in order to reach consensus on the competencies. The authors achieved consensus through the iterative process of questioning that was executed via electronic transmission using the Web-based software Qualtrics. The Qualtrics platform was chosen because it required no specialized training for users and had no associated cost (i.e., the quality improvement project authors had institutional access to the software). Through each round of Qualtrics surveys and consistent with execution of the Delphi technique, participants were asked to (1) evaluate potential competency statements for neuro-oncology advanced practice on a seven-point Likert scale (where 1 = strongly agree and 7 = strongly disagree); (2) provide feedback and have the opportunity to suggest alternate phrasing; and (3) make suggestions for elements perceived to be missing that should be included.
The first-round survey included all the potential competencies identified from pertinent literature and professional organizations. A unique web address created by Qualtrics was included within an email sent to participants. When participants clicked on the link, they were taken to the survey, and Qualtrics tracked the unique user IP address to ensure each respondent completed the survey only once (Bohnenkamp, Pelton, Rishel, & Kurtin, 2014). The first survey remained open for approximately 3 weeks. Participants had the ability to write in competencies for inclusion in the next round. Survey responses were collected and analyzed. A summary report was produced and sent to all participants. The authors integrated the responses and feedback into the second-round survey.
Subsequent survey rounds were created and distributed in the same manner. Poor response rates are an inherent risk with surveys. Strategies to mitigate attrition of participants included maintaining up-to-date communication with the project participants and sending email reminders to complete surveys (Day & Boveva, 2005; Donohoe, Stellefson, & Tennant, 2012). The minimum anticipated number of rounds for this project was two and the maximum was three. Figure 2 details the Delphi process for this project.
Figure 2.
Delphi process map. APP = advanced practitioner; NP = nurse practitioner; PA = physician assistant. Adapted from Cole, Donohoe, & Stellefson (2013).
Privacy, Data Storage, and Confidentiality
Participants remained anonymous; no identifying information was collected or stored from the expert panel. Data was stored electronically on a password-protected encrypted network computer stored in a locked office. The project authors had sole access to the data. No protected health information was used in this project.
RESULTS
Participant Demographics
The participating experts practice neuro-oncology and have clinical expertise in neurology, medical oncology, and neuro-oncology. Twenty-three participants were recruited to participate in the Delphi rounds. Participants included doctors of medicine, a doctor of osteopathy, a doctor of nursing practice, physician assistants, nurse practitioners, and an administrative director. Of the sample (n = 23), 39% were physicians, 57% were APs, and 4% were administrative. The majority of participants were female (70%).
Delphi Rounds
An extensive review of the literature and critical analysis of existing professional and clinical competencies identified six domains of practice. Seventy-eight competencies that correspond to the six domains of practice for neuro-oncology advanced practice were developed. The authors conducted two rounds of questioning from March 2016 through May 2016.
Delphi Round 1. For the first electronic round, participants were asked to evaluate the domains of practice with corresponding competency statements on a seven-point Likert scale (where 1 = strongly agree and 7 = strongly disagree). An individual email was sent to each participant with a secure link to the survey on the Qualtrics platform. During Round 1, participants were also asked to provide suggestions for language refinement or to add competency statements they felt should be present but were not.
Seventy-eight percent (18 of 23) of participants completed the Delphi Round 1 survey. There was overall agreement with all competency statements in the first round, with at least 60% of participants agreeing with all statements. No statements had more than one person who disagreed; therefore, no competencies were deleted. Consensus was reached with all statements during Round 1; therefore, Round 2 was used to incorporate all feedback received and to validate the results of the first round.
Delphi Round 2. In response to the suggested language refinement and the addition of a competency, the Round 2 survey was developed. The same format was used: Participants were again asked to evaluate six domains of practice with corresponding competency statements on a seven-point Likert scale (where 1 = strongly agree and 7 = strongly disagree). Six domains of practice and 50 competency statements were included in the second round. The surveys were distributed electronically to all 23 participants on May 7, 2016, and left open until May 20, 2016. An email reminder was sent to participants after the survey had been open for 1 week.
Fifty-two percent (11 of 21) of participants completed the second round of the Delphi survey. Two participants left the institution prior to starting the second round and declined further participation. Consistent with the literature, there was a 40% drop in response rate between rounds; however, the sample size remained above seven, so the results remain significant (Day & Boveva, 2005). Additionally, all participants had expertise in the subject matter, achieving content validity of results.
The Delphi process validated the final competency framework, and within each practice domain experts reached consensus on all competency statements (Table 1). No additional edits or suggested revisions were offered in the second round. Six domains of practice with 50 corresponding competency statements for neuro-oncology APs were agreed upon. Using the information gleaned from the SWOT analysis and competencies, the authors created a 90-day orientation program for neuro-oncology APs (Figure 3).
Table 1.
Neuro-oncology Advanced Practitioner Competencies for The Preston Robert Tisch Brain Tumor Center at Duke (Delphi Round 2 Consensuses)
Table 1.
Neuro-oncology Advanced Practitioner Competencies for The Preston Robert Tisch Brain Tumor Center at Duke (Delphi Round 2 Consensuses) (cont.)
Table 1.
Neuro-oncology Advanced Practitioner Competencies for The Preston Robert Tisch Brain Tumor Center at Duke (Delphi Round 2 Consensuses) (cont.)
Table 1.
Neuro-oncology Advanced Practitioner Competencies for The Preston Robert Tisch Brain Tumor Center at Duke (Delphi Round 2 Consensuses) (cont.)
DISCUSSION
Neuro-oncology practice is a subspecialty that combines the practice of several disciplines and requires training and education beyond generalist advanced practice graduate programs. The primary purpose of this project was to utilize the Delphi technique to gain consensus on professional competencies for APs in neuro-oncology at an academic medical center. The competency framework offered a baseline to develop a standardized onboarding and transition-to-practice policy for an academic medical center neuro-oncology program. A group of neuro-oncology experts developed the framework, which includes six domains of practice and, within those domains, 50 professional competencies. The orientation program includes both onboarding (basic institutional activities) and a 90-day neuro-oncology–specific orientation (Figure 3). The program outlines the anticipated clinical conditions and procedures for AP orientation and allows for modification depending on the specific needs of the new provider. This work has educational, practice, and additional quality improvement implications.
Figure 3.
The Preston Robert Tisch Brain Tumor Center 12-week advanced practitioner orientation timeline. DUMC = Duke University Medical Center; LMS = learning management system; OESO = Occupational and Environmental Safety Office; AP = advanced practitioner; MNO = medical neuro-oncology; ASCO = American Society of Clinical Oncology. aTo be completed within 6 months of employment.
Implications
No national specialty standards or certifications for advanced practitioners specific to neuro-oncology exist. Development of the competency model emphasizes the unique set of skills that combines several specialties and provides a framework for a professional practice model. Aside from the development of a standard onboarding and orientation model, the competencies offer a standard against which to evaluate job performance and support professional development. The practice implications of this project offer several quality improvement opportunities.
Retention strategies can help organizations reach realistic financial goals. It will be important to monitor the AP turnover rate over the next 5 years in the brain tumor center to evaluate if a standardized, rigorous orientation increases retention. The financial return on investment could be significant. It would be interesting to compare the longevity of APs who have completed the standard orientation in the brain tumor program with that of APs who did not go through the program.
Other quality improvement opportunities include assessing the impact of the program on job satisfaction as well as successful transition to practice. Additionally, it could be postulated that a stable medical team will improve the patient’s experience. This project pursues the triple aim by aspiring to improve the patient experience through the assurance of competent, patient-centered care and to indirectly decrease per capita cost by improving the retention of highly qualified providers, thereby saving the health system thousands of dollars. By continually evaluating the impact of this project on the people who participate in the program as well as how the outcomes may affect the health system, sustainability of this work will be maintained.
Limitations
The outcomes from this quality improvement project are generalizable only to the setting in which the project was implemented. However, there is the absolute potential for the work to be duplicated or adapted to other neuro-oncology teams and subspecialty settings. The Delphi technique is a particularly useful avenue to gain consensus on sensitive or complex topics. It allows for participants to more freely express views that may be different from the majority or that otherwise may not be shared. It is structured communication that fosters inclusiveness. Utilization of the Delphi technique for the neuro-oncology advanced practice competency development was a viable technique that would be used again by the authors. Suggestions for improvement in the execution of the Delphi technique include using the research portal (in this case, the Qualtrics platform) to share information in order to validate the process with the participants rather than communicating results via email (Donohoe et al., 2012). It is important to note that there is no standard application procedure of the Delphi method, especially electronically; however, an abundance of literature describes best practices for executing the technique with rigor. This is acknowledged as a limitation, and if the technique is applied without thoughtful consideration, the internal validity of the findings will be affected.
Conclusions
This quality improvement project successfully created the foundation of advanced practice in a neuro-oncology program of an academic medical center. An application of a structured communication technique with an interprofessional group reached consensus after two rounds on 50 professional competencies for the APs. The competencies served as a framework to create a standard orientation model to support advanced providers new to neuro-oncology. This project supports AP practice and provides the avenue to maintain a unique skill set that ensures the provision of safe, quality, competent patient care.
Footnotes
The authors have no conflicts of interest to disclose.
References
- 1.American Academy of Neurology. American Academy of Neurology Neuro-Oncology Fellowship Core Curriculum. 2013 Retrieved from https://www.aan.com/siteassets/home-page/tools-and-resources/academic-neurologist--researchers/teaching-materials/aan-core-curricula-for-program-directorstor/neurooncology-fellowship-core-curriculum_tr.pdf.
- 2.American Nurses Credentialing Center. 2014 Application manual: Practice transition accreditation program. Silver Spring, MD: American Nurses Credentialing Center; 2014. [Google Scholar]
- 3.American Society of Clinical Oncology. The state of cancer care in america, 2015: a report by the american society of clinical oncology. Journal of oncology practice. 2015;11:79–113. doi: 10.1200/JOP.2015.003772. [DOI] [PubMed] [Google Scholar]
- 4.Bohnenkamp Susan, Pelton Nicole, Rishel Cindy J, Kurtin Sandra. Implementing evidence-based practice using an interprofessional team approach: part two. Oncology nursing forum. 2014;41:548–550. doi: 10.1188/14.ONF.548-550. [DOI] [PubMed] [Google Scholar]
- 5.Cole Zachary Douglas, Donohoe Holly M, Stellefson Michael L. Internet-based Delphi research: case based discussion. Environmental management. 2013;51:511–523. doi: 10.1007/s00267-012-0005-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Coombs L A. The growing nurse practitioner workforce in specialty care. Journal for Nurse Practitioners. 2015;11(9):907–909. [Google Scholar]
- 7.Coniglio David. Collaborative practice models and team-based care in oncology. Journal of oncology practice. 2013;9:99–100. doi: 10.1200/JOP.2012.000859. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Day J, Boveva M. A generic toolkit for the successful management of Delphi studies. Electronic Journal of Business Research Methodology. 2005;3(2):103–116. [Google Scholar]
- 9.Donohoe H, Stellefson M, Tennant B. Advantages and limitations of the e-Delphi technique: Implications for health education researchers. American Journal of Health Education. 2012;43(1):38–46. [Google Scholar]
- 10.Duke Human Resources. Pay ranges & job classifications: Compensation. 2015 Retrieved from https://hr.duke.edu/compensation/classifications/index.php.
- 11.Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. Journal of advanced nursing. 2000;32:1008–1015. [PubMed] [Google Scholar]
- 12.Hsu C-C, Sandford B A. The Delphi technique: Making sense of consensus. Practical Assessment, Research and Evaluation. 2007;12(10) Retrieved from http://pareonline.net/pdf/v12n10.pdf. [Google Scholar]
- 13.Interprofessional Education Collaborative Expert Panel. Core competencies for interprofessional collaborative practice. Washington, DC: Interprofessional Education Collaborative; 2011. Retrieved from http://www.aacn.nche.edu/education-resources/ipecreport.pdf. [Google Scholar]
- 14.Knopf Kristin Elizabeth. Core competencies for bone marrow transplantation nurse practitioners. Clinical journal of oncology nursing. 2011;15:102–105. doi: 10.1188/11.CJON.102-105. [DOI] [PubMed] [Google Scholar]
- 15.Moran K. Developing the scholarly project. In: Moran K, Burson R, Conrad D, editors. The doctor of nursing practice scholarly project a framework for success. Burlington, MA: Jones & Barlett Learning.; 2014. pp. 113–140. [Google Scholar]
- 16.National Commission on Certification of Physician Assistants. Competencies for the physician assistant profession. 2012 Retrieved from http://www.nccpa.net/Uploads/docs/PACompetencies.pdf.
- 17.National Organization of Nurse Practitioner Faculties. Nurse practitioner core competencies. 2012 Retrieved from http://c.ymcdn.com/sites/www.nonpf.org/resource/resmgr/competencies/npcorecompetenciesfinal2012.pdf.
- 18.Oncology Nursing Society. Oncology nurse practitioner competencies. 2007 Retrieved from https://www.ons.org/sites/default/files/npcompentencies.pdf.
- 19.Opperman C, Liebig D, Bowling J, Johnson C S, Harper M. Measuring return on investment for professional development activities: A review of the evidence. ournal of Nurses in Professional Development. 2016;32(3):122–129. doi: 10.1097/NND.0000000000000262. [DOI] [PubMed] [Google Scholar]
- 20.Sargent Lana, Olmedo Miguel. Meeting the needs of new-graduate nurse practitioners: a model to support transition. The Journal of nursing administration. 2013;43:603–610. doi: 10.1097/01.NNA.0000434506.77052.d2. [DOI] [PubMed] [Google Scholar]
- 21.Shulman L N. Efficient and effective models for integrating advanced practice professionals into oncology practice. 2013 doi: 10.14694/EdBook_AM.2013.33.e377. Retrieved from https://meetinglibrary.asco.org/record/80399/edbook. [DOI] [PubMed] [Google Scholar]
- 22.Stewart-Amidei C, Villanueva N, Schwartz R R, Delemos C, West T, Tocco S, Haymore J. American Association of Neuroscience Nurses Scope and Standards of Practice for Neuroscience Advanced Practice Nurses. Journal of Neuroscience Nursing. 2010;42(3):E1–E8. [Google Scholar]
- 23.US Department of Health and Human Services. Quality improvement. 2015 doi: 10.3109/15360288.2015.1037530. Retrieved from https://www.hrsa.gov/sites/default/files/quality/toolbox/508pdfs/qualityimprovement.pdf. [DOI] [PubMed]