Evaluation of the Veterans Affairs Pharmacogenomic Testing for Veterans (PHASER) clinical program at initial test sites

Abstract

Aim:

The first Plan-Do-Study-Act cycle for the Veterans Affairs Pharmacogenomic Testing for Veterans pharmacogenomic clinical testing program is described.

Materials & methods:

Surveys evaluating implementation resources and processes were distributed to implementation teams, providers, laboratory and health informatics staff. Survey responses were mapped to the Consolidated Framework for Implementation Research constructs to identify implementation barriers. The Expert Recommendation for Implementing Change strategies were used to address implementation barriers.

Results:

Survey response rate was 23–73% across personnel groups at six Veterans Affairs sites. Nine Consolidated Framework for Implementation Research constructs were most salient implementation barriers. Program revisions addressed these barriers using the Expert Recommendation for Implementing Change strategies related to three domains.

Conclusion:

Beyond providing free pharmacogenomic testing, additional implementation barriers need to be addressed for improved program uptake.

Over 9 million Veterans receive healthcare services from the Department of Veterans Affairs (VA), the largest integrated healthcare system in the USA [1]. The VA’s robust biomedical research-based infrastructure contributes to the ability for the healthcare system to remain at the forefront of patient care innovations and bring quality care to Veterans. One area of healthcare innovation that continues to gain momentum is the VA’s genomic medicine clinical and research programs, which broadly aim to improve health outcomes through the use of genomic information. The most recent nationwide genomic medicine initiative within the VA is the 2019 VA Pharmacogenomic Testing for Veterans (PHASER) clinical program. As a partnership with Sanford Health, the PHASER program is offering free, multigene pharmacogenomic (PGx) testing for up to 250,000 Veterans at 50 sites from 2019 to 2023. Incorporating PGx information into the drug-prescribing process can improve patient outcomes by improving drug efficacy and decreasing adverse side effects and toxicities. PHASER focused on pre-emptive testing (i.e., testing without a specific need or indication for a medication) as a part of its initial roll out as this has been cited as a preferred strategy for PGx implementation [2,3]. Pre-emptive testing is relevant to the VA given that almost a third of Veterans receiving pharmacy benefits from 2011 to 2017 were prescribed at least one drug with PGx guidelines from the Clinical Pharmacogenetics Implementation Consortium (CPIC) [4]. In addition, 99% of VA pharmacy users are projected to carry at least one actionable PGx variant per the CPIC guidelines [4].

The PHASER program relies on resources external to VA to generate panel-based PGx test results (Sanford Health) and provide interpretation into clinical decisions (translational Software to implement the CPIC guidelines [5]). The program is guided by the PGx implementation best practices recommended by the Implementing Genomics in Practice (IGNITE) consortium network [6]. These practices described key resources identified by the PGx field as central when implementing PGx testing into a healthcare system. While these resources reflect the significant strides made in the implementation of clinical PGx testing, a better understanding of PGx implementation strategies is needed for successful uptake of pre-emptive PGx testing in large, complex, multi-institutional healthcare systems such as the VA.

One of PHASER’s goals is to lead the PGx field in better understanding what drives the successful implementation of a PGx testing clinical program that includes pre-emptive PGx testing as an approach. By largely accounting for the costs of testing through a donation to the VA, the PHASER program has a unique opportunity to understand what other factors are important for optimal uptake of this application of precision medicine. To this end, quality improvement and implementation frameworks were employed to identify strategies for optimizing implementation of a remote, multisite, multigene and pre-emptive PGx testing program. The PHASER program is initiated at VA sites in a staggered manner and allows for the integration of Plan-Do-Study-Act (PDSA) cycles, a quality improvement process that tests and assesses small-scale change at a few sites before making it available systemwide [7]. In line with the principles of PDSA cycles, the PHASER program is developing PGx resources and testing them at a limited number of sites before deploying them more widely over the 5-year span of the program. Implementation science frameworks were used to guide data collection and analysis to inform program development, deployment and revisions within the PDSA cycles. Implementation science frameworks have not been widely incorporated within genomic medicine despite the recognized importance for informing optimal uptake of evidenced-based practices into routine clinical care [8–10].

Developed by implementation scientists affiliated with the VA, the well-cited Consolidated Framework for Implementation Research (CFIR) was used as a framework in these PDSA cycles to identify barriers and facilitators while implementing the PHASER program. This framework was used because it provides a comprehensive list of constructs associated with effective program implementation [11]. Discrete implementation strategies per the Expert Recommendation for Implementing Change (ERIC) study [12,13] were then used to address the identified CFIR implementation barriers and served as the basis of program revisions that will be deployed in the next iteration of the PHASER program. The objective of this report is to describe the first PDSA cycle and the results of its Study and Act stages. These findings will inform future program iterations (i.e., Plan and Do stages of the second PDSA cycle) of the PHASER program.

Methods

The PDSA cycle includes four stages: the Plan stage identifies changes that are being tested and outlines a plan for data collection to assess those changes; the Do stage implements the changes and data collection methods; the Study stage analyzes the collected data; and the Act stage identifies areas of improvement to be implemented in the next PDSA cycle.

Plan stage

Identify changes for evaluation

With the initial rollout of the PHASER program at VA sites, the two major clinical practice changes PHASER sought to achieve included: establishing a PGx testing workflow at VA sites to be able to provide PGx testing to Veterans (preimplementation phase); and engaging providers and Veterans in the program (implementation phase). The national PHASER Project Office was tasked with creating resources and implementation processes to support these changes at VA sites.

Data collection plan

To plan for evaluating the effectiveness of program resources and the implementation process among personnel groups at VA sites, two evaluators with doctoral degrees (i.e., PhD, MD) external to the national PHASER Project Office team who are health services researchers with specialization in genomic medicine observed weekly, in-person national coordinating program operations meetings for 6 months. The national PHASER Project Office team discussed program developments and implementation progress during these program operations meetings. The evaluators developed a plan to administer staff surveys at local VA sites, which focused on evaluation of program resources from the national PHASER Project Office, experience with the program implementation process, and participation rationale and perceptions.

Do stage

Program implementation

The national PHASER Project Office developed resources to support the PHASER program with some flexibility for sites to individualize resources as needed (Table 1). Beyond resources from the national PHASER Project Office, program tasks also relied on resources from local sites and Sanford Health. Descriptions of these resources have been previously published [14]. To establish a PGx testing workflow at each site during the preimplementation phase, the PHASER Project Office sought buy-in from local leadership, identified core teams (i.e., required site champion and optional site coordinator), confirmed site participation through a formal agreement and worked with core teams to set up the laboratory and electronic medical records (EMR) workflows and develop a plan to engage providers and Veterans in the program. During the implementation phase, core teams were responsible for executing outreach strategies to educate providers and Veterans on the program, encourage providers to order PGx testing for Veterans and return PGx results to Veterans and providers. Sites were required to have at least one site champion to facilitate local implementation, have leadership commitment from chiefs of staff, laboratory and pharmacy, and implement a standard PGx testing workflow. Sites had flexibility in hiring a PHASER-funded site coordinator to help with program activities, selecting specific clinical services to adopt PGx testing and implementing outreach strategies to promote program participation among Veterans and providers. Site coordinators were required to have a bachelor’s degree and have at least 1 year of experience as a project manager or coordinator. The role of the site coordinator within local core teams was to support the site champion and be the local expert in the operations of the PHASER program. Site coordinators received local training and support from their site champions. Site coordinators also received training and support from the national PHASER Project Office, including orientation meetings, training on program features and regular meetings to evaluate their site’s progress. The national PHASER Project Office also facilitated opportunities for core teams across sites to discuss their progress and challenges together as a way for sites to learn from other sites. Personnel groups at local sites directly involved with preimplementation and implementation activities of the PHASER program included the core team, providers, laboratory staff, health informatics (clinical application coordinators, [CACs] and laboratory information managers [LIMs]).

Table 1.  . Description of initial Pharmacogenomic Testing for Veterans program resources evaluated as part of the first Plan-Do-Study-Act cycle.

Program phase and process step	Site-specific resources	National PHASER Program Office resources	Sanford Health resources	Resource user(s)
Preimplementation phase steps
1. Build buy-in from site leadership	Site leadership personnel	• PHASER program overview presentation • Available for discussion with site leadership • Sample test result report • Documentation of validation of laboratory testing, VA Ethics assessment, nonresearch determination by VA • Flow diagram for PGx samples and orders	None	Site leadership
2. Identify site champion	Site champion personnel	• Discussion with incumbent champion and/or site leadership • Description of roles and responsibilities	None	Site leadership, site champion
3. Identify optional site coordinator†	Site coordinator personnel	• Description of roles and responsibilities • Funding for site coordinator (12 months 0.5 full-time equivalent)	None	Site coordinator
4. Confirm site participation	Documentation of leadership commitment	• Draft site commitment letter	None	Site leadership, site champion
5. Lab setup for PGx testing	Lab personnel, equipment and space	• Guides on setting up lab workflow with Sanford Health • End-to-end testing with dry run	Personnel, equipment, space	Labs, Sanford Health
6. EMR setup for PGx testing	LIM and CAC personnel, computer equipment	• Health informatics setup guides • Installation guides for clinical decision support tools • Workflow to store PGx reports within the health information system • End-to-end testing with dry run	Personnel, equipment, space	LIM, CAC, Sanford Health
7. Develop plan to engage providers and patients in program†	Develop roll-out plan	• Discussion with sites about their roll-out plan as needed	None	Site champions, site coordinators
Implementation phase steps
1. Educate providers, provide support†	In-service meetings, tailor or create program materials	• PHASER listserv (for provider engagement) • Monthly news letter/off-cycle articles • Resources kept on an internal website • YouTube videos about program procedures • Executive medication summaries	None	Providers
2. Veterans receive optional preappointment mailings†	Tailor or create new materials	• Preappointment mailing introductory letter • Pretesting brochure • Sample PGx report • YouTube video • VA PHASER website	None	Veterans
3. Providers consent Veterans for PGx testing	None	• EMR tool to capture patient’s decision • Information sheet about the consent process	None	Providers
4. Patients submit blood sample	Blood sample collected	• Forms for blood sample submission	None	Labs
5. Sites ship blood samples to Sanford Health for processing	Ship blood samples to Sanford Health	None	None	Labs
6. Sanford Health processes blood samples and generates genetic test results	None	None	Personnel, PGx tests, reagents, equipment, PGx reports	Sanford Health
7. Sanford Health distribute PGx results back to VA sites	Workflow to receive PGx results	None	Personnel to send PGx reports to sites	Site coordinator, site champion, Sanford Health
8. Sites integrate PGx results into the EMR system for future use and mailed to patients	Personnel to enter PGx results in EMR and mail patients copy of report	• Quality control checks on PGx results entered into the EMR	None	Site coordinator, site champion

^†Steps of the implementation process where sites have the flexibility to tailor proposed strategies from the national PHASER Program Office to ensure optimal adoption of the program occurs at the local level.

CAC: Clinical application coordinator; EMR: Electronic medical records; LIM: Laboratory information manager; PGx: Pharmacogenomic; PHASER: Pharmacogenomic Testing for Veterans; VA: Veterans Affairs.

Data collection

Surveys were piloted with the national PHASER Project Office for content accuracy and coherency. Surveys contained 19–37 questions and took 5–15 min to complete, depending on the personnel group. Response options to survey questions varied: we asked respondents to select one answer with and without write-in response options; select one answer along a 5-point response scale; or write in a free response to open-ended questions. Respondents had the choice to select decline a response to most items. An overview of question types for each personnel group is provided in Supplementary Table 1 (Supplementary materials I); the surveys are provided in Supplementary Methods 1–6 (Supplementary materials I). Institutional Review Board (IRB) approval was not required for distribution of this survey because these activities were considered quality improvement rather than research.

The web-based, self-administered surveys were distributed via Qualtrics (UT, USA) to the six VA test sites in 2020 that had completed the preimplementation phase in 2019. Implementation surveys were administered when sites were at least 4 months into the implementation phase. Separate surveys were administered to staff involved during the preimplementation and implementation phases (preimplementation survey only: health informatics personnel; implementation survey only: providers; survey for both phases: core team, laboratory staff). Providers were eligible for the survey if they had ordering privileges and either attended a PHASER educational in-service and/or had ordered a PGx test through the PHASER program. The core team, laboratory staff and health informatics staff at one test site were excluded from receiving surveys since the core team held dual roles on the national PHASER Project Office team and program materials were piloted with the health informatics and laboratory staff at that site before disseminating to other test sites. Survey respondents had 2 weeks to submit surveys and received two email reminders during that time frame. The survey administration schedule is summarized in Supplementary Table 2 (Supplementary materials I).

Study stage

Data analysis

Where applicable, survey questions were categorized as providing information about program implementation barriers and/or facilitators. Implementation barriers did not necessarily inhibit implementation from occurring, but addressing them would improve program implementation (e.g., additional resources that would help with implementation). Implementation facilitators were aspects of the program or processes that aided in successful implementation. Supplementary Tables 3–6 (Supplementary materials I) provide the survey questions for each personnel group relevant in understanding program implementation barriers and/or facilitators.

Survey responses for questions that provided information about implementation barriers and/or facilitators were mapped to the CFIR [11]. CFIR consists of 37 constructs (e.g., complexity, adaptability) categorized under five domains (i.e., intervention characteristics, outer setting, inner setting, characteristics of the individuals, implementation process) that have been associated with effective program implementation. CFIR constructs related to implementation barriers and facilitators were aggregated across preimplementation and implementation surveys and ranked by how frequently they were mentioned by each personnel group. The five most salient barriers for each personnel group were entered separately into the CFIR-ERIC strategy matching tool [15] to identify the implementation strategies recognized as being best able to mitigate those CFIR barriers. Data analysis was completed using Microsoft Excel (WA, USA), SAS 9.4 software (NC, USA) and the gplots package v 3.1.1 [16] for R version 4.0.4 (Vienna, Austria) [17].

Act stage

Identifying areas for improvement

The 73 ERIC strategies are further grouped into nine thematic clusters using concept mapping, which allow for recognition of broader themes [12]. The three clusters with the highest number of ERIC strategies that had an output of ≥75% for any personnel group from the CFIR-ERIC strategy matching tool informed program revisions and/or new program resources. The national PHASER Project Office discussed the feasibility of revising and/or creating new program resources to address the identified ERIC strategies and focused their efforts on resources of the highest importance at the program level.

Figure 1A summarizes the timeline for when the Plan, Do, Study and Act phases occurred at the national level and Figure 1B summarizes the timeline for when the preimplementation and implementation of the Do stage occurred at the site level.

Figure 1. . Timeline of Pharmacogenomic Testing for Veterans’ Plan-Do-Study-Act cycle 1.

(A) The timeline of Plan-Do-Study-Act phase program activities across all test sites from 2018 to 2020 is shown. (B) Preimplementation activities at the first six test sites were completed in 2019 and sites completed 4 months of implementation phase activities before completing the implementation survey.

*Planning of the program components is an ongoing activity that goes well beyond the timeline highlighted in this figure. The activities that define this stage only included the bare minimum program resources needed to begin the preimplementation and implementation activities at sites.

**Preimplementation and implementation initiation occurred at different times across the six VA test sites.

***Site number 1 piloted program materials before they were disseminated to the other sites, and the national PHASER Project Office team represented the core team at this site. Given this circumstance, only providers at this site were surveyed as part of PDSA cycle 1.

****Implementation surveys were sent to sites once they were in the implementation phase for at least 4 months.

PDSA: Plan-Do-Study-Act; PHASER: Pharmacogenomic Testing for Veterans; VA: Veterans Affairs.

Results

Here, we report the findings from the Study and Act stages, including identification of most salient implementation barriers and facilitators, strategies to address these barriers and resource development to apply these strategies.

There were 115 completed surveys out of the 405 surveys (28%) administered across all personnel groups for both program phases. Survey respondents were from VA test sites that completed preimplementation activities for the PHASER program in 2019. All VA test sites have academic affiliations and rate as most complex per the VA complexity model level [18]. The response rates for preimplementation surveys were 69% (4/5 sites represented), 63% (5/5 sites represented) and 70% (5/5 sites represented) for core teams, laboratory staff and health informatics, respectively. The response rate for implementation surveys were 73% (5/5 sites represented), 38% (2/5 sites represented) and 23% (5/6 sites represented) for core teams, laboratory staff, and providers, respectively. Supplementary Table 1 (Supplementary materials II) summarizes the survey response rates by the personnel groups. Table 2 summarizes demographic characteristics of the survey respondents.

Table 2.  . Demographic information of survey respondents at six Veterans Affairs test sites.

	Providers (n = 83); (55 ordering, 28 nonordering) N (%)	Core teams (site champions and site coordinator) (n = 17)† N (%)	Health informatics (CAC (n = 4), LIM (n = 3)) N (%)	Laboratory staff (n = 8)† N (%)	Total (n = 115) N (%)
Gender   Male   Female Decline to answer	30 (36.1%) 52 (62.7%) 1 (1.2%)	9 (52.9%) 7 (41.2%) 1 (5.9%)	1 (14.3%) 5 (71.4%) 1 (14.3%)	3 (37.5%) 3 (37.5%) 2 (25%)	43 (37.4%) 67 (58.3%) 5 (4.3%)
Age   20–29 years   30–39 years   40–49 years   50–59 years   60 years and above   Decline to answer	1 (1.2%) 26 (31.33%) 29 (34.9%) 20 (24.1%) 3 (3.6%) 4 (4.8%)	2 (11.8%) 9 (52.9%) 1 (5.9%) 4 (23.5%) 0 (0%) 1 (5.9%)	1 (14.3%) 0 (0%) 2 (28.6%) 0 (0%) 3 (42.9%) 1 (14.3%)	0 (0%) 2 (25%) 1 (12.5%) 0 (0%) 0 (0%) 5 (62.5%)	4 (3.5%) 37 (32.2%) 33 (28.7%) 24 (20.9%) 6 (5.2%) 11 (9.6%)
Ethnicity   Hispanic or Latino   Non-Hispanic or Latino   Decline to answer	5 (6.0%) 70 (84.3%) 8 (9.6%)	2 (11.8%) 13 (76.5%) 2 (11.8%)	0 (0%) 6 (85.7%) 1 (14.3%)	0 (0%) 1 (12.5%) 7 (87.5%)	7 (6.1%) 90 (78.3%) 18 (15.7%)
Race   American Indian or Alaska Native   Asian   Black or African American   Native Hawaiian or other Pacific Islander   White   Multiracial   Decline to answer	1 (1.2%) 15 (18.1%) 2 (2.4%) 0 (%) 55 (66.3%) 2 (2.4%) 8 (9.6%)	0 (0%) 0 (0%) 4 (23.5%) 0 (0%) 9 (52.9%) 0 (0%) 4 (23.5%)	0 (0%) 1 (14.3%) 0 (0%) 0 (0%) 5 (71.4%) 0 (0%) 1 (14.3%)	0 (0%) 0 (0%) 0 (0%) 0 (0%) 1 (12.5%) 0 (0%) 7 (87.5%)	1 (0.9%) 16 (13.9%) 6 (5.2%) 0 (0%) 70 (60.9%) 2 (1.7%) 20 (17.4%)
Years since last training program   Still in a training program   <10 years   10–19 years   20–29 years   30+ years   Decline to answer	5 (6.0%) 29 (34.9%) 29 (34.9%) 16 (19.3%) 2 (2.4%) 2 (2.4%)	NA	NA	NA	NA
Percentage of full-time equivalent spent in clinical care   ≤25%   26–50%   51–75%   >75%   Decline to answer	11 (13.3%) 13 (15.7%) 9 (10.8%) 41 (49.4%) 9 (10.8%)	NA	NA	NA	NA

^†The same individuals within personnel groups may have been surveyed at preimplementation and implementation time points and each response is treated as a unique data entry.

CAC: Clinical application coordinator; LIM: Laboratory information manager; NA: Not applicable.

Most salient implementation barriers & facilitators

CFIR construct definitions and example survey answers that were mapped to each construct are provided in Supplementary Table 2 (Supplementary materials II). Survey results and response categorization as CFIR implementation barriers or facilitators are provided for in Supplementary Tables 3–11 (providers), Supplementary Tables 12–23 (core team), Supplementary Tables 24–30 (health informatics) and Supplementary Tables 31–42 (laboratory staff) in the Supplementary materials II. Figure 2 summarizes the CFIR constructs that were referenced in survey responses by the group and how frequently constructs were mentioned. Four CFIR domains and 19 constructs were mentioned as implementation barriers or facilitators. Nine unique CFIR constructs were the most salient implementation barriers and ten unique CFIR constructs were the most salient implementation facilitators across the four groups.

Figure 2. . Survey results: frequency of the Consolidated Framework for Implementation Research constructs referenced as program implementation barriers and facilitators*.

Survey results were mapped to CFIR constructs. This figure displays the frequency of how often each CFIR construct was referenced. The CFIR domains and constructs are listed on the y-axis. The personnel groups are shown on the x-axis. Each CFIR construct was categorized as falling in the following categories: top five referenced constructs, top six to ten referenced constructs, below the top ten referenced constructs, and constructs that were not referenced. CFIR constructs were categorized as implementation barriers and implementation facilitators.

*Survey questions included responses that were open-ended, select multiple answer choices and select one answer choice. Survey questions are provided in Supplementary Methods 1–6 (Supplementary materials I), CFIR definitions are provided in Supplementary Table 2 (Supplementary materials II) and survey results that were mapped to CFIR constructs are provided in Supplementary Tables 3–42 (Supplementary materials II).

CFIR: Consolidated Framework for Implementation Research.

Strategies to address implementation barriers

ERIC strategies addressing the five most referenced CFIR barriers for each personnel group are shown in Supplementary Table 43 (Supplementary materials II); ERIC strategies that had an output of ≥75% for any personnel group in these three thematic clusters are shown in Table 3. The three thematic clusters with the highest number of ERIC strategies with an output of ≥75% were identified as the focus for program revisions and/or new program resources. These clusters were: use of evaluative and iterative strategies, develop stake-holder interrelationships and train and educate stakeholders.

Table 3. . CFIR-ERIC Strategy Matching Tool results (per ERIC group)¹⁵: The three clusters with the highest number of ERIC strategies scoring ≥75% in at least one personnel group^†.

ERIC strategy organized by cluster	Providers	Core team	Health informatics	Laboratory staff
Use evaluative and iterative strategies cluster
Assess for readiness and identify barriers and facilitators	110%	115%	120%	86%
Develop a formal implementation blueprint	40%	106%	79%	76%
Conduct local needs assessment	80%	50%	60%	39%
Conduct cyclical small tests of change	70%	27%	95%	57%
Develop stakeholder interrelationships cluster
Identify and prepare champions	107%	124%	97%	76%
Build a coalition	75%	49%	59%	39%
Conduct local consensus discussions	110%	78%	105%	63%
Capture and share local knowledge	105%	84%	108%	94%
Use an implementation adviser	44%	83%	54%	44%
Train and educate stakeholders cluster
Conduct ongoing training	62%	78%	87%	87%
Provide ongoing consultation	39%	77%	55%	52%
Develop educational materials	110%	77%	87%	83%
Distribute educational materials	75%	69%	62%	62%
Conduct educational meetings	153%	90%	110%	100%
Create a learning collaborative	87%	92%	104%	90%

^†Results for all 73 ERIC strategies are listed in Supplementary Table 43 (Supplementary materials II). Up to the five most salient implementation barriers were entered into the CFIR-ERIC Strategy Matching Tool for each personnel group. The CFIR-ERIC Strategy Matching Tool does not provide ERIC strategies for the Engaging CFIR construct, which was one of the most referenced barriers for Core teams, so the next most frequently CFIR construct was used instead (i.e., Relative Priority). The resulting percentage score indicates the number of implementation experts (per ERIC group) endorsing each ERIC strategy to address the CFIR barrier entered. An output of 100% is possible for each CFIR construct entered. Since up to five CFIR constructs were entered for each personnel group, 500% is the highest output score possible.

CFIR: Consolidated Framework for Implementation Research; ERIC: Expert Recommendations for Implementing Change.

Applying strategies through resources

The national PHASER Project Office mapped current program resources to each of the ERIC strategies in the three clusters listed in Table 3. ERIC strategies that were not being addressed or needed to be further adapted were identified, and for each of the items, a lead person was designated to direct the team’s efforts and report back on progress made. The national PHASER Project Office held weekly meetings to work through the logistics of making the necessary changes and reviewed any work product needed to implement those strategies. The national PHASER Project Office created 18 unique resources, which mapped to nine ERIC strategies within the three thematic clusters (Table 4). For instance, a new resource that addressed the ‘use evaluative and iterative strategy’ cluster included creation of a protocol for site coordinators to conduct direct-to-Veteran education regarding PGx to reduce the burden on providers to engage in the program and make it more compatible with their work flow. To be able to rely more heavily on site coordinators to complete program activities, funding availability was increased from part-time to full-time positions for site coordinators. A new resource that addressed the ‘develop stakeholder interrelationship’ cluster included formation of formalized site steering committees to strengthening critical leadership engagement at sites to ensure successful involvement with the program is sustained. The steering committee is tasked with successful deployment of the program and is comprised of the core team, clinical pharmacy specialists and providers from various specialties (e.g., pain, cardiology, primary care). The program established a way to invite providers to serve on the steering committee based on high interest in the local success of the program. A new resource that addressed the ‘train and educate stakeholder’ cluster included development of educational materials that provide thorough training on PGx and the PHASER program for site leadership, providers, site champions and site coordinators.

Table 4.  . New resources to be implemented as part of the second Plan-Do-Study-Act cycle of the Pharmacogenomic Testing for Veterans program.

ERIC strategy	New implementation strategy or resource	Program phase	User(s)
Use evaluative and iterative strategies cluster
Develop a formal implementation blueprint	• More developed guidance on engaging patients and providers • Protocol for direct-to-Veteran education	• Preimplementation • Implementation	• Site champions/site coordinators • Site coordinator
Conduct local needs assessment	• Identify high-impact patients at sites, help providers see value of PGx, enrich tested populations for those most likely to benefit from PGx • Increase funding availability to 1.0 full-time equivalent for site coordinators	• Implementation • Preimplementation	• Site champions, coordinators, providers • Site coordinator
Develop stakeholder interrelationships cluster
Identify and prepare champions	• Structured regular site check-in meetings with national PHASER Project Office about implementation activities • PGx training module • More developed guidance on engaging patients and providers • Direct-to-Veteran education • Include pharmacy site champions as part of the core team	• Implementation • Preimplementation • Preimplementation • Implementation • Preimplementation	• Site champion/site coordinator • Site champion/site coordinator • Site champion/site coordinator • Site coordinator • Site pharmacy champion
Build a coalition	• Formalized steering committee at facilities • Improve lab engagement early on in the program to increase collaboration • Increase engagement with local clinical leaders, so they become advocates of the program	• Preimplementation • Preimplementation • Preimplementation	• Site leadership • Site leadership • Site leadership
Identify early adopters	• Identify informal clinician leaders who have high orders of PHASER PGx tests and leverage their interest to help other clinicians order	• Implementation	• Providers
Train and educate stakeholders cluster
Conduct ongoing training	• Monthly brown bag seminars • Answer frequently asked questions in the newsletter or off-cycle listserv communication	• Implementation • Implementation	• Providers • Providers
Distribute educational materials	• Individualize provider outreach with more targeted resource distribution	• Implementation	• Providers
Provide ongoing consultation	• Provide sites with regular reports of their progress; more transparency of PHASER program roll-out within facility and between national PHASER Project Office • PHASER telehealth clinic for post-test discussion of results • Technical support hotline	• Implementation • Implementation • Implementation	• Site champions/site coordinators • Veterans • Providers
Develop educational materials	• PGx training modules • PHASER-targeted program training module • Monthly brown bag seminars	• Preimplementation • Preimplementation • Implementation	• Site champion/site coordinator • Site leadership • Provider

PGx: Pharmacogenomic; PHASER: Pharmacogenomic Testing for Veterans.

Discussion

The PHASER program is the VA’s initial attempt at pre-emptive, panel-based PGx testing. The first PDSA cycle was conducted as part of the initial rollout of the PHASER program at the first six VA test sites. During this first PDSA cycle, we identified implementation barriers and facilitators to uptake of pre-emptive PGx testing, as well as implementation strategies and resources to employ in subsequent cycles. Most implementation barriers and facilitators were associated with the CFIR domains of inner setting (access to knowledge and information, available resources, compatibility) and process (engaging, executing and formally appointed internal implementation leaders). The implementation strategies associated with these CFIR barriers fell under three ERIC-related conceptual clusters: use evaluative and iterative strategies, develop stakeholder interrelationships, and train and educate stakeholders. Programs such as PHASER aiming to optimize the uptake of pre-emptive PGx testing can use these data to refine current resources and developed new ones to improve implementation of their programs.

While many resources provided during the initial rollout (e.g., workflow, EMR tools, program materials) were seen as implementation facilitators, our findings demonstrate that additional resources (e.g., program materials, staff, funding) will need to be offered in the next iteration of the program. A central theme to the newly developed resources is improving provider engagement – a key need the surveys highlighted. Core team surveys revealed that a main difficulty in implementing PHASER was engaging providers in the program despite provider survey results indicating that most providers view PGx testing favorably. Providers reported that main barriers preventing them from more engagement in the program included wanting additional training on PGx and reducing the time required to implement PGx testing within clinical practice (e.g., shifting Veteran education responsibilities away from providers to accommodate their heavy clinic workload). In response to these survey findings, the national PHASER Program Office will be providing core teams with more guidance and additional resources to engage providers. Expanding the core team to encourage leadership from a site pharmacy champion will help solidify greater collaboration within and across provider groups where pharmacists are already practicing alongside providers. In addition, the national PHASER Program Office will create a protocol for site coordinators to engage in direct-to-Veteran education, implement monthly seminars on PGx topics for providers, develop a technical support hotline for providers, among other resources. To support these additional activities, site coordinator funding will increase from part-time to full-time positions. Implementation barriers relevant for the health informatics and laboratory staff were addressed through better upfront engagement with leadership from these sectors to ensure the PHASER program remained a priority throughout the life cycle of the program.

In comparing the implementation barriers that we identified for the PHASER program to the ones reported from the six diverse genomic projects of the IGNITE consortium network (three PGx-focused and three disease-focused genomic projects) [19], there are some key similarities and differences to highlight. Relative clinical priority was reported in all six IGNITE project as an implementation challenge that impacted the integration of the program into the EMR system [19]. Similarly, relative clinical priority was one of the most salient implementation barriers for the PHASER program, especially among providers, health informatics and laboratory staff personnel. For the PHASER program, this implementation challenge did not impact the integration of the PHASER program into the EMR system as it did in the IGNITE programs since resources were successfully integrated into the EMR system, but rather, this barrier impacted optimal participation among VA providers and other staff members. Access to knowledge and information was another implementation challenge that all six IGNITE projects reported [19]. More specifically, providers lacked knowledge necessary to interpret and apply genomic test results within patient care, which was also a salient implementation barrier among providers in the PHASER program. Last of all, engaging patients to participate in genomic testing was an implementation barrier that all IGNITE projects reported [19]. The PHASER program did not find engaging Veterans an implementation barrier but rather found engaging Veterans one of the easier components of implementing PGx testing. Instead, the PHASER program found engagement among providers more challenging for various reasons, including competing clinical priorities and lack of adequate training in PGx. These differences and similarities in implementation barriers reported in our analysis and the IGNITE projects illustrate that some implementation barriers may be inherent when implementing a genomic medicine program while others may be context and institution specific.

There are several strengths of this analysis. First, there is limited knowledge on how a remote, multisite, multigene, pre-emptive PGx testing program such as PHASER can be optimally implemented within a healthcare system. This critical knowledge gap reflects the limited research conducted in the translational space, as less than 2% of all genomic research is focused on translating genomic discoveries into real-world settings [20]. This lack of translational research contributes to the 17-year time lag it takes for evidenced-based practices to get incorporated into routine practice and contributes to the 50% of evidenced-based practices that never end up making it to routine clinical care [8]. Our analysis expands on the PGx field’s understanding of how to implement a program like PHASER through employing well-established implementation science methodologies and frameworks in our data collection and analysis to systematically assess and identify sources of implementation barriers and facilitators. The surveys included the perspective of key personnel groups at informative time points in order to inform the national PHASER Program Office about program revisions that would be needed to optimally implement the program at current and future sites. These program revisions will be tested and assessed in the next PDSA cycle to inform further refinement of the program. Finally, although the PHASER program largely eliminates implementation barriers related to the financial cost of PGx testing since it is free for Veterans, findings from the PDSA cycle show that this cost coverage is still not sufficient for promotion of optimal prescriber and patient uptake.

There are also limitations to note when interpreting the findings of this analysis. First, survey response from providers was low (23%) and findings from this group and may not necessarily reflect the opinions of all providers involved in the program. Additionally, Veterans represent a key stakeholder group not included in this cycle of data collection. Engagement with Veterans will be valuable in subsequent PDSA cycles once the program is more established. Furthermore, the findings of this analysis are specific to the context of the VA, and generalizability to external health systems may be limited particularly in relation to issues of reimbursement and fragmented healthcare. Finally, the output of the CFIR-ERIC strategy matching tool provides recommended ERIC strategies that may address CFIR barriers [13]. A subsequent PDSA cycle will need to be conducted to evaluate whether the new resources that were identified by the national PHASER Project Office is adequate in addressing the identified CFIR barriers.

Despite these limitations, this analysis provides concrete steps for program improvement. As the PHASER program continues to develop new resources to aid in the integration of PGx testing for Veterans, these PDSA cycles will serve as one avenue to collect feedback from key stakeholders in a systematic way. The PHASER program is set to be active at 40 sites by the end of 2021. Findings from these analyses will be vital in shaping the implementation process. Data from these PDSA cycles will be combined with the formal and informal feedback the national PHASER Program Office receives while working directly with sites to help identify the most pressing and impactful resources that are needed to optimally implement the program within the VA.

Conclusion

PDSA cycles and implementation science frameworks can help PGx-based interventions identify implementation barriers and facilitators, along with strategies to address identified barriers. The first PDSA cycle was completed for the PHASER program, and implementation science methodologies were used to identify program revisions for incorporation in future iterations of the program. Broadly, even though PHASER overcomes many known implementation barriers such as the cost of PGx testing, patient/provider education and electronic health record tools, the findings of this PDSA cycle indicated more support is needed, especially to improve provider engagement in the PHASER program through better PGx training and making the PGx testing process more compatible with clinic workflow. The next PDSA cycle will evaluate whether the identified resources were adequate in addressing these resource gaps, if additional resources are needed, and if new barriers emerge.

Summary points.

• The Veterans Affairs (VA) Pharmacogenomic Testing for Veterans (PHASER) is a remote, multisite, multigene, pre-emptive pharmacogenomic (PGx) clinical testing program. This report describes the first Plan-Do-Study-Act cycle and results of its Study and Act stages that will inform future iterations of PHASER.

• Surveys evaluated implementation resources and processes, and participation rationale and perceptions. Surveys were distributed to implementation core teams, providers, laboratory and health informatics staff.

• Survey responses were mapped to the Consolidated Framework for Implementation Research (CFIR) constructs to identify barriers affecting optimal program implementation for each group. Strategies to address implementation barriers were identified using the Expert Recommendation for Implementing Change (ERIC) typology.

• Survey response rate was 23–73% across survey phases and personnel groups at six VA test sites. Across personnel groups, nine CFIR constructs were identified as the five most salient implementation barriers.

• Program revisions addressed these barriers using ERIC strategies related to three thematic domains (i.e., using evaluative and iterative strategies, developing stakeholder interrelationships, training and educating stakeholders).

• Beyond providing free PGx testing, implementation barriers remain (e.g., providers viewed PHASER favorably but requested more education and personnel support to fully engage) that need to be addressed for improved PGx testing uptake.