An Evaluation of the Spread and Scale of PatientToc™ from Primary Care to Community Pharmacy Practice for the Collection of Patient-Reported Outcomes: A Study Protocol

Abstract

Background

Medication non-adherence is a problem of critical importance, affecting approximately 50% of all persons taking at least one regularly scheduled prescription medication and costing the United States more than $100 billion annually. Traditional data sources for identifying and resolving medication non-adherence in community pharmacies include prescription fill histories. However, medication possession does not necessarily mean patients are taking their medications as prescribed. Patient-reported outcomes (PROs), measuring adherence challenges pertaining to both remembering and intention to take medication, offer a rich data source for pharmacists and prescribers to use to resolve medication non-adherence. PatientToc™ is a PROs collection software developed to facilitate collection of PROs data from low-literacy and non-English speaking patients in Los Angeles.

Objectives

This study will evaluate the spread and scale of PatientToc™ from primary care to community pharmacies for the collection and use of PROs data pertaining to medication adherence.

Methods

The following implementation and evaluation steps will be conducted: 1) a pre-implementation developmental formative evaluation to determine community pharmacy workflow and current practices for identifying and resolving medication non-adherence, potential barriers and facilitators to PatientToc™ implementation, and to create a draft implementation toolkit, 2) two plan-do-study-act cycles to refine an implementation toolkit for spreading and scaling implementation of PatientToc™ in community pharmacies, and 3) a comprehensive, theory-driven evaluation of the quality of care, implementation, and patient health outcomes of spreading and scaling PatientToc™ to community pharmacies.

Expected Impact

This research will inform long-term collection and use of PROs data pertaining to medication adherence in community pharmacies.

INTRODUCTION

Medication non-adherence is a problem of critical importance, affecting approximately 50% of all persons taking at least one regularly scheduled prescription medication and costing the United States more than $100 billion annually.^1,2 Medication non-adherence is associated with clinical outcomes including hospitalizations and mortality.^3,4 It is a complex, multi-faceted problem with many causes such as forgetfulness, access/affordability concerns, and avoiding medication due to bothersome side effects. Johnson’s Medication Adherence Model (MAM) summarizes these causes by theorizing that patients must both “remember” and “intend” to take medication.⁵ The importance of reducing medication non-adherence is reflected in the Healthy People 2020 goals and objectives and recognized in the Centers for Medicare and Medicaid Services (CMS) star ratings program for Medicare Part D prescription drug plans (PDPs).^6–7 Plans are rated annually as achieving 1 (lowest quality) to 5 (highest quality) stars. Plans receiving 5 stars are rewarded through quality bonus payments and the ability for patients to switch to the plan outside of the annual open enrollment period.⁸ Several measures used in determining star ratings are based on beneficiary medication adherence.⁷

Community pharmacists are uniquely positioned to intervene on medication non-adherence. They are widely accessible and visited frequently by patients with chronic conditions, including the elderly and those without a regular source of primary care.⁹ Community pharmacists can provide support for challenges commonly faced by their patients such as limited health literacy, being un/underinsured, and limited English proficiency.¹⁰ Further, community pharmacies nationwide have increased efforts to improve measures influencing PDP star measures to ensure the pharmacy is positioned for financial reward through inclusion in the PDPs preferred pharmacy network and possible bonus payments.¹¹ Traditional data sources for identifying and resolving medication non-adherence in community pharmacies include prescription fill histories.^12–16 However, medication possession does not necessarily mean patients are taking their medications as prescribed. Patient-reported outcomes (PROs), measuring adherence challenges pertaining to both remembering and intention to take medication, offer a rich data source to help pharmacists and prescribers resolve medication non-adherence.

Although the value of collecting and utilizing PROs for clinical and research purposes has been more widely recognized in recent years, to the authors’ knowledge, there are no examples of widespread electronic collection and use of PROs data 1) in community pharmacy settings, or 2) pertaining specifically to medication adherence in ambulatory settings.^17–21 In December 2016, the Agency for Healthcare Research and Quality (AHRQ) released Funding Opportunity Announcement PA-17–077 which provides funding for research projects to “scale and spread” successful health information technology models that use PROs in ambulatory settings. Consequently, the authors received funding in April 2019 to conduct research to inform long-term collection and use of PROs data pertaining to medication adherence in community pharmacies by spreading and scaling a successful model (PatientToc™, described below²²) for health information technology-enabled PROs collection. This research is currently in progress and is expected to be complete in March of 2022. This paper provides an overview of the study aims, conceptual frameworks guiding this work, and a summary of the methodology employed for Aim 1 and planned for Aims 2 and 3.

STUDY AIMS

The initial “spread and scale” of PatientToc™ to community pharmacies for the collection and use of PROs data pertaining to medication adherence will be achieved through the completion of 3 study aims:

Aim 1

Conduct a pre-implementation developmental formative evaluation to determine community pharmacy workflow and current practices for identifying and resolving medication non-adherence, potential barriers and facilitators to PatientToc™ implementation, and create a draft implementation toolkit.

Aim 2

Conduct two plan-do-study-act cycles to refine an implementation toolkit for spreading and scaling implementation of PatientToc™ in community pharmacies.

Aim 3

Conduct a comprehensive, theory-driven evaluation of the quality of care, implementation, and patient health outcomes of spreading and scaling PatientToc™ to community pharmacies.

METHODS

Conceptual Frameworks

Three conceptual frameworks are being integrated to guide this study: 1) Curran et al.’s approach to Evidence-Based Quality Improvement (EBQI)²³ for developing an implementation intervention, 2) The Consolidated Framework for Implementation Research (CFIR)²⁴, and 3) the Conceptual Framework for Implementation Outcomes described by Proctor et al.²⁵ The integration of these frameworks and their place in the proposed study is depicted in Figure 1. First, in Aim 1, as described by Curran, a developmental formative evaluation will be conducted prior to PatientToc™ implementation, followed by an evidence-based iterative process²³ to adapt the intervention and implementation supports (toolkit) as needed. This “diagnostic” assessment of the implementation context will consist of semi-structured interviews and observations informed by the CFIR. The CFIR is a well-established framework that classifies implementation constructs across five domains which research has indicated influence implementation, providing a structure to systematically assess implementation contexts. The five domains are: intervention characteristics (e.g., evidence strength, adaptability), outer setting (e.g., health policy, patient resources), inner setting (e.g., clinic culture, leadership engagement), characteristics of the individuals involved (e.g., knowledge and beliefs, self-efficacy, attributes such as motivation and learning style), and the processes of implementation (e.g., training, mentoring, prompting, facilitating).²⁴ The CFIR domains will also form the basis of a deductive-inductive analytic approach (described below) in Aim 1. A multi-stakeholder advisory panel, comprised of small advisory groups from each of the three states, will also be convened and will consider the data collected in Aim 1 and help to develop an initial PatientToc™ implementation toolkit to be used and refined in Aim 2. Based on the EBQI process^26–27 the panel will be comprised of pharmacist, pharmacy technician, and patient representatives (“end users”), experts in PatientToc™, and experts in implementation. They will meet on a regular basis during Aim 2. As the two EBQI plan-do-study-act (PDSA) cycles will be completed with a small number of pharmacies, the multi-stakeholder panel will receive the data generated in the PDSA cycles and work to iteratively refine the approach to PatientToc™ implementation prior to scaling to more community pharmacies (Aim 3). Data collected during the PDSA cycles (and during the Aim 3 scale out) will cover a range of implementation outcomes as recommended by Proctor et al.’s Conceptual Framework for Implementation Outcomes.²⁵ Specifically, in the Proctor model, “client outcomes” are influenced by “service outcomes,” which are influenced by “implementation outcomes.” Implementation outcomes include: acceptability, feasibility, appropriateness, adoption, fidelity, penetration, sustainability, and cost; these are measured at different stages of implementation (e.g., early stage for appropriateness, late stage for sustainability.)²⁵

Figure 1.

Study Overview and Roles of Integrated Conceptual Frameworks

Description of PatientToc: Current Use in Physician Offices

PatientToc™ is a PROs collection software developed by investigators from the L.A. Net Community Health Resources Network, a primary care PBRN in California.^22,28 PatientToc™ was developed to facilitate collection of PROs data from low-literacy and non-English speaking patients in Los Angeles. L.A. Net provided a design for the product based on experience collecting PROs from more than 10,000 patients in L.A. Net practices speaking 42 different languages. The system was developed over a period of 4 years with continuous input from clinicians, community health workers, patients and researchers. PatientToc™ is used in waiting rooms, pre-visit areas, exam rooms, and educator rooms. Patients interact with a 10-inch android tablet that is either hand held or installed in a case or holder attached to a table. Consistent with research on low-literacy,^29–30 the system presents one question at a time, and read aloud functionality for multiple languages is available. Patients use disposable ear buds to maintain confidentiality when they use the read aloud function. The system can deliver any PROs and responses are transmitted real time to the PatientToc™ server where staff and clinicians can access the results both as a pdf replica of a paper version of the completed survey, and as an aggregated SQL or Excel database. PatientToc™ integrates with EHRs via Health Level Seven (HL7) standards or Fast Healthcare Interoperability Resources (FHIR) interfaces, and through third party integration with service provider systems. Currently, PatientToc™ is being used in over 36 practices including 2 Federally Qualified Health Centers. Two California health plans also used the system to transmit mandatory initial health assessments. It is an estimated that approximately 10,000 patients have completed PROs on PatientToc™ tablets, including the: PHQ-9, Medicare Health Risk Assessments, SBIRT screening, and others.

Description of Planned Spread of PatientToc™ to Community Pharmacies & Pharmacist Intervention

While specific implementation features will be informed by the findings of Aims 1 and 2, we anticipate implementation and scaling of a two-fold intervention. First, patients will complete PROs (described below) in PatientToc™ upon arrival at the pharmacy to drop off or pick up a prescription. Data from PatientToc™ will then be either transmitted electronically to pharmacists through integration of PatientToc™ and the pharmacy’s dispensing system or printed via a wireless printer. Second, the pharmacist will review the PROs data and immediately use this information to inform patient counseling, making any relevant interventions to improve medication adherence at that time. For example, patients may report non-adherence due to medication cost. During counseling, the pharmacist may be able to identify discount coupons for the medication and/or assess and recommend less expensive options for consideration by the patient’s physician.

Aim 1

As noted, community pharmacies offer an excellent, novel, ambulatory setting for the collection and immediate use of PROs data pertaining to medication non-adherence. The thoughtful, systematic spreading of a successful model for health information technology-enabled PROs collection and utilization from primary care to community pharmacies in diverse settings, guided by stakeholders to ensure consideration of local context, provides a critical “proof of concept” for other community pharmacies. Starting with a pre-implementation developmental formative evaluation will enable us to better understand the current context of community pharmacy practice and potential barriers, facilitators, and recommendations for PatientToc™ implementation. Aim 1 activities were approved as an exempt research protocol by the Indiana University Institutional Review Board and partnering organizations have either also approved the protocol as exempt research or indicated that review and approval is not required.

Outcome Measures & Products

1. Formation of a multi-stakeholder (pharmacist, pharmacy technician, and patient) advisory panel, with representatives from Indiana, Minnesota, and Wisconsin, to guide the implementation of PatientToc™ for the duration of the project period.

2. Qualitative themes pertaining to potential barriers, facilitators, and recommendations for PatientToc™ implementation in community pharmacies.

3. Creation of a draft toolkit for adapting and implementing PatientToc™ in community pharmacies.

Participants & Sampling

Description of Participating Practice Sites/PBRNs

1. L.A. Net Community Health Resource Network (L.A. Net) and PatientToc™: Established in 2002, L.A. Net practices are comprised of private practices, federally qualified health centers, and community health centers.²⁸ L.A. Net staff led the development of PatientToc™ and since 2012, PatientToc™ has been used in 36 of L.A. Net’s 116 practices across L.A. to collect PROs from safety net patients.

2. Medication Safety Research Network of Indiana (Rx-SafeNet): Launched in 2010 as an Affiliate Network registered with the AHRQ PBRN Resource Center, Rx-SafeNet is one PBRN administered by the Indiana Clinical and Translational Sciences Institute and is comprised of approximately 145 community pharmacy locations throughout Indiana.^31–35

3. Minnesota Pharmacy Practice-Based Research Network (MPPBRN): The MPPBRN was established in 2008 as a collaboration between pharmacists, the Minnesota Pharmacists Association, and the University of Minnesota. MPPBRN is comprised of 366 pharmacists located throughout Minnesota.³⁶

4. Selected community pharmacies in Wisconsin: Members of the study team previously founded and directed a PBRN and maintain close working relationships with many of those pharmacies and others in Wisconsin. A small sample of these pharmacies are participating.

Recruitment of L.A. Net Practices and Community Pharmacies for Participation

To better understand how PatientToc™ has been implemented in primary care and likely barriers, facilitators, and recommendations for spreading PatientToc™ to community pharmacies, a purposeful sample of L.A. Net, Rx-SafeNet, MPBRN, and Wisconsin locations to visit were recruited. Specifically, two L.A. Net practices with differing approaches to PatientToc™ implementation and three pharmacies from each state, representing a wide range of community pharmacy practice types (e.g., independent vs. health-system outpatient pharmacy, urban vs. rural, etc.) were recruited following usual practices of each PBRN (mirrored for Wisconsin.)

Data Collection Procedures

Semi-Structured Interviews, Rapid Ethnography, and Contextual Inquiries

A purposeful sample (targeting n=5 clinicians/staff and n=5 patients per site) of practitioners, staff, and chronically ill patients from participating practices/pharmacies were invited to participate in one-day site visits, including 30 to 60 minute one-on-one semi-structured interviews. Field notes were taken. Interviews and contextual inquiries³⁷ occurred on-site during visits. Contextual inquiries occurred while routine pharmacy tasks were conducted. Questions centered on the tasks being performed, decisions made, and alternatives considered, with conversations focused on pharmacy workflow and how tasks could be supported or impeded by future implementation of PatientToc™. Interview guides were designed to elicit opinions pertaining to experiences with PatientToc™ implementation (for L.A. Net stakeholders) and anticipated barriers and facilitators, as well as recommendations pertaining to future PatientToc™ implementation at community pharmacies within CFIR domains. Interview guides were pilot tested and refined prior to use. Example interview questions related to each broad CFIR domain are provided in Box 1. All interviews and contextual inquiries were audio-recorded with permission of the participant and subsequently transcribed by a professional company and reviewed for accuracy. Field notes were also reviewed by investigators following each visit in order to create an audio-recorded site observation debrief which was also transcribed for analysis. Study team members met after completing site visits to each state to review transcripts and discuss plans for subsequent site visits. Participant demographics were collected at the conclusion of each interview, and entered into SPSS [v. 23, Cary, NC.]³⁸ All data (both written and audio) were collecting using iPads through secure, HIPAA-compliant mobile applications. Specifically, field notes and demographic data were collected using Research Electronic Data Capture (REDCap) software³⁹ and audio-recordings were captured using the transcriptionist company’s secure, mobile dictation application. Pharmacies were offered $500 to offset time spent by staff in data collection activities; individual staff were not compensated. Patient participants were offered a $10 gift card. Aim 1 data collection procedures were completed in November 2019 and data analyses are in progress.

Box 1. Example interview questions (for pharmacists), by CFIR domain.

CFIR Domain	Example Questions
I. Intervention Characteristics	For PatientToc™ to be implemented at your pharmacy, who would need to make the decision? What would you think about the possibility of implementation? How does PatientToc™ compare to other alternatives that may have been considered or used previously? What advantages does it have? Disadvantages? Are there other interventions you believe would be more beneficial? Why is that? What kind of changes or alterations to PatientToc™ would be needed to make it work in your pharmacy? Why is that?
II. Outer Setting	How do you think patients would respond to PatientToc? Any barriers to use? Examples? What alterations do you think would be needed made to meet specific needs/preferences? Can you tell me what you know about any other organizations that have implemented PatientToc or other similar programs? To what extent would implementing PatentToc provide an advantage for your organization compared to other organizations in your area? Examples? What kind of local, state, or national performance measures, policies, regulations, or guidelines might influence the decision to implement PatientToc? Why?
III. Inner Setting	How would the infrastructure of your organization (social architecture, age, maturity, size, or physical layout) affect the implementation of PatientToc? How would the infrastructure facilitate/hinder implementation? Examples? What kind of infrastructure changes would be needed? What would this process entail? What would you expect the general level of receptivity in your organization to implementing PatientToc? Why? How would you prioritize getting PatientToc implemented relative to other initiatives that are happening? Why is that? How does PatientToc align with goals of the pharmacy? What resources would be needed to successfully implement PatientToc?
IV. Characteristics of Individuals	How prepared do you feel to use PatientToc? Why?
V. Process	What plan would you suggest for implementing PatientToc? Who would be the key influential individuals to get on board with PatientToc? What steps would need to be taken to encourage individuals to commit to using PatientToc? Who would be the key individuals to engage? How does word get out to them?

Formation of Multi-Stakeholder Advisory Panel

A multi-stakeholder advisory panel is being formed to represent patient and pharmacist/pharmacy staff perspectives. The panel, which will participate in the EBQI process (described further below) will consist of 1–2 participants (pharmacists, pharmacy technicians, patients) from each pharmacy participating in Aim 1, as well as a sub-set of investigators.

Analytic Procedures

7Interview transcripts, contextual inquiries, and observation notes were coded using accepted qualitative methods. Specifically, data coding was conducted by three trained research assistants with coding decisions reviewed for a subset of transcripts by three investigators. A combination of deductive (e.g., constructs from the CFIR) and inductive (emergent from the data) approaches were used to establish the coding structure and care was taken to modify, create, or collapse codes as necessary.⁴⁰ SPSS [v. 23, Cary, NC]³⁸ was used to summarize descriptive statistics for participant demographic data to better understand potential implementation contexts and to guide qualitative analysis (e.g., exploring any differences in findings across stakeholder type.)

Synthesis of Findings & Creation of Draft Implementation Toolkit Using the Evidence-Based Quality Improvement Process (EBQI)

Data synthesis and identification of emergent themes is ongoing. Qualitative coding results from observations, contextual inquiries, and semi-structured interviews are being examined to identify overarching themes. Through this process, the intent is to also examine data for differences in findings across methods used (e.g., did observations identify differing themes as compared to what was communicated during interviews) and across different types of pharmacies/clinics/implementation approaches. Resulting themes will inform the EBQI process to create a draft implementation toolkit (i.e., detailed description of implementation considerations) for refinement in Aim 2.

As previously described by others,^41–42 EBQI is a quality improvement approach which leverages the unique expertise of each stakeholder involved in the process of intervention implementation: 1) the “end users” from the implementation context (i.e., community pharmacists, pharmacy technicians, and patients from pharmacies interested in PatientToc™ implementation), 2) intervention experts (i.e., L.A. Net leadership with expertise in PatientToc™) and 3) implementation experts, comprised of experts in implementation science and community pharmacy practice from the study team. Throughout the EBQI discussions, the “evidence-base” and rationale for considering PatientToc™ implementation will be presented by L.A. Net leadership to help secure buy-in among the end users. Findings from the diagnostic analysis (Aim 1) will then be presented and pharmacists, pharmacy technicians, and patients will be asked to comment and prioritize the information gleaned on potential facilitators, barriers, and implementation recommendations. This process ultimately will guide decision-making about 1) how the intervention needs to be adapted, and 2) what implementation strategies are to be considered for the initial implementation toolkit. Parameters for adaptations and implementation strategies will be informed by the intervention and implementation experts to ensure feasibility and alignment with scientific literature.

Aim 2

The pre-implementation developmental formative evaluation in Aim 1 will provide critical learnings pertaining to the context for spreading PatientToc™ to community pharmacies. The next step in the spread and scale process will be to implement PatientToc™ in a small number of community pharmacies, using a plan-do-study-act (PDSA) approach.⁴³ This will facilitate resolution of implementation challenges and refinement of the implementation toolkit for use in subsequent scaling. Specific changes cannot be fully elucidated until PatientToc™ has been implemented and initial observations and interviews conducted. That said, it is expected that ongoing adjustments through the PDSA cycles will be made for at least three components of implementation: 1) data integration between PatientToc™ and the pharmacy dispensing systems (e.g., ensuring accurate and complete population of medication data in PatientToc™), 2) PatientToc™ logistics pertaining to medication adherence PROs measures (e.g., skip patterns, automatic computation of the medication regimen complexity index (MRCI, described below), and 3) considerations for optimizing when and how PROs data should be presented in the pharmacy dispensing system for use in counseling and whether/what types of decision support should be provided along with the PROs data to facilitate pharmacist intervention. Ethics review for Aims 2 and 3 is pending but approval using a single IRB process, as well as registration with clinicaltrials.gov, will occur prior to the initiation of Aim 2 activities.

Outcome Measures

This Aim will focus on Implementation outcomes as PDSA findings will inform subsequent scaling of PatientToc™. Quality of Care and Patient Health Outcomes will be secondary outcomes. Collection of the latter outcomes during the PDSA cycles will simulate collection for the final evaluation in Aim 3. Planned outcome measures are summarized in Box 2; additional secondary outcomes might be added. Plans for operationalizing these outcomes are still being finalized and adjustments to these plans might be made after the completion of Aim 1.

Box 2. Summary of Implementation, Quality of Care, and Patient Health Outcomes.

Primary Outcomes
Type	Outcome	Description of Planned Measurement Approach
Implementation	Acceptability	Perceptions among stakeholders regarding satisfaction with PatientToc™ implementation, as measured by qualitative themes identified through direct observation and interviews as well as descriptive statistics on refusal/completion data (patients declining participation, skipping items)
	Adoption	The initial decision to utilize PatientToc™ in pharmacy practice, as measured by qualitative themes identified through direct observation and interviews as well as descriptive statistics on patient utilization (number of patients consented, measures completed)
	Appropriateness	Perceived relevance and compatibility of PatientToc™ for community pharmacies, as measured by qualitative themes identified through direct observation and interviews.
	Costs	Descriptive statistics of costs of implementing PatientToc™ at participating pharmacies.
	Feasibility	The extent to which PatientToc™ can be successfully used in community pharmacies, as measured by qualitative themes identified through direct observation and interviews as well as descriptive statistics on the number of unscheduled contacts made with practices to discuss problems/issues.
	Fidelity	The degree to which PatientToc™ was implemented as intended, as measured by qualitative themes identified through direct observation and interviews as well as descriptive statistics on pharmacy adherence to PatientToc™ implementation as directed per implementation toolkit (e.g., number of days per PDSA when PatientToc™ was accessed; proportion of patients for whom PatientToc™ data was reviewed and used by the pharmacist during (as opposed to after) patient visit to pharmacy.)
Secondary Outcomes
Quality of Care	Prescription wait times	Proportion of prescriptions filled within pharmacy’s goal time for filling (when available)
	Prescription transfers	Monthly transfers in and out, per pharmacy
	Prescription volume	Number of prescriptions filled per month per pharmacy
	Patient satisfaction	Descriptive statistics of scores on patient satisfaction questionnaire (TBD)
Patient Health Outcomes	Medication adherence (PRO)	Self-reported, measured by scores on the Brief Medication Questionnaire (BMQ) regimen screen
	Medication adherence (PRO)	Self-reported, measured by scores on the Merck Medication Adherence Estimator (Adherence Estimator)
	Medication regimen complexity (PRO, automated)	Automatically computed score on Medication Regimen Complexity Index (MRCI)
	Side effect burden (PRO)	Self-reported, measured by scores on the Adverse Drug Reaction Event Side Effect Screener (ADDRESS)
	Pharmacy-level population medication adherence	Adherence measures from pharmacy-level EQuIPP™ data, when applicable

Participants & Sampling

Recruitment of Community Pharmacies for Participation

As described for Aim 1 (same approach), a purposeful sample of 2–3 practices will be recruited from each state (Indiana, Minnesota, Wisconsin) to implement PatientToc™ and participate in the PDSA cycles. The current plan is to begin recruitment by reaching out to the same pharmacies that participated in Aim 1.

Patient Recruitment

The specific pharmacy workflow for introducing patients to PatientToc™ will be informed by Aim 1 findings and the EBQI process, and may vary by participating pharmacy. For example, all pharmacy patients might have the opportunity to complete PROs in the tablet if desired with only a subset included in the evaluation. For the purposes of this evaluation, data will be sought from patients who are 1) ≥ 50 years of age, and 2) have one or more specific chronic conditions (i.e., hypertension, Type 2 diabetes, and dyslipidemia) requiring routine, oral, prescription medication filled by the study pharmacy as 30-day supplies. Approximately 15 patients per participating pharmacy per PDSA cycle will be recruited. Patients meeting these criteria will be required to provide informed consent and HIPAA authorization prior to their data being used in analysis. The study team is also considering the potential for caregivers and/or pharmacy staff to complete PROs on behalf of patients; this decision will be informed by Aim 1 findings.

Data Collection Procedures

Procedures for PDSA Cycles

Guided by the draft implementation toolkit created through Aim 1 (“Plan”), PatientToc™ will be implemented (“Do”) at the pharmacy locations recruited for this Aim. While specific touch points and resources will be informed by Aim 1, the following are planned to support implementation:

• Assignment of specific research assistant/practice facilitator to each participating pharmacy to serve as their primary point of contact for implementation questions and concerns

• Weekly, individual phone calls and bi-weekly in-person visits between investigators/project staff and each participating pharmacy

• Bi-weekly webinars open to all participating pharmacies to share implementation success stories and challenges and receive feedback from investigators and project staff. A portion of these webinars will be accredited as continuing education for pharmacists and pharmacy technicians.

• Continued quarterly meetings with advisory panel

• Compensation to pharmacies to support participation in project activities

Implementation and data collection for the first PDSA cycle will occur over a three-month period. Using these findings (“Study”), the implementation toolkit will be refined (“Act”) through the EBQI approach described in Aim 1 to guide implementation of a second PDSA cycle (three months) and this process will be repeated.

Implementation Outcomes

Following the same general procedures described for Aim 1, above, qualitative data will be collected during phone calls, webinars, and visits to participating pharmacies. The following administrative data will be collected from the PatientToc™ system and/or practice facilitator records: number of unscheduled contacts made with sites to discuss problems/issues, number of patients approached/enrolled/consented, number of PROs measures completed/skipped items, number of days during PDSA when PatientToc™ accessed, whether PROs data reviewed by pharmacist while patient in pharmacy, and costs associated with PatientToc™ implementation.

Quality of Care Outcomes

All outcome measures (Box 2) will be collected for each participating pharmacy on approximately the last day of each month for the duration of each PDSA cycle (3 months per cycle).

Patient Health Outcomes-PROs

Four PROs pertaining to medication adherence will be collected using PatientToc™ during the PDSA cycles. It is envisioned that these will be completed by patients monthly. The order in which these PROs will be presented to patients, as well as which specific medications in a patient’s regimen they will be requested for, are still being discussed by the study team and will be informed by the Aim 1 EBQI process. The PROs measures include:

Measures focused on “Remembering” to take medications:

• 1Brief Medication Questionnaire (BMQ) regimen screen: The BMQ regimen screen is scored from responses to 5 items asking patients to consider their medication adherence for each regularly scheduled prescription medication over the past 7 days. If non-adherence is reported for any medication, the patient is considered “non-adherent.” To facilitate electronic capture of patient responses to the BMQ in PatientToc™ and minimize redundancy with other PROs collected, fewer items from the regimen screen and/or select items from other screens on the BMQ might be used. However, the final BMQ score will be computed as a dichotomous measure of adherence. The BMQ has been widely published and is correlated with adherence measured by prescription fill data.⁴⁴
• 2Merck Medication Adherence Estimator^®: The Adherence Estimator^® is a 3-item 6-point Likert-type scale for measuring self-reported adherence barriers pertaining to cost, concerns, and commitment.⁴⁵ Those scoring ≥ 8 are considered at “high likelihood for non-adherence.” It is validated for use by patients with common chronic, asymptomatic conditions (e.g., diabetes).
• 3Medication Regimen Complexity Index (MRCI) (automated): The MRCI is a 65-item tool measuring regimen complexity across three components: form/route, dosing frequency, and special instructions.⁴⁶ Patients will be prompted to verify their medication regimen on the tablet. PatientToc™ will be programmed to automatically compute a score for the MRCI. The lowest possible score is 1.5 with no maximum score, and increasing scores are associated with worsening medication adherence.⁴⁶ Others have described experience with automating the MRCI and automation considerations.⁴⁷

Measure focused on “Intending” to take medications:

• 4Adverse Drug Reaction Event Side Effect Screener (ADDRESS): ADDRESS scores will provide a measure of total medication side effect distress burden as defined by summing the product of side effect frequency and side effect severity for each medication it is completed for.^48–49

Patient Health Outcomes-Pharmacy Population Level Medication Adherence

When available, EQuIPP™ medication adherence data will also be collected for participating pharmacies. EQuIPP™ is a program, administered by Pharmacy Quality Solutions, Inc., that provides community pharmacies with report cards for their performance contributing to PDP star measures. We will specifically collect data for measures of medication adherence, including the percent of Part D beneficiaries achieving ≥80% adherence on oral diabetes, hypertension, and cholesterol medications.⁵⁰

Analytic Procedures

Analysis of qualitative data will follow the same general procedures as described for Aim 1. Descriptive statistics will be computed for all quantitative data including: administrative data from PatientToc™, quality of care outcomes, scores on PROs, and EQuIPP™ data. Pharmacy means for each quality of care outcome and EQuIPP™ data will be compared using paired t-tests or Wilcoxon signed-rank tests to pharmacy data for the same time period one-year prior.

Synthesis of Findings

Similar to Aim 1, qualitative findings will be synthesized and triangulated with quantitative findings for consideration during the EBQI process. This will result in further refinements to the implementation toolkit for subsequent scaling and final evaluation in Aim 3.

Aim 3

Gaining experience with both the spread of PatientToc™ to community pharmacies, as well as the scale to approximately 30 pharmacies diverse in geography and patient populations, is critical in understanding the potential sustainability in community pharmacies.

Outcome Measures

This Aim will focus on Patient Health Outcomes to evaluate the impact of scaling PatientToc™ to a greater number of community pharmacies and patients.

1. Primary Outcomes (select Patient Health Outcomes):

   1. Self-reported medication adherence, measured by the BMQ regimen screen (described above)
   2. Self-reported total side effect distress burden, measured by ADDRESS (described above)

2. Secondary Outcomes

Implementation Outcomes

Measures collected for Aim 2 will be collected again with the addition of:

1. Penetration (i.e., integration of PatientToc™ within community pharmacies), measured by descriptive statistics on pharmacy-level utilization of PatientToc™ (e.g., percent of pharmacist shifts)

2. Sustainability (i.e., extent to which PatientToc™ is maintained within community pharmacies), measured by the number of consecutive months goal level of patient participation achieved

Quality of Care Outcomes & Additional Patient Health Outcomes

Same as described for Aim 2, above

Participants & Sampling

Recruitment of Community Pharmacies & Patients

Using the same procedures as for Aim 2, up to 10 total community pharmacies per state will be recruited to participate. Frequency of touch points will be adapted based on Aim 2 findings. Pharmacies and patients recruited for Aim 2 will also be followed for the final (Aim 3) evaluation period (7 months, described below) but possibly with fewer touch points. Patient eligibility criteria is expected to be the same as for Aim 2.

Sample Size Justification

There are two primary endpoints for this study, overall medication adherence (yes/no) (measured by the BMQ) and total medication side effect distress burden (measured by ADDRESS), thus an alpha level of 0.05/2=0.025 is used in the power calculation. For overall adherence, the study will be powered based on using a McNemar’s Test at the patient level. Performing the analyses at the visit level (PROs are collected at each visit) will improve power as will taking the modeling approach below that adjusts for covariates. A change in overall adherence rates of 10% is considered clinically important. Assuming the probability of switching from adherent to not adherent is 0.1 and not adherent to adherent is 0.2 from pre- to post-intervention (overall discordance proportion of 0.3), power of 80%, alpha level of 0.025, 40 patients per pharmacy, and intraclass correlation of 0.35 to account for the clustering of patients within pharmacy, a sample size of 1127 patients are needed when using a two-side McNemar’s Test.⁵¹ As the distress measure is based on a quantitative scale, there should be more than ample power to detect changes with this outcome.

Data Collection Procedures

Data collection procedures will mirror those described for Aim 2. Patients will be enrolled over a 1-month timeframe and PROs collection/pharmacist intervention will occur monthly at subsequent pharmacy visits.

Analytic Procedures

The general approach for visit-level PROs will be to fit generalized linear mixed multi-level models that include both patient and pharmacy level factors. These models can flexibly fit various distributions of interest such as binary for overall medication adherence and quantitative for total distress and medication complexity. Patient visit will be the primary unit of analysis for most PROs, though medication-specific adherence will also be explored when feasible. Time period (pre vs. post) intervention will be the primary explanatory variable of interest to assess overall pre vs post differences. The pre-post evaluation of medication adherence uses baseline (i.e., first time patient interacts with PatientToc™, before the pharmacist has used the data in counseling) PROs data collection as the “pre” comparator. All subsequent PROs completions in PatientToc™ serve as the “post” data collection. Time from first intervention visit will also be considered to look for time trends. Random effects for pharmacy and patient nested with pharmacy will be included. Covariates will include: age, sex, race, number of regularly scheduled prescription medications, type of community pharmacy (e.g., independent vs. chain), location of pharmacy (e.g., rural vs. urban), pharmacy prescription volume, and state (Indiana, Minnesota, or Wisconsin.) For pharmacy level outcomes, all variables will first be aggregated to the pharmacy level as needed via either means, medians, or proportions depending on the distribution of the outcome. Next, paired t-tests or Wilcoxon signed-rank tests will be use to compare pre vs. post outcomes such as proportion of prescriptions filled on time, prescription transfers, prescriptions filled, patient satisfaction, and EQuIPP™ measures. Analytic procedures for qualitative data will occur as described in Aim 1 and analysis of implementation outcomes will occur as described in Aim 2.

DISCUSSION

Study Strengths

This study has many strengths. The systematic collection of timely and actionable PROs data can be challenging, particularly for patient populations with limited literacy and/or health literacy. For example, paper-based data collection can be burdensome for data management/analysis and data quality concerns may be evident.⁵² Electronic data collection offers advantages but technology must ensure privacy and security standards are in place to support the reliability and validity of the data.⁵² In addition, data must be accessible to providers in a timely fashion for clinical decision making.^53–54 Technology exists to facilitate the transfer of PROs data into electronic health records (EHRs) and providers want these data to populate with laboratory results, but few electronic systems do so.⁵³ Prior studies have demonstrated that PROs data collected electronically and on paper have similar psychometric estimates, and electronic collection is just as well-received by patients.^54–56 Furthermore, PatientToc™ is capable of, and has experience integrating, PROs data from its system into EHRs, specifically populating in the laboratory results section of the record. This experience will inform planned integration with pharmacy dispensing systems and EHR interfacing could be explored for future information exchange between community pharmacists and physicians. Therefore, the intervention being evaluated offers novel solutions to identified challenges with the collection of PROs across diverse populations.

Moreover, a systematic review of implementation outcomes and evaluation strategies used in community pharmacy services literature was recently completed.⁵⁷ This review of 237 articles meeting inclusion criteria found very few reported data for penetration and sustainability (implementation outcomes being measured in Aim 3); 1 and 12 articles respectively. Therefore, comprehensive evaluations, such as the study described herein, of community pharmacy interventions guided by implementation science are greatly needed.

Potential Challenges

The EBQI process does not have a natural “timeline” for its work to be complete. The process can be lengthy if the intervention and/or implementation strategies are complex. Further, disagreements sometimes occur in the process of coming to consensus which can lengthen the process. While it is common for the EBQI process to attempt to produce the same adapted intervention and implementation toolkit to be used at all participating locations, it is possible for sites to employ “micro-tailoring” (i.e., site-specific plans tailored to the needs of the site) within acceptable parameters (set by the group) to meet local needs. This tailoring can help to resolve any differences that arise during the process to reach consensus. Curran et al. have used this process in numerous research studies over the last 15 years and each time, the panels were able to reach consensus (with minimal local micro-tailoring).^23,58–59 However, the approach taken by each pharmacy to patient targeting (i.e., only patients eligible for inclusion in this evaluation have the opportunity to complete PROs vs. inclusion/exclusion criteria are applied after the pharmacy uses PatientToc™ as they see fit) may make implementation more challenging as review of PROs data may or may not become a routine part of the pharmacist’s workflow.

It is also recognized that challenges may arise regarding the technical needs pertaining to PatientToc™ implementation. Required data pulls and integration across PatientToc™ and pharmacy vendors may also be challenging. However, active engagement by the PatientToc™ team and early conversations with dispensing system vendors, coupled with funds budgeted toward IT support/data integration needs to assist with trouble-shooting issues identified during the PDSA cycles lends confidence to the research team.

Future Research

This work will inform subsequent scaling and evaluation of PatientToc™ in community pharmacy practice. Future research will focus on a) examining the effect of further scaling on PDP star measures pertaining to medication adherence, b) integrating medication adherence PROs collected in PatientToc™ (and summaries of pharmacist intervention) into patients’ EHRs, and c) linking PROs data collected across PBRNs with EHR data for observational research. These efforts could be funded through future AHRQ awards (e.g., PA-14–291). In the current evaluation, the study team is also considering sub-studies to examine the role of PatientToc™ in capturing social determinants of health data in community pharmacies and in facilitating the collection of PROs for use in specific patient care services (e.g., medication therapy management.)

Conclusion

To the authors’ knowledge, this research-in-progress is the first example of planned widespread electronic collection and use of PROs data in community pharmacy settings for the improvement of medication adherence. This research will inform long-term collection and use of PROs data pertaining to medication adherence in community pharmacies and has the potential to positively impact patient health outcomes as well as performance metrics of importance to community pharmacists and payers.

Figure 2.

Screenshots of PatientToc™