Abstract
BACKGROUND:
A comprehensive medication review (CMR) is an annual service offered to eligible Medicare Part D beneficiaries as a component of the Medication Therapy Management program. However, little is known about the most meaningful aspect of CMRs from the patient’s perspective. This information is necessary to help improve the service.
OBJECTIVE:
To conduct concept elicitation interviews with patients who recently received a CMR to guide quality improvement efforts.
METHODS:
Those who recently received a telephonic CMR were invited to participate in semistructured interviews to provide their insights on the CMR service. An interview guide was used and contained the following 6 key questions (with additional probing questions) exploring: (1) overall experience, (2) medication knowledge, (3) concerns, (4) management, (5) satisfaction, and (6) experience. Interviews were transcribed and analyzed thematically.
RESULTS:
Interviews were conducted with 42 patients and resulted in the identification of themes related to the CMR service that were most meaningful to patients. The resulting framework contained 3 themes related to the content of the CMR (eg, medication review), the characteristics of the pharmacy professional (eg, professionalism), and the interaction during the CMR (eg, the telephonic experience). Intrinsic patient factors (eg, prior experiences) were also identified as important to contextualize patients’ experiences.
CONCLUSIONS:
The framework provides concrete examples of the need for continued quality improvement of the CMR service and can be illustrated using the structure-process-outcome model. Patient perspectives should be accounted for in future quality improvement activities.
Plain language summary
This study identified patient-centered opportunities for improving comprehensive medication reviews (CMRs) by interviewing recipients of this Part D benefit for eligible beneficiaries. Patients stated aspects of the experience they perceived to impact the service, such as the responsiveness and demeanor of pharmacy professional with whom they interacted, and the processes surrounding the experience, such as the timing of the call and understanding the why the call was occurring.
Implications for managed care pharmacy
This study focused on patients’ perceptions of the CMR experience. With significant expansion of the eligible population for the Medication Therapy Management program in 2025, it is critical to understand how to improve the quality of these services with the patient voice guiding program improvements. The results of this study enabled the creation of a framework that provided concrete examples of patient-recommended needs for continued quality improvement of CMR service and is illustrated using the structure-process-outcome model.
Comprehensive medication reviews (CMRs) are a type of Medication Therapy Management (MTM) service that involves interactive encounters between a patient and a qualified provider, primarily a licensed pharmacist in the United States. CMRs aim to (1) improve patient medication knowledge, (2) identify and address patient medication–related and health-related concerns, and (3) empower patient self-management.1 Through a systematic process of collecting patient-specific information, a CMR assesses medication therapies to compose a prioritized list of medication-related problems or concerns and creates a plan to resolve them with the patient, caregiver, and/or prescriber.1 Overall, CMRs and accompanying MTM-related activities are designed to improve medication use, optimize therapeutic outcomes, and reduce the risk of adverse drug events.1 Because of methodological inconsistencies and study design limitations, it is difficult to confidently conclude whether CMRs are effective at improving downstream outcomes, such as emergency department visits.2 Still, given the aims of a CMR and MTM as a whole, they are hypothesized to encourage improved downstream health outcomes and health care resource utilization when administered appropriately. Studies suggest that pharmacist-delivered MTM services can help improve medication adherence and clinical outcomes and reduce medication-related problems, emergency department visits, hospital readmissions, and health care expenditures.3-9
Per the Centers for Medicare & Medicaid Services (CMS), Part D plan sponsors are responsible for performance on the quality measure, MTM Program Completion Rate for CMR, which calculates the percentage of eligible MTM program beneficiaries who have received a CMR within a specified reporting time frame.10 However, this performance measure focuses on the quantity of CMRs completed and fails to capture the quality of the CMR. This is an important aspect to consider as the United States moves toward a value-based health care system.
Previous research has attempted to assess patient-reported CMR quality through the Health-Systems Alliance for Integrated Medication Management instrument; however, there were limitations with the instrument used.11 Specifically, the Health-Systems Alliance for Integrated Medication Management 1.0 instrument assessed only one of the CMR domains (ie, patient medication knowledge), lacked content validity, and was unable to detect a meaningful change nor an overall score.11
To better understand real-world CMR delivery and quality, a recent scoping review of 15 articles reported that the key characteristics of CMRs are content, coverage, eligibility, frequency, process, and responsiveness.12 Another recent study conducted a thematic analysis of CMR call transcripts from 3 organizations and reported 21 themes across 4 stages, including the call opening, medication reconciliation, clinical assessments and guidance, and call closing. The findings suggest that organizations that provide CMRs in accordance with CMS’s goals have variation in the depth in which the CMR components are covered.13 Finally, a further study conducted 10 semistructured interviews with payers, providers, and standards-setting organizations to understand the CMR landscape. Thematic analysis identified 10 themes, including the following: (1) CMR definition, (2) organizational goals for CMRs, (3) CMR content, (4) CMR eligibility, (5) CMR frequency, (6) CMR acceptance and completion, (7) CMR process, (8) CMR quality assurance, (9) CMR preparation & training, and (10) future directions for CMRs.14 These findings culminated in a checklist that can be used to help improve the consistency of CMRs.15
Despite the efforts made to better understand aspects of CMR quality from payer and provider perspectives, little is known about patient perspectives of CMR services. There is a need to amplify the patient’s voice and incorporate relevant perspectives into future quality improvement activities and CMR quality assessment. This is particularly important given the call for a new CMR patient-reported outcome performance measure.16
The objective of this study was to conduct concept elicitation interviews with patients who recently received a CMR to identify the most meaningful aspects of the CMR from the patient perspective.
Methods
DEVELOPMENT OF AN INTERVIEW GUIDE
An interview guide (Supplementary Exhibit 1 (218.6KB, pdf) , available in online article) for the concept elicitation interviews was iteratively developed with input from all members of the research team. Collectively the team brought complementary perspectives from clinical pharmacy, academic pharmacy, Pharmacy Quality Alliance, and the pharmacy industry. Then, the guide was revised until consensus was reached on the final version.
The opening section of the guide included introductions, an overview of the study purpose and procedures, agreement to participate, and an opportunity for questions. Next, 6 key questions with prompts focused on the following: (1) overall experience, (2) medication knowledge, (3) addressing concerns, (4) management, (5) satisfaction, and an (6) experience survey. Lastly, a closing section allowed participants the opportunity to share final thoughts and ask questions.
DEMOGRAPHIC QUESTIONNAIRE
At the end of the interview, participants were asked to complete an 11-item demographic questionnaire (Supplementary Exhibit 2 (218.6KB, pdf) ). The questionnaire collected age, sex, ethnicity, race, state of residence (to determine US census region), health insurance plan, and number of medications taken daily (to determine medication complexity), and used the BRIEF health literacy screening tool to assess patient health literacy.17 The BRIEF contained 4 items, with each scored on a scale from 1 through 5. Each patient could therefore be scored in total between 4 and 20. A score of 4-12 implied limited literacy, a score of 13-16 implied marginal literacy, and a score of 17-20 implied adequate literacy.17
STUDY ELIGIBILITY AND RECRUITMENT
Eligible participants were Medicare Part D beneficiaries who had completed a telephonic CMR service within the past 14 days with a pharmacy professional from a national MTM service provider, aged at least 18 years of age, spoke English or Spanish, and had a recollection of the specified CMR service. Participants were not eligible if they had incomplete or discontinued CMR calls.
Participants were recruited from a national MTM service provider contracted to provide telephonic CMRs (the primary method for providing CMRs) to eligible beneficiaries via the University of Arizona. MTM services, including CMRs, were administered by the MTM service provider via a team of pharmacy professionals consisting of licensed pharmacists, pharmacy technicians, and pharmacy interns. The MTM service provider periodically furnished the research team with lists of eligible participants who had recently completed a CMR within the past 14 days.
The protocol specified minimum sampling thresholds for several variables (ie, race, ethnicity, health literacy, age, sex, geographical region) to make the sample somewhat representative of all patients eligible to receive CMRs in the United States. Throughout the recruitment and data collection period, the representativeness of the participants was assessed, and attempts were made to recruit individuals whose characteristics were not well represented.
This study received approval from the University of Arizona Institutional Review Board (IRB). A verbal informed consent process adherent to the University of Arizona IRB requirements was administered to all participants before commencing the interviews. Participants who completed the study were compensated for their time with a gift card that was mailed to them at the IRB-approved amount.
DATA COLLECTION
One public health researcher was trained to conduct telephonic semistructured concept elicitation interviews by an academic pharmacist with expertise in qualitative research methods. The interviews were conducted between December 2021 and March 2023. All interviews were audio recorded and later transcribed verbatim for analysis through a transcription service.18
DATA COLLECTION AND ANALYSIS
Each interview was analyzed by 2 independent reviewers through an iterative approach. Each transcript was reviewed line-by-line to create a codebook, which was updated throughout the review process. Saturation of codes was reached when no new codes were identified in 3 consecutive transcripts.19 The 2 reviewers compared their code lists and resolved differences through discussion with the research team. Six transcripts were randomly selected for the assessment of inter-rater reliability by calculating the initial percent agreement between coders prior to discussion and reaching consensus. Similar codes were grouped together to form themes. The analysis was aided by Dedoose qualitative data management software (Dedoose SocioCultural Research Consultants [SCRC]). The COREQ (Consolidated criteria for Reporting Qualitative research) checklist was used in the preparation of this research report.20
Results
PARTICIPANT CHARACTERISTICS
Concept elicitation interviews were conducted with 42 patients who had received a CMR within the past 14 days. Patients had a mean age of 74.1 (SD = 9.4). There were 19 females, 21 males, 15 participants identified as Hispanic, 25 identified as non-Hispanic, 3 participants who identified as Asian, and 8 participants who identified as African American. All participants were Medicare beneficiaries. Medication complexity ranged from 5 to 30 medications. Calculated BRIEF score results showed 10 participants who had limited health literacy, 9 participants with marginal health literacy, and 21 participants with adequate health literacy (see Table 1). Saturation of ideas was met, and the overall percent agreement between the 2 coders was 76.1% before discussion to resolve conflicts until consensus was reached.
TABLE 1.
Summary of Participant Characteristics
| Characteristic | Total (N = 42) |
|---|---|
| Age, mean (SD) | 74.1 (9.4) |
| Sex, n (%) | |
| Male | 21 (50.0) |
| Female | 19 (45.2) |
| Missing | 2 (4.8) |
| Ethnicity, n (%) | |
| Hispanic | 15 (35.7) |
| Non-Hispanic | 25 (59.5) |
| Missing | 2 (4.8) |
| Race, n (%) | |
| White | 28 (66.7) |
| Black or African American | 8 (19.0) |
| Asian | 3 (7.1) |
| Missing | 3 (7.1) |
| Region, n (%) | |
| Midwest | 9 (21.4) |
| Northeast | 9 (21.4) |
| South | 12 (28.6) |
| West | 10 (23.8) |
| Missing | 2 (4.8) |
| Number of daily medications, mean (SD) | 11.0 (5.0) |
| BRIEF score, n (%) | |
| Limited | 10 (23.8) |
| Marginal | 9 (21.4) |
| Adequate | 21 (50.0) |
| Missing | 2 (4.8) |
States that constitute each US census region: Midwest: Indiana, Illinois, Michigan, Ohio, Wisconsin, Iowa, Kansas, Minnesota, Missouri, Nebraska, North Dakota, and South Dakota; Northeast: Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, Vermont, New Jersey, New York, and Pennsylvania; South: Delaware, District of Columbia, Florida, Georgia, Maryland, North Carolina, South Carolina, Virginia, West Virginia, Alabama, Kentucky, Mississippi, Tennessee, Arkansas, Louisiana, Oklahoma, and Texas; and West: Arizona, Colorado, Idaho, New Mexico, Montana, Utah, Nevada, Wyoming, Alaska, California, Hawaii, Oregon, and Washington.
BRIEF Score (ranges from 4 to 20): 4-12 = limited, 13-16 = marginal, 17 - 20 = adequate literacy.
CONCEPTUAL FRAMEWORK
Identification of themes related to the CMR service that were most meaningful to patients were grouped to develop the CMR Experience Framework (see Figure 1). The framework consisted of the following 3 main themes: (1) CMR content, (2) characteristics of the pharmacy professional, and (3) the interaction. The 3 main themes had partial overlap with one another creating a Venn diagram in which to understand the patients’ experiences with CMRs.
FIGURE 1.

The CMR Experience Framework
Although not characterized as distinct themes to describe patients’ experiences with the CMR service, 2 intrinsic patient factors emerged as important to help with contextualizing the 3 main themes. Patient perceptions of CMR content, the interaction, and the characteristics of the pharmacy professional were influenced by whether the CMR was conducted with the patient themselves, via a caregiver acting as a translator, or as the representative responsible for managing the patient’s medications. Additionally, a patient or caregiver’s familiarity with the medication regimen, chronicity of the medical condition, quality of relationship with the usual care providers, having preexisting health care training or background, and prior exposure to CMRs influenced how patients perceived the CMR experience.
Select quotes associated with the 3 main themes are reported throughout the text. A more complete list of quotes is available in Supplementary Exhibit 3 (218.6KB, pdf) .
CMR CONTENT
All patients recalled telephonically reviewing their medication list with a pharmacy professional. Patients had a tendency to report that the pharmacy professional reviewed the name, dose, purpose, and schedule of each medication on the medication list: “We took out all of the medicine that I take, how many daily, how many in the morning, how many at night, and they explained what each of them were for and how they helped me.” – P27. Patients also reported reviewing side effects of medications, drug interactions, diet and lifestyle, as well as alternative medications to address effectiveness or cost concerns. These components of the CMR were listed as CMR Content in the framework but not grouped further under a subtheme because they are the core elements of the CMR service.
The process of reviewing the medication list and its impact on patients’ perceptions of helpfulness or usefulness, as well as its impact on addressing patient’s concerns, was influenced by the main theme of characteristics of the pharmacy professional. Therefore, this is represented by the overlap of these 2 themes in the framework.
Whether the information was perceived to be helpful or useful was influenced by several factors. Some patients felt validated or reaffirmed, despite not learning anything new from the pharmacy professional, and viewed the exchange of familiar information as a good refresher or reminder: “I’m a retired nurse, so I understand most of the stuff about my medication. But it was a good reminder, and it was okay.” – P41. Several patients described the process of reviewing their medication regimen as time-consuming: “It took up quite a little bit of time, which was irritating.” – P41 or “tedious” – P5. At times, the medication list that the pharmacy professional had access to was not up-to-date and patients felt burdened by having to correct the pharmacy professional or explain the difference. Several patients felt that the information reviewed during the CMR was redundant given similar discussions that had occurred in the past with the patient’s usual care providers (ie, health care provider prescribing the medication and/or the pharmacist dispensing the medication). Thus, these patients did not find the CMR useful, especially if no gaps or errors in their knowledge or usage were identified: “I’ve been doing this for years, taking the same medications and I’m aware of all their side effects and stuff. So, yeah. There wasn’t any new information for me.” – P20. Certain patients expressed a preference to continue reviewing this type of information with their usual care providers rather than through a CMR.
In addition, when asked whether their medication-related and health-related concerns were addressed by the pharmacy professional, many participants did not recall raising any questions or concerns during the CMR. Participants who shared additional details expressed not being prepared with questions because they were not expecting the CMR phone call when it took place: “I wasn’t expecting that would be one of the functions that this person would perform so I didn’t have any expectations that that would happen.” – P13. Still, several patients reported feeling comfortable asking the pharmacy professional questions, if they had any, whereas others reported a preference to discuss with their usual care providers. For a few patients who recalled raising questions or concerns during the CMR, they did not always perceive that the pharmacy professional was able to adequately address their questions or concerns.
Despite this, patients expressed interest in gaining a better understanding of how their medications impacted their health, a better understanding of how to use their medications correctly in relation to the time of day and meals, and awareness of potential drug interactions and medication-related side effects: “Just the fact that they took the time to make sure I was aware of all of the possibilities of the effects, and taking it on time, and everything.” – P42. Others expressed interest in discussing alternative medications that may be more effective or affordable: “Yes, if they had information that there are other, more effective treatments with possibly less side effects.” – P20.
CHARACTERISTICS OF THE PHARMACY PROFESSIONAL
Pharmacy professional characteristics influenced how patients perceived the CMR experience, including how the pharmacy professional conducted themselves or how they made the patients feel. One subtheme related to professionalism included communication skills and style. During the review of the medication list, patients sometimes described the process as a verification exercise: “It gave me an opportunity to go back over each one of the medications and I wrote them down to make sure that I had it, and then how many times and verified that. So, it was good for data verification and for just making sure I was organized.” – P13. Based on the pharmacy professional’s communication style, some patients described the medication list review as an interrogation: “Don’t just start interrogating me. That’s the word I’m thinking of. It wasn’t a survey – I felt like I was being interrogated with my own information.” – P7.
Also related to the professionalism subtheme was the pharmacy professional’s knowledge of the patient’s medication list. Patients perceived the pharmacy professional as knowledgeable based on the accuracy of the medication list. Despite the potential for the medication list to be out-of-date, patients gravitated toward describing their experience based on whether the pharmacy professional was nice, pleasant, or polite: “They were real polite and nice and I didn’t mind at all.” – P12.
Responsiveness is listed as a separate subtheme in the framework influencing how patients experienced the CMR. Patients expressed the value of those who went above and beyond the review of the medication list. The process of verifying the medication list and speaking slowly, speaking loudly, or reiterating discussion points showed the pharmacy professional’s responsiveness toward the patient’s unique condition and needs. Certain patients felt like someone else other than their usual care providers cared for them: “Well, that at least they took an interest in me and my personal medications and stuff. That I know there’s someone out there caring about what I take and don’t take and everything. Not only my doctor but someone else.” – P30.
THE INTERACTION
Patients also described the interaction during their most recent telephonic CMR experience and shared perspectives of how the CMR was implemented.
Certain aspects influenced the patient CMR experiences. The subtheme of understanding the purpose of the call was often interwoven with other aspects of the interaction. Patients expressed that the callers did not adequately clarify the purpose of the call before initiating the medication list review. This was further influenced by lack of trust of familiarity with the caller. Some patients were unclear of the caller’s identity, and several reported not knowing where the call came from. Patients who assumed that the phone call was from their insurance company felt coerced to complete the CMR to avoid losing medication coverage from their health plan.
Timing of the CMR was another aspect that influenced the patient experience. Certain patients shared how the CMR encounter from an unfamiliar caller was unexpected without advanced notice: “Well, I was really surprised to get it. Actually, I thought it was a little strange, because I didn’t know CMR and all that, so it was disclosing personal information to a stranger on a telephone.” – P1. As such, these patients did not anticipate the phone call, were preoccupied either at work or running errands, and could not pay full attention to the discussion with the pharmacy professional: “They come on without warning, and they don’t have – I take about 12 different meds. So, I’m diabetic. I’m a heart patient. You know, there’s just a lot of stuff to go over there. And it’s just – they call, and a lot of times, I’m not home, or I’m not ready.” – P11.
Also influencing the telephonic experience was the expediency the patients experienced from the beginning of the call to the review of the medication list. Patients described their experience as unpleasant for those who spoke to more than 1 pharmacy professional during the CMR and were put on hold between speakers.
Another subtheme of the CMR interaction that correlated with characteristics was the spoken language used by the pharmacy professional. Language was a concern for patients who preferred to speak something other than English, or for caregivers who had to act as translators: “I would just say the language. The person didn’t speak Spanish and we needed an interpreter.” – P32.
STRUCTURE, PROCESS, OUTCOME
The Donabedian structure-process-outcome orientation can be used to apply these results to provider quality improvement.21 The results of the interviews represent specific structures and processes that were influential to patients during their experience with the CMR service. Figure 2 illustrates this by applying the CMR Experience Framework to applicable structures, processes, and outcomes (SPOs). Although SPOs are often diagrammed linearly, this depiction aims to take a step forward in visualizing how the dynamics of the SPO model can apply to the real world. For example, the funnel represents the structure (eg, characteristics of the pharmacy professional) of which is leveraged to input specific ingredients, or processes (eg, the content and interaction), and the outcomes shown at the egression of the funnel are influenced by both structure and process as well as other inputs such as antecedents.
FIGURE 2.

Applying the CMR Experience Framework for Provider Quality Improvement
Discussion
To assist in improving the quality of the CMR service, this project conducted 42 interviews to identify aspects that affected the patient experience and created a conceptual framework that could be used to guide quality improvement activities. The resulting framework contained 3 themes related to the CMR content, characteristics of the pharmacy professional, and the overall interaction.
Patients provided an array of insights ranging from negative to positive experiences. These results are consistent with previous studies that found mixed results such as CMRs having a valuable impact on patients’ perceptions of health or ability to manage their medications but simultaneously stating that the CMR was not inclusive of all their medications.22 Patients tended to value the CMR provider’s demeanor, which was also found in a previous study finding that many negative comments from patients pertained to “lack of customer service”23 and positive experiences with the service because of the provider’s interpersonal manner.24 The importance of the usual care provider in the process of reviewing and making changes parallels other studies.22,25
Results were organized into a conceptual framework to assist researchers and CMR providers interested in improving the quality and patient experience during the CMR. For example, patients specified structural factors or aspects that relate to pharmacy professionals that influenced their experience, despite accuracy of the medication list or perceived level of knowledge held by the pharmacy professional. Interventions to improve or enhance demeanor and telephonic presentation could include training on emotional intelligence, communication skills, and cultural competency. Additionally, to allow pharmacy professionals to be able to provide positive experiences as described by patients, workload and burnout are areas that cannot be ignored.
Process factors were reported as significantly influencing the patients’ experiences with CMRs. Clarity of who and why the call occurred seemed to be a fundamental step in the CMR process. Further research could explore approaches, framing, and specific language that resonates with patients to dissuade distrust and foster an environment to enhance the patient-provider exchange.
A very important and critical insight, though not necessarily a novel finding, was the patients’ preference and trust of their usual care providers. Patients reported preferences to review medications, discuss questions and concerns, and implement diet and lifestyle recommendation changes after discussion with their usual care providers. This underscores, yet again, the need for interprofessional collaboration between health care professionals, including CMR providers. There is also room for improvement in relaying to patients that the CMR is an independent review inclusive of medications from all providers to improve medication use quality. Patients may not always realize that providers are often inhibited in exchanging patient health information with one another despite the advances in health information technology.
There were several process element opportunities identified for providers to improve the patient experience, including the timing of the call, expediency of getting to the medication review, and the language spoken. Although many efforts have likely been made to assist patients with these aspects of the CMR, these results suggest further improvements are needed. Prescheduling the CMR with advanced knowledge of patients’ language preferences can alleviate many concerns expressed by patients. Additionally, incentivizing providers to deliver CMRs in alignment with the needs and preferences of patients could improve the quality of the service.
Patients often perceived certain essential elements of the CMR as helpful, such as the review of the medication list, addressing questions and concerns, and specifics related to the review. These elements relate directly to the intended outcomes of a CMR, as defined by CMS, which are to improve medication knowledge, address questions and concerns, and empower self-management.1 With the expansion of the Medicare Part D MTM program eligible population,26 providers should simultaneously focus on the enhanced quantity and quality of the services provided.
LIMITATIONS
The potential for interviewer bias existed but was minimized through use a standard interview guide and the same interviewer for all interviews. Recall bias was a possible limitation given the variability in time between the CMRs and the interviews. Although these patients were assessed by the interviewer to have recollection of the recent CMR service, the depth and accuracy of their recollection could vary. Another limitation was that some participants were caregivers responding on behalf of patients; although this only occurred in instances in which patients were incapable of responding for themselves, it is possible that differences between patients and caregivers could exist that may influence the results. It is worth noting that this study focused on telephonic CMRs; hence, patient perspectives of in-person CMRs may be different. Finally, although efforts were made to represent several characteristics (including age, sex, ethnicity, race, region, medications, and health literacy), it is possible that variations in responses may exist between people with different characteristics that were not adequately captured in these interviews. A strength of this study was obtaining an appropriate sample size for qualitative research and evidence that saturation was reached.27
As CMR quality improvement work continues, it may be warranted to stratify the results of studies, such as this, by certain factors (eg, health literacy). This may offer insights into important differences between patients who received CMRs with different characteristics (eg, those with low health literacy and those with greater health literacy).
Conclusions
The results of these interviews uncovered variations in patient perceptions of the CMR experience, sometimes describing it as a verification process, and at other times, an interrogation. Nevertheless, this work uncovered clear structure, processes, and outcome opportunities for improvements in the service. It may behoove CMR providers to implement quality improvement strategies given the recent expansion of eligibility criteria,26 the call for a patient-reported outcome measure for the CMR service,16 and the national attention on overall MTM quality.28,29 This framework can guide CMR providers to address various aspects perceived by patients to impact the adequacy and quality of the CMR service.
ACKNOWLEDGMENTS
The authors acknowledge Dr DeBran Tarver and Uche Agu for their support during data collection, and Jenny Bingham for her review of the revised manuscript.
Funding Statement
Dr Axon received funding from the American Association of Colleges of Pharmacy, Arizona Department of Health Services, National Council for Prescription Drug Programs Foundation, and Tabula Rasa Healthcare Group outside this study. Drs Campbell and Black are employed by Merck Sharp & Dohme LLC, a subsidiary of Merck & Co., Inc. Dr Warholak received funding from Arizona Department of Health Services and Tabula Rasa Healthcare Group outside this study.
REFERENCES
- 1.Centers for Medicare & Medicaid Services. CY 2022 MTM Program Guidance and Submission Instructions. Baltimore, MD: U.S. Department of Health and Human Services; 2021. Accessed August 4, 2021. https://www.cms.gov/files/document/memo-contract-year-2022-medication-therapy-management-mtm-program-submission-v-043021.pdf [Google Scholar]
- 2.Viswanathan M, Kahwati LC, Golin CE, et al. Medication therapy management interventions in outpatient settings: A systematic review and meta-analysis. JAMA Intern Med. 2015;175(1):76-87. doi:10.1001/jamainternmed.2014.5841 [DOI] [PubMed] [Google Scholar]
- 3.Isetts BJ, Schondelmeyer SW, Artz MB, et al. Clinical and economic outcomes of medication therapy management services: The Minnesota experience. J Am Pharm Assoc (2003). 2008;48(2):203-14. doi:10.1331/JAPhA.2008.07108 [DOI] [PubMed] [Google Scholar]
- 4.Barnett MJ, Frank J, Wehring H, et al. Analysis of pharmacist-provided medication therapy management (MTM) services in community pharmacies over 7 years. J Manag Care Pharm. 2009;15(1):18-31. doi:10.18553/jmcp.2009.15.1.18 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Iyer R, Coderre P, McKelvey T, et al. An employer-based, pharmacist intervention model for patients with type 2 diabetes. Am J Health Syst Pharm. 2010;67(4):312-6. doi:10.2146/ajhp090047 [DOI] [PubMed] [Google Scholar]
- 6.Wittayanukorn S, Westrick SC, Hansen RA, et al. Evaluation of medication therapy management services for patients with cardiovascular disease in a self-insured employer health plan. J Manag Care Pharm. 2013;19(5):385-95. doi:10.18553/jmcp.2013.19.5.385 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Taylor AM, Bingham J, Schussel K, et al. Integrating Innovative Telehealth Solutions into an Interprofessional Team-Delivered Chronic Care Management Pilot Program. J Manag Care Spec Pharm. 2018;24(8):813-8. doi:10.18553/jmcp.2018.24.8.813 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Axon DR, Taylor AM, Vo D, Bingham J. Initial assessment of an interprofessional team-delivered telehealth program for patients with epilepsy. Epilepsy Res. 2019;158:106235. doi:10.1016/j.eplepsyres.2019.106235 [DOI] [PubMed] [Google Scholar]
- 9.Axon DR, Kloster J, Eckert B, et al. Evaluation of a pharmacist-led telephonic medication therapy management program in rural Arizona: Implications for community health practice. Clin Pract. 2022;12(3):243-52. doi:10.3390/clinpract12030029 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Pharmacy Quality Alliance. 2023 PQA Measure Manual: Completion Rate for Comprehensive Medication Review (CMR). Alexandria, VA; 2023. Accessed October 30, 2023. https://www.pqaalliance.org/ [Google Scholar]
- 11.Campbell PJ, Axon DR, Anderson EJ, Ekstrand MJ, Brummel A, Warholak T. Psychometric evaluation of a comprehensive medication management service experience instrument. J Am Pharm Assoc (2003). 2022;62(1):218-23. doi:10.1016/j.japh.2021.08.024 [DOI] [PubMed] [Google Scholar]
- 12.Vaffis S, Dhatt H, Anderson EJ, et al. Trends in telephonic comprehensive medication review in the United States from 2000 to present: A scoping literature review. J Am Pharm Assoc (2003). 2022;62(2):406-12 e1. doi:10.1016/j.japh.2021.12.010 [DOI] [PubMed] [Google Scholar]
- 13.Dhatt H, Vaffis S, Le D, et al. Qualitative analysis of telephonic comprehensive medication review content and delivery. J Am Pharm Assoc (2003). 2003;63(2):555-65. doi:10.1016/j.japh.2022.11.006 [DOI] [PubMed] [Google Scholar]
- 14.Anderson EJ, Dhatt H, Vaffis S, et al. Key informant perspectives about telephonic comprehensive medication review services in the United States. J Am Pharm Assoc (Wash DC). 2022;62(3):817-25.e1. doi:10.1016/j.japh.2022.01.006 [DOI] [PubMed] [Google Scholar]
- 15.Vaffis S, Dhatt H, Anderson E, et al. A checklist for a consistent, comprehensive medication review. J Am Pharm Assoc (2003). 2023;63(5):1504-7.e1. doi:10.1016/j.japh.2023.06.020 [DOI] [PubMed] [Google Scholar]
- 16.Castora-Binkley M, Hines L, Vaffis S, et al. It is time for a new comprehensive medication review quality measure. J Manag Care Spec Pharm. 2023;29(6):680-4. doi:10.18553/jmcp.2023.29.6.680 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Wallston KA, Cawthon C, McNaughton CD, Rothman RL, Osborn CY, Kripalani S. Psychometric properties of the brief health literacy screen in clinical practice. J Gen Intern Med. 2014;29(1):119-26. doi:10.1007/s11606-013-2568-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.GMR Transcription. 2024. Accessed: January 24, 2024. https://www.gmrtranscription.com [Google Scholar]
- 19.Saunders B, Sim J, Kingstone T, et al. Saturation in qualitative research: Exploring its conceptualization and operationalization. Qual Quant. 2018;52(4):1893-907. doi:10.1007/s11135-017-0574-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349-57. doi:10.1093/intqhc/mzm042 [DOI] [PubMed] [Google Scholar]
- 21.Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(3 suppl):166-206. doi:10.2307/3348969 [PubMed] [Google Scholar]
- 22.Doucette WR, Zhang Y, Chrischilles EA, et al. Factors affecting Medicare Part D beneficiaries’ decision to receive comprehensive medication reviews. J Am Pharm Assoc (2003). 2013;53(5):482-7. doi:10.1331/JAPhA.2013.12233 [DOI] [PubMed] [Google Scholar]
- 23.Cardosi L, Hohmeier KC, Fisher C, Wasson M. Patient satisfaction with a comprehensive medication review provided by a community pharmacist. J Pharm Technol. 2018;34(2):48-53. doi:10.1177/8755122517752158 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Moczygemba LR, Barner JC, Brown CM, et al. Patient satisfaction with a pharmacist-provided telephone medication therapy management program. Res Social Adm Pharm. 2010;6(2):143-54. doi:10.1016/j.sapharm.2010.03.005 [DOI] [PubMed] [Google Scholar]
- 25.Zhang Y, Doucette WR. Consumer decision making for using comprehensive medication review services. J Am Pharm Assoc (2003). 2019;59(2):168-77.e5. doi:10.1016/j.japh.2018.11.003 [DOI] [PubMed] [Google Scholar]
- 26.Centers for Medicare & Medicaid Services. Medicare Program; Changes to the Medicare Advantage and the Medicare Prescription Drug Benefit Program for Contract Year 2024—Remaining Provisions and Contract Year 2025 Policy and Technical Changes to the Medicare Advantage Program, Medicare Prescription Drug Benefit Program, Medicare Cost Plan Program, and Programs of All-Inclusive Care for the Elderly. PACE; 2024:30448-848. [Google Scholar]
- 27.Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18(1):59-82. doi:10.1177/1525822X05279903 [Google Scholar]
- 28.Pharmacy Quality Alliance. PQA convenes: Advancing medication therapy management quality measurement. Alexandria, VA: Pharmacy Quality Alliance; 2023. Accessed January 24, 2024. https://www.pqaalliance.org/mtm-convenes [Google Scholar]
- 29.Pharmacy Quality Alliance. Advancing medication therapy management quality measurement. 2024. Accessed September 13, 2024. https://www.pqaalliance.org/assets/Research/PQA_QuIRC_MTM_Report.pdf
