Development and Validation of a Patient Satisfaction Survey for Comprehensive Medication Management

Abstract

BACKGROUND:

Comprehensive medication management (CMM) services are a relatively new standard for clinical practice. A patient satisfaction tool for pharmacists providing comparable pharmacy services is essential for measuring quality and sustainability.

OBJECTIVE:

To develop a psychometrically valid questionnaire for measuring patient satisfaction for CMM services.

METHODS:

A patient satisfaction survey tool was developed through a multiphase development process. Validation studies were conducted across 2 urban ambulatory care health system settings providing CMM services. The survey consisted of 10 items related to 3 domains: medication-related needs, pharmacist-patient engagement, and overall satisfaction. Using a 4-point scale, the surveys were mailed, collected, and analyzed for descriptive statistics, internal consistency, and factorial composition.

RESULTS:

Total surveys returned for analysis numbered 195, with an overall survey response rate of 19.2%. Factor analysis and item analysis identified 1 factor of pharmacists’ patient care services. The factor was named “patient satisfaction.”

CONCLUSIONS:

The instrument that was developed provided 1 factor of CMM services. This brief patient satisfaction tool appears to be reliable and valid and may serve other CMM providers to assess 1 measure of quality assurance upon further evaluation.

What is already known about this subject

• Patient satisfaction is a key component for measuring the quality of a service.

• Comparison and validation of patient satisfaction tools have been limited because of varied patient care processes and a recently defined standard for clinical pharmacy practice.

What this study adds

• This study describes the development and validation of a patient satisfaction tool for comprehensive medication management services.

• The tool development used collaborative input from administrators, practitioners, patients, student pharmacists, university faculty, and a health literacy committee.

Along-established and vital aspect of measuring the quality and sustainability of a service includes evaluating patient satisfaction.^1,2 Although an increasing focus of patient satisfaction in pharmacy is being shifted from dispensing functions, error rates, and costs to the quality of pharmacy performance, including patient care services,³ available literature on pharmacist performance of patient care services in comprehensive medication management (CMM) is limited. Without appropriate patient satisfaction measures demonstrating value, pharmacists can be at a disadvantage when requesting reimbursement for these clinical services.⁴ Many of the published examples of pharmacy-patient satisfaction tools for patient care services have not focused on CMM, and few have used a consistent tool or have been tested for reliability.^2,5 In a review of 24 community pharmacy-patient satisfaction articles, sites varied widely in the delivery of the patient care process (e.g., telephone or in person) and type of patient care process used (e.g., comprehensive vs. care specific to 1 medical condition).² This variability is not surprising. Among those publications found that were conducted within a health system setting for pharmacy-patient care services, 1 was specific to a rheumatology clinic; another was in a hospital evaluating anticoagulation services; and 2 were performed (in part) within the Veterans Affairs health system—1 evaluating pharmaceutical care in a hyperlipidemia clinic and 1 was performed with a combination of disease state and medication therapy management sevices.^4,6-8 Only 1 study found, from Minnesota, evaluated CMM services.⁹

Recently, the Joint Commission of Pharmacist Practitioners (JCPP) defined a pharmacy standard for the patient care process.¹⁰ In the future, this 2014 standard may unify clinical pharmacy services. However, for now, the profession of pharmacy has been impeded by lack of a consistent patient care process making comparisons of patient satisfaction of pharmacy services in the literature unreliable.^11,12

In their 2002 study, Kucukarslan and Schommer describe how patient satisfaction tools should be designed to specifically assess the type of service provided.¹³ They point out that a growing body of research suggests cognitive types of pharmacy services should be evaluated from the patient’s perspective and that patient beliefs are the ideal referent, while tangible services, such as wait times and cost, should be compared with prior experiences.¹⁴ In a different study, the same authors (Schommer and Kucukarslan, 1997) indicate that patient satisfaction for pharmacy-patient care services has 4 conceptualizations: performance evaluation, disconfirmation of expectations, affect-based assessment, and equity-based assessment.¹⁴ Although this is a useful framework, CMM services delivered by a clinical pharmacist has not been the professional standard of care, so developing a comparable measurement tool using this methodology presents challenges, given the large number of patients who have never experienced these kinds of services. This potentially limits the “disconfirmation of expectations” (i.e., the patient’s ability to assess the expectation of the experience from the actual experience) and influences having an ideal referent.

Vision for Comprehensive Medication Management

CMM, as defined by the Patient-Centered Primary Care Collaborative (PCPCC), ensures that each patient’s medications are individually assessed to determine that each medication is appropriate for the patient, effective for the medical condition, safe given the comorbidities and other medications being taken, and able to be taken by the patient as intended.¹⁵ CMM also includes an individualized care plan outlining the intended goals of therapy and the appropriate follow-up to determine actual patient outcomes. CMM preceded and is aligned with the JCPP-defined patient care process. However, the PCPCC-defined process is a specific resource guide within the patient-centered medical home, while the JCPP is a standard for all clinical pharmacy patient care services.

In 2012, ambulatory care pharmacy directors, administrators and practitioners from 8 health systems in Minnesota came together with a vision to advance pharmacists’ contributions to ensuring safe and effective use of medications within team-based care models. Members of this administrative group shared a common patient care process and philosophy centered on CMM.

This primarily administrative group later expanded into 13 health systems that shared the same vision of integrated ambulatory patient care and pharmacy services and was named the Health-systems Alliance for Integrated Medication Management (HAIMM). A primary objective of HAIMM was to develop and demonstrate measurable quality standards for CMM services. Since pharmacists are increasingly providing team-based care, HAIMM wanted to demonstrate the contributions of pharmacists in these settings. HAIMM felt consistent quality measurement was lacking for clinical pharmacist interventions as determined by an evaluation of patient satisfaction. Without a measurable, consistent survey tool, there was no opportunity to compare quality markers in patient satisfaction across CMM practice sites.

The objective of this study was to collaboratively develop a reliable and brief patient satisfaction survey specific for pharmacists providing CMM services. This report describes the process of the development of a patient satisfaction tool and the assessment of the tool’s reliability and validity.

Methods

The University of Minnesota Institutional Review Board and the Park Nicollet Institutional Review Board deemed this project was exempt from approval.

Survey Development

At the time of initiation, the 8 health systems involved in HAIMM were either utilizing self-created patient satisfaction tools (n = 5) or had indicated a desire to develop a tool (n = 3). Based on previous experiences, an overarching goal was to create a consistent, easy to administer, and brief survey specific to CMM. During Phase I, the existing surveys were compared and categorized for common themes in patient demographics, pharmacist attributes, integration of team care, use of change-talk, and overall patient satisfaction. Over multiple sessions, a total of 23 pharmacists provided feedback. This feedback group consisted of 17 CMM practitioners, 3 resident pharmacists, 2 student pharmacists, and 3 pharmacists who held faculty positions at the University of Minnesota College of Pharmacy. For survey composition, the determined intent was to gauge 3 conceptual areas related to the pharmacist’s performance addressing medication related-needs, engaging patient-related outcomes, and overall satisfaction. Specifically, questions were detailed across the conceptual areas: (1) addressing patients’ medication-related needs: knowledge gained about indication, effectiveness, safety, and adherence of medications; (2) pharmacist performance for engaging patient-related outcomes: feelings about the pharmacist, increasing confidence in managing medications, participation in teams, and demonstration in caring; and (3) overall satisfaction with recommending others to the pharmacist and rating the overall quality. Eight HAIMM members, 1 from each health system, peer reviewed each question for clarity and content. The resulting survey was then peer reviewed to establish content validity.

During Phase II, the literacy level of the survey instrument was assessed, and items were revised by the Hennepin County Medical Center (HAIMM member) Health Literacy Committee. The committee feedback consisted of the following: (a) clearly defining the pharmacist who is evaluated; (b) offering clear definitions of the intent of the questions; (c) gauging knowledge learned about indication for medications, effectiveness and safety of medications, and addressing adherence barriers; and (c) gauging general feelings about confidence to manage their medications, teamwork, caring, and overall satisfaction. The resulting Flesch-Kincaid Grade Level score using Microsoft Office Professional Plus 2010 (Microsoft Corp., Redmond, WA) was 8.5, indicating that students between the eighth and ninth grades would be able to understand the tool in English.

In Phase III, the survey was piloted within 2 already existing patient partner focus groups at Park Nicollet Health Services (HAIMM member) in order to provide patient feedback on the tool. Conducted 3 weeks apart, patients in each group were asked similar questions as was asked during the peer-review process. The first group consisted of 5 men, aged 59-70 years. Only 1 patient had established care with a CMM pharmacist; the rest were new to the process. The second focus group consisted of 6 patients: 2 men and 4 women, aged 55-74 years; 1 patient had established care with a CMM pharmacist. The 2 focus group sessions were audio-recorded and lasted approximately 35 minutes each. Patients were asked to independently review the survey for 5 minutes and then were asked a series of questions. Feedback was summarized in writing immediately following the session. Minor changes, based on the patient partner feedback, were made to the tool. These changes included clarification of the pharmacist title used (changed from “pharmacist” to “clinical pharmacist”), more language about describing the function of the clinical pharmacist in the survey description, and an explanation about the CMM services provided. Participants also provided explicit feedback that “finding easier ways” (i.e., using a pill box) was different from understanding the “best ways” (i.e., taking a medication with food) to take medicines, so both phrases were included in the tool.

Survey Composition

The final survey consisted of 10 questions, 9 of which used a scale from 1 to 4 (strongly agree, agree, disagree, strongly disagree) and asked patients to evaluate their experiences with the clinical pharmacist (Figure 1). “Neutral” was intentionally not an available option on the tool; however, 6 questions included “not applicable” as an available response. The final question asked patients to rate their overall quality of care and services on a Likert-scale from 1 to 5 (excellent to poor). The survey also included a final, open-ended free-text question on how to improve services, which was not included in this analysis.

FIGURE 1.

Patient Satisfaction Survey

Number	Question
1	My clinical pharmacist helped me to understand why I am taking each of my medicines.
2	My clinical pharmacist helped me understand how to know if my medicines are working.
3	My clinical pharmacist made certain that my medicines are safe (knowing possible side effects of my medicines and avoiding drug interactions).
4	My clinical pharmacist helped me find easier ways to take my medicines.
5	My clinical pharmacist helped me understand the best ways to take my medicines.
6	My clinical pharmacist is working as a team member with my other health care providers.
7	After talking with my clinical pharmacist, I feel more confident to manage my medicines.
8	My clinical pharmacist listened to concerns about my medicines.
9	I would recommend my clinical pharmacist to a family member or friend.
10	Overall, how would you rate the quality of care and services you received from the clinical pharmacist?

© 2014 by Health-system Alliance for Integrated Medication Management. All rights reserved. May not be printed or reprinted without written permission made by one of the authors. Contact: Dr. Jean Moon at jmoon@umn.edu, University of Minnesota College of Pharmacy 7-103 Weaver-Densford Hall, 308 Harvard St. SE, Minneapolis, MN 55455.

Survey Distribution

The survey was primarily distributed within 2 HAIMM member health systems for the purpose of reliability testing: Park Nicollet (PN) Health Services and Fairview (FV) Health Services. PN is a network of 1 wholly owned hospital and 2 affiliated hospitals, 20 primary care clinics, 18 specialty clinics, and 13 retail pharmacies that serve Minneapolis and the surrounding suburbs. PN began its medication program in 2011 and currently has 10 (9.6 full-time equivalent) clinical pharmacists practicing CMM in 18 locations. FV, in partnership with the University of Minnesota, is a network of 7 hospitals, 44 primary care clinics, 55 specialty clinics, and over 30 retail pharmacies that serves the greater Minneapolis-St. Paul area, as well as communities throughout Minnesota and the upper Midwest. FV began its medication therapy management (MTM) program in 1998. Currently, 23 MTM pharmacists (approximately 18 full-time equivalent) practice CMM in 30 locations.

The PN survey was mailed via U.S. Postal Service first class to patients who had received clinical pharmacy services in the previous 3 months. The survey was mailed to the patient’s home address in a generic PN envelope, not from the individual clinic. The address label was computer generated and the envelope contained a postage prepaid and addressed return envelope. The mailing did not contain a cover letter or any specific scoring instructions or directions in addition to the introduction on the survey instrument. The FV mailing was conducted in a similar manner. However, the FV mailing contained a cover letter on FV letterhead or the letterhead of the independent clinic where the patient was seen. The cover letter did not have any specific scoring instructions or directions, but the FV survey did contain a barcode to ease tabulation of results.

Data Analysis

Descriptive statistics for PN demographic information were calculated. FV did not collect demographic information on the administered tool. A Kaiser-Meyer-Olkin measure of sample adequacy was used to determine the appropriateness of factor analysis. Factor analysis was then performed on the PN and FV surveys using the principal component factor analysis (PCA) method. PCA is an extraction method that provides results similar to exploratory factor analysis. Missing values for factor analysis were handled via pairwise deletion. Cronbach’s alpha was used to test the internal consistency reliability of the instrument. Missing values for reliability were handled using listwise deletion. Statistical analysis was done using SPSS version 21 (SPSS, Chicago, IL).

Results

Demographics

The survey was distributed to 289 PN patients, and 88 surveys were returned, giving an overall response rate of 30.4%. The demographic data from PN is presented in Table 1. One survey was returned blank so was excluded from analysis. The updated survey was sent to 727 FV patients, and 107 were collected for analysis. The response rate by the collection time frame was 14.7%. Although FV did not collect demographic data, patients surveyed were 42.8% male and 57.2% female and ranged from aged 21 to 94 years. The overall response rate for PN and FV was 19.2%.

TABLE 1.

Demographic Information for Park Nicollet Health Services Survey Respondents

Patient Factors	Respondents, n (%)
Age, years (n = 88)
25-34
35-44	1 (1.1)
45-54	6 (6.8)
55-64	16 (18.2)
65-74	31 (35.2)
75 or older	34 (38.6)
Gender (n = 88)
Male	32 (36.4)
Female	56 (63.6)
Years visiting clinic (n = 87)
Under 1	7 (8.0)
1-5	13 (14.9)
6-10	4 (4.6)
Over 10	63 (72.4)
Number of times met with clinical pharmacist (n = 87)
1	52 (59.8)
2	21 (24.1)
3	5 (5.7)
4 or more	9 (10.3)

Reliability and Validity

The Kaiser-Meyer-Olkin measure for sample adequacy was 0.87 for the PN survey and 0.93 for the FV survey. PCA revealed only 1 factor and accounted for 66.4% of the variance in the PN survey (eigenvalue = 6.6) and 69.3% in the FV survey (eigenvalue = 6.9). No other components had eigenvalues > 1. In addition, the scree plot showed a break after the first factor for both surveys. All 10 items loaded to the first factor, with a loading greater than 0.75 for PN and 0.76 for FV. The revealed factor was labeled “patient satisfaction,” and the single factor model was used for further analysis of the PN and FV surveys. Cronbach’s alpha of the PN survey was 0.95, and the FV survey was 0.96.

Discussion

Although other validated tools for pharmacy-patient satisfaction exist, this brief collaboratively developed tool was specifically peer and patient reviewed for evaluation of pharmacist-provided CMM services within 2 different health systems. With a recent national patient care process from JCPP for pharmacy services and the ongoing expansion of pharmacist-provided care, there is a continuing need for comparable patient satisfaction research in order to draw conclusions about the quality of services provided by pharmacists in the future. This work may benefit the Medicare Part D program, since CMM is recognized as an “industry standard of care” by the Centers for Medicare & Medicaid Services for delivering MTM and comprehensive medication reviews.¹⁶ In 2015, there were 36,560 plans offering a Medicare Part D benefit to eligible beneficiaries, and each of those plans were required to offer MTM services.¹⁷ However, there is no patient satisfaction evaluation associated with this service.¹⁸

In the development of our patient satisfaction survey, clinical and academic pharmacy experts were used along with external constituents (the HCMC Health Literacy Committee and patient partner focus groups) to develop and validate the instrument. This method is similar to other accounts in the literature where patients, pharmacists, and academic faculty experts have been used to construct a tool.^4,6,19 The development of this CMM patient satisfaction survey tool is unique because it included administrators, practitioners, patients, student pharmacists, university faculty, and a health literacy committee. However, patient satisfaction is only one measurement of quality for services provided, and additional ways to measure the impact of pharmacy patient care services are needed.

After the peer-review process and the patient partner focus groups, the terminology describing how the pharmacist was referred to was modified. Since most of the patients in the focus groups had not used a pharmacist for CMM, a meaningful portion of the focus group time consisted of defining the pharmacist role and services provided. The term “clinical pharmacist” best identified CMM pharmacists and minimized confusion with pharmacists who primarily provide medication distribution functions. We recommend consistent terminology such as this, if this survey is replicated elsewhere to ensure that patients respond to the pharmacy services provided by a CMM pharmacist when they complete the survey.

Despite differences in distribution, collecting patient demographic information, and a minor wording adjustment between the PN and FV distributed surveys, the survey items were found have high internal consistency. Both versions of the survey were independently found to be highly reliable in 2 different health systems via Cronbach’s alpha. The FV version of the survey included the updated language, and this updated version will be used by HAIMM health system members in the future. Data from the health system members will continue to be analyzed and compared in order to advance this instrument as a tool for measuring quality assurance in pharmacy patient care services.

The instrument was relatively simple to administer, although administrative and financial resources may be a consideration when replicating this distribution of the tool. Alternative methods, such as a survey administered by paper at the time pharmacy services are provided might be an alternative used by CMM programs with fewer resources. It is important to recognize that any method of administration (e.g., given at the time of care, mailings at a later time, or cross-sectional sampling) is likely to introduce sampling bias and should be carefully considered.²⁰ Also, this pharmacy-specific survey is anticipated to be administered separately from the overall institution satisfaction survey, since patients receiving CMM would be identified differently.

Limitations

This study has several limitations that should be considered. The response rates for this survey ranged from 14.7% to 30.4% and represented only a portion of the patients who received CMM within the previous 3 months. In addition, differences existed in the patient populations who used CMM pharmacy services at each clinic within the PN and FV health systems and the referral mechanisms to see the CMM pharmacist. The instrument was found to be reliable for PN and FV, although differences may exist as the survey is administered to other patient populations within other health system or CMM settings. Although surveyed patients had experienced patient care services delivered by a pharmacist, the patient partner focus groups used in the development of the tool had little experience with this type of service and could have affected the feedback.

Conclusions

With inconsistencies existing between clinical pharmacy services provided across the country, the HAIMM collaborative allowed an opportunity for programs with similar CMM services to develop and pilot a brief patient satisfaction tool. The results of this initial evaluation indicate that this tool is reliable and valid. Patient satisfaction is an integral part of measuring a quality service, and this tool may serve other programs following the industry standard CMM model with a reliable mechanism to assess and compare performance. Additional analysis is recommended to evaluate the instrument as a measure of quality assurance across additional programs.