Utility of a Brief Screening Tool for Medication-Related Problems

Abstract

Background

Medication therapy management (MTM) services position pharmacists to prevent, detect, and resolve medication-related problems (MRPs.) However, selecting patients for MTM who are most at risk for MRPs is a challenge. Using self-administered scales that are practical for use in clinical practice are one approach.

Objective

The objective of this study was to estimate the psychometric properties of a brief self-administered scale as a screening tool for MRPs.

Methods

This was a non-randomized study utilizing questionnaires administered cross-sectionally. In Phase 1, patients (n=394) at community pharmacies and outpatient clinics completed 78 items, provided to the study team by item authors, assessing perceived MRPs. These data were used to select items for further investigation as a brief, self-administered scale, and estimate the reliability and construct validity of the resulting instrument. In Phase 2, a convenience sample of patients (n=200) at community pharmacies completed a nine-item, self-administered scale. After completion, they were engaged in a comprehensive medication review by their pharmacist who was blinded to questionnaire responses. The main outcome measure for estimating the criterion-related validity of the scale was the number of pharmacist-identified medication-related problems (MRPs.) Item statistics were computed as well as bivariate associations between scale scores and other variables with MRPs. A multivariate model was constructed to examine the influence of scale scores on MRPs after controlling for other significant variables.

Results

Higher scores on the questionnaire were positively correlated with more pharmacist-identified MRPs (r = 0.24; p= 0.001) and scores remained as a significant predictor (p= 0.031) when controlling for other relevant variables in a multivariate regression model (R²= 0.21; p < 0.001.)

Conclusions

Patient responses on the scale may have a modest role in predicting MRPs. The use of self-administered questionnaires such as this may supplement other available patient data in developing patient eligibility criteria for MTM, however, additional research is warranted.

Introduction

Medication therapy management (MTM) services are intended to address medication-related problems (MRPs) among high-risk Medicare patients.^1-3 Specifically, Medicare Part D prescription drug plans are required to target beneficiaries for MTM who use multiple Part D drugs, have multiple comorbidities, and have Part D drug costs exceeding approximately $3,000 annually.⁴

Multiple medications, multiple comorbidities, and similar criteria were previously found by Koechler et al. to predict MRPs⁵ and, in addition to being reflected in Medicare Part D MTM eligibility criteria, have been incorporated into various medication risk assessment tools designed for provider use. For example, Levy validated a 10-item self-administered questionnaire that identified older (≥ 60 years) ambulatory adults at risk for MRPs.⁶ This instrument incorporated the risk factors described by Koecheler et al. Subsequently, Langford et al. modified Levy's questionnaire to create a 5-item medication risk assessment tool for use in all adult patients taking at least two medications. During its pilot, this tool was successful in increasing the medication review referral rate of patients potentially at risk for MRPs.⁷

While these criteria, and tools incorporating them, can be useful, they might not tell the entire story. Anecdotally, clinicians have seen individuals outside of the eligibility criteria experiencing MRPs and individuals meeting the criteria who are not. Furthermore, recent studies have found somewhat limited utility of these criteria as predictors of MRPs and opportunities for improvements in MTM eligibility criteria.^8-10

Therefore, examining alternative risk factors and MTM eligibility criteria is warranted. One strategy could include assessments using items that extend beyond criteria, such as those described above, which are commonly captured in the medical record. However, many of the risk assessment tools described in the literature might have limited application in clinical practice, because they rely on pharmacy claims data,¹¹ provider observation of patient behavior^12-14 or provider assessment of medication regimen complexity or appropriateness using patient medical records or interviews.^15-16 A brief, practical tool that can be self-administered by patients as part of an online health assessment, or while waiting for a prescription or provider visit, might have greater utility.

Several existing self-administered tools incorporate items that extend beyond information available in the medical record.^17-21 For example, Pit et al. used a 31-item medication risk assessment form to determine the prevalence of risk factors for MRPs among older adults in Australia.¹⁸ Similarly, the Beliefs about Medicines Questionnaire (BMQ),¹⁹ a self-assessment questionnaire described by Rovers et al.,²⁰ and the Drug Therapy Concerns (DTC) Scale,²¹ all contain items that capture patient perceptions of their medications. These perceptions might offer additional screening utility for MRPs. However, these tools have not been psychometrically evaluated to examine their predictive validity of all types of MRPs, or they have been studied only in patient populations limited by age or specific disease state.

Therefore, the objective of this study was to examine the psychometric properties and utility of a self-administered questionnaire assessing patient perceptions related to all MRP types as an indicator of MRPs in a general, ambulatory patient population. As part of this assessment, the incremental utility of scale scores as an MRP predictor beyond information routinely available in a medical chart was evaluated.

Methods

Items from Blalock's Drug Therapy Concerns Questionnaire (DTC) were selected for further investigation as a self-administered medication risk assessment tool.²¹ This was because DTC item domains included those from a well-accepted taxonomy for MRPs,²² it was found in early literature searches, and the DTC investigators were willing to discuss and share information about the instrument.

Phase 1: Use of Drug Therapy Concerns Item Pool to Identify Brief, Self-Administered Scale for Further Testing

Data Collection

The original 78-item pool for the DTC (provided by the DTC authors) was administered to a sample of adults (n=394, ages 18 years and older) using at least one prescription medication daily. All data were collected anonymously, and the study protocol received exempt approval status by the University of Pittsburgh Institutional Review Board. Using quota sampling techniques,²³ potential respondents were recruited at eight community pharmacies and two community health centers located within two counties in Pennsylvania over a period of approximately four months in 2009. At each location, a member of the research team was seated at a table with a sign indicating that interested persons could approach to inquire about participation. Patients were screened for participation by completing an eight-question demographics survey. Eligibility included 18 years of age or older, taking at least one regularly scheduled prescription medication, and within an age and sex category necessary to fill the quotas determined a priori. Age and sex quotas were developed using 2006 United States census data as a guide.²⁴ Recruitment procedures and completion of the scale required approximately 10-20 minutes. Respondents who completed at least part of the DTC received a $5 gift card as compensation for their time. Data were coded, entered, and reviewed for accuracy. Analyses were performed using SPSS.

Data Analysis

Three investigators (MES, KSP, KSH) reviewed the original 78-item pool and selected 15 items that maximized clinical utility (i.e., applicable to most chronic medications including all dosage forms) and reduced between-item redundancy. Notes made by patients on their questionnaires were also taken into consideration. Exploratory principal components analyses, examining both one- and two-component solutions, were then computed for this item set and used to remove poorly performing items based on item loadings and remaining conceptual redundancy. The alpha-if-item-deleted statistic was also considered, to ensure positive contributions by each item to the scale's internal consistency. Descriptive statistics were also computed to characterize the study population.

Phase 2: Estimating the Utility of the Brief Scale for use as MRP Predictor

Data Collection

A convenience sample of patients was recruited from two independent community pharmacy locations (under common ownership) in Lancaster County, PA. Enrollment occurred over a sixteen month period from December 2011-March 2013. Eligible patients were 18 years of age or older, taking at least one scheduled prescription medication for a chronic condition, able to complete all study procedures, and were not pregnant. Initial contact with patients was made by a member of the pharmacy staff; only patients believed (from staff knowledge of patient or information from dispensing records) to meet all study eligibility criteria were approached. Interested patients were then asked to provide authorization for their pharmacist to request laboratory records from their primary care provider for the purposes of the study and were scheduled to meet with their pharmacist. After providing written informed consent, patients completed a demographics questionnaire and the nine-item scale and placed their completed documents in a sealed envelope to blind the pharmacist to their responses. They then participated in a comprehensive medication review with their pharmacist, who documented the presence and type of any MRPs identified. The pharmacy was compensated for completing the medication reviews and study documentation.

Problem documentation followed the taxonomy described by Cipolle et al., categorizing individual problems into four main groups: indication, effectiveness, safety, and adherence.²⁵ (Table 1)

Table 1.

Participant Characteristics

Characteristic	Resulta
	Phase 1 (n=394)		Phase 2 (n=200)
Age, mean (range)	50.9 (18-90)		64.8 (19-93)
Male sex, n (%)	174 (44.2)		69 (34.5)
Race, n (%)b	Caucasian	313 (80.1)	Caucasian	192 (96.5)
	African American	65 (16.6)	African American	3 (1.5)
	Hispanic	7 (1.8)	More than one race	4 (2)
	Asian	3 (0.8)
	Other	3 (0.8)
Marital status, n (%)	N/A		Single, living alone	19 (9.5)
			Single, living with partner	6 (3)
			Single, living with friend/relative	4 (2)
			Married	123 (61.5)
			Separated/divorced	14 (7)
			Widowed	34 (17)
Highest level of education completed, n (%)b	Grade School	10 (2.6)	Grade/Middle School	18 (9.1)
	High School	156 (40.8)	High School or GED	96 (48.7)
	College	139 (36.4)	Some College or Community College/Trade School	53 (26.9)
	Graduate/Professional School	77 (20.2)	Bachelor's	19 (9.6)
			Graduate or Professional School	11 (5.6)
Income, n (%)b,c	0-$20,000	125 (35.4)	Comfortable	108 (54.5)
	$20,001-$40,000	91 (25.8)	Just enough to make ends meet	74 (37.4)
	$40,001-$60,000	58 (16.4)
	$60,001 +	79 (22.4)	Do not have enough to make ends meet	16 (8.1)
Difficulty paying for medications, n (%)d	Strongly Agree	62 (15.9)	Strongly Agree	21 (10.7)
	Agree	94 (24.2)	Agree	46 (23.4)
	Not Sure	16 (4.1)	Not Sure	19 (9.6)
	Disagree	171 (44.0)	Disagree	77 (39.1)
	Strongly Disagree	46 (11.8)	Strongly Disagree	34 (17.3)
Total number of medications used, median (range)b,e	4 (1-22)		9 (3-26)
Number of prescription medications, median (range)	N/A		6 (1-19)
Number of non-prescription medications, median (range)	N/A		1 (0-7)
Number of vitamins, herbals, supplements, median (range)	N/A		1 (0-23)
Total number of daily doses of medication, median (range)	N/A		9 (1-32)
Number of medication-related problems, median (range)	N/A		2 (0-8)
One or more indication problems, n (%)f	N/A		156 (78)
One or more effectiveness problems, n (%)g	N/A		61 (30.5)
One or more safety problems, n (%)h	N/A		60 (30)
One or more adherence problems, n (%)i	N/A		68 (34)

^aWill not always sum to total n as not every respondent answered every item

^bData collected differently in Phase 1 vs Phase 2

^cItem for Phase 2: “How would you describe your household income?”

^dItem: “It is difficult to pay for my medication.”

^eIn Phase 1, this was asked as, “How many different medications do you take every day?”; In Phase 2, this was computed by summing the total number of prescription, non-prescription, and herbal medications recorded by the pharmacist.

^fIncludes: No valid medical indication for the drug therapy at this time; Multiple drug therapies are being used for a condition that requires single drug therapy; Medical condition is more appropriately treated with nondrug therapy; Drug therapy is being taken to treat an avoidable adverse reaction associated with another medication; Drug abuse, alcohol use, or smoking is causing the medical problem; Medical condition requires the initiation of drug therapy; Preventive drug therapy is required to reduce the risk of developing a new condition; Medical condition requires additional drug therapy to attain synergistic or additive effects.

^gIncludes: Drug product is not the most effective product for the indication being treated; Medical condition is refractory to the drug product; Dosage form of the drug product is inappropriate; Drug is not effective for the medical problem; Dose is too low to produce the desired response; Dosage interval is too infrequent to produce the desired response; Drug interaction reduces the amount of active drug available; Duration of drug therapy is too short to produce the desired response.

^hIncludes: Drug product causes an undesirable reaction that is not dose-related; Safer drug product is required due to risk factors; Drug interaction causes an undesirable reaction that is not dose-related; Dosage regimen was administered or changed too rapidly; Drug product causes an allergic reaction; Drug product is contraindicated due to risk factors; Dose is too high; Dosing frequency is too short; Duration of drug therapy is too long; Drug interaction occurs resulting in a toxic reaction to the drug product; Dose of the drug was administered too rapidly.

ⁱIncludes: Patient did not understand instructions; Patient prefers not to take medication; Patient forgets to take medication; Drug product is too expensive for the patient; Patient cannot swallow or self-administer the drug product appropriately; Drug product is not available for the patient.

Laboratory data, when available, were used by the pharmacist during the medication assessment. For completing the questionnaires and participating in the medication review, patients received a $25 gift certificate. Pharmacists forwarded all study documents to the principal investigator; no protected health information or other identifiable data were shared. Upon receipt, study documents were reviewed for completeness and data were entered into SPSS v 19.0 for analysis. The protocol was approved by the University of Pittsburgh and Purdue University Institutional Review Boards.

A total of three community pharmacists (who routinely practice together) participated in the data collection process. Prior to commencement of data collection, each pharmacist independently completed five fictitious patient cases and documented identified problems in the same manner as required for study patients. Responses were reviewed, and specific discrepancies were discussed in a conference call between the pharmacists and the principal investigator.

Data Analysis

Descriptive statistics were computed to characterize the study population. For the scale, item statistics, including means, medians, frequencies of endorsement of each response option, item-total correlations, item-criterion correlations, and a reliability estimate (Cronbach's alpha) were computed. The construct validity of the scale was evaluated through exploratory and confirmatory factor analyses. To examine the relationship between scores on the scale and the presence of MRPs, a Spearman's correlation coefficient was computed.

Relationships between potential predictors of MRPs (determined a priori) and the presence of MRPs were assessed. These variables included: age, sex, race, education level, marital status, number of prescription, non-prescription, and vitamin/herbal medications, total number of medication doses per day, description of household income, and self-reported ability to pay for medications. These associations were evaluated using correlations and t-tests as appropriate. Variables associated with MRPs at p < 0.2 were then entered into a multivariate regression model to model associations between the variables and the number of MRPs detected. Non-significant predictor variables in the model were then removed manually in a stepwise progression (starting with the variable with the largest p-value) until the simplest model, explaining the most variability in MRPs, was determined. This model was then evaluated sequentially to determine the change in R² when scores on the scale were entered into the model, after controlling for other significant predictors.

Additionally, potential predictors of MRPs (same predictor variables as above) were examined for each broad category of problem (i.e., indication, effectiveness, safety, adherence). First bivariate statistics (t-tests, Wilcoxon rank sum tests, and chi-squared tests, as appropriate) were computed and then logistic regression models were constructed where the presence of one or more problems within the specified category was the dependent variable. Variables with p-values < 0.2 on bivariate tests were entered into these models. Finally, a receiver-operator characteristic (ROC) curve was constructed to examine the sensitivity and specificity of various cut-points (i.e., scores) on the scale (with items reverse scored, as appropriate) for use as a “positive screen” for MRP risk. For all analyses, except when noted, p-values of < 0.05 were considered statistically significant.

Results

Phase 1

In Phase I, participants (n=394) were generally Caucasian females, using approximately four medications, with at least a high school education, with an average age of approximately 51 years (Table 1). In evaluating the structure of the nine-item scale using exploratory principal components analyses, a one-component solution was found that was clinically logical and exhibited reasonable communalities and factor loadings (Table 2). The Cronbach's alpha estimate of internal consistency was 0.79. This item set was then utilized in Phase 2.

Table 2.

One-Component Solution of the Nine-Item Scale

Itemc	Phase 1a		Phase 2b
	Communality	Factor Loadings	Communality	Factor Loadings
My medication is helping improve my condition.	0.23	0.48	0.18	0.42
My medication does not seem to help that much.d	0.32	0.56	0.39	0.62
Sometimes I think I take too many medications.d	0.37	0.61	0.35	0.59
Sometimes I think I may not be taking the right medication for my condition.d	0.39	0.63	0.60	0.78
Sometimes my medication has effects I do not like.d	0.41	0.64	0.41	0.64
Sometimes I feel worse after I take my medication.d	0.50	0.71	0.63	0.79
I worry about drug interactions between the medications I take.d	0.35	0.59	0.42	0.65
My medication interferes with my routine daily activities.d	0.54	0.74	0.60	0.77
I have trouble taking my medication the way I am supposed to.d	0.31	0.56	0.33	0.57

^aTotal n=394, however not every respondent answered each question. For this analysis, n=358. Accounted for 38% of variance

^bTotal n=200, however not every respondent answered each question. For this analysis, n=199. Accounted for 43% of variance

^c1=Strongly Agree, 2=Agree, 3=Not Sure, 4=Disagree, 5=Strongly Disagree

^dReverse-scored so that higher scores on scale are associated with more MRPs.

Phase 2

Structure of Questionnaire

Participants (n=200) were generally married Caucasian females, with at least a high school education, using approximately six prescription medications (Table 1). For most patients (n=187, 93.5%), at least one MRP was identified by the pharmacist. The most prevalent type of MRP was indication-related problems, which were detected in 78% of participants. Item statistics are reported in Table 3. The best performing items were 3 and 4, with means close to 3.0 and statistically significant correlations between these item scores and total scale scores and the number of MRPs detected (0.26 and 0.23 for items 3 and 4, respectively).

Table 3.

Item Statistics

Phase 1a			Phase 2b
Item	Mean (SD)	Alternatives (n, % Endorsing)d	Item-Total Correlation (Corrected)	Mean (SD)	Alternatives (n, % Endorsing)d	Item-Total Correlation (Corrected)	Item-Criterion (Number of MRPs) Correlation
1. Sometimes my medication has effects I do not like.e	2.99 (1.31)	1(65, 17.0) 2(94, 24.5) 3(41, 10.7) 4(144, 37.6) 5(39, 10.2)	0.50c	2.69 (1.33)	1(48, 24.1) 2(55, 27.6) 3(23, 11.6) 4(57, 28.6) 5(16, 8.0)	0.51c	0.12
2. My medication does not seem to help that much.e	1.89 (0.98)	1(160, 40.8) 2(158, 40.3) 3(41, 10.5) 4(24, 6.1) 5(9, 2.3)	0.43c	2.12 (1.05)	1(64, 32.2) 2(78, 39.2) 3(31, 15.6) 4(22, 11.1) 5(4, 2.0)	0.50c	0.02
3. Sometimes I think I take too many medications.e	2.63 (1.31)	1(92, 23.6) 2(123, 31.5) 3(49, 12.6) 4(91, 23.3) 5(35, 9.0)	0.48c	2.89 (1.31)	1(36, 18.1) 2(52, 26.1) 3(30, 15.1) 4(59, 29.6) 5(22, 11.1)	0.50c	0.26c
4. I worry about drug interactions between the medications I take.e	2.84 (1.25)	1(58, 15.1) 2(126, 32.9) 3(52, 13.6) 4(112, 29.2) 5(35, 9.1)	0.46c	2.73 (1.35)	1(46, 23.1) 2(56, 28.1) 3(23, 11.6) 4(53, 26.6) 5(21, 10.6)	0.54c	0.23c
5. I have trouble taking my medication the way I am supposed to.e	2.11 (1.05)	1(113, 29.0) 2(194, 49.9) 319 (4.9) 4(54,13.9) 5(9, 2.3)	0.43c	1.93 (1.02)	1(76, 38.2) 2(89, 44.7) 3(9, 4.5) 4(21, 10.6) 5(4, 2.0)	0.45c	0.13
6. Sometimes I feel worse after I take my medication.e	2.22 (1.11)	1(106, 27.0) 2(182, 46.4) 3(30, 7.7) 4(60, 15.3) 5(14, 3.6)	0.58c	1.98 (0.98)	1(70, 35.2) 2(89, 44.7) 3(15, 7.5) 4(24, 12.1) 5(1, 0.5)	0.67c	0.09
7. Sometimes I think I may not be taking the right medication for my condition.e	2.35 (1.07)	1(77, 19.9) 2(181, 46.9) 3(62, 16.1) 4(49, 12.7) 5(17, 4.4)	0.50c	2.17 (1.04)	1(60, 30.2) 2(76, 38.2) 3(37, 18.6) 4(22, 11.1) 5(4, 2.0)	0.67c	0.17c
8. My medication interferes with my routine daily activities.e	2.03 (1.06)	1(138, 35.6) 2(165, 42.5) 3(31, 8.0) 4(44, 11.3) 5(10, 2.6)	0.61c	1.93 (0.92)	1(68, 34.2) 2(98, 49.2) 3(14, 7.0) 4(17, 8.5) 5(2, 1.0)	0.65c	0.08
9. My medication is helping improve my condition.	2.02 (0.79)	1(89, 23.1) 2(222, 57.5) 3(57, 14.8) 4(13, 3.4) 5(5, 1.3)	0.36c	2.11 (0.90)	1(49, 24.6) 2(98, 49.2) 3(38, 19.1) 4(10, 5.0) 5(4, 2.0)	0.32c	0.19c
Total Score on DTC, median (range)f	21 (9-41)			20 (9-40)

^aCronbach's alpha: 0.79

^bCronbach's alpha: 0.82

^cp < 0.05

^dMeasured on scale where 1=Strongly Agree, 2=Agree, 3=Not Sure, 4=Disagree, 5=Strongly Disagree

^eReverse-scored so that higher scores are associated with more MRPs

^fPossible scores range from 9 to 45

The reliability estimate for the scale was adequate, with a computed Cronbach's alpha of 0.82 (Table 3). In examining both 1- and 2-component solutions for the items, a one-component solution (Table 3) represented the most appropriate fit when examining the items for clinical meaning and their communalities and factor loadings. Subsequently, confirmatory factor analysis was used to examine the model fit of a unidimensional model consisting of one latent factor representing drug therapy concerns and nine observed indicators representing the items on the nine-item scale. The scale of the latent factor was set by fixing one of the unstandardized item factor loadings. Model fit statistics were examined when fixing alternative unstandardized item factor loadings and the findings were unchanged (Figure 1). Item loadings ranged from 0.33 to 0.80, and all loaded significantly (p < 0.001). Model fit indices suggested a marginal overall fit. The chi-squared test was significant (102.69, 27 df, p < 0.001), the comparative fit index was 0.86, and the RMSEA was 0.12. Modification indices were also computed and reviewed but did not identify opportunities to improve model fit.

Figure 1.

One-Factor Confirmatory Factor Analysis (standardized)

Criterion-Related Validity

Scale scores were significantly correlated with the number of MRPs detected, with higher scores correlating with more problems (r= 0.24, p = 0.001). In addition to scale scores, variables significantly associated (p < 0.2) with number of MRPs detected on bivariate tests included: sex, marital status, education level, income, ability to pay for medications, total number of prescription medications, total number of vitamins/supplements/herbals, and total number of daily medication doses. After removing the total number of prescription medications variable from the regression model, due to its significantly high correlation (r > 0.7) with the total number of daily medication doses variable, a statistically significant multivariate regression model demonstrating associations among predictor variables and the dependent variable, number of MRPs detected was constructed (Table 4). Total daily doses rather than number of medications was used in the model due to its increased specificity.

Table 4.

Predictors of Medication-Related Problems^a

Dependent Variable	Significant Predictorsb		P-value	Parameter Estimate of B (95% Confidence Interval)	Beta	R2	R2Adj	Change in R2
Total number of MRPs	Step 1	Total number of vitamins, herbals, supplements	0.002	−0.17 (−0.28, −0.06)	−0.24	0.19	0.18	0.19
		Household incomec	0.008	0.51 (0.13, 0.89)	0.19
		Total daily medication doses	< 0.001	0.10 (0.06, 0.14)	0.36
	Step 2	Scale score	0.031	0.04 (0.00, 0.07)	0.15	0.21	0.20	0.02

^aOverall model, p < 0.001

^bOther variables included but removed step-wise: sex, marital status, education, ability to pay for medications

^cItem asked, “How would you describe your household income?; 1=Comfortable, 2=Just enough to make ends meet, 3=Do not have enough to make ends meet.”

Scale scores were associated with whether the patient had one or more effectiveness problems (p= 0.075), safety problems (p =0.011), and adherence problems (p= 0.01). Other significant associations (p < 0.2) on bivariate tests were observed for--effectiveness problems: total number of prescription medications and total number of daily medication doses; safety problems: total number of prescription medications, income, and total number of daily medication doses; adherence problems: age, sex, marital status, total number of prescription medications, income, ability to pay for medications, and total number of daily medication doses. After removing total number of prescription medications as described above, these predictor variables were used to construct logistic regression models with presence of one or more effectiveness, safety, or adherence problems as the dichotomous dependent variables. Scale scores did not remain a significant predictor of any problem type when controlling for other predictor variables in the regression models (data not shown). Scale scores were not associated on bivariate tests with indication-related problems.

The ROC curve produced a significant area under the curve (area=0.78, p=0.001; 95% CI: 0.66-0.90). At a score threshold of 15 (items reverse scored as required), the scale exhibits an approximate sensitivity and specificity of identifying patients with at least one MRP of between 81-86% and 61%, respectively.

Discussion

These findings suggest scores on the nine-item scale are a statistically significant, although fairly modest, predictor of MRPs when controlling for other significant predictors of problems. While scores were not a strong predictor, by psychometrically evaluating the utility of the questionnaire in this fashion, this work builds on previously published descriptions of self-administered questionnaires measuring patient perceptions of medications. Previous evaluations either did not examine criterion-related validity^18,20-21 or did not evaluate actual MRPs identified by pharmacists.¹⁹

Other studies have suggested that using self-administered screening tools such as this may increase the number of patients identified for MTM.⁷ The use of the scale in practice to identify patients for MTM requires additional research to evaluate its impact on numbers of patients identified, resulting MRPs detected and resolved, and cost savings. There is a potential for clinical utility; highlighting that patient-reported data (i.e., information not routinely available in a medical chart) might offer some benefit in identifying patients at risk for MRPs beyond MTM eligibility criteria used routinely. However, these findings suggest the role of scale scores might be limited and further study of the instrument is warranted to provide refined estimates of its criterion-related validity. This is especially the case if the goal of the user is to ultimately predict indication-related problems, as scale scores were not associated with these problems in particular. Additionally, although others have begun to explore stakeholder perceptions regarding the use of these types of tools in practice, additional research in this area is needed.²⁵

Some of the items on the scale performed inadequately; therefore, more research is needed to examine the psychometric properties of this scale in other populations. In addition, more research is needed in general with regards to identifying predictors of MRPs, because the regression model only explained approximately 20% of the variability in numbers of MRPs in this sample. Furthermore, as described, due to high correlations among the “total number of medications” and “total daily doses” variables, the former was removed from the models. This could have impacted resulting estimates.

The findings of the confirmatory factor analysis suggest a need for future research pertaining to the item pool and resulting scale to identify a potentially better-fitting model. These findings suggest that drug therapy concerns is likely a multidimensional construct, which is how the DTC instrument was originally conceptualized and designed by its authors.²¹ A non-significant chi-squared value would indicate good model fit, however, this is difficult to achieve as this test is statistically powerful. In a CFA, the chi-squared test assesses the discrepancy between actual and reproduced covariance matrices. Values above a 0.95 on the comparative fit index and values less than 0.05 on the RMSEA would indicate good model fit. ²⁶ However, while drug therapy concerns as a whole is a broad underlying construct, the proposed utility of this tool (as a screen for MRPs) positions estimates of criterion-related validity (i.e., correlation between scores and identified MRPs) as a greater focus of the current research. The modest incremental utility of the scale is worthy of further investigation. Individuals choosing to utilize the scale in practice, as a potential screening tool for MRPs, could consider a score of 15 as a reasonable starting place for a definition of a “positive screen” of heightened risk for MRPs.

There were several limitations to this study. First, the sampling approach for Phase 1 was quota sampling. Although this improved the heterogeneity of the sample as reported in previous assessments of the DTC scale,²¹ it was not without problems. Due to the concurrent collection of data at multiple sites, quotas were not precisely met and meeting enrollment targets for younger males in particular was a challenge. Furthermore, Phase 2 enrolled a convenience sample. Therefore, the sample was rather homogenous, particularly in terms of education level, marital status, and race/ethnicity. In addition, as data for Phases 1 and 2 were collected at some of the same sites, it is possible that some patients participated in both phases. Both samples were also collected primarily in rural areas and insurance carrier information was not collected from patients; therefore, the generalizability of these findings to any particular payer group (such as Medicare Part D) for MTM is unknown.

Finally, although pharmacists completed fictitious patient cases for practice prior to data collection to maximize consistency, it is important to note that the outcome measure (i.e., number and type of MRPs detected) could be dependent on both the taxonomy used for this study and the pharmacists participating. Any differences in MRP classification among the participating pharmacists were not documented or considered. In addition, numerous taxonomies for categorizing these problems are available²⁷ and it is possible that different results could have been found in a study with other pharmacists conducting the medication assessments or using a different system for documenting problems. Furthermore, the extent to which laboratory data was available to the pharmacists upon request to patients’ prescribers, and subsequently used in MRP determination, was not tracked and could have impacted the type and number of MRPs detected.

Conclusion

Psychometric evaluation of a nine-item, self-administered scale suggests patient-reported concerns might provide modest enhancements to other strategies in the identification of MRPs, however, their role may be limited. This or other questionnaires capturing patient concerns or behaviors could be explored further to better understand their potential utility in identifying patients in need of MTM services.

Article Synopsis.

This paper describes a 2-phase effort to develop a brief, self-administered scale and estimate the psychometric properties of the resulting scale as a screen for medication-related problems (MRPs). Item statistics, factor analyses, and bivariate associations were computed, and multivariate models were constructed. Scores on the modified scale was correlated with MRPs (r= 0.24, p = 0.001), and a modest incremental benefit to MRP prediction was found when controlling for other variables. The use of the scale or other instruments measuring patient perceptions pertaining to their medications as a screening tool for MRPs warrants further research.

Highlights.

• This paper describes a 2-phase effort to develop a brief, self-administered scale and estimate the psychometric properties of the resulting scale as a screen for medication-related problems (MRPs).

• Item statistics, factor analyses, and bivariate associations were computed, and multivariate models were constructed.

• Scores on the modified scale was correlated with MRPs (r= 0.24, p = 0.001)

• A modest incremental benefit to MRP prediction was found when controlling for other variables.

• The use of the scale or other instruments measuring patient perceptions pertaining to their medications as a screening tool for MRPs warrants further research.