An Instrument to Evaluate Pharmacists’ Patient Counseling on Herbal and Dietary Supplements

Abstract

Objective

To develop a measure of pharmacists’ patient counseling on herbal and dietary supplements.

Methods

A systematic process was used for item generation, testing, and validation of a measure of pharmacists counseling on herbal and dietary supplements. Because a pharmacist-patient encounter may or may not identify an indication for taking an herb or dietary supplement, the instrument was bifurcated into 2 distinct components: (1) patient counseling in general; and (2) patient counseling related to herbal and dietary supplements.

Results

The instrument demonstrated high reliability and desirable construct validity. After adjusting for item difficulty, we found that pharmacists tended to provide more general patient counseling than counseling related to herbal and dietary supplements.

Conclusion

This instrument can be applied to assess the quality of counseling provided by pharmacists and pharmacy students, and the outcomes of pharmacist and pharmacy student education on herbal and dietary supplements.

INTRODUCTION

Compared to prescription and nonprescription drugs, herbal and dietary supplements have less stringent regulations and looser manufacturing oversight in the United States¹ and are available at many retail outlets, including community pharmacies, grocery stores, and health food stores, as well as on the Internet. Consumers often obtain information on herbal and dietary suppelements from their family, friends, magazines, books, or infomercials rather than from healthcare professionals.^2,3 Because evidence on the efficacy of herbal and dietary supplements often is lacking or conflicting, providing informed and consistent recommendations about these products to their patients can be challenging for healthcare professionals.⁴

An estimated 16% to 42% of patients use prescription drugs concomitantly with herbs and dietary supplements,^5-7 and 22% to 50% take herbs and dietary supplements prior to surgeries and/or during preoperative care.^8-10 The extensive use of herbal and dietary supplements among higher-risk patients poses a potential safety concern that could be mitigated by pharmacist counseling on the appropriate use of these products. Pharmacists have been encouraged to integrate professional counseling and education services on herbal and dietary supplements into their daily practice,^11,12 yet pharmacist counseling services on the appropriate use of these products are often lacking.^13-18 Possible explanations for insufficient counseling on herbal and dietary supplements by pharmacists include an insufficient education in and knowledge of herbal and dietary supplements, unfavorable attitude towards these products, a limited evidence base on herbal and dietary supplements in the literature, and a lack of time dedicated to patient counseling.

Although patient self-care instruction regarding herbal and dietary supplements by pharmacists is indispensable,¹⁹ concerns about pharmacists’ ability to address issues related to the use of herbal and dietary supplements have been expressed by pharmacy educators, individual pharmacists, pharmacy managers, patients, and other health professionals.^12-13,20 The amount of information on herbal and dietary supplements provided to students in colleges and schools of pharmacy and to pharmacy practitioners through postgraduate continuing education is not commensurate with the widespread use of these products in the United States. As of 2005, some colleges and schools of pharmacy still did not offer course instruction in natural products or herbal supplements.^12,14,21-24 Furthermore, practicing pharmacists perceived they were not satisfactorily educated about herbal products and other types of alternative medicines during pharmacy school.²⁵ Shah and colleagues recommended the establishment of guidelines for curricular content on herbal supplements in pharmacy education.²² Revised 2007 Accreditation Council for Pharmacy Education (ACPE) guidelines for evaluating and improving pharmacy curricula include topics regarding knowledge and expected competencies on complementary and alternative medicine, including herbs and dietary supplements.²⁶

Several instruments that assess patient counseling regarding medication-related issues are available, with varying evidence to support their validity.^27-30 The focus of these instruments is on the performance of medication-related counseling or on counseling for specific self-care products. Most of the studies either do not describe the conceptual framework used to guide instrument development and item generation, or they provide a checklist rather than an instrument with a meaningful metric.

The purpose of this study was to describe the development of an instrument that assesses the extent to which counseling activities regarding herbal and dietary supplements are performed by pharmacists. This study was part of a broader study included in the development and implementation of knowledge, attitude, and behavior (KAB) surveys to gather information, describe current practice, and potentially assess the effects of initiatives to refine the practice, education, and training of pharmacists with respect to herbal and dietary supplements.

METHODS

Instrument Development and Refinement

The goal of the instrument was to quantify the provision of patient counseling about herbal and dietary supplements by pharmacists. Specifically, the performance of patient counseling on these products was the construct of interest, and patient counseling was defined as an individualized counseling process when the pharmacist helps the patient treat his/her problems related to herbal and dietary supplement use by providing appropriate guidance using a problem-solving approach.³¹ Several relevant measures were identified in the literature including Kemper's communication practice scale,²⁸ the USP Medication Counseling Behavior Guideline,^27,32 and others.^28-30 Upon further evaluation, we determined that these instruments did not satisfy our measurement goals. Specifically, the pharmacist-patient encounter may or may not identify an indication for taking an herb or dietary supplement. Consequently, the performance of counseling regarding prescription drugs or herbal and dietary supplements was conceptualized as 2 measures: a general patient counseling measure (PC-G) and an herbal and dietary supplement-specific counseling measure (PC-HDS).

When considering herbal and dietary supplements for an individual patient, a special report by the American Dietetic Association and the American Pharmacists Association recommends that health professionals use the SOAP (ie, subjective, objective, assessment, plan) format.^33,34 This approach was followed in conceptualizing the patient-pharmacist consultation process for a drug/HDS-related encounter (Figure 1).³⁵ The counseling process included 6 factors that characterized effective patient-centered communication (eg, fostering relationships, exchanging information, responding to patients’/consumers’ emotions, managing uncertainty, making a decision, enabling patient/consumer self-management).³⁶ When operationalized, the construct was conceptualized as having 3 main domains (ie, foster relationships/information gathering, assessment, and plan/follow-up), each with 2-3 components (Figure 2). Items were generated by the authors or taken either from the literature or established measures.^{29-31,34,37-43}

Figure 1.

Effective patient/consumer-pharmacist consultation process for a clinical encounter in a community pharmacy related to herbal and dietary supplement (HDS) use. Abbreviation: OTC= over-the-counter.

Figure 2.

Conceptualization of a measure of performance of counseling regarding herbal and dietary supplements (HDS). S₁, S_n= items created for “subjective” component; O₁, O_n=items created for “objective” component; C₁, C_n=items created for “content” component; P₁, P_n=items created for “process” component; A₁, A_n=items created for “recommendation” component; D₁, D_n=items created for “documentation” component; M₁, M_n=items created for “monitoring” component.

Item pools were created with a sufficient number of items to cover a broad spectrum of pharmacists’ provision of counseling related to herbal and dietary supplements. Once the item pool was generated, internal expert panel discussions were conducted to identify irrelevant or redundant items, and those items that might have different meanings for some potential respondents. Two lay persons without a pharmacy background reviewed the structure and wording of the instrument as a final check. An external reviewer assessed the draft of the Web-based survey instrument and provided suggestions on each item and the survey administration process.

Item Performance Assessment

Each item on the pharmacists’ herbal and dietary supplement counseling measure used a 5-point Likert-type scale labeled as “none of the time,” “a little of the time,” “some of the time,” “most of the time,” and “all of the time.” A proportion score was derived by dividing the maximum obtainable total score into the total observed score, where the expected total score was equal to the number of items multiplied by 5 (5-point Likert type scale). The higher the score, the more frequently a pharmacist performed patient counseling tasks related to herbal and dietary supplements.

The properties of each measure were evaluated using kernel regression-based nonparametric item response theory (KIRT) models^44,45 to examine the properties of each measure (ie, testing of items with monotonic response pattern, variant item ordering, sufficient number of functional category responses) along with classical test theory-based approaches (ie, internal consistency [Cronbach's alpha], exploratory factor analysis [EFA]).⁴⁶ Items with poor psychometric properties were revised for content and/or eliminated, resulting in a final set of items related to each patient counseling instrument. At least 2 strata of difficulty (ie, participant ability strata) were deemed necessary for the measure to have a sufficient span of ability to interpret the construct defined by the items.⁴⁷ Only items contributing to an acceptable level of reliability (ie, α ≥ 0.7) were retained.⁴⁸

Subject Recruitment

Practicing pharmacists with various levels of knowledge about and practice behaviors regarding herbal and dietary supplements were asked to complete the instrument so that the appropriateness and difficulty of items could be examined in the pilot study. In a subsequent larger scale validation study, pharmacists who were employed in healthcare settings and had regular patient contact were recruited. The studies were approved by the Institutional Review Board of the University of Illinois at Chicago.

Data Analysis

The proportion score and the logit score for the 2 measures (PC-G and PC-HDS), were compared using an independent t test, or if not appropriate, the Mann-Whitney test, based on the results obtained from 1 sample Kolmogorov-Smironov test.⁴⁹ Differences in participant characteristics across the 2 administrations of the survey instrument (ie, pilot and large-scale study) were examined using t tests to identify differential item functioning, and 95% confidence intervals were computed using the bootstrap method.⁵⁰ Person measure scores (ie, in logit units) obtained in the validation study were used when conducting bivariate analyses, ie, Pearson correlations. The corresponding nonparametric analyses were performed whenever the parametric assumption of normality was violated. These approaches were performed to examine the association between background information (eg, including personal, workplace information, herbal and dietary supplement information/training resources) and logit scores of pharmacists’ performance of patient counseling related to herbal and dietary supplements, respectively. A significance level of α ≤ 0.05 was set a priori. All analyses were performed using SPSS for Windows, version 16.0. (SPSS Inc., Chicago, IL) and in using R: a language and environment for statistical computing (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Twenty (58.8%) of the 34 participants in the pilot study completed the KAB survey instrument in its entirety, and 179 (54.4%) of the 330 participants in the larger-scale validation study completed the revised survey instrument. Of 46 items in the initial item pool, 17 were revised and retained for the general measure on patient counseling. Eleven items were generated for the performance measure on patient counseling on herbal and dietary supplements (Table 1). The proportion of items measuring each component of patient counseling in the general construct was similar to that for the initial item pool. However, no items were retained that measured pharmacists’ tasks related to gathering subjective or objective information from patients within the herbal and dietary supplement counseling construct.

Table 1.

Item Content Included in a Pilot Study of Pharmacists’ Patient Counseling Measures Regarding Herbal and Dietary Supplements (HDS)

Abbreviations:PC-G= Patient Counseling - General Measure; PC-HDS= Patient Counseling - Herbal and Dietary Supplement Measure

^a Items were retained in the instrument prepared for the larger scale validation study

^b Item ID represents item order in the instrument prepared for the pilot study

The psychometric properties of the 2 measures are presented in Table 2 and Table 3. With the removal of poorly performing items, the Cronbach's α for the general patient counseling measure in the pilot study was reduced from 0.91 (17-item measure) to 0.86 (10-item measure). In the larger validation study with sufficient sample size to apply the EFA, 2 factors were extracted for general patient counseling measure and were intuitively named as “information gathering” for factor 1 and “facilitating decision making” for factor 2. The 2 factors were strongly correlated (r _f-f = 0.68), which inferred these items might account for only 1 construct. While the items on general patient counseling measure possessed a fairly good level of reliability, 2 items (PC-G2 and PC-G9) did not demonstrate monotonicity. Upon removal of these items, the 8 remaining items were able to distinguish at least 3 groups of respondents’ performance behavior (participant strata = 3.09). The internal consistency of the general patient counseling measure was acceptable (α = 0.81).

Table 2.

Psychometric Properties of Patient Counseling - General Measure (PC-G) in Two Stages of Validation Studies

Abbreviations:SD= standard deviation; KIRT=kernel regression-based nonparametric item response therapy

^a proportion respondents endorsing higher item levels not increasing as the latent trait level increases

95% interval refers to the 2.5% and 97.5% of the corresponding scores

Table 3.

Psychometric Properties of Patient Counseling – HDS Measure (PC-HDS) in Two Stages of Validation Studies

Abbreviations:SD= standard deviation; KIRT=kernel regression-based nonparametric item response therapy

^a proportion respondents endorsing higher item levels not increasing as the latent trait level increases

95% interval refers to the 2.5% and 97.5% of the corresponding scores

For the PC-HDS construct, the α slightly increased from 0.84 to 0.86 in the pilot study after removing 4 items. Two items related to the tasks of subjective information gathering and recommendations were removed from the measure of patient counseling on herbal and dietary supplements. In the larger validation study, the remaining 7 items possessed a good level of reliability (α = 0.89 ± 0.01) and satisfied other psychometric criteria, and were able to stratify patient counseling on herbal and dietary supplements into more than 4 ability groups (participant strata = 4.13). Upon the EFA, 2 items indicating the monitoring (ie, PC-HDS10, PC-HDS11) were loaded in 1 standalone factor and the remaining 5 items (PC-HDS1, PC-HDS3, PC-HDS6, PC-HDS7, PC-HDS8) assessing pharmacists’ performance on the tasks associated with assessment and plan of herbal and dietary supplement counseling were loaded into another factor. These 2 factors were strongly correlated (r _f-f = 0.93) and could be recognized as 1 construct, which corresponded to 1 of the KIRT assumptions, “unidimensionality.”

Quantifying Pharmacists’ Provision of Herbal and Dietary Supplement Counseling

The proportion scores of the 2 measures (ie, PC-G, PC-HDS) that quantified pharmacists’ provision of general and herbal and dietary supplement counseling, were significantly higher in the pilot study than in the larger study (p < 0.05). After adjusting for item difficulty, the person logit scores on the PC-G and PC-HDS items were significantly higher in the pilot study than in the larger study (p < 0.05), which indicated that respondents in the pilot study seemed more likely to perform more activities than respondents in the large-scale study.

Table 4 presents the strength of correlations between the pharmacists’ background information and their person scores for each measure based on the correlation coefficients. There were zero to weak correlation values (r ranged from the absolute value of 0.001 to 0.2) between the majority of pharmacists’ demographic information and their person logit scores, even if there were some exceptions. Experience with disease state management was positively correlated with the provision of general patient counseling (r _s ≥ 0.2). Provision of counseling on herbal and dietary supplements was positively correlated with the following: (1) pharmacists’ personal experience with taking herbs and other supplements, (2) practicing in community pharmacy, (3) providing counseling on nonprescription and self-care products, and (4) with recommending herbal and dietary supplements and offering information on herbal and dietary supplements (r_s ≥ +0.2).

Table 4.

The Strength of Correlations Between the Performance of Herb and Dietary Supplement Counseling and Pharmacists’ Background Information

Abbreviations:HDS = herbal and dietary supplements; PC-G= Patient Counseling - General Measure; PC-HDS= Patient Counseling – HDS Measure

^a the correlation coefficient was greater than |±0.2|

Strong correlations were found between pharmacists’ total scores of PC-G and PC-HDS (mean of r [95% interval] = 0.53 [0.40, 0.64] for logit unit and 0.55 [0.45, 0.73] for proportion score, respectively).

DISCUSSION

Pharmacists and other clinicians are expected to help consumers by evaluating the effectiveness and safety evidence for herbal and dietary supplement products and sharing this information with consumers.¹² In this study, a newly developed instrument for assessing the performance of counseling by pharmacists was supported by the evidence of validity and reliability. We propose that this well-validated instrument can be used for self-assessment by pharmacists, evaluation of student competencies by pharmacy faculty members and preceptors, and quality improvement initiatives by pharmacy managers. This instrument fills a gap that existed in the assessment of the quality of herbal and dietary supplement counseling.

The instrument for the assessment of pharmacists’ provision of counseling was bifurcated into 2 measures: general counseling and herbal and dietary supplement-specific counseling. This was necessary because the pharmacist-patient encounter may or may not begin with an inquiry regarding use of herbs and/or dietary supplements. It may, however, result in the recommendation for herbal and dietary supplements. This instrument implicitly captures the quality of patient counseling encounters by focusing on the main components of the counseling process and the frequency in which they are conducted. Although not all tasks performed by the pharmacist related to a counseling encounter are captured, the instrument quantifies key elements involved in patient counseling associated with herbal and dietary supplements.

Previous measures related to counseling have demonstrated evidence of validity and reliability using primarily classical test theory-based approaches that assumed, for instance, that scores were normally distributed and that the measure was unidimensionality.^28-32 In contrast, the KIRT used in the present study does not invoke assumptions of normality, and the results demonstrated good construct validity and internal consistency for each measure. Additional convergent validity of the patient counseling measures was presented based on significant relationships with related factors.

In this study, pharmacists were requested to indicate the frequency with which they provided herbal and dietary supplement counseling by responding to statements that described different activities related to herbal and dietary supplement counseling. In contrast, previous studies used different approaches to operationalizing item responses related to specific or general activities about herbal and dietary supplement counseling. Some studies used 1 or more dichotomous questions.^{14,16-17,22,51-53} For example, in Dolder's study, pharmacists were asked whether they had received inquiries from patients about natural products on a weekly or monthly basis.¹⁷ Some studies inquired about the length of pharmacists’ encounters ^13,16 or asked pharmacists to rate the frequency using a Likert-type scale. For example, in Brown's study, pharmacists’ average rating on how often they inquired about the patients’ use of complementary and alternative medicine was 2.5 ± 0.9 (scale: 1 = never, 5 = always).⁵¹

In this study, the average proportion of total score on the provision of general patient counseling of 0.65 would be analogous to endorsing 26/40 points, or participants responding “some of the time” on all 8 items. The average person proportion score on the provision of herbal and dietary supplement counseling of 0.46 would be equivalent to endorsing 16/35 points, or participants responding “a little of the time” to all 7 items. This implies that respondent pharmacists tended to spend at least “a little of the time” in their daily practice either providing general patient counseling, or providing patient counseling specifically about herbal and dietary supplements. After adjusting for item difficulty, the average person logit score on the PC-G scale was dramatically higher than the scores on the PC-HDS scale. Given the strong correlation between pharmacists’ person total scores on the PC-G and PC-HDS items, these results imply that pharmacists who provided more general counseling tended to provide more counseling on herbal and dietary supplements and vice versa. Moreover, pharmacists who personally used herbs, amino acid, vitamins, and other supplements; worked in a community setting; or worked in a setting that offered the herbal and dietary supplement products and provided related information on these products tended to spend more time counseling patients about herbal and dietary supplement-related matters. These results intuitively mimic documented pharmacy practice behaviors.

Many patients tend not to disclose their experience with using herbal and dietary supplements to their primary health care professionals.^54,55 The results in this study can help to guide the development of interventions, specifically for pharmacy managers and preceptors, that target pharmacists’ awareness and pharmacy students’ professional performance regarding patient care with respect to the appropriate use of herbal and dietary supplement products. The 15-item patient counseling measure could be used for quality and performance assessment in pharmacy practice, experiential training (eg, introductory and advanced pharmacy practice experiences), postgraduate continuing education, and by allied health clinicians. The 8-item general counseling scale could be used as a standalone measure for assessing pharmacists’ general patient counseling on medications, while the 7-item herbal and dietary supplement scale could be modified for use in patient counseling concerning other self-care nonprescription products as well. Nevertheless, the authors encourage further validation and application of the instrument to evaluate educational outreach, experiential training, and quality assurance in diverse settings. Regardless, these results were exploratory in nature and further use of the instrument in other samples of pharmacists, pharmacy trainees, or clinicians would further determine its generalizability.

CONCLUSION

Although no single, standardized patient counseling measure or instrument applicable for all pharmacy practice or experiential training scenarios exists, the psychometric properties of the measure described here support its use in assessing pharmacists' counseling behaviors on herbal and dietary supplement products. The instrument can be used as a tool for quality assurance and training assessment to ensure that a standard level of patient counseling on herbal and dietary supplements is being provided.