Expected health benefits of e-Therapeutics Highlights according to pharmacists and physicians

Introduction

The purpose of this research brief is to describe the expected health benefits of e-Therapeutics Highlights,^* as evaluated by Canadian pharmacists and family physicians. The Canadian Pharmacists Association (CPhA), the College of Family Physicians of Canada (CFPC) and the Information Technology Primary Care Research Group (McGill University) collaborated to create an innovative continuing education program, called e-Therapeutics Highlights. Highlights are key treatment recommendations from e-Therapeutics+.^** CPhA produces and maintains e-Therapeutics+, a web portal comprising clinical topic summaries that are authored and peer reviewed by subject matter experts (www.e-therapeutics.ca). Once a week, CPhA and CFPC members receive an e-Therapeutics Highlight delivered by e-mail, which they have the option of evaluating by means of a reflective learning activity based on the Information Assessment Method (IAM).¹ For each rated Highlight, pharmacists receive continuing education units, and family physicians receive Mainpro credits.

We present the aggregated results of IAM ratings as a way of summarizing the wisdom of the crowd of Highlight raters. Crowdsourcing is a force multiplier, defined as the release of online material to a crowd of users who may be interested in contributing ideas or performing a task such as rating a Highlight (voting) and submitting their work to a platform or an organization such as the CPhA, for the profit of the entire community.^2,3 Large groups can be collectively wise in identifying relevant information.⁴ This principle of “crowdsourcing” has been applied to the development of innovative learning network approaches such as those seen in Wikipedia and various voting systems.^5,6 Crowdsourcing allows the traditional “ask-the-user” approach to reach a wider audience.^7,8

Methods

Upon viewing each Highlight, participants in this continuing education program were provided with access to the chapter from which it was derived. All CPhA and CFPC members with valid e-mail addresses were invited to participate in the program. Members who accepted this invitation completed a short demographic survey (once). After clicking the “Earn CEU” button (pharmacists) or “Earn Mainpro Credits” button (physicians) adjacent to each Highlight, participants completed an IAM questionnaire (www.mcgill.ca/iam).

The IAM allows clinicians to systematically document potential outcomes of the learning activity, categorized as follows⁹: 1) the cognitive impact of the information (e.g., 1 item concerns learning), 2) the clinical relevance of the information for at least 1 patient (e.g., totally relevant), 3) information use for a specific patient (e.g., 1 item concerns the modification of treatment), and 4) expected patient health benefits (e.g., 1 item concerns avoiding unnecessary treatment). The IAM is based on a model derived from information studies, and the IAM questionnaire has been content validated.^10-12 In addition, the IAM questionnaire allows clinicians to provide constructive feedback that is used to further improve e-Therapeutics+.^13,14

Data from IAM questionnaires completed up to December 31, 2014, were analyzed using descriptive statistics and the benefit of information index (BII). Raw data are available on request. A minimal number of ratings per Highlight (cutoff) was applied to exclude a small number of ratings of Highlights delivered late in the study period. Regarding data for pharmacists, Highlights with at least 40 ratings were included (n = 137 of 145), while for physicians, Highlights with at least 400 ratings were included (n = 245 of 263). For each Highlight, the BII was calculated using “yes/no/possibly” answers to the final IAM question, “Do you expect any health benefits from applying this e-Therapeutics Highlight to a particular patient?” The BII formula is as follows: $\mathrm{BII}=\frac{Y+0.5P}{\left(Y+P+N\right)}$. It considers the number of

“yes” (Y), “possibly” (P) and “no” (N) expected health benefits ratings associated with a Highlight. As each Highlight is not rated by the same number of clinicians, the BII takes into account the proportion of ratings. For each Highlight, the BII considers the proportion of “yes” ratings plus half of the proportion of “possibly” ratings. The BII is proportional to the number of ratings with expected health benefits and inversely proportional to the number of ratings with no expected benefit. BII values fall into a 0 to 1 range, where 0 means all ratings reported no benefit and 1 means all ratings reported expected benefits.

Ethics

This work was conducted according to the ethical principles stated in the Declaration of Helsinki. Ethics approval was obtained from the McGill University Institutional Review Board.

Results

Pharmacists (2012-2014)

Across Canada, 1907 pharmacists (CPhA members) completed at least 1 IAM questionnaire to rate and comment on Highlights between March 8, 2012, and December 31, 2014. Among the participants who completed the demographic questionnaire, 62.2% were women (1087/1747) and 60.4% were community pharmacists (916/1516). Pharmacists submitted 16,400 ratings on 137 Highlights (average 120 ratings per Highlight; range: 73-209). They indicated that the clinical information from Highlights was totally or partially relevant for at least 1 patient in 59.6% of the ratings (9772/16,400) and “will be used for at least one patient” in 30.6% of ratings (5025/16,400). In 19.7% of the ratings (3223/16,400), pharmacists expected patient health benefits from using the Highlight information, such as “avoiding an unnecessary treatment, or preventive intervention.” The BII value ranged from 0.06 to 0.51 (mean 0.23), indicating that, on average, each Highlight led to expected health benefits in 23% of 120 ratings. This translates to a potential health benefit for at least 27 patients per Highlight.

The 5 most potentially beneficial Highlights for pharmacists (highest BII value) are presented in Table 1.

Table 1.

Most beneficial Highlights according to pharmacists

e-Therapeutic Highlights	Number of ratings		BII
	Pharmacists	Physicians	Pharmacists	Physicians
Constipation in adults: Osmotic laxatives (e.g., polyethylene glycol 3350, lactulose) are safe and effective for long-term use. Polyethylene glycol (PEG) is also safe and effective for use in pediatric and geriatric patients. Whenever possible, use PEG instead of lactulose for the treatment of chronic constipation since it results in greater improvements in stool frequency, stool form and relief of abdominal pain and reduces the need for additional laxatives.	137	2173	0.51	0.53
Osteoarthritis: The risk of increased cardiovascular events for both celecoxib, the only remaining COX-2 inhibitor, and nonselective nonsteroidal anti-inflammatory drugs (NSAIDs) is acknowledged by experts. The American Heart Association’s statement on the use of NSAIDs concludes that the risk of cardiovascular events is proportional to COX-2 selectivity and underlying patient risk. They suggest that stepped-care therapy, guided by patient relief, should start with acetaminophen, small doses of opioids, nonacetylated salicylates or nonselective NSAIDs at the lowest effective dose for the shortest possible time. A meta-analysis suggested that naproxen may be the least harmful NSAID for cardiovascular risk, but further data are needed.	178	1417	0.50	0.50
Sinusitis: Avoid antihistamines in acute sinusitis because of their tendency to cause excessive dryness with thickening of secretions and crusting. The second-generation antihistamines may, however, have a role in chronic sinusitis where a clear allergic component is demonstrated.	130	1051	0.46	0.46
Osteoarthritis: Topical diclofenac solution is as effective for pain relief of hand and knee osteoarthritis as oral NSAIDs. In addition, the incidence of gastrointestinal bleeds is lower than in patients treated with oral NSAIDs. Therefore, begin NSAID therapy with topical rather than oral agents in persons 75 years and older. Monitor for serious adverse effects that may occur in elderly patients, including gastrointestinal upset and local irritation.	115	1328	0.46	0.54
Smoking cessation: Be aware of the effect of smoking on hepatic metabolism. Induction of CYP1A2 by cigarette smoking can increase the clearance and potentially necessitate higher doses of drugs such as caffeine, clozapine, diazepam, estrogens, fluvoxamine, methadone, nifedipine, olanzapine, rasagiline, theophylline, trifluoperazine or warfarin. Conversely, a reduction of the dose may be required if the patient quits smoking. Patients should be made aware of the potentially increased effects (and side effects) of caffeine when they quit smoking.	125	1120	0.45	0.46

Physicians (2010-2014)

Across Canada, 13,444 physicians (CFPC members) submitted at least 1 Highlight rating between January 19, 2010 and December 31, 2014. All completed the demographic questionnaire: 51.2% were women (n = 6877), and 90.5% were practising family physicians (n = 12,161). Physicians submitted 261,146 ratings on 245 Highlights (average 1066 ratings per Highlight; range: 421-2650). They indicated that the clinical information from Highlights was totally or partially relevant for at least 1 patient in 88.2% of the ratings (230,220/261,146) and “will be used for at least one patient” in 46% of ratings (120,189/261,146). The BII value ranged from 0.12 to 0.62 (mean 0.37), indicating that, on average, each Highlight led to expected health benefits in 37% of 1066 ratings. This translates to a potential health benefit for at least 391 patients per Highlight.

Discussion

Overall, 1907 pharmacists and 13,444 physicians reported that the informational content of Highlights was valuable. They reported that Highlights can be beneficial to patients in 23% and 37% of IAM ratings, respectively. The use of IAM offers a friendly automated solution to integrate a continuing education program and its evaluation. While other programs use the IAM, the present results cannot be easily compared, as the information content and participants are different (e.g., InfoPOEMs are e-mailed synopses of original clinical research and rated by members of the Canadian Medical Association). There are limitations, as data were self-reported, and participants were not representative of the Canadian populations of pharmacists and physicians. In addition, since we had no access to those who received Highlights but did not read the content or who read the content but did not rate it, our results are not generalizable. Nonetheless, the wisdom of the crowd was employed to identify the most beneficial Highlights according to pharmacist and physician raters. These Highlights cover common topics such as constipation and osteoarthritis. The higher average value of BII for physicians (0.37) as compared with pharmacists (0.23) might be explained by differences in professional roles (e.g., in terms of prescribing authority).

Conclusion

In the context of a popular continuing education program, Canadian pharmacists and family physicians rated Highlights. Programs such as Highlights are typically associated with a change in knowledge. It remains to be determined whether the increase in awareness of Highlights and knowledge is associated with practice improvement. Further research could compare self-reported IAM data with an audit of electronic medical records and examine any Highlight-related practice improvement. Finally, raising awareness of the most beneficial Highlights (Table 1) may be of interest to pharmacists, regardless of their participation in the Highlights program.