Clinical decision-making and dispensing performance in pharmacy students and its relationship to executive function and implicit memory

Abstract

Background

When providing pharmaceutical care, the pharmacist relies upon a clinical decision-making process that involves information gathering, clinical reasoning, and clinical judgment. Typically, pharmacists have to identify, retain and recall numerous pieces of key information arranged spatially in medical records and prescriptions or verbally from colleagues when making decisions. Executive function, including spatial working memory and verbal reasoning, along with other cognitive domains, will likely contribute to the elements that comprise this process.

Objective

To establish the predictive utility of markers of executive function and implicit memory on clinical decision-making and dispensing performance in pharmacy students.

Methods

MPharm students from two sites completed a battery of cognitive tasks designed to measure elements of executive and other cognitive functions (e.g., verbal working memory (VWM), visuospatial working memory (VSWM), and implicit memory (IM)). Performance on 2 clinical case studies was used to assess clinical decision-making ability (n = 16), and a prescription screening and dispensing assessment was used to assess dispensing accuracy (n = 32). A statistical model was built to establish whether executive and other cognitive functions markers can predict clinical decision-making and dispensing performance.

Results

Performance in VSWM test and IM tests were found to explain approximately 63% of the deviance in clinical decision-making ability (null residual deviance = 49.4, deviance explained by variables = 31.0; Matrix Model p < 0.01, Dot-clearing test p < 0.01). Performance is the VSWM, and VWM tests explained approximately 30% of the deviance in the dispensing task (null residual deviance = 7596.7, deviance explained by variables = 2099.3; Matrix Model*Baddeley Reasoning Model, p < 0.05).

Conclusion

The results suggest that specific cognitive domains contribute to the clinical decision-making process. This adds to a growing body of literature that highlights the importance of person-specific factors in predicting clinical competence.

1. Introduction

Over the course of their studies, pharmacists acquire a broad knowledge of pharmaceutical science and therapeutics, which underpins their ability to assess prescriptions for safety and efficacy and dispense items legally and accurately.¹ However, in order to execute these tasks effectively, they must also develop skills in information gathering, identifying medicine-related problems, critical appraisal of evidence, and making judgments on risks and benefits.² When screening a clinical case and a prescription, the initial stages of data gathering and problem identification involve (i) observing the key pieces of information, (ii) recognising their importance, (iii) retaining that information, (iv) recalling, and finally (v) responding appropriately in the context of the entire case. For example, when presented with a set of blood results and a prescription for a patient with headache, confusion and decreased Glasgow Coma Scale score, a pharmacist will need to (i) observe the [Na⁺]_plasma, (ii) recognize that it is low and may be pharmacological in origin, (iii) retain that information, along with other data, when reviewing the prescription for drugs which can cause hyponatremia, and (iv) recall and respond appropriately using clinical reasoning and judgment to determine the most appropriate course of action. Importantly, there may be multiple pieces of information to identify and retain, increasing the cognitive load and complexity of the task. Therefore, it is likely that the clinical decision-making process draws upon cognitive domains such as working memory, implicit memory, and other executive functions.³

Working Memory (WM) is a complex neuronal network located primarily in the pre-frontal cortex, enabling the temporary storage and manipulation of information.⁴ Working memory is thought to be comprised of a number of distinct systems: the Phonological Loop (verbal working memory, VWM), the Visuospatial Sketchpad (visuospatial working memory, VSWM), the Episodic Buffer (which modulates and integrates sensory information), and the Attention Controller.⁵ Together, these systems constitute an aspect of cognitive processing that allows humans to solve problems, reason, and make decisions.⁴ Implicit memory concerns the retrieval of unconscious skills-based memories and is processed in regions of the brain such as the cerebellum.⁶

Executive function shows not only variation across the population but also temporal variation, especially under certain environmental conditions such as stress.⁷ Such variation could therefore lead to fluctuations in performance on tasks that rely on these cognitive domains – perhaps going some way to explain why errors may occur in clinical practice or why performance in clinical examinations differs between individuals and under certain conditions. Evidence shows that factors known to have a negative impact on working memory function can affect pharmacists' dispensing performance. In a study conducted in a community pharmacy setting, researchers identified a link between high workload and self-reported dispensing errors.⁸ In a separate study involving a simulated pharmacy dispensing task, psychosocial factors, such as social group stress, combined with the level of objective and perceived workload, predicted errors.⁹ To our knowledge, however, there are no peer-reviewed publications that have investigated the direct relationship between WM, or implicit memory capacity, and dispensing accuracy, clinical appropriateness, or clinical decision-making by pharmacists or pharmacy students. We, therefore, set out to establish the predictive utility of markers of executive function and implicit memory on these pharmaceutical skills: clinical decision-making and dispensing performance in pharmacy students.

2. Methods

2.1. Clinical decision making and cognition

2.1.1. Participants

The study was conducted in two UK University Pharmacy Schools. Pharmacy students studying at one UK University were recruited into the Clinical Decision-Making arm of the study following an email advertisement. The inclusion criterion was current enrolment into Year 3 of the Master of Pharmacy (MPharm) course, with no exclusion criteria. Participation was voluntary, and after gaining informed consent, participants were asked to attend one of several cognitive and Clinical Decision-Making assessment sessions. Participants were recruited from Year 3, and not earlier in the degree, as, by this stage of the course, they had developed a broad knowledge of clinical pharmacy and acquired basic skills in clinical decision-making. Measurements of 4 cognitive domains were assessed at the beginning of this session: WM: verbal reasoning (Baddeley Reasoning Model), WM: visuospatial (Matrix Model), Executive Function (5-point test), and Implicit memory (dot-clearing test). The tests are described briefly in section 2.3 below.

2.1.2. Assessment of clinical decision-making

Following assessment of cognitive function, participants sat two clinical decision-making (CDM) Objective Structured Clinical Examination (OSCE) stations. OSCEs are an established tool to assess clinical skills and have been used in undergraduate and postgraduate education in Medicine, Pharmacy, and Nursing for over 30 years. They provide a consistent, objective approach to assess clinical skills in a safe clinical environment.¹⁰ Each CDM OSCE station lasted 5 min and assessed problem identification, clinical reasoning, and clinical judgment (Clinical Decision-Making). Specifically, each station consisted of a detailed patient history (with clinical and laboratory investigation) followed by 3 questions. These questions asked each participant to (a) identify any clinical problems, (b) comment on the cause/s of any clinical problems, and (c) what further action should be taken. The first case involved a patient with rheumatoid arthritis prescribed methotrexate but was self-medicating with ibuprofen and subsequently developed methotrexate toxicity (blood dyscrasias). The second case involved a patient who was prescribed ramipril, with blood results indicating renal impairment and hyperkalemia. A CDM OSCE score was obtained based on standardized marking criteria. The supplementary material section provides an example of a typical OSCE station and marking criteria.

2.2. Dispensing accuracy and cognition

2.2.1. Participants

Pharmacy students studying at a different UK University were recruited into the Dispensing Accuracy arm of the study following an email advertisement. The inclusion criterion was current enrolment into the Master of Pharmacy (MPharm) course, with no exclusion criteria. After gaining informed consent, volunteers were asked to attend one of several WM assessment sessions. Measurements of WM were made using 2 different cognitive tests, which assessed either the phonological loop or visuospatial sketch pad element of WM. The tests are described briefly in section 2.3 below.

2.2.2. Assessment of dispensing accuracy

At the time of recruitment, all MPharm student participants had taken a summative dispensing assessment in their previous year of study. This assessment involved students checking a prescription presented to them, addressing any clinical and/or legal problems identified within the script, and then dispensing those items accurately. The mark each student gained in this exam measured their dispensing accuracy and the appropriateness of their clinical and legal check.

Because of slight differences in the exam format and the level of difficulty between year groups, the distribution of marks (mean and standard deviation) was found to be different. To correct for this, the exam marks for students within each year group were first normalized and then transformed into marks that were distributed around an arbitrary mean with an arbitrary standard deviation. Normalization involved calculating z-scores for each student from their individual mark (x), the population mean exam score (from the whole year group, m), and the standard deviation (from the whole year group, s) using Eq. 1 (below). Z-scores for all students across the year groups were transformed, using eq. 2, into a new dispensing accuracy mark with an arbitrary population mean of 55 (X) and a standard deviation of 20 (S). Transformed dispensing marks and working memory scores were subsequently statistically tested for an association using a Pearson's correlation.

$$z\mathit{score}=\frac{\left(x-m\right)}{s}$$

$$\mathit{transformed\; score}=X+\left(z\mathit{score}\times S\right)$$

where X = 55 and S = 20.

2.3. Cognitive tests

Details are provided below of the battery of cognitive tests used in the study. Participants for the Clinical-Decision Making arm sat all 4 cognitive tests, whereas volunteers for the dispensing arm of the study sat cognitive test the Baddeley Reasoning Model and Matrix Model only.

2.3.1. The Baddeley reasoning model

The Baddeley Reasoning Model (BRM)¹¹ is an established measure of verbal working memory. The 60-question assessment involved participants answering up to 60 true/false questions in 3 min relating to a statement about a sequence of 2 letters. For example, for question 1, the statement read: ‘B precedes A,’ and the sequence of letters read BA. For this sequence, the statement is true. In subsequent questions, the sequence of letters and the statements differ. The number of correctly answered questions (out of a maximum of 60) in 3 min provided the student's score.

2.3.2. The matrix model

The Matrix Model (MM)¹² is a cognitive test that assesses visuo-spatial working memory. Participants were presented with an image of a 5 × 5 matrix visualized on a projector screen. An object was displayed at a random location in the grid, and the participants were asked to remember its location prior to the image being turned off. Participants were then provided with a sequence of 3 directions indicating the object's movement in the grid – for example, up 2 spaces, left 3 spaces, down 1 space. Participants were then shown a blank grid (labeled with x, y coordinates) and asked to record the object's new location in their answer booklet (i.e., the x, y coordinates). The task was repeated a further 5 times; however, in each subsequent test, the object's location was altered, and the number of directions increased sequentially to a maximum of 8. The student's score was determined by adding the number of correctly identified directions for each test where they successfully located the object. For example, if a participant correctly identified the object's location in each test, they would achieve a maximum score of 33 (3 + 4 + 5 + 6 + 7 + 8 = 33). If they only correctly identified the object's location following the tests with 3 directions and 6 directions, but not the other tests, they would achieve a total score of 9.

2.3.3. Dot-clearing test

Dot-Clearing Test (DCT).¹³ This computer-based test measured participants' implicit memory. It comprised two sections. In the first section, participants were presented with a series of individual words, each present on the screen for 2 s. Participants were told that they did not need to remember these words. In the second part of the test, participants were shown another series of words individually on-screen, hidden behind a mass of dots. The words were a combination of some of the previously seen ‘test’ words and ‘new’ words. Over a short period of time (seconds), the dots gradually disappeared to reveal the words. Participants were asked to begin typing as soon as they recognized the word. The theory of the test is that participants should identify ‘test’ words (i.e., words that they have seen in the first part of the test) more quickly than ‘new’ words. Following on from this, individuals with better implicit memory should identify ‘test’ words more quickly than those with worse implicit memory. The score for this test is calculated as the ratio of time to identify previously seen work/time to identify a new word. The inference is that the lower the score, the better the implicit memory function.

2.3.4. Five-point test (FPT)

The Five-point Test¹⁴ measures the participant's executive function. During the test, participants were asked to draw as many patterns as possible in 3 min by joining together dots arranged in the style of the five points on a dice. Participants were told that all the lines in each pattern needed to be joined together but could join as many or as few dots as they liked. The number of original patterns generated in 3 min determined the participant's score. The more patterns generated, the better the executive function.

2.4. Statistical analysis

Generalised linear models (GLM), which are less sensitive to the distribution of error around the explanatory variables, were used to identify significant models of CDM OSCE performance and Dispensing Accuracy, using R statistical software [R Core Team].¹⁵ Analyses started with a global model which included all main effects and interactions: Y = β0 + β1x1 + β2x2 + …, where the parameters β0 … βn were estimated by maximum likelihood. The significance of each of the terms was assessed by comparing the difference between the deviance values of the model before and after the term was fitted. This provided the minimum adequate model of significant effects. For each significant term, the deviance explained refers to the change in deviance attributed to the term in question when fitted last, as a proportion of the total deviance explained by the main effects in the minimum adequate model. P-values were estimated by comparison with the reduced model not containing the term in question. The results provide only estimates of coefficients having a significant influence on the model for each analysis.

2.5. Ethics approval

Ethical approval was obtained from the university research ethics committee.

3. Results

3.1. Clinical decision-making and cognition

3.1.1. Assessment of cognition

Sixteen MPharm Year 3 students volunteered to participate in this arm of the study and attended a 1.5-h assessment session during which they undertook 4 short psychometric assessments and 2 CDM OSCE stations. The mean scores for each cognitive test (±SEM) were: BRM, 19.63/60 ± 2.33; FPT, 31.88 ± 2.36; MM, 9.44/15 ± 1.24; DCT, 1.09 ± 0.016.

3.1.2. Cognition and clinical decision-making

The mean combined CDM OSCE score (stations 1 and 2) was 6.66/12 ± 0.45. To assess whether scores on a battery of cognitive tests could predict CDM OSCE performance, we constructed a model using a multivariate Generalised Linear Model approach. Details of the final model, in which 2/4 of the initial predictors were retained in model (MM and DCT), are shown in Table 1. The model reduced the null deviance of 49.39 (df 15) by approximately 63% (residual deviance = 23.94 df 13). A Spearman's correlation between these two variables (MM and DCT) produced an r = 0.142 (p = 0.597), suggesting no multicollinearity exists.

Table 1.

Cognition and Clinical Decision-making: multivariate Generalised Linear Model. *p < 0.05, **p < 0.01.

Predictors	Estimate	Std. Error	Deviance	df	Pr(>Chi)
Intercept	17.87	5.04
MM	0.23	0.072	18.56	13	0.001498**
DCT	−14.51	5.58	12.47	13	0.009246**

3.2. Dispensing accuracy and cognition

3.2.1. Assessment of working memory function

A total of 32 students across years 2, 3, and 4 of the pharmacy course volunteered to participate in the study. The median age of the volunteers was 21 (range 19–24). There was no difference in the mean ± SEM Baddeley reasoning scores between year groups (one-way ANOVA, p > 0.05). Scores in the matrix model showed a trend to being higher in year 4 compared with years 2 and 3, although they did not reach statistical significance (one-way ANOVA, p > 0.05). No correlation was found between the BRM and the MM scores across all years (Pearson's, p > 0.05, R² = 0.04).

3.2.2. Working memory and dispensing safety and accuracy

Dispensing examination marks for participants were first normalized and then transformed into a new mark to allow comparison between year groups (see methods). The series of transformed marks from the combined year groups followed a normal distribution (P > 0.05, D'Agostino & Pearson omnibus normality Fig. 1A). No association was found between scores in the BRM, which is a test of verbal reasoning involving the phonological loop, and dispensing assessment scores across all year groups (Figure 1Bi). There was, however, a significant association between performance in the MM test of WM (which utilizes the visuo-spatial sketchpad) and transformed dispensing scores (Pearson's, p < 0.05, R² = 0.15, Figure 1Bii). A generalised linear model was built to determine whether cognitive assessments could predict dispensing assessment performance. A significant model with both BRM and MM reduced the deviance by approximately 30%, with a significant interaction between BRM and MM (Table 2).

Fig. 1.

Distribution of dispensing (A) marks and association with working memory scores (Bi and Bii).

Table 2.

Working memory and dispensing safety and accuracy: multivariate Generalised Linear Model. *p < 0.05, **p < 0.01. Null residual deviance: 7596.7.

Predictors	Estimate	Std. Error	Deviance	df	Pr(>Chi)
Intercept	73.8	15.9
MM	−3.8	4.0	1152.04	30	0.01542*
BRM	−75.3	41.0	0.19	29	0.64524
BRM:MM	24.4	11.1	947.26	28	0.02805 *

4. Discussion

In this piece of work, we have shown that cognitive performance can be incorporated into a model to successfully predict dispensing accuracy and clinical decision-making performance in student pharmacists. Our final models were able to explain approximately 30% and 63% of the deviance in dispensing and CDM OSCE performance, respectively.

4.1. Visuo-spatial working memory

The MM used in this experiment is an adapted version of the Brooks Matrix Task used by Baddeley et al.¹² and assesses visuo-spatial working memory through the visuo-spatial sketchpad. The relationship observed between this cognitive domain and dispensing and clinical decision-making performance may be explained by the participant's requirement to identify, retain, interpret and recall visually displayed patient data to identify a clinical problem. The total quantity of information on a case history form or drug chart, the location of key parameters, and whether or not they are highlighted in some way may affect the ability of pharmacy students to identify a pharmaceutical problem. Therefore, an important avenue to explore is whether medical clerking proformas and drug charts could be designed in such a way as to reduce the cognitive load and requirements of working memory to reduce the risk of clinical decision-making errors. There is little consistency between hospitals and other care providers in the UK in terms of the layout of clinical notes, biochemical and therapeutic drug monitoring data, paper and electronic drug charts. The EQUIP study reported drug chart design as a contributory factor to medication harm.¹⁶ Therefore, any attempt to provide a universal drug chart on medical notes should consider these findings. It may also be advisable to use a verbal rather than a visual mnemonic should be employed to prevent overloading the visuo-spatial working memory.¹⁷

One limitation of our study is that participants underwent only a single psychometric assessment. As working memory function can vary temporally in an individual and across the population, we may find that clinical decision-making ability also varies in such a way. Various factors, including stress, are known to precipitate changes to WM function in a particular individual.¹⁸ It is interesting to hypothesize that changes to WM capacity/function under certain conditions may explain error-prone behavior (making poor clinical decision / dispensing error). Consequently, tools that enhance or maintain cognitive function under these conditions may be of value, especially in a practice-based clinical setting. Additionally, psychometric analysis at the point at which students qualify and first enter the workplace may highlight individuals who need more intense training during induction. Further studies to investigate this would be advisable.

4.2. Implicit memory

Implicit memory, which is unconscious, skills-based memory, is likely to have been utilized during the process of learning the skills of analyzing patient data and identifying associated pharmaceutical problems that took place before the clinical decision-making experiment. When these skills were learned, the necessary steps in the process would have transitioned into the implicit memory store. This transition may be more effective in participants that scored highly in the DCT. The findings suggest that greater practice in exercises that guide students through the clinical decision-making process, strengthening synaptic connections, could increase student performance in these tasks. Therefore, we would advocate optimizing opportunities for practical based elements of clinical-decision making, using relevant clinical paraphernalia, and the narrative of a clinical expert.

Implicit memory, along with explicit (semantic, episodic), is also known to be utilized when humans search for items in everyday activities and experimental scenarios.¹⁹ For example, the shape, size, and dimensions of an object or stimuli need to be unconsciously recalled to provide an update on whether that object has been found. In the scenarios used in our current work, students were required to ‘search’ for pharmaceutical problems amongst detailed clinical notes, prescriptions, and pharmaceutical products. Reduced capacity of this store may therefore also explain our findings.

4.3. Limitations

Although the final models produced in this study are significant, they fail to explain approximately 27% of the deviance in clinical decision-making scores and a greater proportion of the variance in dispensing accuracy. This deviance may be due to the differences in academic performance between candidates, which will need to be tested in future iterations of this model. The study was also limited in its size and the psychometric tests' sensitivity. Using a battery of tests that could probe the subtleties of each cognitive domain, along with the use of fMRI and eye-tracking glasses, could provide further insight could be gained into the role of executive function and implicit memory in clinical decision-making and dispensing performance.

Finally, although OSCEs are an established method of assessing clinical skills, we recognize that there is some debate about their effectiveness in assessing some of the elements of clinical-decision making – notably clinical reasoning. However, rather than solely examining the suitability of the students' final choice of intervention, the clinical decision-making stations used here also explored students' understanding of the mechanisms underpinning the scenario and the reasoning behind their judgments. This approach is recommended to ensure that clinical reasoning is assessed at the appropriate level of Miller's pyramid.²⁰ Therefore, we are confident that our systems provide an effective measure of clinical decision-making.

5. Conclusions

These data establish the predictive utility of cognitive tests of WM and implicit memory on certain clinical skills. In particular, VSWM capacity appears to be a key component of student pharmacists' effective clinical decision-making and dispensing performance.

Declaration of Competing Interest

The authors declare no actual, or perceived conflicts of interest.

Appendix A. Supplementary data

Supplementary material