Assessing the ability of the Drug-Associated Risk Tool (DART) questionnaire to stratify hospitalised older patients according to their risk of drug-related problems: a cross-sectional validation study

Abstract

Objectives

The Drug-Associated Risk Tool (DART) has been developed as a self-administered questionnaire for patients with the aim of stratifying patients according to their risk of drug-related problems (DRPs). We aimed to validate the ability of the questionnaire to distinguish between hospitalised patients showing lower and higher numbers of DRPs.

Design

Cross-sectional study assessing the questionnaire’s concurrent criterion validity.

Setting

Five geriatric and the associated physical and neurological rehabilitation wards of a Swiss regional secondary care hospital with 617 beds.

Participants

We recruited 110 patients from a total of 437 admissions. Exclusion criteria were insufficient knowledge in spoken or written German, medical conditions preventing meaningful conversations and already receiving pharmacy services.

Interventions

Comprehensive pharmacist-led clinical medication reviews were performed, including patient interviews, to identify potential and manifest DRPs. A cluster analysis was conducted to assess the discriminatory potential of the DART to group patients according to number (low and high) of identified DRPs. A subsequent discriminatory function analysis was performed to reduce the number of items. We determined which DART items may be used to trigger what type of medication review.

Results

Recruited patients had a median age of 79 years and were prescribed a median of 11 drugs. Patients with a median DART score of 10 and a median of 3 DRPs represented one cluster, whereas patients with a median DART score of 15 and a median of 8 DRPs represented another cluster. Discriminatory function analysis reduced the questionnaire to five items with a moderate to strong correlation with the number of DRPs per patient (Spearman’s rank correlation ρ=0.44). Additional items were associated with patients benefiting from interviews.

Conclusions

As a self-administered questionnaire for patients, the DART may be used to stratify hospitalised non-acute older patients in groups of having low and high likelihood of DRPs. The analyses showed that a short form of the DART can be used instead of the full tool to identify older inpatients at risk for DRPs. Additional eight items from the DART may be used to initiate additional clinical pharmacy services. The linkage between certain DART questions and type of medication review enables pharmacist resource allocation.

Strengths and limitations of this study.

• The performed comprehensive clinical medication reviews were performed by one pharmacist to ensure consistency and repeated by a second pharmacist to ensure their validity.

• Item reduction was possible following a cluster analysis and a subsequent discriminant function analysis making the Drug-Associated Risk Tool less time consuming.

• However, the questionnaire is currently only validated in older non-acute patients with the ability to engage in conversation hospitalised on geriatric and associated rehabilitation wards.

• Patients with cognitive impairments (eg, dementia) had to be excluded, which further restricts the generalisability of the results.

Background

When pharmacists take responsibility for the optimisation of medicines use, they are practising pharmaceutical care.¹ This care includes the prevention, identification and resolution of drug-related problems (DRPs).² DRPs are defined as ‘events or circumstances involving drug therapy that actually or potentially interfere with desired health outcomes’ and are distinguished by preventability, presence and cause.³ The DRPs that pharmacists are able to avert are preventable potential or manifest DRPs, which are erroneously caused by deviations from accepted guidelines or by patients’ behaviour. Within hospitals, clinical pharmacists practise pharmaceutical care and advise on appropriate, safe and economic use of medicines.⁴ Hospitals are a point of care where new medicines are introduced to treat acute illnesses in addition to existing treatment for chronic conditions—a process growing more complex with each added medicine.⁵ Besides focusing on newly introduced medicines, hospitalisation can be an opportunity for clinical pharmacists to perform medication reviews on the patient’s whole drug therapy while having access to the vast amount of information provided by medical records, laboratory measurements and patients’ opinions and experiences.^{5 6} Drug therapy evaluations including this information are classified as Medication Reviews Type 3 (MRT3).⁷ MRT3s take into account the circumstances in which patient interviews are necessary for the identification of the majority of DRPs. Whereas Medication Reviews Type 2a (MRT2a) rely on information from medication histories and patient interviews, MRT3s also take clinical data and laboratory measurements into consideration.⁶ They are conducted using a combination of methods for DRP identification, are structured and multifaceted and therefore yield optimal results.^{5 6 8} The combination of methods needs to include explicit as well as implicit criteria of inappropriate prescribing to balance the benefits and drawbacks of each method.⁹ Explicit criteria are rigid statements that enable quick evaluation of the therapy appropriateness with little clinical judgement but do not allow for individual patient tailoring. Implicit criteria are individual assessments that enable a patient-specific evaluation of complex drug therapy regimens but require clinical knowledge and time.⁹ Such comprehensive drug therapy evaluations can reduce the number of days spent in hospital for selected patients¹⁰ but require human resources currently not available in Swiss hospitals: the 35.9 full-time equivalent clinical pharmacist positions stand in strong contrast to 1.4 million hospitalisations per year.¹¹

Prioritising patients in order to structure the daily workload is an accepted requirement to maintain effective clinical pharmacy services with finite resources.^{12 13} The systematic review of Falconer and colleagues comprehensively evaluated published algorithms to prioritise patients according to their risk for DRPs.¹³ Of the 11 models in their final assessment, only the Brighton Adverse Drug Reactions Risk model by Tangiisuran and colleagues¹⁴ remained as tool with fair discrimination, sufficient validation and reasonable performance. The score requires laboratory measurements further necessitating effort by a healthcare professional (ie, high white cell count on admission). Contrary to a model based on clinical data, a well-developed paper-based questionnaire may be distributed to the patients while they are waiting to be seen by their physician or pharmacist.¹⁵ Patient-filled questionnaires also help in identifying additional patient-centred DRPs, for example, low adherence and poor health literacy.

The Drug-Associated Risk Tool (DART) is a 35-item questionnaire about risk factors deemed to be associated with DRPs.¹⁶ The DART was developed to assist clinical pharmacists in stratifying their patients for medication reviews and to tailor clinical pharmacy services according to available resources. The items of the questionnaire were identified by triangulation, including quantitative and qualitative methods, described elsewhere.¹⁷ The tool is designed to be a self-administered questionnaire for patients in order not to increase the workload of either clinical pharmacists or other healthcare professionals. The applicability of the DART used as a self-administered questionnaire has been shown for hospitalised patients; compared with documented medical records, patients from geriatric, medical and orthopaedic wards with a median age of 81, 65 and 67.5 years, respectively, were able to adequately reproduce their medical information, limitations being renal and hepatic insufficiency.¹⁶

In order to evaluate the DART as a risk stratification tool, the goal of the present study was to validate its ability to distinguish between hospitalised patients showing lower and higher numbers of currently present potential and manifest DRPs.

Methods

Setting and study design

Prospective patient enrolment and data collection were conducted in a Swiss regional secondary care hospital with 617 beds from February to November 2016. The hospital administration gave permission to recruit patients for 10 months; we aimed to recruit at least 100 study participants, as discussed for self-administered questionnaires by Barenholtz Levy.¹⁵ We chose to prove the concurrent criterion validity for the DART,¹⁸ as there is currently no gold standard for the risk assessment of DRPs. The concurrent criterion validity assessment correlates a new tool with another measure of the trait under study, both administered at the same time.

Recruitment

We recruited patients from five geriatric and the associated physical and neurological rehabilitation wards with approximately 60 beds and a reported mean hospital stay of 17 days.¹⁹ In this hospital, patients get transmitted from other wards to the rehabilitation wards after acute care and generally mark less acutely ill patients. Patients were included in the study when admitted to one of the participating wards and were approached within 72 hours of admission. Exclusion criteria were insufficient knowledge in spoken or written German, medical conditions preventing meaningful conversations (eg, deliria, acute psychosis, dementia, aphasia and cognitive impairment), patients treated within palliative care and patients who were already subject to other clinical pharmacy services (ie, ward rounds and phone consultation). Ethical considerations required the approval of the ward physician or responsible caregiver before patient contact.

Data collection

After giving informed consent, the patients received the DART questionnaire to complete in self-admission. The DART consists of 29 questions with dichotomous answers and 6 questions with Likert scale answers. The questionnaire is divided in sections on health, polypharmacy, self-medication, specific drugs, adherence, concerns over the medication, medication literacy and medication application. The study pharmacist collected clinical data on medical conditions, drug treatment (inpatient medication list) and laboratory values (ie, renal and hepatic function, nutritional state, health and disease markers and drug-monitoring values) from the electronic patient charts. The completed questionnaires were collected and stored separately without evaluation. The collected clinical data were entered on a case report form.

Tools and measures

As criterion measure we chose MRT3s⁷ to identify DRPs, using the implicit criteria on potentially inappropriate medications (PIMs) ‘Medication Appropriateness Index’ (MAI)²⁰ and current Swiss treatment guidelines,²¹ and the explicit criteria on PIMs ‘STOPP/START criteria version 2’ (STOPP, Screening Tool of Older People’s Prescriptions; START, Screening Tool to Alert to Right Treatment)²² as part of the review. Feasibility, content validity, predictive validity and reliability have all been demonstrated for the MAI.²³ Improvements in drug therapy appropriateness have been shown to decrease the total MAI score.⁸ For each medicine, 10 criteria are judged to be ‘appropriate’, ‘marginally appropriate’ or ‘not appropriate’. A weighted score is applied for evaluations deemed to be ‘partially appropriate’ or ‘marginally appropriate’.²⁴ The medicine’s total score indicates its appropriateness, whereas each contributing criterion indicates an opportunity for optimisation. The STOPP/START criteria are shown to improve medication appropriateness and to reduce adverse drug reactions, whereas the STOPP statements are associated with adverse drug events.²² The drug regimens were screened for drug–drug interactions by the commercial online database mediQ.²⁵ A structured interview for detection of patient-centred DRPs was newly developed as part of the MRT3s; the interview was based on the Polymedication Check (PMC),²⁶ a reimbursed cognitive service provided by Swiss community pharmacists that focuses on adherence problems, patients’ knowledge and handling problems. We supplemented the PMC with items from the Beliefs about Medicines Questionnaire in order to identify drug-related concerns.²⁷ The interviews took place within 24 hours of study inclusion. Thus, the assessments took place within 4 days of ward admission. All answers to the interview questions were dichotomous. The study pharmacist performed the patient interviews with each patient using an iPad Air, V.2 (Apple, Cupertino, California, USA), where interview guide and data entry were combined within the online questionnaire form Flexiform, V.2.7.1 g (IT-services University of Basel).

Data classification

We coded the medication using the anatomical therapeutic chemical (ATC) classification.²⁸ DRPs were documented with the GSASA (Swiss Association of Public Health Administration and Hospital Pharmacists) classification system for DRPs, for which inter-rater reliability has been shown.²⁹ We assessed the potential clinical relevance of the DRPs by using a German version of CLEO, CLEO_de, which was tested for inter-rater and test–retest reliability.³⁰ CLEO assists pharmacists in assessing the potential relevance of their own interventions and the underlying DRPs identified in the three distinct dimensions: clinical/patient, economic/hospital and organisational/staff. For our research, we focused on the clinical dimension, with its six levels: ‘harmful’, ‘null’, ‘minor’, ‘moderate’, ‘major’ and ‘lifesaving,’ which achieved good inter-rater (intraclass correlation ICC=0.63) and excellent test–retest reliability (mean ICC=0.76). Levels equal to or higher than moderate describe interventions that prohibit potential damage requiring additional treatment or that recommend further surveillance.

Based on all collected data, the study pharmacist performed MRT3s and then a second clinical pharmacist repeated the medication reviews independently. Identified potential and manifest DRPs and their estimated relevance were compared; divergence was resolved by discussion until consensus was reached.

Patient and public involvement

Patients were not involved in the development of the research question or the execution of the study. The interview was pilot tested with two inpatients to sort out any issues regarding its understandability. Study participants were encouraged to contact the study pharmacist in case they wanted to be informed on the results. Contact information was available on the study information.

Statistical analysis

The total DART risk score was calculated by assigning points to each answer. Dichotomous answers were assigned a risk score with one (1) point being assigned to each ‘Yes’ answer and zero (0) points given to each ‘No’ answer. Ordinal answers were assigned a corresponding dummy variable. As studies reported means of 8–10 identified DRPs per patient,^{31 32} the dependent variable (ie, numbers of DRPs) was expected to lack a predefined cut-off value, that is, low-risk patients having zero DRPs. Instead of defining an artificial cut-off for a tolerable number of DRPs (eg, low-risk patients having four DRPs), we used a cluster analysis. Cluster analyses group observations into collectives with respect to all defined variables (ie, DART score and number of DRPs) without necessitating previous categorisation: the analysis was expected to form patient clusters with high DART scores and high numbers of DRPs and low DART scores and low numbers of DRPs in the absence of an artificial definition of high and low. We performed a Ward’s hierarchical cluster analysis with squared Euclidian distance using the variables DART score and number of DRPs. This analysis necessitated the elimination of statistical outliers in order to be performed. For this reason, Backhaus and colleagues recommend a preceding single linkage hierarchical cluster analysis with squared Euclidian distance. Statistical outliers can be objectively identified from the resulting dendrogram.³³ We assessed the appropriateness of the clusters generated by the Ward’s hierarchical cluster analysis by homogeneity and calculated the F-values for each cluster and variable (ie, DART total risk-score and number of DRPs). F-values of below 1 represent a homogeneity that is lower within the proposed cluster than within all observations.³³ We performed effect size calculations using Pearson’s correlation coefficient r, which were interpreted according to Gignac: r=0.1 as small,. 0.2 as medium and 0.3 as large.³⁴ We compared the obtained clusters concerning their total number of identified DRPs and their total DART risk score by a Mann-Whitney U test.³⁵ Furthermore, we performed a stepwise discriminant function analysis in order to investigate the discriminatory potential of subsets of items of the questionnaire for the generated clusters and hence to identify possibilities to reduce the number of items in the questionnaire. We calculated Wilks’ lambda (λ) for the whole DART questionnaire and the subsets of items (‘reduced items’) to report on discriminatory values. Lower Wilk’s λ values indicate a higher differential potential.³³ For scale correlations, we used Spearman’s rank correlation coefficient ρ, which we interpreted as follows: ρ=0.1 as weak, 0.3 as moderate and 0.5 as strong.³⁶ An area under the receiver operating characteristic curve (AUROC) analysis for the reduced items was performed.

As described above, cluster analysis and subsequent discriminant factor analysis were again used to additionally determine discriminatory DART items concerning DRPs identified within the patient interviews only.

Additionally, we calculated Spearman’s rank correlation coefficient ρ to assess the correlation between the score of the reduced items and the potential relevance of the detected DRPs. For the single items, we used a Mann-Whitney U test. These additional tests were used to assign the necessary type of medication review (ie, MRT3 or MRT2a) to certain items of the DART.

We defined statistical significance as p values <0.05. The statistical analysis was performed using IBM SPSS Statistics for Windows, V.24.0. Interview data preparation was performed using RStudio, V.1.0.136 (RStudio, Boston, Massachusetts, USA) and running R, V.3.3.2 (R Foundation for Statistical Computing, Vienna, Austria).

Results

Dataset

We recruited 110 patients from a total of 437 admissions to the five wards during the enrolment period from February to November 2016. Figure 1 summarises the recruitment flow. Study population characteristics are presented in table 1.

Figure 1.

Recruitment flow.

Table 1.

Dataset characteristics

Characteristic	Value
Demographic (n=110)
Age, median (IQR) (years)	79.0 (15.0)
Female, n (%)	76 (69.1)
Clinical status
Glomerular filtration rate* (CKD-EPI), mean±SD (mL/min)	70.5±20.5
Patients with moderate renal insufficiency up to renal failure, n (%)	29 (27.7)
Diagnosed heart failure, n (%)	4 (3.6)
Diagnosed liver insufficiency, n (%)	0 (0.0)
Diagnosed COPD, n (%)	10 (9.1)
Diagnosed diabetes, n (%)	24 (21.8)
Medication
Prescribed drugs†, median (IQR)	11.0 (6.0)

*Missing values n=5.

†As reported on the inpatient medication list, including already used as needed medications.

CKD-EPI, chronic kidney disease epidemiology collaboration; COPD, chronic obstructive pulmonary disease.

We identified a total of 595 DRPs, averaging 5.4 DRPs per patient (range: 0–17). One hundred and eight patients had at least one DRP (98.2%). Thirty-four DRPs were deemed to be manifest, that is, patients showing signs of adverse drug events. Identified DRPs are presented in table 2. The most frequent causes of DRPs were ‘Insufficient knowledge of the patient’ (n=138, 23.2%; for example, not knowing the indication of the drug) and ‘Incomplete patient documentation’ (n=118, 19.8%; for example, missing diagnoses or treatments), summating in the most frequent problems ‘Safety of treatment’ (n=182, 30.6%) and ‘Patient dissatisfaction’ (n=154, 25.9%) as classified by the GSASA documentation tool. Analgesics were the drug class linked to most DRPs (71 DRPs, 13.7%), with acetaminophen causing most of the potential issues within this class (26 DRPs). On a drug level, pantoprazole was accountable for most DRPs (35 DRPs, 6.3%), followed by acetaminophen (26 DRPs, 4.7%) and then calcium and cholecalciferol (19 DRPs, 3.4%). With the help of the CLEO_de tool, we estimated the potential clinical relevance of the DRPs to be ‘null’ (n=47, 7.9%), ‘minor’ (n=399, 67.1%), ‘moderate’ (n=106, 17.8%), ‘major’ (n=40, 6.7%) and ‘lifesaving’ (n=3, 0.5%).

Table 2.

Identified drug-related problems (DRPs) as classified by the GSASA classification system²⁹

Description	Total=595, n (%)
Detected problem
Safety of treatment	182 (30.6)
Patient dissatisfaction	154 (25.9)
Treatment effectiveness	107 (18.0)
Untreated indication	52 (8.7)
Treatment costs	7 (1.2)
Classification not possible	93 (15.6)
Cause of problem
Insufficient knowledge of the patient	138 (23.2)
Incomplete patient documentation	118 (19.8)
No concordance with guidelines or contraindication	60 (10.1)
Drug not indicated or duplication	46 (7.7)
Treatment not received	46 (7.7)
Interaction	36 (6.1)
Overdose	29 (4.9)
Underdose	25 (4.2)
Inappropriate therapy duration	20 (3.4)
Insufficient adherence	19 (3.2)
Inappropriate timing or frequency of administration	14 (2.4)
Adverse effect	12 (2.0)
Dose not adjusted to organ function	9 (1.5)
Inappropriate dosage form	7 (1.2)
Error in medication process	3 (0.5)
Insufficient knowledge of healthcare professionals	2 (0.3)
Prescribed drug not available	1 (0.2)
Classification not possible	10 (1.7)
Manifest	34 (5.7)
Potential	561 (94.3)

GSASA, Swiss Association of Public Health Administration and Hospital Pharmacists.

Validation of the DART

We analysed the datasets for correlation of risk factors identified by the DART questionnaire with the number of DRPs. Seven datasets were excluded from this analysis: two patient cases had incomplete data and another five cases were identified as statistical outliers by the single-linkage cluster analysis. Two of these five patients had a DART score of 16 and 15 and a number of DRPs of 13 and 17, respectively; two patients had a low DART score (0 and 3) and a high number of DRPs (9 and 8); one patient had a high DART score (21) and a low number of DRPs (2). These preliminary steps reduced the analysed dataset to a total of 103 patients.

The DART total risk score showed a weak to moderate correlation with the number of DRPs identified (Spearman’s rank correlation ρ=0.27, p<0.01). Using the cluster analysis, we were able to identify two clusters with 61 and 42 observations, respectively. The two clusters may be regarded as completely homogeneous, as all F-values are below 1 (F _{Cluster1, DART risk-score}=0.51; F _{Cluster1, identified DRPs}=0.21; F _{Cluster2, DART risk-score}=0.97; and F _{Cluster2, identified DRPs}=0.72). The effect sizes were large for both variables (DART risk-score Pearson’s: r=0.54; identified DRPs: r=0.79). Comparing the two clusters, a Mann-Whitney U test revealed a statistically significant tendency for the DART total risk-score (U=476.5, p<0.001) and number of identified DRPs (U=100.0, p<0.001). Cluster 1 represented patients with a median of 10 risk factors (range: 3–16) and three identified DRPs (range: 1–6) and cluster 2 represented patients with a median of 15 risk factors (range: 9–23) and 8 DRPs (range: 4–15). The clusters also presented a difference in summated MAI scores per patients: clusters 1 and 2 contained patients averaging at an MAI score of 5.6 and 13.2, respectively. In the discriminant function analysis, the DART total risk score achieved a Wilks’ λ of 0.69 (p<0.001). Stepwise discriminant function analysis identified the DART items on diabetes, polypharmacy (>5 medicines), missing doses, concerns on dependency and heart failure as important discriminators between the two clusters. These items achieved a combined Wilks’ λ of 0.57 (p<0.001), indicating a higher differential potential than the total DART risk score itself. The score of the five items alone showed a moderate to strong correlation with the number of DRPs identified (Spearman’s rank correlation ρ=0.44, p<0.01) and a strong correlation with the original total risk score (ρ=0.714, p<0.01). The AUROC was 0.865 (SE=0.035, p<0.001, 95% CI 0.797 to 0.932), further displaying the discriminatory potential of the summated five items. The r eceiver o perating c haracteristic (ROC) curve is shown in figure 2. The coordinates of the ROC curve presented in table 3 show cut-offs at either one (1) or two (2) ‘Yes’ answers with decreasing sensitivity but increasing specificity.

Figure 2.

Receiver operating characteristic (ROC) curve of the five DART items on diabetes, polypharmacy (>5 medicines), missing doses, concerns on dependency and heart failure. DART, Drug-Associated Risk Tool.

Table 3.

Coordinates of the receiver operating characteristic (ROC) curve

Positive if greater than or equal to	Sensitivity	Specificity
0.5	1.000	0.443
1.5	0.738	0.820
2.5	0.429	0.967
3.5	0.071	1.000
5	0.000	1.000

Concerning the DRPs identified within the patient interviews only, Spearman’s rank correlation showed a moderate correlation between the score of the reduced five items and the number of DRPs identified (ρ=0.45, p<0.01). Cluster analysis and subsequent discriminant factor analysis on the whole questionnaire classified the DART items on the use of drugs (non-steroidal antirheumatics, antidiabetics and digoxin), restricted kidney function, concerns about dependency, concerns at having to use medicines, use of therapeutic skin patches, preparation of medicines by home care and polypharmacy as predictors of DRPs identified during patient interviews.

The score of the reduced items showed a statistically significant correlation with DRPs estimated to be of moderate (ρ=0.40, p<0.001) and minor (ρ=0.23, p=0.02) clinical relevance. In the evaluation tool CLEO, patient-centred DRPs are estimated to be of minor relevance; these include restricted knowledge of medicines, restricted satisfaction, compliance or quality of life and damage that does not require monitoring or treatment. Mann-Whitney U tests showed a statistically significant correlation between DRPs with high clinical relevance (ie, CLEO ‘major’, ‘lifesaving’) and the DART items on issues such as tablet-splitting (U=419.5, p=0.020), heart failure (U=590.0, p=0.018) and use of oral anticoagulants (U=696.5, p=0.004). Use of steroids (U=182.5, p=0.010) was associated with DRPs with moderate clinical relevance. Table 4 presents the synopsis of the statistical results, combining single DART items with two types of medication review.

Table 4.

Combination of DART items and possible triggered type of medication review

DART item (translated from German)	Outcome in statistical analysis	Possible triggered clinical pharmacy service
I have a heart weakness/heart performance weakness.	Correlated with DRPs with high clinical relevance.	Consider immediate MRT3 inclusive of a patient interview.
I have trouble taking my medicine because of splitting tablets.	Correlated with DRPs with high clinical relevance.	Consider immediate MRT3 inclusive of a patient interview.
I use Marcoumar (phenprocoumon), Xarelto (rivaroxaban), Sintrom (acenocoumarol), Eliquis (apixaban), Lixiana (edoxaban) or Pradaxa (dabigatran) at home.	Correlated with DRPs with high clinical relevance.	Consider immediate MRT3 inclusive of a patient interview.
I use cortisone at home.	Correlated with DRPs with moderate clinical relevance.	Consider immediate MRT3 inclusive of a patient interview.
I have diabetes.	Discriminated for cluster of patients with a high number of DRPs.	Consider MRT3 inclusive of a patient interview.
I take more than five drugs every day, which are prescribed by my physician.	Discriminated for cluster of patients with a high number of DRPs.	Consider MRT3 inclusive of a patient interview.
Do you sometimes forget to take your medicine?	Discriminated for cluster of patients with a high number of DRPs.	Consider MRT3 inclusive of a patient interview.
I sometimes worry about becoming too dependent on my medicines.	Discriminated for cluster of patients with a high number of DRPs.	Consider MRT3 inclusive of a patient interview.
I use medicines against rheumatism/inflammation at home.	Discriminated for cluster of patients with DRPs only identified in patient interview.	Consider patient interview (MRT2a).
I use insulin/medicines against diabetes at home.	Discriminated for cluster of patients with DRPs only identified in patient interview.	Consider patient interview (MRT2a).
I use digoxin at home.	Discriminated for cluster of patients with DRPs only identified in patient interview.	Consider patient interview (MRT2a).
I sometimes worry about the long-term effects of my medicines.	Discriminated for cluster of patients with DRPs only identified in patient interview.	Consider patient interview (MRT2a).
Having to take this medicine worries me.	Discriminated for cluster of patients with DRPs only identified in patient interview.	Consider patient interview (MRT2a).
I apply my medication in the form of skin patches.	Discriminated for cluster of patients with DRPs only identified in patient interview.	Consider patient interview (MRT2a).
I have a restricted kidney function/kidney dysfunction/kidney disease.	Discriminated for cluster of patients with DRPs only identified in patient interview.	Consider patient interview (MRT2a).
The preparation of my medicine is done by a homecare institution.	Discriminated for cluster of patients with DRPs only identified in patient interview.	Consider patient interview (MRT2a).

MRT3, Medication Review Type 3; drug therapy evaluations using medical records, laboratory measurements, and the patient’s opinions and experiences. MRT2a, Medication Review Type 2a; drug therapy evaluations using medication history and the patient’s opinions and experiences.⁷

DART, Drug-Associated Risk Tool; DRPs, drug-related problems.

Discussion

The goal of the present study was to validate the ability of the DART questionnaire to distinguish between hospitalised patients showing lower and higher numbers of currently present potential and manifest DRPs. With a weak to moderate correlation (ρ=0.27, p<0.01), the total risk score of the DART allowed for the discrimination of two patient groups as a result of a cluster analysis: patients with a median DART score of 10 presented a median of 3 identified DRPs and an average MAI score of 5.6, whereas patients with a median DART score of 15 presented a median of 8 DRPs and an average MAI score of 13.2, supporting the validity of the generated clusters. We identified the five items, diabetes, polypharmacy (>5 medicines), missed doses, concerns about dependency and heart failure, as important discriminators between these two patient collectives. The weak to moderate correlation between the DART total score and identified DRPs increased when only these five items of the DART were taken into account, resulting in a moderate to strong correlation (ρ=0.44, p<0.01). Reducing the DART items represents a less time-consuming and more valid measure of identifying DRPs compared with the total DART scale. Because we indicated the source of identification in our dataset, we were able to show that the risk score of the reduced items correlated moderately with the number of DRPs identified only in direct patient interviews (ρ=0.45). With a second cluster analysis and subsequent discriminatory function analysis, we identified additional DART items that were of discriminatory value for patients who specifically benefited from the interviews. Furthermore, stated tablet-splitting issues, heart failure and use of steroids or oral anticoagulants showed a statistically significant correlation to DRPs with moderate to high clinical relevance. Table 4 combines these findings into a repository of patient questions that may be used by practitioners with limited resources to tailor two types of medication reviews.

The items identified as being of discriminatory value for the DART questionnaire and the overall number of DRPs are well in line with previous research on DRPs. The symptoms of heart failure may worsen when drugs are not taken as prescribed,³⁷ patients with diabetes mellitus who are non-adherent to their medication regimen are especially prone to hospitalisation,³⁸ polypharmacy and non-adherence in general are established risk factors for medication-related hospitalisations,³⁹ steroids^{40 41} and anticoagulants^41–44 are drugs frequently associated with re-hospitalisation or adverse drug events; tablet-splitting is a known safety issue, as problems with handling and adherence are associated with it.⁴⁵ As many as 52.5% of Swedish patients with prescriptions for split-tablets stated a preference and wished for whole tablets instead.⁴⁶

The risk score of the reduced items was associated with DRPs estimated to be of minor and moderate clinical relevance by the CLEO evaluation tool. CLEO associates the level ‘minor’ with problems being mainly patient centred and without the potential to produce harm that needs further monitoring or treatment.³⁰ These patient-centred problems include restricted knowledge about the medicines and restricted satisfaction, compliance and quality of life, for example, swallowing difficulties. The assignment of patient-centred DRPs like these to a minor clinical relevance is bound to the use of CLEO in estimating the potential clinical relevance of our identified DRPs and might be argued otherwise: health illiteracy and swallowing difficulties are risk factors for DRPs and non-adherence^{47 48} and may cause patient harm. We identified four items of the DART that had a statistically significant correlation with problems deemed to cause patient harm: tablet-splitting issues, heart failure and the use of steroids or use of oral anticoagulants.

As part of ongoing processes to shift medical documentation to electronic datasets, risk stratification tools are currently being developed as automated algorithms.^{22 49} Tools that take advantage of computer-based algorithms allow for the surveillance of the whole hospital, showing a clear advantage over paper-based questionnaires such as the DART. In our analysis, we also identified 11 items of the DART that could seamlessly be integrated into an automated algorithm (see table 4); however, we also present five items that necessitate direct patient contact. The WHO expects ‘seven-star pharmacists’ (caregiver, communicator, decision maker, teacher, life-long learner, leader and manager) to focus on patient-centred care (ie, respect the patients’ opinions and concerns).⁵⁰ Patients’ opinions and sorrows cannot be assessed with automated algorithms processing electronic documentation. With the DART, we present a questionnaire that asks the patients about their medicine use and is intended to be completed by the patient to trigger clinical pharmacy services promoting tailored patient care.

Strengths and limitations

A strength of the study presented here is the validation procedure: (1) the completed questionnaires were not evaluated until after the MRT3; (2) MRT3 were performed by one pharmacist to ensure consistency and repeated by a second pharmacist to ensure their validity; and (3) the cluster analysis with subsequent discriminant factor analysis showed its additional value over a simple scale correlation by highlighting items for item reduction. Our results contribute to the growing evidence on risk factors associated with DRPs. The items we identified as being valuable within our questionnaire are risk factors that have been judged to be potentially harmful elsewhere.^37–45 Past research has been able to show that these risk factors negatively influenced rehospitalisation and occurrence of adverse drug reaction rates,^37–45 and we showed that they should be used to trigger clinical pharmacy services that include patient interviews.

The limitations of this work constitute the generalisability of the results. The 110 medication reviews of the dataset were performed with older patients hospitalised on geriatric and the associated physical and neurological rehabilitation wards, having a median age of 79 years and a median of 11 prescribed drugs. This represents an elderly, highly polymedicated population. Additionally, we included patients from rehabilitation wards, who generally mark less acutely ill inpatients. Furthermore, the exclusion criterion ‘medical conditions preventing meaningful conversations’ was a necessity for patient interviews but excluded patients vulnerable to DRPs, especially cognitively impaired patients that have an independent risk for medication-related hospitalisation.³⁹ Since the DART was developed as a self-administered questionnaire, cognitive impairment is inherently a limitation of this tool. Combining these limitations, the DART currently proved to be suitable for older non-acute patients with the ability to engage in conversation. Furthermore, the use of a Ward’s hierarchical cluster analysis necessitated the exclusion of five statistical outliers as identified by a preceding single linkage hierarchical cluster analysis, which impeded a desirable intention-to-treat analysis. However, the exclusion of two patients who later would have fallen within the cluster of high DART risk score and high number of DRPs demonstrates the objectivity of this outlier identification. The additional three excluded outliers having contradictory DART risk score and numbers of DRPs show that the DART does not perform well for all patients but may be regarded in the lights of restricted specificity and sensitivity of the tool, which are present in any risk assessment. An additional limitation is that we did not correlate the DART with clinical outcomes (ie, rehospitalisation rates). This is because the DART is a screening tool pointing at patients at risk. Identifying risks as such cannot improve outcomes; it has to be followed by appropriate interventions. The DART, however, may help to direct interventions to patients in need of optimising pharmacotherapy and by this improve clinical outcomes.

Implication for practice

The implication of our research for practice is the addition of a self-administered questionnaire to the list of available tools that may be used for risk stratification. Distributed at the beginning of a hospitalisation, the DART may be completed by the patients themselves without increasing the workload for healthcare providers. The results can be used to tailor clinical pharmacy services and to allocate available resources to older non-acute patients who most need them. We suggest as triggers for MRT3 within a hospitalised older population the eight items of the DART on heart insufficiency, tablet-splitting issues, use of anticoagulants or steroids, diabetes, polypharmacy (>5 medicines), adherence and concerns about dependency. If resources permit, the additional eight items on the use of non-steroidal antirheumatic drugs, antidiabetics, digoxin, and restricted kidney function, concerns on dependency, concerns at having to use medicines, use of therapeutic skin patches and preparation of medicines by home care services may be used as indicators for patients who benefit from a patient interview focusing on adherence problems, patients’ knowledge and handling problems (ie, MRT2a). As the items of the original DART were carefully selected by a triangulation process, the remaining items may still be used to shape the contents of an MRT3 or a patient interview.

Conclusion

We present the DART as a validated self-administered questionnaire that may be used to identify a high risk of DRPs in hospitalised older non-acute patients able to engage in a conversation. Subsets of the items may trigger different clinical pharmacy services for patients in need and allow for rational allocation of work resources.