Frequency and Acceptance of Clinical Decision Support System-Generated STOPP/START Signals for Hospitalised Older Patients with Polypharmacy and Multimorbidity

Abstract

Background

The Screening Tool of Older Persons’ Prescriptions (STOPP)/Screening Tool to Alert to Right Treatment (START) instrument is used to evaluate the appropriateness of medication in older people. STOPP/START criteria have been converted into software algorithms and implemented in a clinical decision support system (CDSS) to facilitate their use in clinical practice.

Objective

Our objective was to determine the frequency of CDSS-generated STOPP/START signals and their subsequent acceptance by a pharmacotherapy team in a hospital setting.

Design and Methods

Hospitalised older patients with polypharmacy and multimorbidity allocated to the intervention arm of the OPERAM (OPtimising thERapy to prevent Avoidable hospital admissions in the Multimorbid elderly) trial underwent a CDSS-assisted structured medication review in four European hospitals. We evaluated the frequency of CDSS-generated STOPP/START signals and the subsequent acceptance of these signals by a trained pharmacotherapy team consisting of a physician and pharmacist after evaluation of clinical applicability to the individual patient, prior to discussing pharmacotherapy optimisation recommendations with the patient and attending physicians. Multivariate linear regression analysis was used to investigate potential patient-related (e.g. age, number of co-morbidities and medications) and setting-related (e.g. ward type, country of inclusion) determinants for acceptance of STOPP and START signals.

Results

In 819/826 (99%) of the patients, at least one STOPP/START signal was generated using a set of 110 algorithms based on STOPP/START v2 criteria. Overall, 39% of the 5080 signals were accepted by the pharmacotherapy team. There was a high variability in the frequency and the subsequent acceptance of the individual STOPP/START criteria. The acceptance ranged from 2.5 to 75.8% for the top ten most frequently generated STOPP and START signals. The signal to stop a drug without a clinical indication was most frequently generated (28%), with more than half of the signals accepted (54%). No difference in mean acceptance of STOPP versus START signals was found. In multivariate analysis, most patient-related determinants did not predict acceptance, although the acceptance of START signals increased in patients with one or more hospital admissions (+ 7.9; 95% confidence interval [CI] 1.6–14.1) or one or more falls in the previous year (+ 7.1; 95% CI 0.7–13.4). A higher number of co-morbidities was associated with lower acceptance of STOPP (− 11.8%; 95% CI − 19.2 to − 4.5) and START (− 11.0%; 95% CI − 19.4 to − 2.6) signals for patients with more than nine and between seven and nine co-morbidities, respectively. For setting-related determinants, the acceptance differed significantly between the participating trial sites. Compared with Switzerland, the acceptance was higher in Ireland (STOPP: + 26.8%; 95% CI 16.8–36.7; START: + 31.1%; 95% CI 18.2–44.0) and in the Netherlands (STOPP: + 14.7%; 95% CI 7.8–21.7). Admission to a surgical ward was positively associated with acceptance of STOPP signals (+ 10.3%; 95% CI 3.8–16.8).

Conclusion

The involvement of an expert team in translating population-based CDSS signals to individual patients is essential, as more than half of the signals for potential overuse, underuse, and misuse were not deemed clinically appropriate in a hospital setting. Patient-related potential determinants were poor predictors of acceptance. Future research investigating factors that affect patients’ and physicians’ agreement with medication changes recommended by expert teams may provide further insight for implementation in clinical practice.

Registration

ClinicalTrials.gov Identifier: NCT02986425.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40266-021-00904-z.

Key Points

Clinical decision support system-assisted medication review using Screening Tool of Older Persons’ Prescriptions (STOPP)/Screening Tool to Alert to Right Treatment (START) criteria in hospitalised older patients with polypharmacy and multimorbidity resulted in a median of 6 (interquartile range [IQR] 4–8) generated signals per patient.

The acceptance of signals after clinical evaluation by a pharmacotherapy team was highly variable, ranging from 2.5 to 75.8% for the ten most frequently generated STOPP and START signals.

The country of the participating trial site was the strongest predictor of acceptance, and patient-related characteristics were poor predictors of acceptance.

Background

Polypharmacy poses an increasing challenge in healthcare and is largely driven by the steadily growing multimorbid elderly population and prescribers’ adherence to single-disease-oriented guidelines [1]. Polypharmacy is—as a negative by-product of the benefits of pharmacotherapy—associated with an increased risk of negative health outcomes, such as adverse drug events, falls, decline in cognitive function, hospitalisation, and even death, especially in frailer older people [2]. Therefore, the potential benefits should outweigh the potential risks of pharmacotherapy for each patient, and this balance should be evaluated both on treatment initiation and regularly during long-term follow-up through medication review.

Explicit screening tools, such as the Screening Tool of Older Persons’ Prescriptions (STOPP) and the Screening Tool to Alert to Right Treatment (START), have been developed to facilitate the detection of potentially inappropriate prescribing in the process of regular medication review in older people [3–6]. Research has shown that the use of STOPP/START criteria in patient care can lead to a reduction of polypharmacy, inappropriate prescribing, and adverse drug reactions [5, 6]. However, application of STOPP/START v2—which comprises 114 criteria—is time consuming, which hampers its use in everyday clinical practice [7]. Hence, STOPP/START criteria v2 were converted into software algorithms that can be implemented into a clinical decision support system (CDSS) to facilitate their application [8, 9].

A recent systematic review concluded that the use of CDSS-generated signals is likely to reduce potentially inappropriate prescriptions in older patients. However, studies reported rates of clinician adherence to these signals ranging from 33 to 55% [10]. Too many irrelevant signals can result in alert fatigue and inappropriate alert overrides, decreasing the effectiveness of CDSS in clinical practice [11, 12]. The STOPP/START criteria are population-based recommendations to detect medication overuse, misuse (STOPP), and underuse (START) and require clinicians’ careful consideration concerning their applicability to individual patients. Investigating the relevance of CDSS-assisted detection of potential medication overuse, underuse, and misuse by STOPP/START for individual patients in clinical practice is necessary to gain insight into the applicability of these population-based recommendations to individual patient care.

This study aimed to determine the frequency of CDSS-generated STOPP/START signals and subsequent acceptance by a pharmacotherapy team for use in individual hospitalised older patients with polypharmacy and multimorbidity. In addition, we also investigated measurable determinants that may be associated with acceptance.

Methods

Setting, Design, and Study Population

This study was embedded in the OPERAM (OPtimising thERapy to prevent Avoidable hospital admissions in the Multimorbid elderly) trial—a cluster-randomised controlled trial investigating the effect of a structured medication review on drug-related hospital admissions. As previously described in detail, inpatients were recruited from four hospitals in four countries (Switzerland, Belgium, Ireland, the Netherlands) and randomised to receive usual pharmaceutical care (control group) or a CDSS-assisted structured medication review (intervention group) [13]. Inclusion criteria were age ≥ 70 years, multimorbidity (defined as three or more chronic conditions), and polypharmacy (defined as the use of five or more regular medications for over 30 days prior to admission). The two exclusion criteria were (1) patients admitted to palliative care within 24 h after hospital admission and (2) patients undergoing a structured medication review other than the trial intervention or having received a medication review during the 2 months preceding the index hospitalisation to reduce the risk of contamination bias. Both medical (e.g. internal medicine, cardiology, pulmonology, neurology) and surgical (e.g. general surgery, vascular surgery, orthopaedics, neurosurgery) wards were eligible for inclusion. However, geriatric wards were excluded to comply with the exclusion criteria, because medication optimisation was considered standard of geriatric care in all participating trial sites. The OPERAM trial was approved by the participating hospitals’ medical ethics committees and registered under trial registration number NCT02986425.

In this study, OPERAM intervention patients for whom data from the in-hospital CDSS-assisted medication review were available were included for analysis.

The structured medication review was conducted by a team comprising a physician and a pharmacist (hereafter, pharmacotherapy team) who were trained by standardised operating procedures in all sites. The medication review was performed according to the Systematic Tool to Reduce Inappropriate Prescribing (STRIP) method [14] and consisted of five consecutive steps [15].

1. A structured history taking of medication use (SHiM) [16] and data entry of relevant and available patient information into the CDSS (i.e. current in-hospital medication list updated by information from the SHiM, medical conditions, laboratory values, signs, and patient-reported symptoms).

2. Digitalized screening of the current medication list for medication overuse and underuse by STOPP/START algorithms.

3. A pharmacotherapy analysis by the pharmacotherapy team, who evaluated CDSS-generated signals for clinical applicability to each patient based on the patient’s medical status. Accepted signals were translated into patient-specific medication optimisation recommendations and presented on a feedback report in a standardised format.

4. Discussion of the feedback report with both the attending physician and the patient.

5. Generation of a discharge report for the patient’s general practitioner, which included in-hospital medication changes and recommendations that were agreed upon by the attending physician and the patient but deferred to the general practitioner for implementation.

This research focused on the first three steps of the medication review process and ended at the stage of either acceptance or rejection of CDSS signals by the pharmacotherapy team that resulted in medication optimisation recommendations to be discussed with the attending physician and the patient, prior to the implementation of medication changes. All consecutive steps of the OPERAM intervention and the focus of this study (steps 1–3) are summarised in Fig. 1.

Fig. 1.

Summary of all consecutive steps (1–5) of the medication review within the OPERAM trial and the focus of this study: the acceptance of CDSS-generated STOPP/START signals by the pharmacotherapy team (steps 1–3) prior to discussion with the attending hospital physician and the patient. CDSS clinical decision support system, START screening tool to alert to right treatment, STOPP screening tool of older persons’ prescriptions

Clinical Decision Support System (CDSS) with Integrated STOPP/START Algorithms

The CDSS used for pharmacotherapy analysis was the STRIP Assistant (STRIPA), a web-based CDSS developed to perform a digitalised STRIP analysis with integrated STOPP/START criteria v2 [8, 17]. International coding systems were used for translating the STOPP/START v2 into algorithms, using the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes for diseases, the Anatomical Therapeutic Chemical (ATC) coding system for medication, and the Logical Observation Identifiers Names and Codes (LOINC) database for measurements (e.g. blood pressure, bone mineral density, laboratory values). The MedDRA was used to register patient-reported symptoms (e.g. dizziness, fatigue) [9, 15].

Of the 80 original STOPP criteria, 79 were encoded into algorithms. Only STOPP A2 (Any drug prescribed beyond the recommended duration, where treatment duration is well defined) could not be converted into an algorithm. In total, 34 original START criteria were converted to 33 algorithms as START A1 (Start vitamin K antagonists, direct thrombin inhibitors or factor Xa inhibitors in the presence of chronic atrial fibrillation) and START A2 (Start aspirin if START A1 is contraindicated) were merged into one algorithm (START A1/2). START I1 and I2 (Start influenza and pneumococcal vaccines) were excluded from analysis because CDSS custom settings differed per country for these two criteria based on national vaccination programmes. This resulted in a total of 110 STOPP/START algorithms available for analysis.

Details of the CDSS and the intervention as performed in the OPERAM trial have been published previously [15].

Outcomes

The primary outcome was the frequency and subsequent acceptance of CDSS-generated STOPP/START signals by the pharmacotherapy team (Fig. 1, steps 2–3). Frequency was defined as the number of population-based STOPP/START signals generated by the CDSS. Acceptance was defined as the percentage of STOPP/START signals accepted by the pharmacotherapy team after evaluation for clinical applicability to the individual patient. Accepted signals resulted in recommendations for the attending hospital physicians to initiate a drug based on START signals or in recommendations to discontinue or reduce dosage (e.g. drug tapering of benzodiazepines, antidepressants) based on STOPP signals. Data regarding the STOPP/START signals that the pharmacotherapy team accepted or rejected were saved within the CDSS and available for analysis.

The mean acceptance—namely, the percentage of accepted STOPP and START signals at the patient level—was used to investigate determinants that may affect signal acceptance.

Potential Determinants

Signal type (STOPP vs. START), patient-related factors, and setting-related factors were investigated as potential determinants. Patient-related factors included sex, age, number of co-morbidities, number of medications, history of falls, history of hospital admissions, renal function, systolic blood pressure, and being housebound or not. Setting-related factors included ward type (medical vs. surgical), admission type (elective vs. non-elective), length of hospital stay, and country of inclusion. Potential determinants with continuous values were dichotomised or categorised into tertiles based on patient distribution or based on clinically accepted cut-off values for measurements (renal function < 30, 30–50, and > 50 mL/min; systolic blood pressure < 120, 120–140, and > 140 mmHg). Data on potential determinants were captured during the index hospitalisation in an electronic case report form for all OPERAM patients. The included potential determinants were selected after expert consensus and based on a potential relation with STOPP/START (e.g. falls—section STOPP K; renal function—section STOPP E, STOPP B7, START F1; systolic blood pressure—START A4, STOPP K3) and database availability.

Data Analysis

Data analysis was performed with IBM SPSS Statistics v.25.0.0.2. An unpaired, two-sided Student’s t-test (α = 0.05, β = 0.2) was used to test the difference in percentages of mean acceptance for STOPP versus START signals. The effect of patient-related and setting-related determinants on mean acceptance was investigated separately for STOPP and START signals in a univariate linear regression analysis and entered in a multivariate linear regression model after examination of model assumptions.

Results

Study Population

A total of 2008 patients were included in the OPERAM study, 963 of whom were assigned to the intervention group. Data on the CDSS-assisted structured medication review during hospital admission were incomplete for 137 (14.2%) intervention patients. The study population therefore consisted of 826 patients who underwent a structured in-hospital medication review as part of the OPERAM intervention (Fig. 2).

Fig. 2.

Flowchart of the study population. ¹Reasons why no in-hospital pharmacotherapy analysis was performed in 88 (9%) of the OPERAM intervention patients were not collected at the patient level but included the following: patient discharged or transferred from ward, patient died, patient withdrew from study, and other reasons. ²The pharmacotherapy team had to actively save the results into the CDSS. In 49 (5%) of the OPERAM intervention patients data were not saved in the CDSS

The distribution of patients among the four participating trial sites was 399 (48.3%), 132 (16.0%), 92 (11.1%), and 203 (24.6%) for Switzerland, Belgium, Ireland, and the Netherlands, respectively. The study population had a median age of 78 years (interquartile range [IQR] 74–84); the median number of co-morbidities was 11 (IQR 8–17), and the median number of medications was 10 (IQR 7–13). In total, 8.4% of the study patients were nursing home residents, and the Barthel Index of Activities of Daily Living score [18] was high (median 95; IQR 75–100) (Table 1).

Table 1.

Baseline characteristics of the study population

Characteristics	n = 826
Age, years	78 (74–84)
Sex, female	46.4 (383)
Number of co-morbidities	11 (8–17)
Number of medications	10 (7–13)
Renal function, CKD-EPI; ml/min/1.73 m2	61 (43–79)
Nursing home residents	8.4 (69)
Housebound	13.3 (110)
Barthel Index of ADLa	95 (75–100)
Patients with one or more fall(s) in the previous year	37.9 (313)
Number of falls in the previous year	0 (0–1)
Patients with one or more hospital admission in the previous year	50.1 (414)
Number of hospital admissions in the previous year	1 (0–1)
Length of hospital stay (days)	8 (6–12)
Admission type
Elective	25.3 (209)
Non-elective	74.1 (612)
Ward
Medical	78.1 (645)
Surgical	21.9 (181)
Country of inclusion
Switzerland	48.3 (399)
Belgium	16.0 (132)
Ireland	11.1 (92)
The Netherlands	24.6 (203)

Data are presented as % (n) for categorical variables or median (interquartile range) for continuous variables. Missing data: renal function, 74 (9.0%); nursing home residents, 3 (0.4%); Barthel Index of ADL, 11 (1.3%); housebound, 2 (0.2%); number of falls during the previous year, 9 (1.1%); number of hospitalisations in the previous year, 3 (0.4%); length of stay during index hospitalisation, 2 (0.2%); admission type, 5 (0.6%)

ADL activities of daily living, CKD-EPI chronic kidney disease epidemiology collaboration equation

^aADL as measured by the Barthel Index. Values range from 0 to 100, with higher values indicating higher functional independence [18]

Frequency of STOPP/START Signals

In total, 5080 STOPP/START signals were generated in 826 patients. The median was 6 (IQR 4–8) generated signals per patient. No signals were generated in 0.8% (n = 7) of the patients, whereas 1–3, 4–6, and > 6 signals were generated in 39%, 38%, and 22% of the patients, respectively.

Of the generated signals, 68.2% (n = 3465) were based on STOPP criteria. In 96% (n = 791) of patients, one or more STOPP signals were generated, with a median of 4 (IQR 2–6) per patient, and 31.8% (n = 1615) of the generated signals were based on START criteria. In 82% (n = 681) of cases, one or more START signals were generated, with a median of 2 (IQR 1–3) per patient. The distribution of generated signals per patient was comparable across countries and ranged between 93 and 98% for one or more STOPP signals and 80–87% for one or more START signals.

In total, 68 of the 79 implemented STOPP criteria and 29 of the 31 START criteria generated a signal by the CDSS based on actual medical data on diagnosis, medication use, measurements, and laboratory values. Table 2 lists the ten most frequently generated STOPP and START signals and their subsequent acceptance as well as the 11 STOPP and two START signals that were never generated.

Table 2.

Overview of the frequency and subsequent acceptance of generated STOPP/START signals

Signal		Definition			Frequency, n	Acceptance, %
Top ten most frequently generated STOPP signals
STOPP A1		Any drug prescribed without an evidence-based clinical indication			1412	54.2
STOPP A3		Any duplicate drug class prescription e.g. two concurrent NSAIDs, SSRIs, loop diuretics, ACEI, anticoagulants			503	26.0
STOPP D5		Benzodiazepines for ≥4 weeks			181	64.1
STOPP F2		PPI for uncomplicated peptic ulcer disease or erosive peptic oesophagitis at full therapeutic dosage for >8 weeks			146	34.9
STOPP B6		Loop diuretic as first-line treatment for hypertension			101	22.8
STOPP C3		Aspirin, clopidogrel, dipyridamole, VKA, direct thrombin inhibitors or factor Xa inhibitors with concurrent significant bleeding risk, i.e. uncontrolled severe hypertension, bleeding diathesis, recent non-trivial spontaneous bleeding			75	4.0
STOPP F3		Drugs likely to cause constipation in patients with chronic constipation where non-constipating alternatives are available			75	20.0
STOPP G2		Systemic corticosteroids instead of inhaled corticosteroids for maintenance therapy in moderate-severe COPD			63	6.3
STOPP C5		Aspirin in combination with VKA, direct thrombin inhibitor or factor Xa inhibitors in patients with chronic AF			60	31.7
STOPP L2		Use of regular (as distinct from PRN) opioids without concomitant laxative			56	12.5
		Other STOPP criteria			793	32.2
STOPP signals that were never generated
STOPP C7		Ticlopidine in any circumstances			0	NA
STOPP D3		Neuroleptics with moderate-marked antimuscarinic/anticholinergic effects with a history of prostatism or previous urinary retention			0	NA
STOPP D6		Antipsychotics (i.e. other than quetiapine or clozapine) in those with parkinsonism or Lewy body disease			0	NA
STOPP D7		Anticholinergics/antimuscarinics to treat extrapyramidal side effects of neuroleptic medications			0	NA
STOPP E5		Colchicine if eGFR <10 ml/min/1.73 m2			0	NA
STOPP F1		Prochlorperazine or metoclopramide with Parkinsonism			0	NA
STOPP G1		Theophylline as monotherapy for COPD			0	NA
STOPP H1		NSAID other than COX-2 selective agents with history of peptic ulcer disease or gastrointestinal bleeding, unless with concurrent PPI or H2 antagonist			0	NA
STOPP J2		Thiazolidinediones in patients with heart failure			0	NA
STOPP J4		Oestrogens with a history of breast cancer or venous thromboembolism			0	NA
STOPP M1		Concomitant use of two or more drugs with antimuscarinic/anticholinergic properties			0	NA
		Total			3465	40.1
Top ten most frequently generated START signals
START H1		High-potency opioids in moderate-severe pain, where paracetamol, NSAIDs, or low-potency opioids are not appropriate to the pain severity or have been ineffective			162	2.5
START A6		ACEI with systolic heart failure and/or documented coronary artery disease			133	51.1
START E4		Bone anti-resorptive or anabolic therapy in patients with documented osteoporosis, where no pharmacological or clinical status contraindication exists and/or previous history of fragility fracture(s)			118	43.2
START H2		Laxatives in patients receiving opioids regularly			115	47.8
START E3		Vitamin D and calcium supplement in patients with known osteoporosis and/or previous fragility fracture(s) and/or bone mineral density T-scores more than − 2.5 in multiple sites			110	60.9
START E5		Vitamin D supplement in older people who are housebound or experiencing falls or with osteopenia			99	75.8
START A5		Statin therapy with a documented history of coronary, cerebral, or peripheral vascular disease, unless the patient’s status is end of life or age is >85 years			80	62.5
START G2		5α reductase inhibitor with symptomatic prostatism, where prostatectomy is not considered necessary			79	15.2
START D2		Fibre supplements for diverticulosis with a history of constipation			76	18.4
START A1A2		VKA or direct thrombin inhibitors or factor Xa inhibitors in the presence of chronic AF. If an oral anticoagulant is contraindicated, start aspirin (75–160 mg) instead			72	50.0
		Other START criteria			571	29.4
START signals that were never generated
START C4		Topical prostaglandin, prostamides, or β-blocker for primary open-angle glaucoma			0	NA
START G3		Topical vaginal oestrogen or vaginal oestrogen pessary for symptomatic atrophic vaginitis			0	NA
Total					1615	37.2

The ten most frequently generated signals and their subsequent acceptance as well as signals that were never generated are specified. Detailed information on frequency and acceptance for all STOPP/START signals—in total and per country—can be found in the electronic supplementary material SI1. Some of the original STOPP/START criteria v2 titles are shortened

ACEI angiotensin-converting enzyme inhibitor, AF atrial fibrillation, COPD chronic obstructive pulmonary disease, COX cyclooxygenase, eGFR estimated glomerular filtration rate, NA not applicable, NSAID non-steroidal anti-inflammatory drug, PPI proton pump inhibitor, PRN pro re nata (as needed), SSRI selective serotonin-reuptake inhibitors, START screening tool to alert to right treatment, STOPP screening tool of older persons’ prescriptions, VKA vitamin K antagonist

Acceptance of STOPP/START Signals

Overall, the pharmacotherapy team accepted 39.1% (n = 1990) of all 5080 generated STOPP/START signals, which corresponds to a median of 2 (IQR 1–3) per patient. The team accepted 40.1% (n = 1390) STOPP signals, resulting in a recommendation to the attending hospital physician and the patient. The median number of accepted STOPP signals was 1 (IQR 0–2) per patient. The team accepted 37.2% (n = 600) START signals resulting in a recommendation to initiate a drug (median 0; IQR 0–1).

In general, there was high variability in the acceptance of individual STOPP/START signals. Acceptance of the top ten most frequently generated STOPP/START signals ranged from 2.5 to 75.8%. STOPP A1 (Stop any drug prescribed without an evidence-based clinical indication) covered 28% of all generated signals, with more than half of the signals accepted (54%). Drugs for acid-related disorders were the drug class most often recommended for discontinuation based on STOPP A1 (22.5%), followed by mineral supplements (calcium) (8.0%) and psychoanaleptics (7.3%). Figure 3 shows the drug classes recommended for discontinuation based on STOPP A1.

Fig. 3.

Distribution of drugs on Anatomical Therapeutic Chemical (ATC)-2 level that were recommended for discontinuation because of a lack of an evidence-based clinical indication (STOPP A1). Drugs that resulted in a recommendation < 20 times were categorized as ‘X00 other’. A total of 766 of 1412 generated STOPP A1 signals were accepted by the pharmacotherapy team. STOPP screening tool of older persons’ prescriptions

Other STOPP signals from the top ten that resulted in a recommendation in more than 25% of cases included benzodiazepines (STOPP D5 [64%]), proton pump inhibitors (STOPP F2 [35%]), unindicated dual anticoagulant and antiplatelet therapy (STOPP C5 [32%]), and duplicated drug classes (STOPP A3 [26%]).

The most frequently generated START signal was a high-potency opioid in moderate–severe pain (START H1), but this signal was almost never accepted (3%). From the top ten most frequently generated signals based on START criteria, signals to initiate vitamin D, calcium, or bone anti-resorptive therapy in osteoporosis (START E5 [76%], START E3 [61%], START E4 [43%]); a laxative with concurrent opioid use (START H2 [48%]); statin therapy with known coronary, cerebral, or peripheral vascular disease (START A5 [63%]); an angiotensin-converting enzyme inhibitor (ACEI) with systolic heart failure and/or documented coronary artery disease (START A6 [51%]); or an anticoagulant with chronic atrial fibrillation (START A1A2 [50%]) were accepted in > 25% of cases (Table 2). Detailed information on frequencies and subsequent acceptance for all STOPP/START criteria—in total and stratified per country—are provided in the electronic supplementary material (ESM 1). An overview of the drugs (on ATC-2 level) involved in the medication optimisation recommendations based on accepted STOPP/START signals is provided in ESM 2.

For 9.1% (n = 181) of all accepted signals, the pharmacotherapy team added the advice to defer implementing the recommended action to the patient’s general practitioner. The accepted signals that were most frequently (more than ten times) recommended for deferral were to stop a drug without indication (STOPP A1; n = 43), to stop a benzodiazepine (STOPP D5; n = 22), to start bone anti-resorptive therapy (START E4; n = 19), and to start an ACEI (START A6; n = 16). These deferred recommendations were all included in the top ten most generated signals (Table 2).

Determinants

There was no difference in mean acceptance of STOPP versus START signals (+ 2.1; 95% confidence interval [CI] − 1.5 to 5.7]). Linear regression analysis was performed on potential patient-related and setting-related determinants for STOPP and START signals.

For STOPP signals, mean acceptance significantly decreased after multivariate linear regression analysis for patients with more co-morbidities (more than nine: − 11.8%; 95% CI − 19.2 to − 4.5; Table 3). Admission to a surgical ward was positively associated with acceptance (+ 10.3%; 95% CI 3.8–16.8). The rate of acceptance was higher in Ireland (+ 26.8%; 95% CI 16.8–36.7) and the Netherlands (+ 14.7; 95% CI 7.8–21.7) than in Switzerland as reference country.

Table 3.

Univariate and multivariate linear regression of patient-related and setting-related determinants on mean acceptance

Determinant	STOPP			START
	Patients, n	Univariate	Multivariate	Patients, n	Univariate	Multivariate
Patient-related
Sex
Male	421	37.7	Reference	374	37.0	Reference
Female	370	+ 5.5 (1.0–9.9)*	+ 2.8 (− 1.9–7.5)	307	+ 2.5 (− 3.4–8.3)	− 0.8 (− 7.1–5.5)
Age, years
< 75	226	38.6	Reference	193	37.2	Reference
75–80	249	+ 0.9 (− 4.8–6.7)	+ 1.0 (− 4.8–6.9)	211	+ 0.7 (− 6.8–8.3)	+ 0.9 (− 7.0–8.8)
> 80	316	+ 3.3 (− 2.1–8.8)	+ 2.7 (− 3.1–8.5)	277	+ 1.9 (− 5.2–9.0)	+ 1.9 (− 5.8–9.7)
Number of co-morbidities
< 7	282	48.7	Reference	234	42.6	Reference
7–9	257	− 7.5 (− 12.7 to − 2.2)*	− 5.4 (− 11.6–0.8)	224	− 7.1 (− 14.1 to − 0.04)*	− 11.0 (− 19.4 to − 2.6)*
> 9	252	− 19.0 (− 24.3 to − 13.7)*	− 11.8 (− 19.2 to − 4.5)*	223	− 6.5 (− 13.6–0.5)	− 7.1 (− 17.2–3.0)
Number of medications
< 9	287	39.3	Reference	252	38.7	Reference
9–12	275	+ 2.9 (− 2.4–8.2)	+ 2.7 (− 2.9–8.3)	239	− 0.4 (− 7.2–6.4)	− 2.9 (− 10.3–4.6)
> 12	229	− 0.4 (− 6.0–5.1)	+ 5.2 (− 0.9–11.2)	190	− 1.5 (− 8.8–5.8)	− 2.1 (− 10.2–6.1)
Number of falls in the previous year
0	480	41.1	Reference	403	35.8	Reference
≥ 1	302	− 1.7 (− 6.4–2.9)	+ 0.2 (− 4.6–4.9)	269	+ 5.0 (− 0.9–10.9)	+ 7.1 (0.7–13.4)*
Number of hospital admissions in the previous year
0	386	43.4	Reference	319	34.2	Reference
≥ 1	402	− 5.9 (− 10.4 to − 1.5)*	− 3.5 (− 8.1–1.2)	359	+ 7.2 (1.4–13.0)*	+ 7.9 (1.6–14.1)*
Housebound
No	687	40.0	Reference	589	36.8	Reference
Yes	102	+ 1.2 (− 5.5–7.9)	− 4.9 (− 12.5–2.7)	90	+ 9.1 (0.6–17.6)	− 0.0 (− 10.0–10.0)
Renal function (eGFR; CKD-EPI; ml/min/1.73 m2)
> 50	477	39.4	Reference	407	36.6	Reference
30–50	169	− 1.6 (− 7.2–4.0)	− 2.0 (− 7.6–3.6)	149	+ 2.5 (− 4.7–9.7)	+ 2.1 (− 5.5–9.6)
< 30	76	0.2 (− 7.5–8.0)	+ 1.6 (− 6.0–9.3)	69	− 1.0 (− 10.7–8.8)	− 1.0 (− 11.1–9.1)
Systolic blood pressure (mmHg)
120–140	298	39.8	Reference	261	37.2	Reference
< 120	243	− 2.8 (− 8.1–2.7)	− 0.0 (− 5.5–5.5)	209	− 0.3 (− 7.3–6.7)	− 1.1 (− 8.4–6.2)
> 140	235	+ 3.9 (− 1.6–9.4)	+ 3.0 (− 2.6–8.6)	199	+ 3.3 (− 3.8–10.4)	+ 4.7 (− 2.9–12.2)
Setting-related
Ward
Medical	618	38.6	Reference	535	38.1	Reference
Surgical	173	+ 7.2 (1.8–12.6)*	+ 10.3 (3.8–16.8)*	146	+ 0.2 (− 6.9–7.3)	− 1.8 (− 10.5–6.9)
Admission type
Elective	198	39.1	Reference	163	38.6	Reference
Non-elective	589	+ 1.5 (− 3.7–6.7)	+ 4.8 (− 1.2–10.8)	514	− 0.4 (− 7.2–6.4)	+ 1.4 (− 6.8–9.7)
Length of hospital stay (days)
< 6	194	38.6	Reference	151	35.9	Reference
6–10	332	+ 2.2 (− 3.5–7.9)	− 1.5 (− 7.4–4.4)	385	+ 1.4 (− 6.2–9.0)	− 0.8 (− 8.9–7.3)
> 10	263	+ 2.2 (− 3.8–8.2)	− 3.8 (− 10.2–2.5)	244	+ 4.8 (− 3.0–12.6)	+ 3.9 (− 4.6–12.4)
Country of inclusion
Switzerland	392	30.7	Reference	320	31.3	Reference
Belgium	122	+ 9.6 (3.5–15.8)*	+ 4.2 (− 4.4–12.8)	107	+ 11.6 (3.4–19.9)*	+ 8.8 (− 2.7–20.2)
Ireland	88	+ 27.7 (20.7–34.7)*	+ 26.8 (16.8–36.7)*	78	+ 26.2 (16.9–35.5)*	+ 31.1 (18.2–44.0)*
The Netherlands	189	+ 20.8 (15.6–26.1)*	+ 14.7 (7.8–21.7)*	176	+ 7.8 (0.9–14.8)*	− 2.3 (− 7.1–11.6)

Data are presented as % (95% confidence interval) unless otherwise indicated. All determinants were entered in the multivariate linear regression model for mean acceptance of STOPP and START signals

CKD-EPI chronic kidney disease epidemiology collaboration equation, eGFR estimated glomerular filtration rate, START screening tool to alert to right treatment, STOPP screening tool of older persons’ prescriptions

*p < 0.05

For START signals, mean acceptance significantly decreased by − 11.0% (95% CI − 19.4 to − 2.6) for patients with seven to nine co-morbidities after multivariate analysis. One or more falls (+ 7.1%; 95% CI 0.7–13.4) and one or more hospital admissions in the previous year (+ 7.9; 95% CI 1.6–14.1) were positively associated with acceptance of START signals. Compared with Switzerland, a higher acceptance rate was only found in Ireland (+ 31.1%; 95% CI 18.2–44.0).

Table 3 shows all results of univariate and multivariate linear regression analysis of patient-related and setting-related determinants on mean acceptance of STOPP and START signals.

Discussion

Frequency and Acceptance

In 819 of 826 patients (99%), at least one signal for potential inappropriate prescribing was generated by the CDSS using a set of 110 algorithms based on STOPP/START criteria v2 [3]. One or more STOPP or START signals were generated in 96% and 82% of patients, respectively. The pharmacotherapy team accepted 39% (n = 1990) of the total 5080 CDSS-generated STOPP/START signals. Overall, there was high variability in both the frequency and the acceptance of the individual criteria. The most frequently generated signal (28% of all signals) was to discontinue a drug without a clinical indication (STOPP A1), which was accepted in 54% of cases. Although more STOPP (68%) than START (32%) signals were generated, no significant difference was found between their respective mean acceptance rates.

The detection of potential inappropriate prescribing in older patients has been investigated in several studies using a CDSS in a hospital setting. Heterogeneity in reported frequencies of medication overuse, underuse, and misuse can generally be explained by differences in the study population, types of tools used, and tool applications (e.g. prospective vs. retrospective). For instance, a recent study found a lower prevalence of potential overuse (56%) and potential underuse (58%) after application of STOPP/START v2 algorithms on a database with medical information from older hospitalised patients [19]. Retrospective database studies are often limited by incomplete documentation of relevant medical information, directly affecting the prevalence of STOPP/START signals. Dalton et al. [20] included four controlled studies in a systematic review reporting acceptance (range 29.3–95.0%) of computer-generated recommendations for medication overuse in hospitalised older adults. However, the computerised intervention tools were rather heterogeneous and did not include detection of potential underuse, which impedes comparison with our findings.

More comparable to our research in relation to the study design and population is the SENATOR trial. This multicentre clinical trial investigated the impact of CDSS-generated STOPP/START criteria v2 on the occurrence of adverse drug reactions within 14 days of inclusion in multimorbid older inpatients [21]. The frequency of generated START signals (1.8 vs. 2 per patient) was similar to that in our findings, but we detected higher overuse (2.8 vs. 4.0 per patient), which may be explained by the exclusion of STOPP A1 (no clinical indication for the drug) in the SENATOR trial. In contrast to the medication review process in OPERAM, CDSS-generated signals were directly presented to the attending physicians without assessment for clinical applicability by a pharmacotherapy team. The clinical relevance of the CDSS-generated signals according to attending physicians was not prospectively measured, but a post hoc analysis of the SENATOR trial showed that only 15% of generated signals were implemented by the attending physicians [22]. However, retrospective examination of signals by a pharmacist–physician pair found that 39% of all generated signals were deemed to be of possibly important or very important clinical relevance [22]. This percentage is in line with the rate of signal acceptance by the pharmacotherapy team in our study.

Determinants

Country of recruitment was the most important determinant for which a significant difference in acceptance for both STOPP and START signals was found compared with Switzerland as the country of reference. The higher acceptance of signals by the pharmacotherapy team from Cork (Ireland)—the originator of STOPP/START version 1—may be partly explained by familiarity with applying these criteria in their hospital. However, the STOPP/START criteria are now widely used across Europe, and the pharmacotherapy teams were trained according to standardised operating procedures before performing the intervention. Therefore, site-specific differences in rotation and level of clinical experience of the pharmacotherapy teams may be more likely to explain the variability in acceptance across sites, with Switzerland having a higher turnover of physician–pharmacist pairs performing the intervention than the other countries.

The impact of other significant patient-related and setting-related determinants on acceptance was relatively low, ranging from − 11.8 to 10.3%. Acceptance was positively associated with admission to a surgical ward for STOPP signals (10.3%), which suggests that special attention to deprescribing in patients on surgical wards may be beneficial. Investigation of patient-related factors revealed a negative association between an increased number of co-morbidities and the acceptance of STOPP and START signals. This may indicate that the population-based STOPP/START criteria are less suitable for application to individual patients with multiple conditions, for instance because co-existing relevant contraindications could impede medication changes. From the patient-related determinants, one or more hospital admissions in the previous year and a history of falls were positively associated with acceptance of START signals. The higher acceptance rates in patients with a history of falls could be explained by the high number of accepted signals related to vitamin D, calcium supplements, and bone-antiresorptive therapy. Although these patient-related factors were statistically significant, differences were considered too small to define a clear inpatient population for whom the application of STOPP/START would be of lesser or greater value from a clinical perspective.

CDSS-Related Restrictions

To incorporate guideline recommendations into the CDSS, STOPP/START criteria were converted into algorithms; however, many lacked sufficient clarity for translation [9, 23, 24]. STOPP A2 (Any drug prescribed beyond the recommended duration, where treatment duration is well defined) could not be coded at all, and some elements of other criteria were left out (e.g. for START A5 [… unless the patient’s status is end-of-life]). For other ambiguous criteria (e.g. STOPP M1 [drugs with antimuscarinic/anticholinergic properties]), experts (senior physicians and clinical pharmacists) were consulted to reach consensus on which conditions or drugs should be included in the algorithms. Risk of over-detection rather than under-detection was chosen as a strategy for converting STOPP/START criteria into algorithms within the OPERAM trial. Consequently, simplifying certain criteria probably led to false-positive signals and negatively affected acceptance.

In addition, multiple STOPP and START criteria could be generated recommending medication changes for the same drug, whereas the CDSS allowed the pharmacotherapy team to accept only one recommendation for each drug per patient. For instance, STOPP L2 (use of regular [as distinct from as needed] opioids without concomitant laxative) and START H2 (laxatives in patients receiving opioids regularly) would both be generated in a patient using opioids without a laxative. In such cases, the pharmacotherapy team could either reject both signals, or—if a drug change was clinically indicated—accept the most appropriate signal of the two, which resulted more frequently in a recommendation to initiate a laxative (Table 2; START H2: frequency n = 115; acceptance 47.8%) rather than to discontinue the opioid (Table 2; STOPP L2: frequency n = 56; acceptance 12.5%).

Setting-Related Restrictions

The pharmacotherapy analysis was performed in a hospital setting. Decisions to accept or reject STOPP/START signals may be different in other clinical settings as well as the willingness of patients and phycians to change long-term medication use. Hospitalisations have a significant impact on the continuity of pharmacotherapy, whereas STOPP/START criteria mainly focus on chronic drug use [25–27]. However, the pharmacotherapy team could also decide to accept but defer the implementation (e.g. drug tapering) of a clinically relevant signal until after discharge, and those signals were counted as accepted. In addition, our geriatric population was relatively functionally independent, with only 8.4% of participants living in nursing homes. Results from a study investigating the impact of STOPP/START criteria (v1) in frail geriatric chronic care residents found that 82.4% of STOPP and 92.6% of START recommendations made by a research pharmacist were implemented by the attending physician [28, 29], whereas only 62.2% of all OPERAM patients had one or more STOPP/START recommendations implemented at 2-month follow-up [30]. Interestingly, implementation of recommendations to discontinue benzodiazepines was lower in the geriatric chronic care setting (23%; n = 3/13) than in the OPERAM trial at 2-month follow-up (39.1%; n = 45/115) [28, 30]. These differences may illustrate that decisions to optimise pharmacotherapy in a hospitalised population are likely to differ from decisions made for long-term care facility residents or in primary care.

Strengths and Limitations

In our study, medical information at the time of pharmacotherapy analysis was prospectively collected and assessed for clinical applicability by physicians and pharmacists who had clinical experience in caring for older adults and full access to the patient’s actual medical file. Unlike in retrospective studies, the pharmacotherapy team considered essential factors such as life expectancy, drug exposure length, and time until benefit. Carvalho et al. [31] reported that only one-third of all STOPP criteria and just one START criterion could be adequately applied if only a patient’s medication list is available without diagnostic data. Consequently, applying STOPP/START using medical databases without clinical evaluation is hampered compared with its use on real-time patient data. Our structured prospective evaluation of STOPP/START signals in a large group of in-hospital older people provides accurate insight into clinically relevant signals of over-prescribing and under-prescribing in this population.

A limitation of this study was the relatively large amount of missing data (n=137). After performing a pharmacotherapy analysis, the pharmacotherapy team had to actively save the results into the CDSS. In 49 (5%) of the OPERAM intervention patients data were not saved. No in-hospital pharmacotherapy analysis was performed for the other missing patients because of various reasons, such as early discharge from the hospital, transfer to another ward, or withdrawal before intervention.

The acceptance reflects the pharmacotherapy team’s treatment recommendations regarding presumed overuse, underuse, and misuse; however, information about individualised treatment goals and patient preferences was not always available during the pharmacotherapy analysis. Implementation of the proposed recommendations after discussion with both the attending hospital physician and the patient, and the persistence after discharge, were not included in the design of this study. In the main OPERAM trial results, data on implementation of recommendations at 2 months after index hospitalisation were provided [30]. However, in this substudy, the study population and the term ‘recommendations’ were defined differently than in the OPERAM main trial (see ESM 3).

Lastly, the reasons for rejection of CDSS-generated STOPP/START signals were not collected, which makes it difficult to distinguish whether CDSS-related or setting-related restrictions had a larger impact on low acceptance of signals by the pharmacotherapy teams.

Implications

The use of STOPP/START v2 criteria as algorithms is a helpful approach to detect medication overuse, underuse, and misuse in older patients within a hospital setting, but it may also result in signal overload. Given that more than half of all generated signals were rejected, an expert team’s involvement in translating population-based CDSS signals to individual patients is essential. Furthermore, our most frequently recommended action was ‘to stop a drug without a clear indication’ (STOPP A1), which requires critical clinical evaluation. Without such an expert team, signal overload will probably lead to low implementation rates in usual care, as shown in the SENATOR trial (15%) [22].

Our detailed description of the combined frequency and acceptance of STOPP/START v2 within a large European hospital population could help to differentiate which STOPP/START algorithms provide the highest clinical benefit in a hospital setting. Future research investigating factors that affect patients’ and physicians’ agreement with medication changes recommended by expert teams may provide further insight for implementation in clinical practice. In addition, our results were based on decisions made by a pharmacotherapy team in a hospital setting, which may not be the most appropriate setting in which to change chronic medication. It would be highly interesting to compare the results of this study with those of the OPTICA (Optimising PharmacoTherapy In the multimorbid elderly in primary CAre) trial, in which the application of a similar STOPP/START-based CDSS is being investigated in a primary care setting [32].

Conclusion

Nearly all hospitalised patients with polypharmacy and multimorbidity had at least one signal for potential medication overuse, underuse, or misuse, and 39% of them were accepted by a pharmacotherapy team at the individual patient level. There was a high variability in the frequency and subsequent acceptance of individual STOPP/START v2 signals. In general, the investigated patient-related determinants were poor predictors of acceptance of STOPP/START v2 recommendations in a hospital setting. The moderate overall acceptance and the site-specific differences in acceptance emphasize the important role of a pharmacotherapy team in translating population-based STOPP/START signals to individual patients.

Supplementary Material

Below is the link to the electronic supplementary material.

Declarations

Funding

This work is part of the OPERAM project supported by the EU’s Horizon 2020 research and innovation programme under grant agreement no. 634238 and by the Swiss State Secretariat for Education, Research and Innovation under contract number 15.0137. The work of Katharina Tabea Jungo was supported by the Swiss National Science Foundation, grant no. NFP74 407440_167465. The opinions expressed herein are those of the authors and do not necessarily reflect the official views of the European Commission or the Swiss government. The funder of the study had no role in the study design; data collection, analysis, and interpretation; or writing of the report.

Conflicts of Interest

Denis O’Mahony has a patent ‘A Prescription Decision Support System (based on screening tool of older person’s prescriptions and screening tool to alert to the right treatment [STOPP/START] prescribing rules)’ issued to the European Patent Office (Munich). Marco R Spruit received a 2011 grant and personal fees from Spru IT before the conduct of the study and reports a settlement agreement between Spru IT and Utrecht University, in which all STRIP assistant intellectual property is transferred to Utrecht University in exchange for obtaining a free but non-exclusive right to provide STRIP assistant consultancy or support services, or both, on a commercial basis and to update the STRIP assistant until June 2023. Bastiaan TGM Sallevelt, Corlina JA Huibers, Jody MJ Op Heij, Toine CG Egberts, Eugène P van Puijenbroek, Zhengru Shen, Katharina Tabea Jungo, Nicolas Rodondi, Olivia Dalleur, Anne Spinewine, Emma Jennings, Ingeborg Wilting, and Wilma Knol have no conflicts of interest that are directly relevant to the content of this article.

Availability of data and material

Data for this study will be made available to others in the scientific community upon request after publication. Data will be made available for scientific purposes for researchers whose proposed use of the data has been approved by a publication committee.

Ethics Approval

The OPERAM trial was approved by the independent research ethics committees at each participating site (lead ethics committee: Cantonal Ethics Committee Bern, Switzerland, ID 2016-01200; Medical Research Ethics Committee Utrecht, Netherlands, ID 15-522/D; Comité d’Ethique Hospitalo-Facultaire Saint-Luc-UCL: 2016/20JUL/347–Belgian registration No: B403201629175; Cork University Teaching Hospitals Clinical Ethics Committee, Cork, Republic of Ireland; ID ECM 4 (o) 07/02/17), and Swissmedic as the responsible regulatory authority.

Consent

Written informed consent was obtained from the patients or their legal representatives before enrolment in the OPERAM trial.

Author Contributions

Authorship eligibility is based on the International Committee of Medical Journal Editors authorship criteria. The authors certify that they have participated in the conception and design (BS, CH, TE, EvP, DOM, AS, NR, IW, WK), acquisition and interpretation of data (BS, CH, JoH, IS, TE, EvP, IW, WK, NR, AS, DOM), drafting the article (BS, CH), and revising it critically for important intellectual content (all authors). All authors approved the final article.