STOPP/START Medication Criteria Modified for US Nursing Home Setting

STRUCTURED ABSTRACT

BACKGROUND/OBJECTIVES

A barrier to assessing the quality of prescribing in nursing homes (NH) is the lack of explicit criteria for this setting. Our objective was to develop a set of prescribing indicators measurable with available data from electronic nursing home databases by adapting the European-based 2014 STOPP/START criteria of potentially inappropriate and underused medications for the US setting.

DESIGN

A two-stage expert panel process. In first stage, investigator team reviewed 114 criteria for compatibility and measurability. In second stage, we convened an online modified e-Delphi (OMD) panel to rate the validity of criteria and two webinars to identify criteria with highest relevance to US NHs.

PARTICIPANTS

Seventeen experts with recognized reputations in NH care participated in the e-Delphi panel and 12 in the webinar.

MEASUREMENTS

Compatibility and measurability were assessed by comparing criteria to US terminology/setting standards and data elements in NH databases. Validity was rated with a 9-point Likert-type scale (1=not valid at all, 9=highly valid). Mean, median, interpercentile ranges, and agreement were determined for each criterion score. Relevance was determined by ranking the mean panel ratings on criteria that reached agreement; half of the criteria with the highest mean values were reviewed and approved by the webinar participants.

RESULTS

Fifty-three STOPP/START criteria were deemed as compatible with US setting and measurable using data from electronic NH databases. E-Delphi panelists rated 48 criteria as valid for US NHs. Twenty-four criteria were deemed as most relevant, consisting of 22 measures of potentially inappropriate medications and 2 measures of underused medications.

CONCLUSION

This study created the first explicit criteria for assessing the quality of prescribing in US NHs.

INTRODUCTION

Nearly 3.2 million elderly and disabled Americans receive care in nursing homes (NHs) each year. On average, NH residents who remain beyond a short stay (> 90 days) will receive 10 different medications per month.^{(1, 2)} However, the overall quality of prescribing in NHs is unclear; high use of medications in NHs may include overuse of suboptimal medications but it does not preclude the underuse of beneficial medications. Nearly 60% of NH residents receive unnecessary medications, while the underuse of beneficial medications may be as high as 42%.⁽³⁾ Living in a NH is an independent predictor of underprescribing,⁽⁴⁾ and at least 16 studies have demonstrated underprescribing in NHs in the context of specific conditions.^(5–7) Underuse or overuse of clinically indicated drug therapies has been associated with increased morbidity, mortality and reduced quality of life in NHs.⁽⁸⁾

A critical barrier to assessing the quality of prescribing in NHs has been the lack of explicit criteria tailored to the NH population, particularly the majority who are frail and elderly. Most tools for assessing inappropriate prescribing are developed for older adults living in the community, and few consider the underuse of beneficial medications. The Screening Tool of Older Person’s Prescriptions (STOPP) is a validated, evidence-based list of 80 criteria for potentially inappropriate prescribing in community-dwelling older adults. ^{(9, 10)} The Screening Tools to Alert Doctors to Right Treatment (START) is a set of 34 evidence-based and validated prescribing indicators for common diseases in community-dwelling older adults.^{(9, 10)} Over 45 research studies have used the STOPP/START criteria since it was made available. ⁽³⁾ Both the STOPP and START criteria were developed to improve quality of care and are widely used in Europe, but to our knowledge have not been adapted to the United States (US) NH populations.⁽¹¹⁾ Other criteria also exist including the Beers Criteria of potentially inappropriate medications in the elderly but only the STOPP/START to our knowledge addresses underprescribing of medications.⁽¹²⁾ Developers of the STOPP/START have concluded that the criteria are feasible to apply in NHs, but some indicators may “have limited practical value, given that life expectancy is short.” ⁽³⁾

The primary objective of this study was to adapt the STOPP/START criteria for the US NH setting through an e-Delphi consensus panel. A secondary aim was to create a broadly useful subset of STOPP/START prescribing indicators that could be measured with information captured in widely used electronic NH databases. NHs participating in Medicare and Medicaid are required to assess a core set of data elements on all nursing home residents called the Minimum Data Set.⁽¹³⁾ The work presented here is part of a larger investigation sponsored by the National Institutes of Health (R01AG046341) to investigate the relationship between Medicare Part D prescription drug coverage and medication use in NHs.

METHODS

STOPP and START Criteria

The STOPP/START criteria were originally developed in Ireland after a review of the evidence-base of systematic reviews and randomized clinical trials on best prescribing practices for community-dwelling older adults. The criteria were updated in 2014, which is the version we used in this study.⁽¹⁴⁾

The materials for the Delphi panel were prepared in two phases. In phase one, the investigative team that consisted of the principal investigator (PI), two pharmacists, and a geriatrician reviewed each of the 114 2014 STOPP/START criteria for compatibility and measurability in context of the US setting and NH populations. This process included modifying text to American terminology, deleting references to drugs that were not available in the US market, simplifying conditional qualifiers that were deemed less relevant in NH setting and making some general criteria more targeted in scope (Supplemental Appendix S1). For example, the term “neuroleptics” was replaced with “antipsychotics,” a criterion about maintenance therapy was redefined as therapy exceeding 14 days, and the criterion for any duplicative therapy was limited to duplicative therapy in 4 key drug classes (i.e., hypnotic/sedatives, antidepressants, antipsychotics, anxiolytics). In this stage, we also assessed the measurability of each criterion with the standard data elements available in the Minimum Data Set 3.0, Medicare Part A and B claims, and pharmacy dispensing records. Criteria dependent upon laboratory values or medical histories were deemed as not measurable in these data.

Panel Selection

In phase two, we conducted an e-Delphi panel. We invited 22 experts with recognized reputations in NH care and interdisciplinary skills representing the following professional domains: geriatric medicine, nursing, clinical pharmacy, research, policy, and quality assurance. Other selection criteria included clinical knowledge of NH prescribing issues and active role in caring for NH residents. Individuals indicating interest were asked to provide explicit consent to participate in the study, as well as basic demographic and professional information. A $250 honorarium was offered for participation. These activities were determined to be exempt from review by the Human Subjects Protection Committee at RAND; the overall study was approved by the Institutional Review Board at Northeastern University.

E-Delphi Panel

The expert panel was conducted using an online platform (ExpertLens^™)^{(15, 16)} to identify the most valid STOPP/START criteria for the US NH setting. The Delphi method is a well-established and structured approach for soliciting the opinions of experts and establishing a convergence of opinion.⁽¹⁷⁾ Although not formally compared to its face-to-face counterpart, the e-Delphi panel has been deemed acceptable for developing health services performance measures ⁽¹⁸⁾ and shown to provide a useful and feasible forum for allowing the participants in an expert panel to provide ratings and take part in moderated discussions over the Internet that produces replicable results.⁽¹⁶⁾ This e-Delphi platform has been successfully used in numerous studies on healthcare topics. ^(18–23)

Before the e-Delphi panel, we drafted and pilot tested the protocol to isolate programming or formatting issues and to finalize instructions and question phrasing. After the pilot, STOPP/START criteria examples were developed and added to the protocol to better justify and explain their meaning in the US NH context (Supplemental Appendix S1).

The e-Delphi panel consisted of three rounds: the initial assessment, feedback and discussion, and final assessment. Login and participation training were conducted through online videos. In round one, participants rated the validity of the 52 STOPP/START criteria deemed compatible and measurable in phase one of this study (see Supplemental Table S1, column 3) on a 9-point Likert-type scale (1= not valid at all and 9=highly valid). For each criterion, participants were asked to consider whether (1) there is adequate scientific evidence or professional consensus to support the criterion in the NH population; and (2) there are clear health benefits to NH residents who receive care specified by the criterion. Participants were asked to explain their numeric response using an open-text box displayed below each question. Round one was open September 1–18, 2015.

In round two, the experts received an individualized summary of round one results. Each participant saw a series of bar charts showing the frequency distributions of responses to all round one questions. On each chart, they saw (1) their own rating, (2) the group median rating, (3) interquartile range of group responses. Participants also saw and commented on all rating explanations provided in round one. Finally, they could anonymously participate in an asynchronous discussion around the validity of each criteria; this online discussion was moderated by the PI who posted probing questions around unclear or discordant ratings. Round two was open September 18 – 28, 2015.

In round three, experts were encouraged to revise their round one ratings in light of round two group feedback and discussion. Criteria deemed invalid or of uncertain validity in round one (see below) were re-rated in round three. Round three was open September 28 – October 9, 2015.

Finally, after the e-Delphi process concluded, we conducted two webinar meetings to address comments generated throughout the e-Delphi process indicating concerns that some criteria were clinically valid but less relevant to a NH population of older adults. For example, the criterion of disease-modifying anti-rheumatic drugs for rheumatoid arthritis was rated as valid, but the accompanying comments indicated this would be a rare problem in NH setting. During the first webinar, e-Delphi participants adjudicated the criteria with uncertain validity. During the second webinar, they discussed relevance of the criteria deemed valid during the e-Delphi process in the context of a typical patient-case scenario in the US NH setting.⁽²⁴⁾

Data Analysis

The final validity rating for each criterion was determined by applying the two-step RAND/UCLA Appropriateness Method. ⁽²⁵⁾ In the first step, we determined the existence of disagreement by calculating the value of Interpercentile Range (IPR), or the range of responses that fell between the 70th and the 30th percentiles, and comparing it to the value of the Interpercentile Range Adjusted for Symmetry (IPRAS), which is a measure of dispersion for asymmetric distributions. If IPR>IPRAS, it is an indicator of disagreement among participants.^{(25, 26)} We then used the median value to determine whether the panel considered each criterion to be valid, invalid, or of uncertain validity. A median score between 7 and 9 indicated a criterion as valid; a median score between 4 and 6 indicated uncertain validity; and a median score of 1 to 3 qualified the criterion as invalid.

We tested for differences in means between round three and round one responses and found no differences (analysis not shown). Therefore, to ensure that all data are used in the analysis, we combined round one responses of those experts who could not participate in round three with round three responses of those who contributed to all rounds in tabulating the final ratings (Supplemental Appendix S1).^{(19, 27)}

Finally, for criteria deemed valid by the e-Delphi panel, mean scores were calculated and the criteria ranked from high to low to determine the top 25 of criteria to be discussed during the second webinar to determine their relevance in the US NH context.

RESULTS

Of the 22 invited experts, 17 participated (response rate 77%) in round one (Supplemental Appendix S1). Of the 17 experts, 15 (88%) participated in round two and 11 (65%) participated in round three. Five (30%) of the 17 participants were primarily pharmacists, 5 (30%) were nurses, 4 (24%) researchers, and 3 (18%) physicians. Slightly more than half (9) of participants were principally employed by academic medical centers, 7 (41%) by long-term care facilities, and 1 by a government agency (Table 1). Twelve panel members attended the first webinar and 6 attended the second.

Table 1.

Demographic Characteristics of Expert Panel n=17

Characteristic	N	%
Gender
Female	14	82.4
Primary Training
Health Care Professional	14	82.4
Research	3	17.6
Primary Occupation
Nurse	5	29.4
Pharmacist	5	29.4
Researcher	4	23.5
Physician	3	17.6
Primary Institution
Academic Medical Center	9	52.9
Long-term Care Provider	7	41.2
Government	1	5.9

Table 2 shows the final list of the most highly rated STOPP/START criteria according to the standards of relevance for the US NH setting, measurability with data available in NH electronic databases, and validity. In phase one of this study, 61 of 114 STOPP/START criteria were deemed hard-to-measure. (See full list of criteria and ratings in Supplemental Appendix S1.) Requiring laboratory test results was the main reason for deeming a criterion infeasible to measure. Among the remaining 52 criteria, the e-Delphi panel rated 48 as valid (no disagreement and the median score between 7 and 9) with only 4 criteria receiving an uncertain validity rating (Supplemental Appendix S1).

Table 2.

Final list of modified STOPP/START criteria for US NH population aged 65 or older

Drug Class/Physiologic System	Criteria
Potential Prescribing Omissions (START)
Vaccines	Pneumococcal vaccine at least once after age 65 Seasonal influenza vaccine annually
Potentially Inappropriate Medications (STOPP)
Antiplatelet/Anticoagulant Drugs	Concomitant prescription NSAID and vitamin K antagonist, direct thrombin inhibitor or factor Xa inhibitors Concomitant prescription NSAID and antiplatelet agent without PPI prophylaxis
Central Nervous System and Psychotropic Drugs	Any phenothiazines Anticholinergics/antimuscarinics with delirium or dementia Antipsychotics (with the exception of quetiapine or clozapine) with Parkinsonism or Lewy Body Disease Benzodiazepines for ≥ 4 weeks Concomitant use of two or more drugs with antimuscarinic/anticholinergic properties Any tricyclic antidepressants Any first generation antipsychotics Any duplicate prescription within these drug classes: hypnotics/sedatives, antidepressants, anxiolytics, or antipsychotics
Endocrine System	Sulphonylureas with type 2 diabetes mellitus
Gastrointestinal System	Oral elemental iron doses greater than 200 mg daily Prochlorperazine or metoclopramide with Parkinsonism Proton pump inhibitor for uncomplicated peptic ulcer disease or erosive peptic esophagitis at full therapeutic dosage for > 8 weeks
Musculoskeletal System	COX-2 selective NSAIDs with cardiovascular disease Prescription NSAID and COX-2 selective NSAID with peptic ulcer disease, unless with concurrent proton pump inhibitor or H2 antagonist
Renal System	Digoxin at a dose greater than 125mcg/day Metformin with end-stage renal disease or dialysisa NSAIDs with renal failure, end-stage renal disease, or dialysisb
Urogenital System	Antimuscarinic drugs with dementia, cognitive impairment, glaucoma/cataracts/macular degeneration, or enlarged prostate Selective alpha-1 blockers with orthostatic hypotension
Respiratory System	Systemic corticosteroids instead of inhaled corticosteroids for maintenance therapy (> 14 days) in chronic obstructive pulmonary disease

NSAID: nonsteroidal anti-inflammatory drug

PPI: proton-pump inhibitor

COX-2: cyclooxygenase-2

^aAs defined by ICD-9 codes of 585.4, 585.5, 585.6

^bAs defined by chronic kidney disease stage 3–5 and end stage renal disease

Participants of the first webinar elected to eliminate any criteria deemed of uncertain validity (4 criteria). During the second webinar, the panel approved the top 25 of the valid criteria with the highest mean rating as a relevant and broadly useful subset to be applied in US NHs. One additional criterion was excluded as a duplicate, resulting in a final list of 24 criteria (Table 2). Nearly all of the highest rated criteria (22 out of 24) came from the STOPP list of potentially inappropriate medications. Only the criteria of vaccines from the START criteria of potential prescribing omissions received a rating high enough to be in the top 25 of the rankings. Medications used for treating the central nervous system comprised the largest category of highly rated criteria (8 of 24).

DISCUSSION

To our knowledge, this is the first assessment of relevance, measurability, and validity of STOPP/START criteria in the US NH setting. The e-Delphi expert panel identified 24 criteria (22 STOPP and 2 START) with strong consensus on their potential for improving medication prescribing in the US NH setting. Experts agreed on the validity of the STOPP/START criteria and relevance to the target patient population; however, a greater challenge was the feasibility of applying the criteria to the data elements available in electronic NH databases. Nearly half of the original STOPP/START criteria required patient-level information from laboratory testing or physical patient assessment, which are generally not reliably captured outside the patient medical record.

Our study has a number of limitations. As with all research using the Delphi approach, the consensus decisions are linked to the panel expertise. First, although our panel was diverse and included nurses, pharmacists, physicians, and researchers with NH expertise, a panel with a different range of NH expertise may come to a different consensus decision. Second, not all participants answered round three questions due to time constraints. Even with attrition, our panel was larger than 9 people – panel size recommended by the RAND/UCLA Appropriateness Method. Third, while attrition is not common in in-person panels, 65% participation rate is adequate for online Delphi panels that often have ≤50% participation rates.^{(28, 29)}

Nevertheless, this list of medication use criteria offers an evidence-based and consensus generated tool that is uniquely tailored to the NH population of older US adults. A comparison of our modified STOPP/START list with the 2015 version of the American Geriatrics Society (AGS) Beers criteria of potentially inappropriate prescribing in community-dwelling older adults shows both substantial overlap and some differences (Supplemental Appendix S1). ⁽¹²⁾ The modified STOPP/START list includes 17 criteria that are also in the AGS Beers Criteria. The 5 unique STOPP criteria include: systemic corticosteroids instead of inhaled corticosteroids in chronic obstructive pulmonary disease, proton pump inhibitors >8 weeks, antimuscarinic drugs in chronic prostatism, metformin in end-stage renal disease or dialysis, and oral elemental iron doses of >200 mg/day. The START criteria are also unique because there are no other broad- based medication use tools for recommending the initiation of therapy in older adults. Our modified STOPP/START list includes criteria for the important clinical care domains of initiating of annual influenza vaccinations and the pneumococcal vaccination at least once if aged 65 or older. Lastly, we feel the assessment of measurement feasibility with information captured in widely used electronic NH databases has created a broadly useful subset of STOPP/START prescribing indicators.

In summary, this study offers an assessment of the STOPP/START criteria in terms of the validity, relevance to the US NH setting, and measurement feasibility. Using the e-Delphi method, a panel of NH experts identified 24 criteria with strong consensus on the potential for improving medication prescribing in the US NH setting. The formulation of expert-led, consensus-driven criteria on appropriate medication use is currently a developing area in LTC research. The originators of the STOPP/START criteria are currently convening their own Delphi panel of physicians and pharmacists to create a set of potentially inappropriate prescribing criteria for very frail older adults with less than 1 year survival. Future research should also validate the clinical impact on health outcomes in the NH population. We hope our adapted criteria are a useful tool for improving both the overuse of potentially inappropriate medications and the underuse of potentially beneficial medications among older adults living in NHs.

Supplementary Material

Supp Appendix S1

SUPPLEMENTAL APPENDIX S1: Comparison to original STOPP/START and Beers Criteria and E-delphi additional results