Table 1.

Outcome categories for de-implementation effectiveness

Name	Rationale and definitions	Examples
Health outcomes	De-implementing a clinical practice should improve (or at least have no negative effect on) health outcomes. Health outcomes can therefore be considered measuring the safety of de-implementation	Mortality, morbidity, quality of life, symptoms
Low-value care use	The primary aim of a de-implementation intervention is to reduce low-value care. Predefined low-value care use should therefore be (one of) the primary outcome(s) of de-implementation effectiveness. Typically, the definition of low-value care is based on diagnoses or clinical criteria that represent low-value care in combination with a specific clinical practice. Data is often gathered from individual patient records or administrative databases. Individual patient records usually contain more specific information on clinical decisions and may therefore yield more accurate information	Antibiotic use for viral upper respiratory infections Use of radiological imaging in patients with acute low back pain without “red-flag” symptoms
Appropriate care use	Can be used as an outcome when a medical practice can be either appropriate or inappropriate. For instance, in patients with respiratory infection, use of antibiotics can be either appropriate or inappropriate. Change in appropriate care use measures unintended consequences of de-implementation and can therefore be considered as a measure of safety of de-implementation	Antibiotic use for confirmed pneumonia Use of radiological imaging in patients with low back pain and “red-flag” symptoms
Total volume of care	Total volume includes both appropriate and inappropriate care and is an indirect measure of low-value care. It may sometimes be justifiable to use in very large samples if it is impossible to differentiate between appropriate and inappropriate care and if using individual patient records is not possible. Outcomes that are based on diagnoses often include both appropriate and inappropriate care and should therefore be considered as total volume care, not as low-value care, outcomes	Total use of antibiotics in upper respiratory tract infections Use of radiological imaging in low-back pain
Intention to reduce the use of low-value care	Intention is the first step to change but does not reliably describe actual change in use of low-value care. As intention can be measured earlier than other outcomes, it may sometimes be justifiable to use as a preliminary assessment of the effectiveness of a de-implementation intervention. It is often used after educational interventions and when the data is gathered through surveys	Intention to reduce the use of inappropriate antibiotic use in upper respiratory tract infections Intention to reduce use of inappropriate radiological imaging in low-back pain