Table 1.

Inclusion criteria and typical examples of exclusions

Inclusion criteria	Typical exclusion examples
Population
The system is primarily intended for use by healthcare professionals (including clinicians and nonclinicians eg, managers)	Websites primarily intended to help patients choose healthcare provider
Intervention
The system provides clinical performance feedback to healthcare professionals. Clinical performance includes compliance with pre-defined clinical standards, as well as patient outcomes	Systems that provide feedback primarily regarding nonclinical performance, for example, care costs, patient access, and epidemiological surveillance
Clinical performance data are obtained from medical records, computerized databases, or observations from patients	Clinical performance feedback systems based on peer or supervisor observation
Feedback relates to multiple patients	Highly specific systems that only provide data for a single patient
Feedback to inform quality improvement actions at individual, team, or organizational levels	Intensive care unit dashboards that summarize patients’ current clinical status to primarily inform bedside or point-of care decisions
Feedback is provided via a dynamic interface with which the user can interact, (eg, a web-based portal or desktop application)	Feedback primarily provided on paper, verbally or via static interfaces such as screensavers, e-mail, or electronic documents
Providing clinical performance feedback is a core and essential function of the system, that is, in systems with additional functionalities, it is unlikely these would be offered in the absence of such feedback	Point-of-care reminder systems that additionally provide clinical performance feedback once per year
Outcome
The system primarily aims to improve clinical performance (as defined above)	Systems primarily intended to reduce costs
Study type
Empirical research evaluation studies of systems being used by healthcare professionals as target end-users, reporting findings from primary data collection and analysis (either qualitative or quantitative) focusing on the behavior of end-users using the system, outcomes of their behavior from using the system, or performance of the system	Articles reporting system descriptions, or studies conducted with members or the system development or research team
Peer-reviewed publications in scholarly journals, written in English with abstracts available for review	Conference abstracts, theses, gray literature, and non-English literature