Study authors	Akce et al.52
Bias	Judgement	Support for judgement
Bias due to confounding	Moderate risk of bias	The study authors acknowledge the potential for confounding factors; however, these have not been controlled for by study design or statistical analyses.
Bias in selection of participants into the study	Low risk of bias	Randomised selection of participants into the study.
Bias in classification of interventions	No information	Classification of interventions not explicitly described.
Bias due to deviations from intended interventions	No information	Deviation from intended interventions of the study is not adequately described.
Bias due to missing data	Serious risk of bias	Attrition of data is described but not analysed.
Bias in measurement of outcomes	Low risk of bias	Double data abstraction and data entry to minimise bias in measurement of outcomes.
Bias in selection of the reported result	Serious risk of bias	Retrospective data recorded.
Overall quality assessment	Moderate risk of bias

Study authors	Auyong et al.25
Bias	Judgement	Support for judgement
Bias due to confounding	Low risk of bias	Potential risk of bias due to confounding accounted for by statistical analysis including logistic regression models
Bias in selection of participants into the study	Low risk of bias	Consecutive selection of participants into the study
Bias in classification of interventions	Low risk of bias	Intervention groups clearly defined
Bias due to deviations from intended interventions	No information	Potential deviations from intended interventions not described.
Bias due to missing data	Low risk of bias	No missing data reported.
Bias in measurement of outcomes	Moderate risk of bias	Outcome assessors not blinded to intervention status.
Bias in selection of the reported result	Moderate risk of bias	Multiple logistic regression models used to compare binary secondary outcomes which may increase the risk of bias arising from selective reporting of results.
Overall quality assessment	Low risk of bias

Study authors	Baehren et al.40
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not controlled for by study design or statistical analyses.
Bias in selection of participants into the study	Serious risk of bias	The study authors acknowledge that “Enrolment was based on a convenience sample “
Bias in classification of interventions	Low risk of bias	Clear classification of intervention groups.
Bias due to deviations from intended interventions	Serious risk of bias	As reported in the article, “the number of patients treated by each physician is not the same.”
Bias due to missing data	No information	Attrition of data not reported.
Bias in measurement of outcomes	Low risk of bias	“Subjects were identified only by the research assistants who reviewed the triage information patients were assigned a sequential number”
Bias in selection of the reported result	Moderate risk of bias	Primary outcomes clearly reported. Multiple outcome measurements within secondary outcome domains.
Overall quality assessment	Moderate risk of bias

Study authors	Benditz et al.24
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses. Patients informed of study but all received same intervention; health care professionals had increased experience with protocol over time.
Bias in selection of participants into the study	Low risk of bias	The study authors state “no patients were available on Mondays, which may also represent some kind of selection bias. …Wards to be visited were randomized daily by drawing a number to prevent selection bias.”
Bias in classification of interventions	Moderate risk of bias	Classification of interventions not clearly defined.
Bias due to deviations from intended interventions	No information	No information to describe potential deviations from intended interventions.
Bias due to missing data	Serious risk of bias	As acknowledged in the article, “We have no information about the excluded patients.”
Bias in measurement of outcomes	Low risk of bias	The study authors state “To avoid any interviewer–patient interaction bias, the nurse informed the patients that she was working independently from the health care team, that all information or judgment given in the interview would be treated confidentially, and that participation was voluntary. Data were anonymized after the interview.”
Bias in selection of the reported result	Low risk of bias	Primary and secondary outcome measurements reported in article tables.
Overall quality assessment	Moderate risk of bias

Study authors	Bingle et al.21
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses
Bias in selection of participants into the study	Serious risk of bias	Potential bias introduced by selection of participants into the study not controlled in the study.
Bias in classification of interventions	No information	Limited description of the differences between the 3 detailing interventions
Bias due to deviations from intended interventions	Moderate risk of bias	Intervention groups moderately defined, which may introduce a certain degree of bias due to deviations of intended interventions.
Bias due to missing data	No information	Attrition data not described.
Bias in measurement of outcomes	Serious risk of bias	Retrospective chart audit, limited description of process. Pre and post screening performed by different people.
Bias in selection of the reported result	Low risk of bias	Primary and secondary outcome measurements reported in article tables.
Overall quality assessment	Serious risk of bias

Study authors	Boothby et al.50
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses
Bias in selection of participants into the study	Serious risk of bias	Selection of participants into the study done by convenience sampling.
Bias in classification of interventions	Low risk of bias	Intervention groups clearly defined.
Bias due to deviations from intended interventions	No information	No information to describe potential deviations from intended interventions.
Bias due to missing data	No information	Attrition data not described.
Bias in measurement of outcomes	Low risk of bias	The study authors report that “data were normalized with regard to the varying hospital census”.
Bias in selection of the reported result	Low risk of bias	Primary and secondary outcome measurements reported in article tables.
Overall quality assessment	Moderate risk of bias

Study authors	Bos et al.34
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Co‐morbidities e.g. Charlson's and polypharmacy not accounted for as can increase risk of ADRs—although age, sex etc. were
Bias in selection of participants into the study	Serious risk of bias	Not just index admission, re‐admissions were included—can skew data as some can be predisposed to drug related problems; could not be adjusted for in analysis
Bias in classification of interventions	Low risk of bias	Clear classification of intervention groups, pre and post groups separated by 3 mo.
Bias due to deviations from intended interventions	No information	No information to describe potential deviations from intended interventions.
Bias due to missing data	No information	No attrition of data described.
Bias in measurement of outcomes	Low risk of bias	The authors of the study report that “By blinding all case record forms with respect to the study period before assessment by the experts and by correcting for confounders, the probability of bias was minimized.”
Bias in selection of the reported result	Low risk	Primary and secondary outcome measurements reported in article tables.
Overall quality assessment	Serious risk of bias

Study authors	Chan et al.26
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Postop D2–3 may be insufficient to titrate PRN agents to optimal analgesia so pain ratings may not reflect full potential of the intervention regimen Also unclear if patients aware of involvement in active intervention aimed at improving pain—may have placebo if patients were aware
Bias in selection of participants into the study	Serious risk of bias	Patients not randomised; recruitment periods were different lengths
Bias in classification of interventions	Low risk of bias	Fully prospective study
Bias due to deviations from intended interventions	No information	Not described.
Bias due to missing data	Serious risk of bias	The authors acknowledge that “there are some missing data in both surveys, especially the second questionnaire.” 29.5% missing data in the intervention group (Group B).
qBias in measurement of outcomes	Serious risk of bias	Surveys completed by patients; risk of deviation or subjective interpretation. Outcome assessors were not blinded.
Bias in selection of the reported result	No information	Not described.
Overall quality assessment	Serious risk of bias

Study authors	Chanques et al.36
Bias	Judgement	Support for judgement
Bias due to confounding	Low risk of bias	Accounted for delirium and intensive care unit handicaps in communicating pain. Healthcare professionals blinded to results of assessments in control phase.
Bias in selection of participants into the study	Low risk of bias	Prospective, consecutive recruitment of participants into the study.
Bias in classification of interventions	Low risk of bias	Clear classification of interventions.
Bias due to deviations from intended interventions	No information	Strategies to address potential deviations from intended interventions not adequately described.
Bias due to missing data	Moderate risk of bias	Attrition data described, but no analysis was performed.
Bias in measurement of outcomes	Serious risk of bias	Potential for subjective measurement of pain scores. Duration of intervention period was 8 weeks longer; bigger sample size (but interrater reliability established). Cannot blind healthcare professionals who administer RASS and BPS
Bias in selection of the reported result	No information	Not described.
Overall quality assessment	Moderate risk of bias

Study authors	Cui et al.32
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Postop main medication assessed by may be swayed as surgeons informed of intervention; cannot guarantee that they did not change practices per protocol—which nurses may have picked up during their rounds together
Bias in selection of participants into the study	Serious risk of bias	Selection of participants into the study by convenience sampling.
Bias in classification of interventions	Low risk of bias	Clear classification of interventions.
Bias due to deviations from intended interventions	No information	Strategies to address potential deviations from intended interventions not adequately described.
Bias due to missing data	Low risk of bias	16 and 15 nurses completed, though unclear how many patients were recruited then excluded due to missing data or other reasons
Bias in measurement of outcomes	Serious risk of bias	Questionnaire measurement of outcomes; potential for subjective interpretation and response.
Bias in selection of the reported result	No information	Not described.
Overall quality assessment	Moderate risk of bias

Study authors	Duncan and Pozehl31
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses
Bias in selection of participants into the study	Moderate risk of bias	Retrospective audit of patient medical records.
Bias in classification of interventions	Low risk of bias	Clear classification of interventions.
Bias due to deviations from intended interventions	No information	Strategies to address potential deviations from intended interventions not adequately described.
Bias due to missing data	Moderate risk of bias	Attrition data described, but no analysis was performed
Bias in measurement of outcomes	Serious risk of bias	Self‐reported pain intensity ratings; potential for subjective measurement.
Bias in selection of the reported result	No information	Not described.
Overall quality assessment	Moderate risk of bias

Study authors	Ferguson et al.54
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Confounding factors acknowledged but could not be controlled for due to the nature of the hospital setting.
Bias in selection of participants into the study	Low risk of bias	All patient‐controlled analgesia (PCA) errors in hospital over set period.
Bias in classification of interventions	Low risk of bias	Intervention groups clearly defined.
Bias due to deviations from intended interventions	Serious risk of bias	Acknowledged potential for deviations from intended interventions.
Bias due to missing data	No information	Missing data not reported.
Bias in measurement of outcomes	Low risk of bias	Data PCA errors were objective and carried a lower potential for subjective interpretation.
Bias in selection of the reported result	No information	Not described.
Overall quality assessment	Moderate risk of bias

Study authors	Humphries et al.27
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	No adjustments made to account for potential confounding factors.
Bias in selection of participants into the study	Low risk of bias	All eligible patients included in analysis.
Bias in classification of interventions	Low risk of bias	Intervention group clearly defined.
Bias due to deviations from intended interventions	No information	Potential deviations from intended interventions not described.
Bias due to missing data	Low risk of bias	Study authors report no attrition of data.
Bias in measurement of outcomes	Moderate risk of bias	Outcome assessors not blinded to intervention status.
Bias in selection of the reported result	No information
Overall quality assessment	Moderate risk of bias

Study authors	Juhl et al.28
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Measurement of pain scores depends on the day which postoperative patients were interviewed; was not consistent between patients nor between pre–post intervention periods.
Bias in selection of participants into the study	Moderate risk of bias	Prospective and consecutive patient inclusion. However, nurses' participation was voluntary, which is susceptible to volunteer bias.
Bias in classification of interventions	Low risk of bias	Prospective study and clear pre–post periods.
Bias due to deviations from intended interventions	Low risk	Intervention groups clearly defined to minimise deviation from intended interventions.
Bias due to missing data	Moderate risk of bias	Attrition described but no analysis performed.
Bias in measurement of outcomes	Moderate risk of bias	Nurses who assessed subjective pain could not be blinded; good that McGill Pain Questionnaire translated by 3 qualified people
Bias in selection of the reported result	Moderate risk of bias	Reporting of anaesthetists' vs surgeons' prescribing patterns were nonconsistent (converse figures reported)
Overall quality assessment	Moderate risk of bias

Study authors	Kjeldsen et al.33
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Diff number of patients treated with NSAIDs (one of the risk drugs) which can affect (opioid) analgesia requirements and possibly subsequent recs (Table 1)
Bias in selection of participants into the study	Low risk of bias	Controlled prospective study design, analysis performed on 2 groups to ensure similar cohorts.
Bias in classification of interventions	Low risk of bias	“To ensure standardized interventions, an information sheet was developed for each of the risk medications”
Bias due to deviations from intended interventions	No information	Strategies to address potential deviations from intended interventions not adequately described.
Bias due to missing data	Moderate risk of bias	Reasons for exclusion; differs slightly between the 2 groups
Bias in measurement of outcomes	Low risk of bias	Comprehensive reporting of outcomes
Bias in selection of the reported result	Low risk of bias	Selection of reported results addressed
Overall quality assessment	Serious risk of bias

Study authors	Majumder et al.29
Bias	Judgement	Support for judgement
Bias due to confounding	Moderate risk of bias	Confounding variables are addressed to an extent by study design (e.g. restriction of included patients).
Bias in selection of participants into the study	Low risk of bias	Consecutive selection of participants into the study.
Bias in classification of interventions	Low risk of bias	Intervention groups clearly defined.
Bias due to deviations from intended interventions	No information	Potential deviations from intended interventions not described.
Bias due to missing data	Low risk of bias	Study authors report no attrition of data.
Bias in measurement of outcomes	Moderate risk of bias	Outcome assessors not blinded to intervention status of the participant.
Bias in selection of the reported result	Low risk of bias	Reported results reflect predefined primary and secondary outcome measures.
Overall quality assessment	Low risk of bias

Study authors	Manias et al.45
Bias	Judgement	Support for judgement
Bias due to confounding	Moderate risk of bias	Potential confounding factors addressed to an extent by study design.
Bias in selection of participants into the study	Low risk of bias	Consecutive patient recruitment into the study.
Bias in classification of interventions	Low risk of bias	Clear classification of interventions.
Bias due to deviations from intended interventions	No information	Strategies to address potential deviations from intended interventions not adequately described.
Bias due to missing data	Low risk of bias	No missing data reported.
Bias in measurement of outcomes	Low risk of bias	The study authors report “The research assistant who collected data from both groups was blinded to group assignment. In addition, nurses in each hospital were not informed about their allocated group.”
Bias in selection of the reported result	No information	Not described.
Overall quality assessment	Low risk of bias

Study authors	McAllister et al.41
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses
Bias in selection of participants into the study	Low risk of bias	All eligible participants included into the study.
Bias in classification of interventions	Low risk of bias	Clear classification of interventions.
Bias due to deviations from intended interventions	No information	Strategies to address potential deviations from intended interventions not adequately described.
Bias due to missing data	No information	Missing data not described.
Bias in measurement of outcomes	Low risk of bias	Clear documentation of outcome measurement.
Bias in selection of the reported result	No information	Not described
Overall quality assessment	Moderate risk of bias

Study authors	McEvoy et al.56
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses
Bias in selection of participants into the study	Low risk of bias	Retrospective analysis of electronic medication orders.
Bias in classification of interventions	Low risk of bias	Clear classification of intervention groups.
Bias due to deviations from intended interventions	No information	Strategies to address potential deviations from intended interventions not adequately described.
Bias due to missing data	Low risk of bias	Study authors reported no missing data.
Bias in measurement of outcomes	Low risk of bias	Clear documentation of outcome measurement.
Bias in selection of the reported result	Serious risk of bias	The authors acknowledge that “the duration of postintervention assessment and sample size were not the same as the preintervention assessment. This may have affected the overall results due to variance in prescribing patterns.”
Overall quality assessment	Moderate risk of bias

Study authors	Morisson et al.20
Bias	Judgement	Support for judgement
Bias due to confounding	Low risk of bias	Study design involved propensity score matching with control participants. Statistical analyses to control for confounding factors included logistic regression models.
Bias in selection of participants into the study	Low risk of bias	All eligible participants were enrolled in the study.
Bias in classification of interventions	Low risk of bias	Intervention groups clearly defined.
Bias due to deviations from intended interventions	No information	Potential deviations from intended interventions not described.
Bias due to missing data	Low risk of bias	Details of subject enrolment clearly outlined and no other missing data reported.
Bias in measurement of outcomes	Low risk of bias	The study was double‐blinded. Participants were not informed as to whether they were enrolled in the intervention or control unit and outcome assessors were blinded to participant group.
Bias in selection of the reported result	Low risk of bias	Primary and secondary outcome measurements reported in article tables.
Overall quality assessment	Low risk of bias

Study authors	Moustafa et al.39
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses
Bias in selection of participants into the study	Low risk of bias	Prospective, consecutive selection of participants into the study.
Bias in classification of interventions	Moderate risk of bias	Classification of interventions not explicitly described.
Bias due to deviations from intended interventions	No information	Strategies to address potential deviations from intended interventions not adequately described.
Bias due to missing data	Low risk of bias	No attrition of data (100% pain assessment performed)
Bias in measurement of outcomes	Serious risk of bias	The study authors acknowledge “ED personnel were aware (in the second phase only) of the main outcome of the study and it is likely that the Hawthorne effect had an impact on pain assessment, but it did not so much on analgesics treatment.”
Bias in selection of the reported result	No information	Not described.
Overall quality assessment	Serious risk of bias

Study authors	Muntlin et al.43
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses
Bias in selection of participants into the study	Serious risk of bias	The authors report “Patients were approached to take part in the study 24 h a day, weekdays and weekends until the desired number of patients had been included.” Convenience sampling may introduce bias into the study.
Bias in classification of interventions	Serious risk of bias	Clear classification of interventions, however, phases B and A2 were not as explicitly described as A1.
Bias due to deviations from intended interventions	No information	No information to describe potential deviations from intended interventions.
Bias due to missing data	Moderate risk of bias	Excluded data is described, however, differences between included and excluded patients were not analysed
Bias in measurement of outcomes	Serious risk of bias	Questionnaire reporting of outcomes; potential for discrepancies between researchers
Bias in selection of the reported result	No information	Not described.
Overall quality assessment	Serious risk of bias

Study authors	Neitzel et al.22
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses. Patient pain experience may affect length of stay.
Bias in selection of participants into the study	Serious risk of bias	Selection of participants into the study by convenience sampling.
Bias in classification of interventions	Low risk of bias	Intervention groups clearly described.
Bias due to deviations from intended interventions	No information	Strategies to address potential deviations from intended interventions not adequately described.
Bias due to missing data	Low risk of bias	Reasonably complete data collection
Bias in measurement of outcomes	Moderate risk of bias	Outcome assessors not blinded to the intervention status of the participants. Provider survey is potentially subjective.
Bias in selection of the reported result	No information	Not described
Overall quality assessment	Moderate risk of bias

Study authors	Netherton et al.42
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	The authors acknowledge “We have not been able to adjust for confounders; so the increase in ketorolac use seen in Calgary EDs may be due to secular changes, that is, physician knowledge about ketorolac use for acute pain, or due to an influx of newly trained ED physicians who may have been familiar with its use in their residency training programs”
Bias in selection of participants into the study	Low risk of bias	Random selection of participants into the study.
Bias in classification of interventions	Low risk of bias	Clear classification of interventions.
Bias due to deviations from intended interventions	No information	Not described.
Bias due to missing data	Moderate risk of bias	Attrition data described, but no analysis performed; “Excluding patients arriving by Emergency Medical Service may have led to the exclusion of the most severe presentations of renal colic.”
Bias in measurement of outcomes	Moderate risk of bias	Outcome assessors were not blinded to the intervention status of the study participants.
Bias in selection of the reported result	No information	Not described.
Overall quality assessment	Moderate risk of bias

Study authors	Olsen et al.37
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	The authors acknowledge “A pre–post intervention design has weaknesses, as confounding variables could influence the outcome, but differences in baseline variables between groups were controlled, using regression analysis”
Bias in selection of participants into the study	Serious risk of bias	Non‐consecutive recruitment of eligible patients into the study.
Bias in classification of interventions	Low risk of bias	Clear classification of interventions.
Bias due to deviations from intended interventions	No information	Potential deviation from intended interventions not adequately described.
Bias due to missing data	Moderate risk of bias	Attrition data described, but no analysis was performed.
Bias in measurement of outcomes	Serious risk of bias	Potential for subjective pain assessment; “we did not have independent observers assessing pain in the control group”
Bias in selection of the reported result	Serious risk of bias	No results re pain events or even any of the 3 pain tools; high risk of selective reporting
Overall quality assessment	Serious risk of bias

Study authors	Paul et al.53
Bias	Judgement	Support for judgement
Bias due to confounding	Low risk of bias	The authors report that potential confounding factors were addressed. “Cases where the cause of the critical incident was attributed to patient factors were excluded from this study”
Bias in selection of participants into the study	Low risk of bias	“Two reviewers (B.B., A.M.) obtained and examined all Hamilton Health Sciences Acute Pain Service (HHS APS) critical incident reports (n = 642) dating from 1 February 2002 to 28 February 2009. Each report was reviewed to determine whether the event involved a PCA setup, programming, or administration error. Any discrepancies were resolved through consensus or discussion with a third reviewer”
Bias in classification of interventions	Low risk of bias	Classification of interventions between intervention and control groups explicitly described.
Bias due to deviations from intended interventions	No information	Potential deviation from intended interventions not adequately described.
Bias due to missing data	No information	Missing data not described.
Bias in measurement of outcomes	Moderate risk of bias	Outcome assessors not blinded to the intervention status of participants.
Bias in selection of the reported result	No information	Not described
Overall quality assessment	Low risk of bias

Study authors	Pierik et al.44
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	The study authors acknowledge that “There are a number of important potential confounding factors, e.g. severity of injury, knowledge and experience of pain management, which were not measured and may have differed in both periods No adjustments could be made.”
Bias in selection of participants into the study	Low risk of bias	Consecutive patient recruitment of participants into the study.
Bias in classification of interventions	Low risk of bias	Clear classification of interventions.
Bias due to deviations from intended interventions	No information	Potential deviations from intended interventions not adequately described.
Bias due to missing data	Low risk of bias	No attrition of data.
Bias in measurement of outcomes	Serious risk of bias	The authors report “The percentage of patients who were actually administered analgesics might be underestimated. Even though the ED staff was instructed to list all medications, some may have neglected to do so, especially for over‐the‐counter analgesics.”
Bias in selection of the reported result	Low risk of bias	Objective selection of reported results described.
Overall quality assessment	Moderate risk of bias

Study authors	Shaw et al.51
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses
Bias in selection of participants into the study	Low risk of bias	Analysis of all eligible prescription error rates according to predefined criteria.
Bias in classification of interventions	Low risk of bias	Clear intervention methodology and classification described.
Bias due to deviations from intended interventions	Low risk of bias	Hard copy material provided to study participants, unlikely to produce deviation across participants.
Bias due to missing data	No information	Missing data not described.
Bias in measurement of outcomes	Low risk of bias	Quantitative analysis of prescription error rates.
Bias in selection of the reported result	Unclear	Not described
Overall quality assessment	Low risk of bias

Study authors	Titsworth et al.23
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses. Patient questionnaire performed, no blinding of patients described, potential for patients to answer more positively due to nonanonymity
Bias in selection of participants into the study	Low risk of bias	“Systematic random sampling was used to identify every 10th postoperative neurosurgical patient admitted preintervention and every 17th patient postintervention”
Bias in classification of interventions	Low risk of bias	Explicit classification of interventions.
Bias due to deviations from intended interventions	Low risk of bias	Intended interventions were clearly documented to minimise risk of bias introduced by deviation from intervention.
Bias due to missing data	Moderate risk of bias	Excluded data is described, however, differences between included and excluded patients were not analysed
Bias in measurement of outcomes	Serious risk of bias	“Nurses collected pain scores and were aware of the initiative; this could have potentially influenced patient‐reporting practices, resulting in response bias.”
Bias in selection of the reported result	No information	Not described.
Overall quality assessment	Moderate risk of bias

Study authors	Urfer et al.49
Bias	Judgement	Support for judgement
Bias due to confounding	Low risk of bias	Potential confounding factors addressed by restriction of participant sample to patients aged 65 years and above.
Bias in selection of participants into the study	Low risk of bias	Consecutive selection of participants into the study.
Bias in classification of interventions	Low risk of bias	Clear distinction between use and no use of 5‐point checklist.
Bias due to deviations from intended interventions	No information	Potential deviation from intended interventions not adequately described.
Bias due to missing data	Moderate risk of bias	Excluded data is described, however, differences between included and excluded patients were not analysed
Bias in measurement of outcomes	Low risk of bias	“Data were recorded on a standardized case report form and anonymized before statistical analysis.”
Bias in selection of the reported result	Moderate risk of bias	“it was not possible to blind the 2 investigators assessing medication appropriateness, because charts used for chart review contained hospitalization dates, dates of laboratory examinations etc.”
Overall quality assessment	Low risk of bias

Study authors	Usichenko et al.30
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	History of opioid use and opioid tolerance may affect pain experience and pain scores. Potential confounding factor(s) not adjusted for.
Bias in selection of participants into the study	Low risk of bias	Prospective, consecutive recruitment of participants into the study
Bias in classification of interventions	Low risk of bias	Clear classification of intervention groups.
Bias due to deviations from intended interventions	Low risk of bias	Interventions clearly defined and no deviation from intended interventions reported.
Bias due to missing data	Low risk of bias	91% and 85% survey completion rate; proportions similar across groups
Bias in measurement of outcomes	Moderate risk of bias	Potential for subjective reporting of pain in surveys using visual rating scale
Bias in selection of the reported result	Moderate risk of bias	Power calculations done
Overall quality assessment	Moderate risk of bias

Study authors	VanGulik et al.35
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Nurses not blinded to intervention; bedside manner or attention to detail/pain may become more astute as they gain more experience with pain medication especially over 15 mo.
Bias in selection of participants into the study	Unclear risk of bias	Unclear if patient selection was random or consecutive for total knee arthroplasty patients.
Bias in classification of interventions	Low risk of bias	Clear classification of intervention groups, prospective study design.
Bias due to deviations from intended interventions	Low risk of bias	The study authors report “the patients were neither aware of nor trained in the pain management programme and they were asked exactly the same question in both phases, though by different persons”
Bias due to missing data	No information	Only 1.3% of records missing from retrospective audit.
Bias in measurement of outcomes	Serious risk of bias	The study authors report “The pain scores in the control group were asked when patients were at rest, not undergoing interventions, which may have led to relatively low pain levels at that particular moment … .during the intervention phase, nurses were trained to be more alert of high NRS levels compared to the control phase”
Bias in selection of the reported result	No information	Not described
Overall quality assessment	Moderate risk of bias

Study authors	Whipple et al.55
Bias	Judgement	Support for judgement
Bias due to confounding	Serious risk of bias	Potential confounding factors not addressed by experimental design or statistical analyses
Bias in selection of participants into the study	Serious risk of bias	Retrospective review of PCA errors. No strategies to minimise selection bias reported.
Bias in classification of interventions	Low risk of bias	Clear classification of intervention groups.
Bias due to deviations from intended interventions	Low risk of bias	Objective review of PCA errors using defined criteria.
Bias due to missing data	Low risk of bias	No attrition of data due to the nature of the study.
Bias in measurement of outcomes	No information	Measures to reduce bias in outcome measurement not described
Bias in selection of the reported result	No information	Not described.
Overall quality assessment	Moderate risk of bias