Interventions to reduce waiting times for elective procedures

Abstract

Background

Long waiting times for elective healthcare procedures may cause distress among patients, may have adverse health consequences and may be perceived as inappropriate delivery and planning of health care.

Objectives

To assess the effectiveness of interventions aimed at reducing waiting times for elective care, both diagnostic and therapeutic.

Search methods

We searched the following electronic databases: Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register, the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE (1946‐), EMBASE (1947‐), the Cumulative Index to Nursing and Allied Health Literature (CINAHL), ABI Inform, the Canadian Research Index, the Science, Social Sciences and Humanities Citation Indexes, a series of databases via Proquest: Dissertations & Theses (including UK & Ireland), EconLit, PAIS (Public Affairs International), Political Science Collection, Nursing Collection, Sociological Abstracts, Social Services Abstracts and Worldwide Political Science Abstracts. We sought related reviews by searching the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effectiveness (DARE). We searched trial registries, as well as grey literature sites and reference lists of relevant articles.

Selection criteria

We considered randomised controlled trials (RCTs), controlled before‐after studies (CBAs) and interrupted time series (ITS) designs that met EPOC minimum criteria and evaluated the effectiveness of any intervention aimed at reducing waiting times for any type of elective procedure. We considered studies reporting one or more of the following outcomes: number or proportion of participants whose waiting times were above or below a specific time threshold, or participants' mean or median waiting times. Comparators could include any type of active intervention or standard practice.

Data collection and analysis

Two review authors independently extracted data from, and assessed risk of bias of, each included study, using a standardised form and the EPOC 'Risk of bias' tool. They classified interventions as follows: interventions aimed at (1) rationing and/or prioritising demand, (2) expanding capacity, or (3) restructuring the intake assessment/referral process.

For RCTs when available, we reported preintervention and postintervention values of outcome for intervention and control groups, and we calculated the absolute change from baseline or the effect size with 95% confidence interval (CI). We reanalysed ITS studies that had been inappropriately analysed using segmented time‐series regression, and obtained estimates for regression coefficients corresponding to two standardised effect sizes: change in level and change in slope.

Main results

Eight studies met our inclusion criteria: three RCTs and five ITS studies involving a total of 135 general practices/primary care clinics, seven hospitals and one outpatient clinic. The studies were heterogeneous in terms of types of interventions, elective procedures and clinical conditions; this made meta‐analysis unfeasible.

One ITS study evaluating prioritisation of demand through a system for streamlining elective surgery services reduced the number of semi‐urgent participants waiting longer than the recommended time (< 90 days) by 28 participants/mo, while no effects were found for urgent (< 30 days) versus non‐urgent participants (< 365 days).

Interventions aimed at restructuring the intake assessment/referral process were evaluated in seven studies. Four studies (two RCTs and two ITSs) evaluated open access, or direct booking/referral: One RCT, which showed that open access to laparoscopic sterilisation reduced waiting times, had very high attrition (87%); the other RCT showed that open access to investigative services reduced waiting times (30%) for participants with lower urinary tract syndrome (LUTS) but had no effect on waiting times for participants with microscopic haematuria. In one ITS study, same‐day scheduling for paediatric health clinic appointments reduced waiting times (direct reduction of 25.2 days, and thereafter a decrease of 3.03 days per month), while another ITS study showed no effect of a direct booking system on proportions of participants receiving a colposcopy appointment within the recommended time. One RCT and one ITS showed no effect of distant consultancy (instant photography for dermatological conditions and telemedicine for ear nose throat (ENT) conditions) on waiting times; another ITS study showed no effect of a pooled waiting list on the number of participants waiting for uncomplicated spinal surgery.

Overall quality of the evidence for all outcomes, assessed using the GRADE (Grades of Recommendation, Assessment, Development and Evaluation) tool, ranged from low to very low.

We found no studies evaluating interventions to increase capacity or to ration demand.

Authors' conclusions

As only a handful of low‐quality studies are presently available, we cannot draw any firm conclusions about the effectiveness of the evaluated interventions in reducing waiting times. However, interventions involving the provision of more accessible services (open access or direct booking/referral) show some promise.

Plain language summary

Effects of interventions to reduce waiting times for non‐urgent health procedures

Long waiting times for non‐urgent procedures are common in public healthcare systems, where care is provided free of charge and supply is limited by budget constraints. This may cause distress among patients as well as adverse health consequences.

We reviewed the evidence on the effects of interventions in reducing waiting times. We found eight eligible studies (three randomised controlled trials and five interrupted time series studies) involving 135 primary care clinics, seven hospitals and one outpatient clinic. Different interventions, elective procedures and clinical conditions across included studies made pooling of data unfeasible. The quality of the included evidence (to November 2013) ranged from low to very low, as data were obtained from randomised controlled trials that for the most part suffered from serious bias, and from non‐randomised studies without a control group.

The single study that evaluated an intervention aimed at prioritising demand showed that introducing a system for streamlining elective surgery reduced the number of semi‐urgent patients waiting longer than recommended, but did not affect urgent or non‐urgent groups.

Seven studies evaluated interventions aimed at restructuring the intake assessment/referral process. Three of four studies evaluating effects of open access or direct booking/referral showed beneficial effects: One study showed reduced waiting times for open access to sterilisation through keyhole surgery; another showed that open access to investigative services may lead to reduced waiting times for patients with urinary symptoms (but not for patients with microscopic blood in urine); and one study reported that same‐day scheduling reduced waiting times for those seeking child health outpatient services. One study showed no effect of a direct booking system on the proportion of patients reported to have moderate or severe cell changes on the neck of the womb who received an appointment for further investigation within four weeks.

Two studies of distant consultancy (instant photography for skin conditions and telemedicine for ear, nose and throat conditions) showed no effect on waiting times to see a specialist. One study reported that using a pooled waiting list did not change the number of patients waiting for routine back surgery within the recommended time. We found no studies evaluating interventions aimed at increasing capacity or rationing demand.

As only a handful of low‐quality studies are presently available, we cannot draw any firm conclusions about the effectiveness of the evaluated interventions in reducing waiting times. However, interventions involving the provision of more accessible services (open access or direct booking/referral) show some promise.

Summary of findings

Summary of findings for the main comparison. Summary of findings: interventions aimed at rationing and/or prioritising demand.

Interventions aimed at rationing and/or prioritising demand compared with no intervention
Patient or population: patients scheduled for elective surgery Settings: hospital surgery units Intervention: introduction or suspension of an intervention aimed at prioritising demand Comparison: no intervention
Outcomes	Effect measure	Number of studies (hospitals/practices/health professionals/participants)	Quality of the evidence (GRADE)	Comments
Introduction of interventions aimed at rationing and/or prioritising demand
Number of participants waiting longer than recommended	Number of participants waiting < 90 days: change in slope: ‐27.99 participants/mo (SE 8.58, P value 0.002); change in level: +32.55 participants (SE 54.65, P value 0.55) Number of participants waiting < 30 days: change in slope: ‐1.03 participants/mo (SE 0.51, P value 0.049); change in level: ‐5.40 participants (SE 6.44, P value 0.41) Number of participants waiting < 365 days: change in slope: ‐1.62 participants/mo (SE 2.96, P value 0.59); change in level: +5.50 participants (SE 11.83, P value 0.64)	1 reanalysed ITS study at 1 public hospital	⊕⊝⊝⊝ Very lowa	A single study of very low quality; impossible to draw any conclusions about the effectiveness of the intervention
GRADE Working Group grades of evidence. High quality: Further research is very unlikely to change our confidence in the estimate of effect. Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. Very low quality: We are very uncertain about the estimate. aOne non‐randomised study downgraded for high risk of bias (high risk that the intervention is dependent on other changes and unclear risk of reporting bias).

Summary of findings 2. Summary of findings: interventions aimed at restructuring referral processes.

Interventions aimed at restructuring referral processes compared with no intervention
Patient or population: patients needing elective specialist ambulatory visit or surgery Settings: hospitals and primary care practices Intervention: restructuring referral processes Comparison: no intervention
Outcomes	Effect measure	Number of studies (hospitals/practices/health professionals/participants)	Quality of the evidence (GRADE)	Comments
Direct/open access and direct booking systems
Waiting time	Not possible to give a pooled estimate. 1 RCT showed a reduction in waiting time only for participants with lower urinary tract syndrome (ratio of means 0.7, 95% CI 0.5 to 0.9) and not for those with microhaematuria; the other RCT showed a reduction in waiting time for participants randomly assigned to direct laparoscopic sterilisation if compared with standard procedure (108 vs 127 days, P value 0.003)	2 RCTs (123 general practices and 2 hospitals/1191 participants)	+ + ⊝⊝ Lowa	Only 2 RCTs available targeting different elective procedures. Difficult to draw any conclusions about effectiveness or generalisability
Proportion of participants waiting below a recommended time threshold	Proportion of participants waiting less than 4 weeks: change in level: ‐14.26%, P value 0.50; change in slope: +6.29% each 3 months, P value 0.62	1 reanalysed ITS study (1 community primary care unit and 1 public hospital/2501 women).	+⊝⊝⊝ Very lowb	Extremely scarce evidence of very low quality, impossible to draw any conclusions
Waiting time	Waiting time (days): change in level: ‐25.20 days, SE 3.83, P value < 0.001; change in slope: ‐3.03 days/mo, SE 0.92, P value 0.005	1 reanalysed ITS study (1 outpatient clinic/7594 appointments)	+⊝⊝⊝ Very lowc	Scarce evidence of low quality; impossible to draw any conclusions
Distant consultancy
Waiting time	Waiting time: mean 55 days (SD = 40, P value >0.05)	1 RCT (1 hospital, 10 general practices/136 participants)	+ + ⊝⊝ Lowd	Scarce evidence of low quality; impossible to draw any conclusions.
Waiting time	Waiting time: change in level: ‐0.69 months (SE 0.55, P value 0.23); change in slope: ‐0.21 months each year (SE 0.13, P value 0.15)	1 reanalysed ITS study (1 ENT clinic/1690 participants)	+⊝⊝⊝ Very lowe	Scarce evidence of very low quality; impossible to draw any conclusions
Generic waiting list
Number of participants waiting less than recommended time threshold or within a recommended time period	Number of participants waiting less than 9 months: change in level: ‐20.59 (SE 22.67, P value 0.37); change in slope: 2.75 participants each month (SE 12.69, P value 0.86) Number of participants waiting between 9 and 18 months: change in level: ‐5.28 (SE 16.20, P value 0.75); change in slope: ‐6.59 participants each month (SE 8.73, P value 0.46)	1 reanalysed ITS study (1 hospital)	+⊝⊝⊝ Very lowf	Scarce evidence of low quality; impossible to draw any conclusions
GRADE Working Group grades of evidence. High quality: Further research is very unlikely to change our confidence in the estimate of effect. Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. Very low quality: We are very uncertain about the estimate. aOne RCT with low risk of bias and one RCT with high risk of bias (due to high risk of attrition and contamination bias) and indirectness (low applicability of the intervention); according to the GRADE rule, the overall quality of evidence for this outcome is that of the trial with the lowest quality (i.e. low). bOne non‐randomised study downgraded for high risk of bias due to high risk that the intervention is dependent on other changes. cOne non‐randomised study downgraded for high risk of bias due to intervention affecting data collection, risk of attrition and reporting bias. dOne RCT with unclear risk of bias (high risk of attrition bias) and imprecision of results. eOne non‐randomised study downgraded for high risk of bias due to high risk of reporting bias. fOne non‐randomised study downgraded for high risk of bias due to unclear risk of intervention not being independent of other changes and having affected data collection, and unclear risk of attrition and reporting bias.

Background

Description of the condition

Elective health procedures are procedures that are programmed and are not delivered in emergency or urgent situations. Long waiting times for elective health procedures are observed in most health systems and are thought to occur when demand exceeds supply. They tend to occur in health systems that combine public health insurance with zero or low patient cost‐sharing and constraints on capacity (Siciliani 2013); some view them as a structural feature of those systems (Harrison 2000; Hurst 2003; Kreindler 2010; Siciliani 2013). The mere existence of waiting lists is not necessarily a negative phenomenon, as it is sometimes considered a structural and inevitable way of rationing scarce supply (Appleby 2011; Black 2004; Lindsay 1984). Total absence of waiting times would certainly cause dysfunction, as planning of activities would be jeopardised and efficiency would greatly suffer if services were not used at their full capacity.

However, long waiting times can cause distress for patients, in some cases can have adverse health consequences (Kreindler 2010) and are perceived by patients, the public and policy‐makers as lack of appropriate delivery and planning of health care (Kreindler 2010). This explains in part why long waiting times tend to catalyse the tensions between patients' and citizens' expectations on the one side, and health care providers' ability to meet those expectations on the other. The question is therefore how to find a way to keep waiting times at a safe and acceptable level while ensuring quality, equity and wise use of resources (Kreindler 2010).

Extended studies of this phenomenon have not pinpointed specific health system determinants of waiting lists (Hurst 2003; Siciliani 2003; Siciliani 2013). Besides a link between universalistic access to care and long waiting times, researchers have found associations with lower levels of health expenditure and lessened availability of curative beds and of physicians. However some data contradict these associations, and long waiting times are found in countries where health expenditure and availability of services and resources are high. Great variation between countries demonstrates that it is still very difficult to advance tenable inferences on causal relationships, and only descriptive associations can be put forward.

Evidence on how waiting times for elective procedures affect health outcomes appears less conclusive than for urgent procedures (Hirvonen 2007). This could be explained by the fact that patients on a waiting list whose health deteriorates while waiting tend to be shifted to an emergency list and "lost" from the elective procedure waiting lists.

Description of the intervention

Analysis of determinants and implications of waiting times, as well as of the impact of policies targeted at their reduction, has been the object of a number of reviews (Appleby 2005a; Appleby 2011; Harrison 2000; Hurst 2003; Kipping 2002; MacMillan Press 1993; Siciliani 2003; Siciliani 2013; Yates 1987).

Comparative data between countries and health systems experiencing or not experiencing long waiting times for elective procedures do not lend themselves to hypotheses that go beyond the imbalance between supply and demand. However causes of inadequate supply or excessive demand can be numerous and heterogeneous. An inadequate supply can be due to insufficient capacity or inefficient use of existing capacity, which in turn can be affected by several factors such as important changes in technology (Siciliani 2013). Decreasing healthcare demand is quite problematic; probably only effective prevention of illnesses and/or their effective management within primary care would reduce the number of patients who need treatment. Clinical uncertainty and variations in clinical practice have been associated with inappropriate demand.

During the past decade, waiting time guarantees have become a frequent and popular policy tool to address waiting times. Recommended minimum waiting times are established for different types of elective procedures, and hospital incentives or penalties are associated with meeting the set targets. This policy has been successful in England, where publication of waiting times data was coupled with sanctions for poorly performing hospital managers. This strategy appeared to reduce maximum waiting times (Appleby 2005b; Dash 2004; Department of Health 2002; Mayor 2003; Propper 2008), but, as it happened in other countries, it did not seem to affect average waiting times. To meet target waiting time as set by policies, health systems and organisations adopt specific interventions to reach the set target. These can be of three main types: interventions aimed at rationing and/or prioritising demand, at expanding capacity and at improving the organisational management of waiting lists or restructuring the intake assessment/referral process. Examples of interventions aimed at rationing and/or prioritising demand include patients' financial contributions to healthcare services (co‐payment), development and implementation of explicit referral criteria or practice guidelines to increase appropriateness and use of tools that triage patients according to their clinical conditions (clinical priority scores). Interventions aimed at expanding capacity comprise additional funding to increase appointment slots in the public sector, thus subsidising or facilitating access to the private sector. Examples of interventions aimed at restructuring the intake assessment/referral process include queuing strategies, redesign of clinical pathways, open access (patients seen without an appointment), direct booking/referral (specialty visits booked directly by patients), pooled waiting lists (single and pooled waiting lists for different consultants, with patients assigned to the first available appointment) and telemedicine, among others.

All of the above strategies have been used alone or in combination (Hurst 2003) in different settings and countries with heterogeneous results.

How the intervention might work

A lengthy waiting time is thought to result from a misalignment between the demand for procedures as expressed by citizens and the capacity of health systems to supply such procedures in adequate number and time. Therefore interventions implemented to enforce national or regional policies for reduction of waiting times may act on increasing supply or on reducing demand.

Interventions to increase supply involve raising funding/expenditures to buy additional personnel, equipment or time slots for extra numbers of procedures; to provide incentives for extra activity; or to buy extra activity from other providers.

Other interventions include improving efficiency and shortening patients' pathways by eliminating redundancies or obstacles in the process of care (Kreindler 2010).

Interventions to reduce demand include taking actions to discourage inappropriate requests and promote appropriate use of procedures and prioritising patients, while taking into consideration both clinical (e.g. severity of condition, expected benefit, need, urgency) and non‐clinical (e.g. ability to work) parameters (Kreindler 2010).

Why it is important to do this review

Numerous reports (Rachlis 2005; Siciliani 2013; Willcox 2007) and reviews (Kreindler 2010; Miller 2008) have sought to assess and compare the impact of different national policies for regulating and containing length of waiting times, using mainly national administrative data. These reports represent a fundamental contribution to the debate on management of waiting times and waiting lists, but they rarely provide evidence on the effectiveness of interventions. It is therefore important to summarise and evaluate existing evidence to identify what interventions are most effective in reducing waiting times for elective procedures.

Objectives

To assess the effectiveness of interventions aimed at reducing waiting times for elective care, both diagnostic and therapeutic.

Methods

Criteria for considering studies for this review

Types of studies

We considered randomised controlled trials (RCTs), controlled before‐and‐after studies (CBAs) and interrupted time series (ITS) designs that met the minimum criteria used by the Cochrane Effective Practice and Organisation of Care Group (EPOC) (EPOC 2013). We considered ITS studies as eligible if they had a clearly defined point in time when the intervention occurred, and at least three data collection points before and after the intervention (Ramsay 2003). We included CBAs if they involved at least two (intervention and/or control) sites (EPOC 2013). We included inappropriately analysed ITS studies if they reported data (in graphical or table format) that could be used to reanalyse data while taking into account possible secular trends in the analysis.

Types of participants

We included healthcare providers of any discipline/specialty area and patients referred to any type of elective diagnostic or therapeutic procedure.

Types of interventions

We considered any type of regulatory/administrative, economic, clinical or organisational intervention aimed at reducing waiting times for access to elective diagnostic or therapeutic procedures. We classified interventions according to the following taxonomy.

• Interventions aimed at rationing and /or prioritising demand (e.g. co‐payment, explicit referral criteria or practice guidelines, clinical priority scores, waiting time cap strategies).

• Interventions aimed at expanding capacity (e.g. additional funding to the public sector, ways of subsidising or facilitating access to the private sector).

• Interventions aimed at restructuring the intake assessment/referral process (e.g. different queuing strategies, theatre management strategies, other resource sharing strategies, remuneration schemes, direct access, open access, telemedicine).

We considered as a comparator standard practice (i.e. no intervention) or any kind of active intervention aimed at reducing waiting time.

Types of outcome measures

Commonly reported measures include mean and median waiting times, measured at different points in the patient pathway, which can be the moment at which patients see their general practitioner, the time of referral, the time patients are put on a waiting list, the time at which they undergo the elective procedure or the time they are discharged from hospital. Mean and median waiting times are considered reliable measures, although the mean can be influenced by a small number of patients with long waiting times and tends to be above the median (Siciliani 2013).

We restricted the review to include only studies that provided an objective measure of the impact of the interventions considered, expressed in terms of:

• number or proportion of participants whose waiting times were above or below a specified or recommended time threshold; or

• participants' mean or median waiting times for elective procedures.

We considered safety outcomes, that is, any health outcomes of participants (e.g. mortality, morbidity, complication rates), as well as costs.

Search methods for identification of studies

M. Fiander, Trials Search Co‐ordinator (TSC) for the EPOC Group, wrote the search strategies, in consultation with the review authors. The TSC searched the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects (DARE) for related systematic reviews, as well as the databases listed below for primary studies. Major databases were searched in November 2013; other databases, from which the identification of trials is less likely, were searched in November 2012 (see notations below); exact search dates for each database are included with the search strategies in Appendix 1 (MEDLINE) and Appendix 2 (other).

Neither date nor language limits were applied. Two methodological search filters were used to limit retrieval to appropriate study designs: the Cochrane Highly Sensitive Search Strategy (sensitivity‐ and precision‐maximising version, 2008 revision) (Higgins 2011) to identify randomised trials; and an EPOC methodology filter to identify non‐RCT designs.

Databases

• MEDLINE (1946‐2012), In‐Process and other non‐indexed citations, Ovid SP.

• EMBASE (1947‐2012), Ovid SP.

• Cochrane Central Register of Controlled Trials Evidence‐Based Medicine (EBM) Reviews, Ovid.

• Cumulative Index to Nursing and Allied Health Literature (CINAHL), EbscoHost (1980‐2012).

• EPOC Register, Reference Manager.

• Dissertations and Theses Full Text, UK and Ireland ProQuest.

• Health Technology Assessment, Fourth Quarter 2013, EBM Reviews, Ovid.

• National Health Service (NHS) Economic Evaluation Database, Fourth Quarter 2013, EBM Reviews, Ovid.

• PAIS (Public Affairs International), ProQuest.

• Science, Social Sciences and Humanities Citation Indexes, Conference Proceedings, Web of Science (e.g. Web of Knowledge).

• ABI Inform (January 2013).

• Canadian Research Index (November 2012).

• Communication Disorders Database, ProQuest (November 2012).

• Political Science Collection, ProQuest (November 2012).

• Nursing and Allied Health Source, ProQuest (November 2012).

• Sociological Abstracts and Social Services Abstracts, ProQuest (November 2012).

• Worldwide Political Science Abstracts, ProQuest (November 2012).

Searching other resources

Trial registries

• World Health Organization International Clinical Trials Registry Platform (ICTRP) (http://apps.who.int/trialsearch/AdvSearch.aspx).

Grey literature

We undertook a grey literature search that included, but was not limited to, the following sites.

• AHRQ (Agency for Healthcare Research and Quality) (http://www.ahrq.gov/) (November 2013).

• Centre for Health Services and Policy Research (CHSPR) (http://www.chspr.ubc.ca/pubs/pub‐search) (November 2013).

• Centre for Health Economics and Policy Analysis, McMaster University (CHEPA) (http://www.chepa.org/ ) (November 2013).

• Health Quality Council (HQC), University of Saskatchewan (November 2013).

• Institute for Clinical Evaluative Sciences (ICES) (http://www.ices.on.ca/ ) (November 2013).

• Institute of Health Economics (http://www.ihe.ca/publications/library/) (November 2013).

• Ontario Health Technology Advisory Committee (OHTAC) (http://www.health.gov.on.ca/english/providers/program/ohtac/tech/recommend/rec_mn.html ) (November 2013).

• Organisation for Economic Co‐operation and Development (OECD) (www.oecd.org) (November 2013).

• Public Health Agency of Canada (PHAC) (www.phac‐aspc.gc.ca) (November 2013).

• World Health Organization (WHO) (http://www.euro.who.int/en/what‐we‐do/data‐and‐evidence/health‐evidence‐network‐hen/publications ) (November 2013).

We also searched lists of references from relevant studies and systematic reviews; and contacted the authors of all eligible studies to ask about other relevant studies.

Data collection and analysis

Selection of studies

We downloaded all titles and abstracts retrieved by the electronic searches to the reference management database Reference Manager (Reference Manager 2010) and removed duplicates. One person (AH) sifted through the search results, discarding obviously irrelevant studies, and produced a long list of possibly eligible studies. Thereafter one of the review authors (GF) assessed these citations. Two review authors (GF, IS) independently obtained and assessed full‐text copies of potentially eligible studies. The review authors were not blinded to study author or location. We resolved disagreements through full‐group discussion.

Data extraction and management

At least two review authors (among LB, SM, LV, IS and JH) independently extracted study data using a modified EPOC data collection checklist (EPOC 2013). We resolved disagreements by discussion and, when necessary, through the involvement of an arbitrator (RG). We contacted study authors to ask for additional data/information. If study authors did not respond the first time, we sent two email reminders.

For inappropriately analysed ITS studies, if data needed for reanalyses were reported in tables or graphs in the original paper, we contacted the study authors to request original data to ensure minimum approximation. If data were not available, or if we received no response from the study authors, we used the software xyExtract to extract data from graphs (Wagner 2002).

To evaluate the impact of interventions on outcomes, we classified all included studies, according to the above described taxonomy, into three different intervention categories: rationing and/or prioritising demand; expanding capacity; and restructuring the intake assessment/referral process.

Assessment of risk of bias in included studies

At least two review authors (among SM, LV, LB, IS and JH) independently assessed the risk of bias of each included study using the criteria suggested by EPOC (EPOC 2013) and Davey (Davey 2013). We assessed RCTs for generation of allocation sequence, concealment of allocation, similar baseline outcome measurements, similar baseline characteristics, incomplete outcome data, blinding of participants, blinding of outcome assessors, protection against contamination, selective outcome reporting and other risks of bias. Criteria for assessing ITS design included independence of the intervention from other changes, appropriate analysis of data, prespecified shape of the intervention, intervention unlikely to affect data collection, knowledge of the allocated intervention during the study adequately prevented, incomplete outcome data, selective outcome reporting and other risks of bias.

We scored each study for risk of bias as follows: 'low' if all key domains were scored as 'low risk'; 'unclear' if one or two key domains were scored as 'unclear risk'; and 'high' if more than two key domains were scored 'unclear risk' or 'high risk' (adapted from Davey 2013).

Measures of treatment effect

We calculated the effects of interventions by study design.

For RCTs, when available, we reported preintervention and postintervention values for outcomes of intervention and control groups, and we calculated the absolute change from baseline with 95% confidence interval (CI), or the effect (e.g. mean difference, ratio of means) with 95% CI.

We reanalysed ITS studies that were inappropriately analysed as simple before‐and‐after studies using segmented time‐series regression techniques to estimate the effect of the intervention, taking into account the time trend and autocorrelation among observations. Adjustment for autocorrelation involved estimating the autocorrelation parameter and including it in the segmented regression model if necessary (Wagner 2002). We obtained estimates for regression coefficients corresponding to two standardised effect sizes for each study: a change in step or level, and a change in slope before and after the intervention. A change in step or level was defined as the difference between the predicted level al the first intervention time point and the level predicted by the preintervention time trend. A change in slope was defined as the difference between postintervention and preintervention slopes (Ramsay 2003).

A change in level and/or slope with a negative value may indicate:

• an effect in terms of a reduction in waiting time (i.e. a beneficial intervention effect);

• an effect in terms of a reduction in the number/proportion of participants waiting longer than the recommended time (i.e. a beneficial intervention effect); or

• an effect in terms of a reduction in the number/proportion of participants waiting within the recommended time (i.e. a non‐beneficial intervention effect).

A change in level and/or slope with a positive value may indicate:

• an effect in terms of an increase in the number or proportion of participants treated within the recommended time (i.e. a beneficial intervention effect); or

• an effect in terms of an increase in the number or proportion of participants waiting longer than the recommended time (i.e. a non‐beneficial intervention effect).

We used STATA 12 (Stata 2011) for all analyses.

Unit of analysis issues

Included cluster trials were analysed appropriately, hence there was no need for reanalysis.

Dealing with missing data

For eligible studies, when data on outcomes of interest were missing or were incompletely reported, we contacted study authors to ask for additional information. Authors of only one study (Hofstetter 2010) were able to provide data for the ITS reanalysis. Authors of two RCTs (Leggett 2004; McKessock 2001) were unable to provide data on the outcomes of interest for this review.

Assessment of heterogeneity

We identified too few studies to explore heterogeneity. We descriptively reported heterogeneity of studies by assessing differences in populations of interest, types/categories of interventions, outcomes, study design and measures of effect.

Data synthesis

We summarised separately and qualitatively described the results of RCTs and reanalysed ITS studies.

Although initially planned, we decided against meta‐analysis because of the significant heterogeneity of eligible studies in terms of targeted elective procedures, participant and healthcare provider populations and characteristics and components of the intervention and setting. We performed no subgroup or sensitivity analyses.

'Summary of findings' tables

We assessed the quality of evidence for primary outcomes using the GRADE (Grades of Recommendation, Assessment, Development and Evaluation) approach (Guyatt 2008; Higgins 2011) and reported this information in Table 1 and Table 2. We rated the quality of the body of evidence for each outcome as 'high,' 'moderate,' 'low' or 'very low.'

Results

Description of studies

Results of the search

After duplicates were removed, electronic searches yielded 3040 citations. Of these, 2939 were judged not relevant and were excluded. Of the remaining 101 citations, 93 did not meet our inclusion criteria and were excluded. Eight publications (Hofstetter 2010; Leach 2004; Leggett 2004; Lowthian 2011; Lukman 2004; Mallard 2004; McKessock 2001; Thomas 2003) were found eligible for inclusion in this review. See PRISMA study flow diagram (Figure 1).

1.

Study flow diagram.

The additional search run up to November 2013, resulting in 662 citations. Of these, 468 were judged not relevant and excluded. Of the remaining 194, 189 did not meet our inclusion criteria and were thus excluded. Four studies from this new search are listed in the Characteristics of studies awaiting classification table, and one protocol is listed in the Characteristics of ongoing studies table. The six publications are awaiting final classification.

Included studies

The eight included studies are described in the Characteristics of included studies and are summarised, according to the taxonomy of interventions, in Table 3.

1. Summary of characteristics of included studies.

Reference	Study design	Setting and participants	Intervention (duration)	Control	Outcomes	Risk of bias
Interventions aimed at rationing and/or prioritising demand
Lowthian 2011	Reanalysed ITS study	1 hospital, patients waiting for elective surgery	Redesigning and streamlining of perioperative services (3 years)	Routine practice	Number of participants waiting longer than recommended wait time	High
Interventions aimed at restructuring referral processes
Direct/open access and direct booking systems
McKessock 2001	Cluster‐RCT	1 hospital and 57 general practices, 232 patients referred for elective laparoscopic sterilisation	Direct booking laparoscopic service (10 months)	Routine referral from GP to clinic	Waiting time	High
Thomas 2003	Cluster‐RCT	1 hospital and 66 general practices, 959 patients with lower urinary tract symptoms (LUTS) or microscopic haematuria (MH)	Direct booking investigation service (10 months)	Routine practice, consisting of initial outpatient appointment plus 1 further appointment for routine day case investigation	Waiting time	Low
Lukman 2004	Reanalysed ITS study	1 hospital and 1 community primary care, 2501 patients with cervical cytology abnormality and needing a colposcopy	Direct booking colposcopy service (24 months)	Referral and appointment made by GP	Proportion of participants obtaining an appointment within the recommended time threshold	High
Mallard 2004	Reanalysed ITS study	1 clinic, 7594 appointments for outpatients attending a public health clinic	Same‐day scheduling (12 months)	Routine practice, consisting of complex appointment guidelines and next place available schedule	Waiting time	High
Distant consultancy
Hofstetter 2010	Reanalysed ITS study	1 hospital and 1 community primary care, 1690 patients needing ENT specialty care in a rural area	Telemedicine consultations (6 years)	Face‐to‐face visit in main city hospital	Waiting time	High
Leggett 2004	RCT	1 hospital and 10 general practitioners, 136 patients requiring dermatology referral	Instant photography (unknown duration)	Face‐to‐face first appointment with dermatology consultant	Waiting time	Unclear
Single generic waiting list
Leach 2004	Reanalysed ITS study	1 hospital, patients requiring routine spinal surgery	Generic waiting list (14 months)	Each consultant managing own waiting list	Number of participants waiting less than or longer than the prespecified time threshold	High

Study designs

Among the eight included studies were two cluster‐RCTs (McKessock 2001; Thomas 2003), one RCT (Leggett 2004) and five reanalysed ITS studies (Hofstetter 2010; Leach 2004; Lowthian 2011; Lukman 2004; Mallard 2004).

Geographical location of study

Five studies were conducted in the UK (Leach 2004; Leggett 2004; Lukman 2004; McKessock 2001; Thomas 2003); two in the USA (Hofstetter 2010; Mallard 2004); and one in Australia (Lowthian 2011).

Settings and participants

The review involved a total of seven hospitals, one outpatient clinic and 135 general practices (GPs)/primary care clinics. Two studies (Leach 2004; Lowthian 2011) involved only hospitals. One study involved one outpatient clinic (7594 appointments) (Mallard 2004). Five studies were carried out in general practices/primary care and hospitals: one hospital and one community primary care clinic (1690 participants) (Hofstetter 2010); one hospital and 10 GPs (136 participants) (Leggett 2004); one hospital and one community primary care clinic (2501 female participants) (Lukman 2004); one hospital and 57 general practices (232 female participants) (McKessock 2001); and one hospital and 66 general practices (959 participants) (Thomas 2003).

Targeted elective procedures

The eight included studies reported interventions that targeted elective procedures for different clinical conditions: referrals for ENT (Hofstetter 2010), uncomplicated spinal surgery (Leach 2004). dermatology (Leggett 2004), elective surgery (Lowthian 2011), colposcopy for abnormal cervical cytology (Lukman 2004), any paediatric clinical conditions treated in an outpatient clinic (Mallard 2004), laparoscopic sterilisation (McKessock 2001) and urological investigations (Thomas 2003).

Types of interventions and comparators 

Following our classification, study interventions were grouped according to whether they were aimed at rationing/and or prioritising demand; expanding capacity; or restructuring the intake assessment/referral process.

Interventions aimed at rationing and/or prioritising demand

One study (Lowthian 2011) evaluated the effects of introducing a system for streamlining elective surgery patients according to urgency, and compared this system with routine practice. The introduction of the intervention coincided with the construction of a dedicated elective surgery and procedural facility. The intervention lasted three years.

No studies evaluating interventions aimed at rationing demand were found.

Interventions aimed at expanding capacity

No studies evaluating interventions aimed at expanding capacity were found, although an increase in capacity was introduced as a co‐intervention in one study (Lowthian 2011).

Interventions aimed at restructuring the intake assessment/referral process

Seven studies evaluated interventions aimed at restructuring the referral process (Hofstetter 2010; Leach 2004; Leggett 2004; Lukman 2004; Mallard 2004; McKessock 2001; Thomas 2003). Median duration of intervention was 11.8 months, ranging from 7 months (Mallard 2004) to five years (Hofstetter 2010); for one study (Leggett 2004) the duration of the intervention was not specified.

Three studies (Lukman 2004; McKessock 2001; Thomas 2003) explored direct booking/referral. McKessock et al (McKessock 2001) evaluated the impact of direct access to laparoscopic sterilisation in general practices against routine referral from GP to clinic. Thomas et al (Thomas 2003) evaluated a direct booking urological investigation service for patients referred by their GPs for lower urinary tract symptoms (LUTS) or microscopic haematuria (MH) and compared it versus current practice, consisting of an initial outpatient appointment plus one further appointment for routine day case investigation. Lukman et al (Lukman 2004) evaluated a direct booking system in a colposcopy clinic for women with abnormal cervical cytology versus referral and appointment made by GP: A "fail safe" pathway to retrieve patients failing to respond to the new referral system was set in place.

Distance consultancy interventions were evaluated in two studies (Hofstetter 2010; Leggett 2004). Hofstetter 2010 assessed the introduction of telehealth for participants needing ear nose throat (ENT) specialty care in a rural area and compared the intervention versus ENT face‐to‐face visit in the main city hospital. In Leggett 2004, instant photography was introduced to diagnose and manage dermatology conditions in general practices located near a major teaching hospital, and was compared versus face‐to‐face index appointment with a dermatology consultant.

In one study (Leach 2004), the effects of introducing a generic waiting list and pooling all initial outpatient appointments and dates for routine spinal surgery were compared against current practice, consisting of each consultant managing his or her own waiting list. A second intervention was to integrate the MRI booking system with outpatient review appointments. However, it was unclear when this intervention was introduced.

One study (Mallard 2004) evaluated open access/same‐day scheduling for paediatric outpatients attending a public health clinic and compared this approach with standard routine based on complex appointment guidelines and next place available schedule.

Outcomes

One study that evaluated an intervention aimed at prioritising of demand measured the absolute number of participants waiting longer than a recommended waiting time (Lowthian 2011), that is, the number of urgent elective participants waiting longer than 30 days, semi‐urgent participants waiting longer than 90 days and non‐urgent participants waiting longer than 365 days.

Among the seven studies assessing interventions aimed at improving the organisational management of waiting lists or restructuring the intake assessment/referral processes, five studies measured the effects on waiting time (Hofstetter 2010; Leggett 2004; Mallard 2004; McKessock 2001; Thomas 2003); one study measured the proportion of participants obtaining an appointment within the recommended four weeks waiting time (Lukman 2004) and one study the absolute number of participants waiting less than nine months, between nine and 18 months or longer than 18 months (Leach 2004).

Of the three included RCTs, one (Thomas 2003) reported estimated effect with the 95% CI, while the other two (Leggett 2004; McKessock 2001) reported estimated effect and P value.

No safety outcomes were reported in the included studies. Two RCTs (McKessock 2001; Thomas 2003) measured direct and indirect costs (NHS and participant time and travel costs, respectively).

Excluded studies

Reasons for exclusion of the 93 citations were as follows: ineligible study design (n = 59); inappropriately analysed ITS study with no graphically reported data, and/or lacking baseline data or inappropriately analysed ITS study with graphically reported data but with insufficient number of data points before and/or after the intervention (n = 23); CBA studies with only one intervention and/or control site and without graphically presented data (n = 5), ineligible intervention (n = 4) and one study could not be located. See Characteristics of excluded studies for details. We listed two relevant study protocols (Augestad 2008; Wahlberg 2013;) under ongoing studies (see Characteristics of ongoing studies).

Risk of bias in included studies

See 'Risk of bias' tables in Characteristics of included studies, Figure 2 and Figure 3 for RCTs, and in Figure 4 and Figure 5 for reanalysed ITS studies.

2.

Risk of bias graph for RCTs: review authors' judgements about each risk of bias item for each included study.

3.

Risk of bias graph for RCTs: review authors' judgements about each risk of bias item presented as percentages across all included studies.

4.

Risk of bias graph for reanalysed ITS studies: review authors' judgements about each risk of bias item for each included study.

5.

Risk of bias graph for reanalysed ITS studies: review authors' judgements about each risk of bias item presented as percentages across all included studies.

RCTs

See 'Risk of bias' in Characteristics of included studies table, Figure 2 and Figure 3.

One of the three included trials (Thomas 2003) had an overall low risk of bias, and two trials (Leggett 2004; McKessock 2001) were at high risk of bias. In these trials high attrition and contamination were the main sources of bias: in Leggett 2004, 36.6% of intervention participants could not be diagnosed, and among those who received a diagnosis, 38.0% still needed to see a dermatologist face‐to‐face for management; in McKessock 2001, only 10 out of 75 participants allocated to the intervention group actually received the intervention.The McKessock 2001 trial was also at high risk of contamination, as a large proportion of participants assigned to the intervention group were treated according to standard referral practice. The other two trials were at low risk of contamination.
 
 In two trials (Leggett 2004; Thomas 2003), both the sequence generation process and allocation concealment were adequate and at low risk of bias, and in one trial (McKessock 2001), risk of bias for sequence generation and allocation concealment was unclear. Baseline characteristics were similar in two trials (Leggett 2004; Thomas 2003) and unclear in one trial, as baseline characteristics were not provided (McKessock 2001). All trials had an unclear risk of selective outcome reporting, as no trial protocols were available. All trials were at low risk of bias for baseline outcome measures (as no baseline measure of outcomes can be provided for the outcome of interest), blinding (due to objective outcomes) and other risk of bias.

Reanalysed ITS studies

See 'Risk of bias' in Characteristics of included studies table, Figure 4 and Figure 5.

All five ITS studies (Hofstetter 2010; Leach 2004; Lowthian 2011; Lukman 2004; Mallard 2004) were at overall high risk of bias.

Intervention independent of other changes

Risk of bias was low in two studies (Hofstetter 2010; Mallard 2004): In one, the number of available appointment slots was stable during the study period (Hofstetter 2010); in another (Mallard 2004), extra clinics were organised at the beginning of the intervention period, but as these data were not included in the analysis, risk of bias must be considered low. In one study (Leach 2004), risk of bias was unclear, as it was unclear whether a second intervention was implemented at the same time as the main intervention, or later during the intervention period, which may have affected the results. In two studies, risk of bias was high (Lowthian 2011; Lukman 2004): In one (Lowthian 2011), three different interventions were implemented over time, of which the main intervention (streamlining of services) was one; this complicates interpretation of results; in the other (Lukman 2004), extra clinics (of unclear duration) were introduced after the start of the main intervention to meet extra demand, and a second intervention (introduction of colposcopy nurse) was put in place during the intervention period.

Appropriate analysis and shape of intervention effect prespecified

All included ITS studies were reanalysed by the review authors; therefore risk of bias is low for these items.

Intervention unlikely to affect data collection

Data were retrospectively collected in three studies (Hofstetter 2010; Lowthian 2011; Lukman 2004); thus risk of bias for this item is low; two studies had unclear risk of bias, as no information was provided on how data had been collected (Leach 2004; Mallard 2004).

Knowledge of allocated interventions adequately prevented during the study

All outcomes of interest for this review are objective and are unlikely to be affected by non‐blinding; therefore risk of bias is low.

Incomplete outcome data

Low risk of bias due to incomplete outcome data is reported for one study (Lowthian 2011), and unclear risk of attrition bias is reported for four studies (Hofstetter 2010; Leach 2004; Lukman 2004; Mallard 2004), as no information on number of participants who withdrew or were lost to follow‐up was provided.

Selective outcome reporting

Usually ITS studies do not have a study protocol with prespecified outcomes, and it is sometimes difficult to judge whether all important outcomes have been reported; therefore most of the included ITS studies had an unclear risk of bias for this item (Leach 2004; Lowthian 2011; Lukman 2004; Mallard 2004). Another study (Hofstetter 2010) was at high risk of bias because of different waiting time outcomes measured in preintervention and postintervention periods: For preintervention, waiting time was measured by referral to a face‐to‐face specialist appointment, which, in many cases, involved treatment given to the participant, but in the postintervention period, waiting time was measured with referral until the consultant looked at participant data sent by store and forward telehealth, but not when the participant actually received feedback/treatment.

Other bias

All five ITS studies had unclear risk of bias (Hofstetter 2010; Leach 2004; Lowthian 2011; Lukman 2004; Mallard 2004).
 
 In Hofstetter 2010, it was unclear whether all participants who received a telehealth specialist consultation could be diagnosed and subsequently treated, or if some had to see a specialist face‐to‐face to be diagnosed. Telemedicine techniques and equipment most likely improved over the study years, which may have had implications for whether consultations were successful. In Leach 2004, it was unclear whether preintervention and/or postintervention data related to waiting times for all 10 surgeons, or for only the seven who agreed to the intervention. In Lowthian 2011, it was unclear whether study authors each month added up the number of people who were currently waiting too long and were still waiting. If so, participants may appear on the graph for several consecutive months, that is, from the time they exceed the recommended time to the time they have surgery, and if so, this may have affected the results of the analysis. In Lukman 2004, the source of data for analysis (Figure 2 in Lukman 2004) seems to include data for all types of referrals (inadequate, abnormal or other), but the study aims to assess the impact of direct booking on waiting time from abnormal smear report to colposcopy clinic (direct booking available only for abnormal smears, not for the other referrals), but it is unclear how this may have affected the results. In Mallard 2004, the definition of 'waiting time' was unclear, and it was unclear whether the analysis included also waiting time for prescheduled appointments, for which shorter waiting time presumably was not desired.

Effects of interventions

See: Table 1; Table 2

See Table 1 and Table 2 for the main comparisons.

Effects of interventions aimed at rationing and/or prioritising demand

See Table 4.

2. Interventions aimed at rationing and/or prioritising demand: reanalysed ITS studies.

Study	Outcome	Postintervention period	Secular trend (SE, P)	Change in level (SE, P)	Change in slope (SE, P)
Lowthian 2011	Number of participants waiting longer than recommended time threshold ("urgent" participants waiting less than 30 days) every month	3 years	+0.25 (SE 0.41, P value 0.55)	‐5.40 (SE 6.44, P value 0.41)	‐1.03 (SE 0.51, P value 0.049)
	Number of participants waiting longer than recommended time threshold ("semi‐urgent" patients waiting less than 90 days) every month		+13.72 (SE 6.23, P value 0.032)	+32.55 (SE 54.65, P value 0.55)	‐27.99 (SE 8.58, P value 0.002)
	Number of participants waiting longer than recommended time threshold ("non‐urgent" participants waiting less than 365 days) every month		‐0.15 (SE 1.85, P value 0.94)	+5.50 (SE 11.83, P value 0.64)	‐1.62 (SE 2.96, P value 0.59)

One ITS study (Lowthian 2011) evaluated the effects of interventions aimed at prioritising demand. Results of this study show that streamlining of elective surgery services had an effect on the waiting time of ‘semi‐urgent’ patients only, with 28 (SE 8.58, P value 0.002) fewer participants per month waiting longer than recommended (< 90 days). No effects on waiting times were found for 'urgent' or 'non‐urgent' participant groups (with recommended waiting times of less than 30 days and 365 days, respectively).

No effectiveness data for interventions aimed at rationing demand were included in this review.

Effects of interventions aimed at expanding capacity

No effectiveness data for interventions aimed at increasing capacity were included in this review.

Effects of interventions aimed at improving the organisational management of waiting lists or restructuring the intake assessment/referral process

See Table 5 and Table 6.

3. Interventions aimed at restructuring referral processes: RCTs.

Study	Outcome	Preintervention value	Postintervention value	Effect
Direct/open access and direct booking systems
McKessock 2001	Mean waiting time from referral to operation (days)	Intervention: NA Control: NA	Intervention: 108 days Control: 127 days	Statistically significant difference (P value 0.003)
Thomas 2003	Median waiting time from referral to initial hospital appointment (IQR, days)	LUTS Intervention: 106 (70‐170) Control: 130 (77‐175) MH Intervention: 65 (41‐107) Control: 65 (48‐96)	LUTS Intervention (IQR): 36 (24‐64) Control: 75 (39‐99) MH Intervention: 41 (31‐58) Control: 47 (34‐62)	LUTS Ratio of means (95% CI): 0.7 (0.5 to 0.9) MH Ratio of means (95% CI): 1.0 (0.7 to 1.2)
Distant consultancy
Leggett 2004	Mean waiting time for appointments (days)	NA	NA	"Median waiting time in study and control groups were similar (mean 55 days; SD=40)"

NA: not available.

IQR: interquartile range.

LUTS: lower urinary tract syndrome.

MH: microscopic haematuria

4. Interventions aimed at restructuring referral processes: reanalysed ITS studies.

Study	Outcome	Postintervention period	Secular trend (SE, P)	Change in level (SE, P)	Change in slope (SE, P)
Direct/open access and direct booking systems
Lukman 2004	Proportion of participants waiting less than recommended time threshold (percentage of participants with moderate/severe lesions waiting less than 4 weeks) every 3 months	24 months	+0.86% (SE 3.78 P value 0.83)	‐14.26% (SE 19.83, P value 0.50)	+6.29% (SE 12.26, P value 0.62)
Mallard 2004	Waiting time (days) per month	12 months	+1.40 (SE 0.8, P value 0.13)	‐25.20 (SE 3.83, P value < 0.001)	‐3.03 (SE 0.92, P value 0.005)
Distant consultancy
Hofstetter 2010	Waiting time (months) per year	6 years	‐0.04 (SE 0.06, P value 0.51)	‐0.69 (SE 0.55, P value 0.23)	‐0.21 (SE 0.13, P value 0.15)
Single generic waiting list
Leach 2004	Number of participants waiting less than recommended time threshold (less than 9 months) per month	14 months	+9.44 (SE 10.93, P value 0.40)	‐20.59 (SE 22.67, P value 0.37)	+2.75 (SE 12.69, P value 0.86)
	Number of participants waiting within a recommended time threshold (between 9 and 18 months) per month		‐3.30 (SE 7.78, P value 0.68)	‐5.28 (SE 16.20, P value 0.75)	‐6.59 (SE 8.73, P value 0.46)

Seven studies evaluated the effects of improving organisational management and restructuring of referral processes: three RCTs (Leggett 2004; McKessock 2001; Thomas 2003) and four ITS studies (Hofstetter 2010;Leach 2004; Lukman 2004; Mallard 2004).

Effects of direct/open access and direct booking systems

Four studies (two ITS: Lukman 2004; Mallard 2004; and two RCTs: McKessock 2001; Thomas 2003) evaluated the effects of direct/open access or direct booking systems.

Both trials showed beneficial effects of direct/open access interventions on waiting times: One of the trials (McKessock 2001) enrolling 232 participants showed a reduction in mean waiting time for those randomly assigned to direct access to laparoscopic sterilisation as compared with control participants (108 vs 127 days, P value 0.003), and the other (Thomas 2003) showed that introducing an open access urological investigation service reduced waiting times for participants with lower urinary tract syndrome by 30% (ratio of means 0.7, 95% CI 0.5 to 0.9), although no significant difference was found for those with microscopic haematuria (total n = 959). McKessock 2001 suffered from high attrition, as only 10 out of 75 participants allocated to the intervention group actually received the intervention.

Both trials evaluated costs. In McKessock 2001, evaluation of total costs to patients and total NHS costs showed no differences between intervention and control groups. Thomas 2003 reported no differences in costs between intervention and control groups.

One ITS study (Mallard 2004) showed that open access resulted in a direct reduction in mean waiting times for paediatric patients (total n = 7594) at health clinic appointments (step change: ‐25.20 days, SE 3.83, P value < 0.001; slope change: ‐3.03 days/mo; SE 0.92, P value 0.005).

The other ITS study (Lukman 2004) showed no effect of introducing a direct booking system on the proportion of participants (n = 2501) ‐ with moderate or severe cellular abnormalities of the uterine cervix ‐ who received a colposcopy appointment within the recommended four weeks of waiting time (step change: ‐14.26%; SE 19.83, P value 0.50; slope change: 6.29; SE 12.26, P value 0.62).

Effects of distant consultancy

Two studies ‐ one trial (Leggett 2004) and one ITS study (Hofstetter 2010) ‐ evaluated the effects of distance consultancy on waiting times.

Both studies showed no effect of distance consultancy on waiting times: Leggett 2004 showed no effect of using instant photography to diagnose and manage dermatology referrals on the waiting time of dermatology patients (n = 136) (mean 55 days, SD = 40, P value > 0.05), and the ITS study (Hofstetter 2010) showed no effect of introducing telemedicine to manage rural ENT patients (n = 1690) on waiting times (step change: ‐0.69 months; SE 0.55, P value 0.23; slope change: ‐0.21 months each year; SE 0.13, P value 0.15).

Leggett 2004 suffered from high attrition: 36.6% of intervention participants did not receive the intervention, and among those who did, 38.0% still needed to see a dermatologist face‐to‐face.

Effects of introducing generic waiting lists (pooling of patients)

One ITS study (Leach 2004) showed no effect of introducing a generic waiting list for non‐complex spinal surgery on the number of participants waiting less than nine months (step change: ‐20.59 participants; SE 22.67, P value 0.37) and on the number of participants waiting between nine months and 18 months (step change: ‐5.28 participants; SE 16.20, P value 0.75).

Discussion

The aim of this review was to identify interventions that are effective in reducing waiting time for elective procedures.

Summary of main results

See Table 1 and Table 2 for main results.

The review included eight studies evaluating the effects of interventions aimed at reducing waiting times for elective procedures: three RCTs (Leggett 2004; McKessock 2001; Thomas 2003) and five reanalysed ITS studies (Hofstetter 2010; Leach 2004; Lowthian 2011; Lukman 2004; Mallard 2004). One study evaluated interventions aimed at prioritising demand (Lowthian 2011), and seven studies evaluated interventions aimed at restructuring the intake assessment/referral process (Hofstetter 2010; Leach 2004; Leggett 2004; Lukman 2004; Mallard 2004; McKessock 2001; Thomas 2003). The included studies were heterogeneous in terms of types of interventions, target conditions and elective procedures, study design and outcome measures, thus hindering meta‐analysis.

The Lowthian 2011 study, which evaluated a system using explicit referral guidelines for streamlining patients according to the urgency of their condition, showed a reduced number of semi‐urgent patients waiting longer than the recommended time, but unchanged numbers of urgent and non‐urgent elective patients waiting too long. However, no information was provided on how the number of patients not waiting too long was affected. Of concern in interpreting the results of this study are the discrepancies found between the numbers reported in text and in figures, which suggest that a participant may have been counted more than once. Another problem is that we cannot say how the results are affected by co‐interventions introduced during the intervention period.

Among the seven studies that evaluated interventions aimed at restructuring the intake assessment/referral process, three studies showed decreased waiting time (Mallard 2004; McKessock 2001; Thomas 2003), and four studies reported no effect (Hofstetter 2010; Leach 2004; Leggett 2004; Lukman 2004). However, important caveats were related to all of these studies; their results should therefore be interpreted with caution.

In McKessock 2001, only 14/75 (18.7%) women referred from intervention practices for laparoscopic sterilisation were eligible for direct referral according to the inclusion criteria, and of these, only 10/14 women actually received the intervention. In the discussion, study authors highlighted that participants seemed to prefer the current referral system and suggest the need to conduct preliminary studies before implementing new services based on assumptions of acceptance of revised clinical pathways. In one study (Mallard 2004), it was unclear exactly what the definition of waiting time was, and if the waiting time reported also included prescheduled appointments (i.e. appointments for which shorter waiting time presumably was not desired). In another study (Lukman 2004), extra clinics were introduced after the start of the main intervention to meet extra demand, and a dedicated colposcopy nurse was introduced halfway through the intervention period, which complicates the interpretation of results.

Distant consultancy resulted in no effect on improvement of mean waiting time (Hofstetter 2010; Leggett 2004). However, in Hofstetter 2010, outcomes measured in the preintervention and postintervention periods differed: For control participants, time from referral to specialist appointment and presumably also to treatment was measured, while for intervention participants, time from referral to examination/consultation was measured. It remains unclear when participants who received telehealth consultations received treatment. Also, study authors provided no information on the number of unsuccessful telehealth appointments for which a face‐to‐face appointment was required. In Leggett 2004, a large proportion of intervention participants could not be diagnosed through the use of instant photography, which indicates that this approach may not be suitable for some dermatological conditions. An intervention aimed at restructuring means of queuing using a generic waiting list showed no effect of the intervention on the number of participants waiting less than the recommended time threshold (Leach 2004). In this study, seven out of 10 consultants participated, but it was not clear whether preintervention and/or postintervention data related to waiting times for all 10 surgeons, or for only the seven who agreed to the intervention, which may have affected results of reanalysis of this study.

We found no studies evaluating interventions directly aimed at rationing demand or increasing capacity.

On the basis of available evidence, it is difficult to draw any firm conclusions about the effectiveness of interventions to reduce waiting time.

Overall completeness and applicability of evidence

Despite media and journal coverage given to waiting time policies implemented nationwide in different health systems, we could not find and include studies with usable empirical data measuring their impact. This was disappointing, as it implies failure of researchers to adequately evaluate policy initiatives to improve waiting times.

Most included studies were conducted in the UK (5/8) or in the USA (2/8) ‐ both high‐income countries but with different healthcare systems. No studies were conducted in low‐ and middle‐income countries. All interventions targeted elective therapeutic or test‐and‐treat procedures.

Most of the evaluated interventions were aimed at improving the organisational management of waiting lists or restructuring the intake assessment/referral process. These studies however, did not cover all possible interventions, for example, resource sharing strategies or remuneration schemes. Only one study involved interventions aimed at prioritising demand, but no study evaluated interventions including co‐payments, practice guidelines or clinical priority scores. No study evaluated the effects of interventions aimed at expanding capacity (e.g. providing additional funding to the public sector, subsidising or facilitating access to the private sector).

None of the included studies reported on adverse effects of the interventions (e.g. morbidity, mortality), and only two studies reported on costs.

Quality of the evidence

More than half of the evidence included in this review was derived from non‐randomised low‐quality time series studies with no control groups, involving only one or two intervention sites, which we reanalysed to remove the risk of bias due to secular trends in uncontrolled data.

The overall quality of the evidence for all outcomes ranged from low to very low, which is why no robust conclusions regarding the effectiveness of the evaluated interventions can be drawn. The quality of evidence for the effectiveness of interventions aimed at rationing and/or prioritising demand was low, as only one reanalysed ITS study (Lowthian 2011), conducted at a single site, was included in this review. Even though this study showed a beneficial effect of streamlining services for semi‐urgent patients, the intervention was not independent of other changes, which made it difficult to isolate the effect of the main intervention. This type of intervention needs further investigation, during controlled conditions, to determine its effectiveness in reducing waiting times for elective surgery.

The quality of the evidence on the effectiveness of direct booking/open access or same‐day scheduling was low because of high risk of bias in most studies (3/4). Bias was mainly due to high attrition/contamination (McKessock 2001) and other changes concurrent with the main intervention (Lukman 2004; Mallard 2004). One of the four studies was at low risk of bias (Thomas 2003). These interventions, all of which involve the provision of more accessible services, show some promise, as three of the four studies show a beneficial intervention effect in terms of reduced waiting times.

Data on the effectiveness of distant consultancy on waiting time were limited to two studies, which evaluated two different types of distance consultancy for two different conditions: one providing specialist consultations for ENT patients through telemedicine (Hofstetter 2010), and the other using teledermatology for specialist consultations (Leggett 2004). Both studies were at high risk of bias ‐ the first study because of selective reporting bias, as it appeared to measure and report different things in the preintervention and postintervention periods and did not provide information on the numbers of successful or failed teleconsultations; the latter had high risk of attrition bias, with only 25.4% of intervention participants who received a 'photo‐diagnosis' not needing to be seen by a dermatologist, while 38.0% needed to be seen face‐to‐face for further management, and for 36.6%, photo‐diagnosis was not possible. Neither study showed a significant intervention effect.

Finally, evidence on the effect of introducing a generic waiting list in spinal surgery on the number of patients waiting less than a recommended time threshold or within a recommended time period was limited to only one observational study with high risk of bias (Leach 2004).

Potential biases in the review process

The search strategy used in this review was carefully developed by an experienced information technologist, and a comprehensive search, involving a large number of databases, was performed. One review author sifted all references identified by the electronic searches, excluding papers that clearly were not eligible, and two review authors independently assessed all potentially eligible titles and abstracts against the eligibility criteria to ensure that no important references were missed. We also searched reference lists of included studies and contacted study authors about other published or unpublished studies. In addition, we searched trials registers for ongoing trials, along with a number of sources of grey literature. Despite all this, we cannot exclude the possibility that important references may have been missed.

Few studies were identified for inclusion in this review, and none claimed negative results that could be suggestive of publication bias. Unfortunately, because too few studies were identified for inclusion in this review, we could not assess publication bias.

Agreements and disagreements with other studies or reviews

The impact of different national policies for regulating and containing length of waiting times has been evaluated by reports (Rachlis 2005; Siciliani 2013; Willcox 2007), overviews and reviews (Kreindler 2010; Miller 2008). However these documents do not provide data on the effects of specific interventions used to enforce or implement national and regional policies on waiting times for elective procedures.

Authors' conclusions

Implications for practice.

Decision‐makers should be aware that for interventions aimed at prioritising demand (e.g. co‐payment, explicit referral criteria or practice guidelines, clinical priority scores), evidence is incomplete, and for those aimed at rationing demand or expanding capacity (e.g. providing additional funding to the public sector, subsidising or facilitating access to the private sector), evidence is lacking. Thus, implementation of such interventions should be monitored for both effectiveness and possible drawbacks.

Implications for research.

Despite the importance of long waiting times as a relevant healthcare problem, only scarce evidence of low quality is presently available.

RCTs and cluster‐RCTs are ideally the best study designs to be applied to fill in this knowledge gap. Large and robust experimental studies might be difficult and expensive to set up, and represent unfamiliar ground in policy‐making. However robust and useful evidence on the effectiveness of interventions aimed at reducing waiting time could be obtained with good quality interrupted time series studies, which are both feasible and practical.

Some points must be taken into consideration before one can plan and embark on a study addressing the effectiveness of any intervention to reduce waiting times for elective procedures.

• Greater attention should be paid to the quality of study designs, and cluster‐RCTs should be carefully controlled for contamination bias across interventions among the included clusters.

• Researchers designing ITS studies should adhere to the quality criteria described by the EPOC Group (EPOC 2013), for example, allow a sufficient number of data points before as well as after the intervention to enable reliable statistical inference, and use formal tests for trend, taking into account any secular trends.

• A reliable primary outcome should be chosen: It is still uncertain which could be the most appropriate outcome measure ‐ among the many available ‐ that could best depict the 'long waiting time phenomenon'; however the proportion of patients waiting above a recommended time threshold appears to be suitable in terms of practical relevance, effective communication of results and statistical reliability. As interventions tend to act on supply or on demand, process outcomes ‐ such as increase in supply or decrease in demand ‐ should also be monitored to evaluate tenure of the causal mechanism between variation on supply/demand and waiting time; if the number of participants waiting too long is provided, the number of participants not waiting too long should also be reported.

Future research

• Research evaluating interventions aimed at rationing services and/or prioritising demands or interventions aimed at expanding capacity is lacking and therefore needed.

• Future researchers should make greater efforts to collect and analyse data on undesired consequences of interventions, as well as on economic outcomes in different health settings.

• Interventions showing some promise (e.g. direct booking, open access, same‐day scheduling) but also streamlining of services needs further evaluation.

• Interventions involving advanced technology (i.e. distance consultancy) (telemedicine or photo specialist consultations) may need reevaluation in the light of rapidly evolving new and better technology.

Appendices

Appendix 1. MEDLINE search strategy

Ovid MEDLINE(R) In‐Process & Other Non‐Indexed Citations and Ovid MEDLINE(R) < 1946 to November 2012>.
 
 1 Waiting lists/ (7789)
 2 ((wait or waiting) adj2 (time or times or list or lists)).ti. (2254)
 3 ((wait or waiting) adj2 (time or times or list or lists) adj4 (reduce? or reduction or eliminat$ or lower or fewer or intervention or policy or policies or reform$ or effectiveness or impact or improv$ or organi?ational$ or quality)).ab. (1729)
 4 (eliminat$ patient? wait$ or improv$ patient? wait$ or lower? patient? wait$ or lowering patient? wait$ or reduc$ patient? wait$).ti,ab. (62)
 5 or/1‐4 [Waiting Lists/Waiting Times‐MeSH] (9487)
 6 (waiting room? or waiting area? or watchful wait$ or "wait and see" or "wait until").ti,ab. (4186)
 7 ((wait$ list? or waitlist?) adj2 (control? or controlled or group? or intervention or trial or study)).ti,ab. (1891)
 8 (waiting room? or waiting area? or watchful wait$ or "wait and see" or "wait until").ti,ab. (4186)
 9 Watchful Waiting/ (612)
 10 5 not (or/6‐9) [Waiting Lists/Time] (8961)
 11 exp transplantation/ (391733)
 12 (transplant? or transplantation?).ti,ab,hw. (488983)
 13 Emergency Service, Hospital/ or Trauma centres/ or exp Emergency Medical Services/ or exp Emergency Treatment/ or Emergencies/ or Emergency Medicine/ (194007)
 14 (emergency or emergencies).ti,hw. (130333)
 15 (cell or cells or cellular or molecular$ or animal? or lab or labs or laborator$).ti. (1638971)
 16 patient? delay$.ti,ab. (1129)
 17 or/11‐16 [terms to exclude] (2251755)
 18 (randomized controlled trial or controlled clinical trial).pt. or randomized.ab. or placebo.ab. or clinical trials as topic.sh. or randomly.ab. or trial.ti. (819631)
 19 exp animals/ not humans.sh. (3807583)
 20 18 not 19 [Cochrane RCT Filter 6.4.d Sens/Precision Maximizing] (757438)
 21 intervention?.ti. or (intervention? adj6 (clinician? or collaborat$ or community or complex or DESIGN$ or doctor? or educational or family doctor? or family physician? or family practitioner? or financial or GP or general practice? or hospital? or impact? or improv$ or individuali?e? or individuali?ing or interdisciplin$ or multicomponent or multi‐component or multidisciplin$ or multi‐disciplin$ or multifacet$ or multi‐facet$ or multimodal$ or multi‐modal$ or personali?e? or personali?ing or pharmacies or pharmacist? or pharmacy or physician? or practitioner? or prescrib$ or prescription? or primary care or professional$ or provider? or regulatory or regulatory or tailor$ or target$ or team$ or usual care)).ab. (137446)
 22 (pre‐intervention? or preintervention? or "pre intervention?" or post‐intervention? or postintervention? or "post intervention?").ti,ab. [added 2.4] (8029)
 23 (hospital$ or patient?).hw. and (study or studies or care or health$ or practitioner? or provider? or physician? or nurse? or nursing or doctor?).ti,hw. (670660)
 24 demonstration project?.ti,ab. (1793)
 25 (pre‐post or "pre test$" or pretest$ or posttest$ or "post test$" or (pre adj5 post)).ti,ab. (56179)
 26 (pre‐workshop or post‐workshop or (before adj3 workshop) or (after adj3 workshop)).ti,ab. (511)
 27 trial.ti. or ((study adj3 aim?) or "our study").ab. (533289)
 28 (before adj10 (after or during)).ti,ab. (327726)
 29 ("quasi‐experiment$" or quasiexperiment$ or "quasi random$" or quasirandom$ or "quasi control$" or quasicontrol$ or ((quasi$ or experimental) adj3 (method$ or study or trial or design$))).ti,ab,hw. [ML] (92136)
 30 ("time series" adj2 interrupt$).ti,ab,hw. [ML] (791)
 31 (time points adj3 (over or multiple or three or four or five or six or seven or eight or nine or ten or eleven or twelve or month$ or hour? or day? or "more than")).ab. (7481)
 32 pilot.ti. (34197)
 33 Pilot projects/ [ML] (74430)
 34 (clinical trial or controlled clinical trial or multicenter study).pt. [ML] (594981)
 35 (multicentre or multicenter or multi‐centre or multi‐center).ti. (25360)
 36 random$.ti,ab. or controlled.ti. (669876)
 37 (control adj3 (area or cohort? or compare? or condition or design or group? or intervention? or participant? or study)).ab. not (controlled clinical trial or randomized controlled trial).pt. [ML] (363890)
 38 "comment on".cm. or review.ti,pt. or randomized controlled trial.pt. [ML] (2695157)
 39 (rat or rats or cow or cows or chicken? or horse or horses or mice or mouse or bovine or animal?).ti. (1288069)
 40 exp animals/ not humans.sh. [ML] (3807583)
 41 (or/21‐37) not (or/38,39‐40) [EPOC Methods Filter ML 2.4] (1920409)
 42 (10 not 17) and 20 [WL RCT] (216)
 43 ((10 not 17) and 41) not 42 [WL EPOC] (1773)
 44 ((wait or waiting) adj2 (time or times or list or lists) adj4 (reduce? or reduction or eliminat$ or lower or fewer or intervention or policy or policies or reform$ or effectiveness or impact or improv$ or organi?ational$ or quality)).ti. (211)
 45 (eliminat$ patient? wait$ or improv$ patient? wait$ or lower? patient? wait$ or lowering patient? wait$ or reduc$ patient? wait$).ti,ab. (62)
 46 (or/44‐45) not (or/6‐9,17,42‐43) [Keyword Results] (99)
 47 review.pt. or ((systematic adj3 review?) or metaanalys$ or meta‐analys$ or literature review).ti. (1779206)
 48 (10 not 17) and 47 (187)
 49 42 not 48 [RCT Results] (200)
 50 43 not 48 [EPOC EPOC Filter Results] (1773)
 51 42 not 49 [Reviews Set 1] (16)
 52 46 and 47 [KW Reviews Set 2] (4)
 53 or/51‐52 [Reviews‐‐Results to export] (20)
 54 46 not 53 [Keyword Results to export] (95)

An update of the above reported search was performed in November 2013.

Appendix 2. Additional search strategies

ABI Inform, ProQuest

January 31, 2013

all(waiting list or waiting lists or wait lists or waiting list) NOT (all(organ or organs or transplant or transplantation or emergency room* or emergency department* or trauma ) or ti(therapy))

Canadian Research Index

Jan 5, 2012
 
 (((((((ti("wait* list*") OR ti("wait* time*") OR ab("wait* list*") OR ab("wait* time*")) AND (health OR healthcare OR primary care OR referral* OR physician* OR doctor* OR non‐urgent OR non‐emerg* OR elective)) NOT ((wait* list* n/7 group*) or (wait* list* n/7 control*) or (wait* list* n/7 cohort?) or (wait* list* n/7 random*))) NOT ((transplant* or organ donat*) OR ti(emergency) OR ti(review)))

Communication Disorders Database, ProQuest

Jan 5, 2012 (49 results)
 
 (((((((ti("wait* list*") OR ti("wait* time*") OR ab("wait* list*") OR ab("wait* time*")) AND (health OR healthcare OR primary care OR referral* OR physician* OR doctor* OR non‐urgent OR non‐emerg* OR elective)) NOT ((wait* list* n/7 group*) or (wait* list* n/7 control*) or (wait* list* n/7 cohort?) or (wait* list* n/7 random*))) NOT ((transplant* or organ donat*) OR ti(emergency) OR ti(review))) =49

Dissertations, UK & Ireland; and Dissertations & Theses, ProQuest

November 28, 2013 (35 results)

November 16, 2012 (192 results)

(((((((ti("wait* list*") OR ti("wait* time*") OR ab("wait* list*") OR ab("wait* time*")) AND (health OR healthcare OR primary care OR referral* OR physician* OR doctor* OR non‐urgent OR non‐emerg* OR elective)) NOT ((wait* list* n/7 group*) or (wait* list* n/7 control*) or (wait* list* n/7 cohort?) or (wait* list* n/7 random*))) NOT ((transplant* or organ donat*) OR ti(emergency) OR ti(review))) NOT (((((((ti("wait* list*") OR ti("wait* time*") OR ab("wait* list*") OR ab("wait* time*")) AND (health OR healthcare OR primary care OR referral* OR physician* OR doctor* OR non‐urgent OR non‐emerg* OR elective)) NOT ((wait* list* n/7 group*) or (wait* list* n/7 control*) or (wait* list* n/7 cohort?) or (wait* list* n/7 random*))) AND (randomi?ed or randomly or controlled or trial or study or pilot or effectiveness or organi?ational or improv* or experimental or pre‐intervention or post‐intervention or workshop* or quasiexperiment* or quasi‐experiment* or (time n/10 period?) or (time n/10 series)) )) NOT ((transplant* or organ donat*) OR ti(emergency) OR ti(review)))

ECON‐LIT (ProQuest)

January 31, 2013 (129 results)

November 16, 2012 (158 results)

(ti("wait* list*") OR ab("wait* list*")) AND (health or healthcare or primary care or referral* or physician* or doctor* or non‐urgent or non‐emerg* or elective). No limits

EMBASE Classic + EMBASE < 1947 to 2012 November 08>

1 ((wait or waiting) adj2 (time or times or list or lists)).ti. (2773)
 2 wait target?.ti,ab. (17)
 3 ((wait or waiting) adj2 (time or times or list or lists) adj4 (reduce? or reduction or eliminat$ or lower or fewer or intervention or policy or policies or reform$ or effectiveness or impact or improv$ or organi?ational$ or quality)).ab. (2347)
 4 (eliminat$ patient? wait$ or improv$ patient? wait$ or lower? patient? wait$ or lowering patient? wait$ or reduc$ patient? wait$).ti,ab. (95)
 5 or/1‐4 [Wait Lists/Time‐‐EMTREE not available for this concept] (4802)
 6 (waiting room? or waiting area? or watchful wait$ or "wait and see" or "wait until").ti,ab. (5692)
 7 ((wait$ list? or waitlist?) adj2 (control? or controlled or group? or intervention or trial or study)).ti,ab. (2423)
 8 (waiting room? or waiting area? or watchful wait$ or "wait and see" or "wait until").ti,ab. (5692)
 9 watchful waiting/ (1031)
 10 exp organ transplantation/ or emergency/ or emergency health service/ or emergency ward/ or rescue personnel/ (392259)
 11 (transplant? or transplantation?).ti,ab,hw. (598772)
 12 (emergency or emergencies).ti,hw. (208162)
 13 donor?.ti,ab,hw. (289744)
 14 (cell or cells or cellular or molecular$ or animal? or lab or labs or laborator$).ti. (1982518)
 15 patient? delay$.ti,ab. (1493)
 16 or/6‐15 [terms to exclude] (2840055)
 17 5 not 16 (2796)
 18 controlled clinical trial/ or controlled study/ or randomized controlled trial/ [EM] (3980271)
 19 (book or conference paper or editorial or letter or review).pt. not randomized controlled trial/ [Per BMJ Clinical Evidence filter] (3836581)
 20 (random sampl$ or random digit$ or random effect$ or random survey or random regression).ti,ab. not randomized controlled trial/ [Per BMJ Clinical Evidence filter] (47405)
 21 (animal$ not human$).sh,hw. (3771897)
 22 18 not (or/19‐21) [Trial filter per BMJ CLinical Evidence] (2612246)
 23 intervention?.ti. or (intervention? adj6 (clinician? or collaborat$ or community or complex or DESIGN$ or doctor? or educational or family doctor? or family physician? or family practitioner? or financial or GP or general practice? or hospital? or impact? or improv$ or individuali?e? or individuali?ing or interdisciplin$ or multicomponent or multi‐component or multidisciplin$ or multi‐disciplin$ or multifacet$ or multi‐facet$ or multimodal$ or multi‐modal$ or personali?e? or personali?ing or pharmacies or pharmacist? or pharmacy or physician? or practitioner? or prescrib$ or prescription? or primary care or professional$ or provider? or regulatory or regulatory or tailor$ or target$ or team$ or usual care)).ab. (175866)
 24 (pre‐intervention? or preintervention? or "pre intervention?" or post‐intervention? or postintervention? or "post intervention?").ti,ab. [added 2.4] (10294)
 25 (hospital$ or patient?).hw. and (study or studies or care or health$ or practitioner? or provider? or physician? or nurse? or nursing or doctor?).ti,hw. (1448525)
 26 demonstration project?.ti,ab. (2224)
 27 (pre‐post or "pre test$" or pretest$ or posttest$ or "post test$" or (pre adj5 post)).ti,ab. (80635)
 28 (pre‐workshop or post‐workshop or (before adj3 workshop) or (after adj3 workshop)).ti,ab. (689)
 29 trial.ti. or ((study adj3 aim?) or "our study").ab. (727350)
 30 (before adj10 (after or during)).ti,ab. (439645)
 31 (time points adj3 (over or multiple or three or four or five or six or seven or eight or nine or ten or eleven or twelve or month$ or hour? or day? or "more than")).ab. (9936)
 32 pilot.ti. or (pilot adj (project? or study or trial)).ab. (78137)
 33 (multicentre or multicenter or multi‐centre or multi‐center).ti. (34541)
 34 random$.ti,ab. or controlled.ti. (842407)
 35 (control adj3 (area or cohort? or compare? or condition or design or group? or intervention? or participant? or study)).ab. (555613)
 36 *experimental design/ or *pilot study/ or quasi experimental study/ (5386)
 37 ("quasi‐experiment$" or quasiexperiment$ or "quasi random$" or quasirandom$ or "quasi control$" or quasicontrol$ or ((quasi$ or experimental) adj3 (method$ or study or trial or design$))).ti,ab. (120056)
 38 ("time series" adj2 interrupt$).ti,ab. (913)
 39 or/23‐38 (3656448)
 40 (rat or rats or cow or cows or chicken? or horse or horses or mice or mouse or bovine or animal?).ti. (1594402)
 41 (animal$ not human$).sh,hw. (3771897)
 42 review.ti. (286471)
 43 39 not (or/40‐42) [EPOC Filter 2.4a EMBASE] (3203340)
 44 (17 and 22) not (or/40‐42) [RCT Results] (346)
 45 (17 and 43) not 44 [EPOC Filter Results] (1576)

Cochrane Central Register of Controlled Trials, EBM Reviews, Ovid <October 2012> 
 
 1 Waiting lists/ (204)
 2 ((wait or waiting) adj2 (time or times or list or lists)).ti,kw. (97)
 3 ((wait or waiting) adj2 (time or times or list or lists) adj4 (reduce? or reduction or eliminat$ or lower or fewer or intervention or policy or policies or reform$ or effectiveness or impact or improv$ or organi?ational$ or quality)).ab. (446)
 4 (eliminat$ patient? wait$ or improv$ patient? wait$ or lower? patient? wait$ or lowering patient? wait$ or reduc$ patient? wait$).ti,ab. (4)
 5 or/1‐4 [Waiting Lists/Waiting Times‐MeSH] (658)
 6 (waiting room? or waiting area? or watchful wait$ or "wait and see" or "wait until").ti,ab. (327)
 7 ((wait$ list? or waitlist?) adj2 (control? or controlled or group? or intervention or trial or study)).ti,ab. (1487)
 8 (waiting room? or waiting area? or watchful wait$ or "wait and see" or "wait until").ti,ab. (327)
 9 Watchful Waiting/ (31)
 10 exp transplantation/ (8073)
 11 (transplant? or transplantation?).ti,ab,hw. (14302)
 12 Emergency Service, Hospital/ or Trauma centres/ or exp Emergency Medical Services/ or exp Emergency Treatment/ or Emergencies/ or Emergency Medicine/ (5206)
 13 (emergency or emergencies).ti,hw. (3487)
 14 (cell or cells or cellular or molecular$ or animal? or lab or labs or laborator$).ti. (17817)
 15 patient? delay$.ti,ab. (300)
 16 donor?.ti,ab,hw. (3773)
 17 or/6‐16 [terms to exclude] (40068)
 18 5 not 17 (168)
 19 18 not placebo?.ti,ab,hw. (161)

ACP Journal Club <1991 to October 2012>, Cochrane Database of Systematic Reviews <2005 to October 2012>, Database of Abstracts of Reviews of Effects <4th Quarter 2012>, Health Technology Assessment <4th Quarter 2012>, EBM Reviews, Ovid 
 
 1 Waiting lists/ (22)
 2 ((wait or waiting) adj2 (time or times or list or lists)).ti,kw,hw. (41)
 3 ((wait or waiting) adj2 (time or times or list or lists) adj4 (reduce? or reduction or eliminat$ or lower or fewer or intervention or policy or policies or reform$ or effectiveness or impact or improv$ or organi?ational$ or quality)).ab. (18)
 4 (eliminat$ patient? wait$ or improv$ patient? wait$ or lower? patient? wait$ or lowering patient? wait$ or reduc$ patient? wait$).ti,ab. (0)
 5 or/1‐4 [Waiting Lists/Waiting Times‐MeSH] (58)
 6 (waiting room? or waiting area? or watchful wait$ or "wait and see" or "wait until").ti,ab. (21)
 7 ((wait$ list? or waitlist?) adj2 (control? or controlled or group? or intervention or trial or study)).ti,ab. (44)
 8 (waiting room? or waiting area? or watchful wait$ or "wait and see" or "wait until").ti,ab. (21)
 9 Watchful Waiting/ (0)
 10 exp transplantation/ (257)
 11 (transplant? or transplantation?).ti,ab,hw,kw. (963)
 12 Emergency Service, Hospital/ or Trauma centres/ or exp Emergency Medical Services/ or exp Emergency Treatment/ or Emergencies/ or Emergency Medicine/ (99)
 13 (emergency or emergencies).ti,hw,kw. (411)
 14 (cell or cells or cellular or molecular$ or animal? or lab or labs or laborator$).ti. (987)
 15 patient? delay$.ti,ab. (4)
 16 donor?.ti,ab,hw. (136)
 17 or/6‐16 [terms to exclude] (2297)
 18 5 not 17 (40)
 19 18 not placebo?.ti,ab,hw,kw. (38)

Nursing, ProQuest :

Nov 15, 2012 (231 results)
 
 ((((((ti("wait* list*") OR ti("wait* time*") OR ab("wait* list*") OR ab("wait* time*")) AND (health OR healthcare OR primary care OR referral* OR physician* OR doctor* OR non‐urgent OR non‐emerg* OR elective)) NOT ((wait* list* NEAR/7 group*) OR (wait* list* NEAR/7 control*) OR (wait* list* NEAR/7 cohort?) OR (wait* list* NEAR/7 random*))) AND (randomi?ed OR randomly OR controlled OR trial OR study OR pilot OR effectiveness OR organi?ational OR improv* OR experimental OR pre‐intervention OR post‐intervention OR workshop* OR quasiexperiment* OR quasi‐experiment* OR (time NEAR/10 period?) OR (time NEAR/10 series))) NOT (transplant* OR organ donat*)) NOT ti(emergency)) NOT ti(review)

PAIS, ProQuest

November 15, 2012 (71 results) & Feb 7, 2013 (9 results)

(((ti("wait* list*") OR ti("wait* time*") OR ab("wait* list*") OR ab("wait* time*")) AND (health OR healthcare OR primary care OR referral* OR physician* OR doctor* OR non‐urgent OR non‐emerg* OR elective OR hospital OR hospitals OR hospitalized OR hospitalised OR inpatient* OR outpatient* OR out‐patient* OR surgery)) NOT (emergency OR emergencies OR transplant* OR "organ* donor*" OR donation* OR "wait* list control*")) AND ((quality NEAR/2 improv*) OR (quality NEAR/2 manag*) OR policy OR policies OR organisation* OR organization* OR reform OR reforms OR government* OR qualitativ* OR incentiv* OR theory OR theories)

PILOTS (Published International Literature on Traumatic Stress)

January 5, 2012: Search identified 23 citations, none relevant.

((ti("wait* list*") OR ab("wait* list*")) AND (health or healthcare or primary care or referral* or physician* or doctor* or non‐urgent or non‐emerg* or elective)) NOT (("wait* list*" p/5 group*) OR ("wait* list*" p/5 control*) OR ("wait* list*" p/5 random*))

Political Science, ProQuest

Nov 16, 2012 (106 results)

Social Services Abstracts, ProQuest

November 16, 2012
 January 6, 2012

Strategy 1:

(((((((ti("wait* list*") OR ti("wait* time*") OR ab("wait* list*") OR ab("wait* time*")) AND (health OR healthcare OR primary care OR referral* OR physician* OR doctor* OR non‐urgent OR non‐emerg* OR elective)) NOT ((wait* list* n/7 group*) or (wait* list* n/7 control*) or (wait* list* n/7 cohort?) or (wait* list* n/7 random*))) NOT ((transplant* or organ donat*) OR ti(emergency) OR ti(review))) NOT
 
 (((((((ti("wait* list*") OR ti("wait* time*") OR ab("wait* list*") OR ab("wait* time*")) AND (health OR healthcare OR primary care OR referral* OR physician* OR doctor* OR non‐urgent OR non‐emerg* OR elective)) NOT ((wait* list* n/7 group*) or (wait* list* n/7 control*) or (wait* list* n/7 cohort?) or (wait* list* n/7 random*))) AND (randomi?ed or randomly or controlled or trial or study or pilot or effectiveness or organi?ational or improv* or experimental or pre‐intervention or post‐intervention or workshop* or quasiexperiment* or quasi‐experiment* or (time n/10 period?) or (time n/10 series)) )) NOT ((transplant* or organ donat*) OR ti(emergency) OR ti(review)))

Strategy 2: 
 (((((((ti("wait* list*") OR ti("wait* time*") OR ab("wait* list*") OR ab("wait* time*")) AND (health OR healthcare OR primary care OR referral* OR physician* OR doctor* OR non‐urgent OR non‐emerg* OR elective)) NOT ((wait* list* n/7 group*) or (wait* list* n/7 control*) or (wait* list* n/7 cohort?) or (wait* list* n/7 random*))) AND (randomi?ed or randomly or controlled or trial or study or pilot or effectiveness or organi?ational or improv* or experimental or pre‐intervention or post‐intervention or workshop* or quasiexperiment* or quasi‐experiment* or (time n/10 period?) or (time n/10 series)) )) NOT ((transplant* or organ donat*) OR ti(emergency) OR ti(review)))

Sociological Abstracts, ProQuest

November 16, 2012
 
 (((((((ti("wait* list*") OR ti("wait* time*") OR ab("wait* list*") OR ab("wait* time*")) AND (health OR healthcare OR primary care OR referral* OR physician* OR doctor* OR non‐urgent OR non‐emerg* OR elective)) NOT ((wait* list* n/7 group*) or (wait* list* n/7 control*) or (wait* list* n/7 cohort?) or (wait* list* n/7 random*))) AND (randomi?ed or randomly or controlled or trial or study or pilot or effectiveness or organi?ational or improv* or experimental or pre‐intervention or post‐intervention or workshop* or quasiexperiment* or quasi‐experiment* or (time n/10 period?) or (time n/10 series)) )) NOT ((transplant* or organ donat*) OR ti(emergency) OR ti(review))) = 115

Web of Knowledge Science & Social Sciences Citation Indexes

November 28, 2013 & November 13, 2012

		Web of Knowledge Science & Social Sciences Citation Indexes Search Strategy; Databases=SCI‐EXPANDED, SSCI Timespan=All Years
Search Date		November 13, 2012 to November 28, 2013
Set	Results	Search query
	183	#25 OR #28 [2013 results]
29	733	#25 OR #28 [2012 results]
28	688	(#9 not #16) and #17
		Refined by: [excluding] Web of Science Categories=( TRANSPORTATION SCIENCE TECHNOLOGY OR ENGINEERING MULTIDISCIPLINARY OR FOOD SCIENCE TECHNOLOGY OR OPERATIONS RESEARCH MANAGEMENT SCIENCE OR COMPUTER SCIENCE HARDWARE ARCHITECTURE OR STATISTICS PROBABILITY OR COMPUTER SCIENCE SOFTWARE ENGINEERING OR MATHEMATICAL COMPUTATIONAL BIOLOGY OR ENGINEERING INDUSTRIAL OR PHYSICS FLUIDS PLASMAS OR PERIPHERAL VASCULAR DISEASE OR PHYSICS ATOMIC MOLECULAR CHEMICAL OR ENGINEERING CIVIL OR PLANNING DEVELOPMENT OR PHYSICS CONDENSED MATTER OR SOCIAL SCIENCES MATHEMATICAL METHODS OR TRANSPORTATION OR TRANSPLANTATION OR ENGINEERING MANUFACTURING OR BUSINESS FINANCE OR ENGINEERING ELECTRICAL ELECTRONIC OR AUTOMATION CONTROL SYSTEMS OR ERGONOMICS OR PHYSICS MATHEMATICAL OR HOSPITALITY LEISURE SPORT TOURISM OR COMPUTER SCIENCE INFORMATION SYSTEMS OR INSTRUMENTS INSTRUMENTATION OR NUTRITION DIETETICS OR COMPUTER SCIENCE CYBERNETICS OR PHYSICS MULTIDISCIPLINARY OR BIOTECHNOLOGY APPLIED MICROBIOLOGY OR CELL BIOLOGY OR MATERIALS SCIENCE MULTIDISCIPLINARY OR COMPUTER SCIENCE ARTIFICIAL INTELLIGENCE OR MATHEMATICS OR ENGINEERING CHEMICAL OR COMPUTER SCIENCE THEORY METHODS OR MATHEMATICS INTERDISCIPLINARY APPLICATIONS OR SPORT SCIENCES OR TELECOMMUNICATIONS OR MATHEMATICS APPLIED OR ZOOLOGY )
27	988	(#9 not #16) and #17
26	356	((#9 NOT #16) AND #17) NOT #25
25	632	#23 or #24
24	632	((#9 NOT #16) AND #17) AND TS=(health or healthcare or medicine or hospital* or surgery or surgical or inpatient* or outpatient* or "out‐patient*" or family doctor* or family practitioner* or family physician* or family practice or general practitioner* or general practice or primary care or treatment or therapy or physio* or cancer or oncolog* or pediatrician* or specialist* or orthopedic* or orthopaedic* or hip or hips or joints)
23	397	((#9 NOT #16) AND #17) AND TI=(health or healthcare or medicine or hospital* or surgery or surgical or inpatient* or outpatient* or "out‐patient*" or family doctor* or family practitioner* or family physician* or family practice or general practitioner* or general practice or primary care or treatment or therapy or physio* or cancer or oncolog* or pediatrician* or specialist* or orthopedic* or orthopaedic* or hip or hips or joints)
22	39	(#20 OR #21) AND Document Types=(Review)
21	1	((#9 not #16) AND #19) AND Document Types=(Review)
20	39	((#9 not #16)) AND Document Types=(Review)
19	45613	TI=((systematic NEAR/3 review?) or metaanalys* or meta‐analys* or "literature review")
18	988	(#9 not #16) and #17
17	8525837	(TI=(random* OR trial OR study OR pilot OR piloted or piloting or comparative OR tool OR tools OR innovat* OR organisation* OR organization* OR impact OR influence OR changing OR quality OR implement* or intervention*)) OR TS=(random* OR controlled OR "control group" or "control groups" OR pilot OR innovat* OR organisational* OR organizational* OR impact OR "quality improv*" or quality manag* OR implement* or team or teams or team‐based or multifaceted or multi‐faceted or complex intervention* or cooperat* or "patient focus*" or "physician led" or "nurse led" or "pharmacist led" or "nurse practitioner" or "skill mix")
16	2845710	#15 OR #14 OR #13 OR #12 OR #11 OR #10
15	26	TI=("patient? delay*")
14	2376784	TI=(cell or cells or cellular or molecular* or animal? or lab or labs or laborator*)
13	52661	TI=(emergency or emergencies)
12	127493	TS=("hospital emergency service" or "trauma centres" or "emergency medical services" or "emergency treatment" or emergencies or "emergency medicine" OR "emergency room*")
11	309957	TI=(transplant* or transplantation or organ or organs or cadaver*) OR TS=((organ near/3 donor*) or (organ* near/3 donat*) or (cadaver* near/3 organ*))
10	308791	TS=(transplantation)
9	4348	#5 not (#6 or #7 or #8)
8	1299	TS=("watchful waiting")
7	135	TI=(("wait* list*" or waitlist*) NEAR/2 (control or controlled or group* or intervention or trial or study)) OR TS=(("wait* list?" or waitlist?) NEAR/2 (control or controlled or group* or intervention or trial or study))
6	3716	TI=("waiting room*" or "waiting area*" or "watchful wait*" or "wait and see" or "wait until") OR TS=("waiting room*" or "waiting area*" or "watchful wait*" or "wait and see" or "wait until")
5	4485	#4 OR #3 OR #2 OR #1
4	45	TI=("eliminat* patient* wait*" or "improv* patient* wait*" or "lower* patient* wait*" or "lowering patient* wait*" or "reduc* patient* wait*") OR TS=("eliminat* patient* wait*" or "improv* patient* wait*" or "lower* patient* wait*" or "lowering patient* wait*" or "reduc* patient* wait*")
3	226	Title=((wait or waiting) NEAR/2 (time or times or list or lists) NEAR/4 (reduce? or reduction or eliminat* or lower or fewer or intervention or policy or policies or reform* or effectiveness or impact or improv* or organi?ational* or quality))
2	3537	TI=((wait or waiting) NEAR/2 (time or times or list or lists))
1	1399	Topic=("waiting lists")

Worldwide Political Science Abstracts, ProQuest

Nov 16, 2012 (11 results)
 
 (ti("wait* list*") OR ab("wait* list*")) AND (health or healthcare or primary care or referral* or physician* or doctor* or non‐urgent or non‐emerg* or elective) [No limits]

CINAHL EbscoHost

Run November 12, 2012

#	Query	Results
1	(MH "Quasi‐Experimental Studies")	5,248
2	TI ( intervention* or multiintervention* or multi‐intervention* or postintervention* or post‐intervention* or preintervention* or pre‐intervention* ) or AB ( intervention* or multiintervention* or multi‐intervention* or postintervention* or post‐intervention* or preintervention* or pre‐intervention* )	131,711
3	TI ( pre‐test* or pretest* or posttest* or post‐test* ) or AB ( pre‐test* or pretest* or posttest* or "post test* ) OR TI ( preimplement*" or pre‐implement* ) or AB ( pre‐implement* or preimplement* )	6,201
4	MH Experimental Studies or Community Trials or Community Trials or Pretest‐Posttest Design + or Quasi‐Experimental Studies + Pilot Studies or Policy Studies + Multicenter Studies	30,735
5	TI ( (comparative N2 study) or (comparative N2 studies) or evaluation study or evaluation studies ) or AB ( (comparative N2 study) or (comparative N2 studies) or evaluation study or evaluation studies )	9,402
6	MH "Multiple Time Series" or MH "Time Series"	1,201
7	TI pre w7 post or AB pre w7 post	8,019
8	TI ( ( quasi‐experiment* or quasiexperiment* or quasi‐random* or quasirandom* or quasi control* or quasicontrol* or quasi* W3 method* or quasi* W3 study or quasi* W3 studies or quasi* W3 trial or quasi* W3 design* or experimental W3 method* or experimental W3 study or experimental W3 studies or experimental W3 trial or experimental W3 design* ) ) or AB ( ( quasi‐experiment* or quasiexperiment* or quasi‐random* or quasirandom* or quasi control* or quasicontrol* or quasi* W3 method* or quasi* W3 study or quasi* W3 studies or quasi* W3 trial or quasi* W3 design* or experimental W3 method* or experimental W3 study or experimental W3 studies or experimental W3 trial or experimental W3 design* ) )	10,891
9	TI ( (time point*) or (period* n4 interrupted) or (period* n4 multiple) or (period* n4 time) or (period* n4 various) or (period* n4 varying) or (period* n4 week*) or (period* n4 month*) or (period* n4 year*) ) or AB ( (time point*) or (period* n4 interrupted) or (period* n4 multiple) or (period* n4 time) or (period* n4 various) or (period* n4 varying) or (period* n4 week*) or (period* n4 month*) or (period* n4 year*) )	44,180
10	AB ( before* n10 during or before n10 after ) or AU ( before* n10 during or before n10 after )	29,013
11	TI time series	218
12	AB time series	1,570
13	AB "before‐and‐after"	15,296
14	(MH "Pilot Studies")	26,572
15	TI pilot	10,293
16	TI ( collaborativ* or collaboration* or tailored or personalised or personalized ) or AB ( collaborativ* or collaboration* or tailored or personalised or personalized )	33,761
17	(intervention n6 clinician*) or (intervention n6 community) or (intervention n6 complex) or (intervention n6 design*) or (intervention n6 doctor*) or (intervention n6 educational) or (intervention n6 family doctor*) or (intervention n6 family physician*) or (intervention n6 family practitioner*) or (intervention n6 financial) or (intervention n6 GP) or (intervention n6 general practice*) Or (intervention n6 hospital*) or (intervention n6 impact*) Or (intervention n6 improv*) or (intervention n6 individualize*) Or (intervention n6 individualise*) or (intervention n6 individualizing) or (intervention n6 individualising) or (intervention n6 interdisciplin*) or (intervention n6 multicomponent) or (intervention n6 multi‐component) or (intervention n6 multidisciplin*) or (intervention n6 multi‐disciplin*) or (intervention n6 multifacet*) or (intervention n6 multi‐facet*) or (intervention n6 multimodal*) or (intervention n6 multi‐modal*) or (intervention n6 personalize*) or(intervention n6 personalise*) or (intervention n6 personalizing) or (intervention n6 personalising) or (intervention n6 pharmaci*) or (intervention n6 pharmacist*) or (intervention n6 pharmacy) or (intervention n6 physician*) or (intervention n6 practitioner*) Or (intervention n6 prescrib*) or (intervention n6 prescription*) or (intervention n6 primary care) or (intervention n6 professional*) or (intervention* n6 provider*) or (intervention* n6 regulatory) or (intervention n6 regulatory) or (intervention n6 tailor*) or (intervention n6 target*) or (intervention n6 team*) or (intervention n6 usual care)	36,648
18	TI ( demonstration project OR demonstration projects OR preimplement* or pre‐implement* or post‐implement* or postimplement* ) or AB ( demonstration project OR demonstration projects OR preimplement* or pre‐implement* or post‐implement* or postimplement* )	1,191
19	TI ( pre‐workshop or preworkshop or post‐workshop or postworkshop or (before n3 workshop) or (after n3 workshop) ) or AB ( pre‐workshop or preworkshop or post‐workshop or postworkshop or (before n3 workshop) or (after n3 workshop) )	283
20	TI ( trial or (study n3 aim) or "our study" ) or AB ( (study n3 aim) or "our study" )	73,438
21	TI random* OR controlled	30,013
22	TI ( multicentre or multicenter or multi‐centre or multi‐center ) or AB random* or AB ( (multicent* n2 design*) or (multicent* n2 study) or (multicent* n2 studies) or (multicent* n2 trial*) )	90,896
23	TI ( (control w3 area) or (control w3 cohort*) or (control w3 compar*) or (control w3 condition) or (control w3 group*) or (control w3 intervention*) or (control w3 participant*) or (control w3 study) ) or AB ( (control w3 area) or (control w3 cohort*) or (control w3 compar*) or (control w3 condition) or (control w3 group*) or (control w3 intervention*) or (control w3 participant*) or (control w3 study) )	41,234
24	TI ( (time points n3 over) or (time points n3 multiple) or (time points n3 three) or (time points n3 four) or (time points n3 five) or (time points n3 six) or (time points n3 seven) or (time points n3 eight) or (time points n3 nine) or (time points n3 ten) or (time points n3 eleven) or (time points n3 twelve) or (time points n3 month*) or (time points n3 hour*) or (time points n3 day*) or (time points n3 "more than") ) or AB ( (time points n3 over) or (time points n3 multiple) or (time points n3 three) or (time points n3 four) or (time points n3 five) or (time points n3 six) or (time points n3 seven) or (time points n3 eight) or (time points n3 nine) or (time points n3 ten) or (time points n3 eleven) or (time points n3 twelve) or (time points n3 month*) or (time points n3 hour*) or (time points n3 day*) or (time points n3 "more than") )	1,347
25	S1 or S2 or S3 or S4 or S5 or S6 or S7 or S8 or S9 or S10 or S11 or S12 or S13 or S14 or S15 or S16 or S17 or S18 or S19 or S20 or S21 or S22 or S23 or S24	380,833
26	(MM "Clinical Trials+")	7,527
27	TI ( “clinical study” or “clinical studies” ) or AB ( “clinical study” or “clinical studies” )	6,281
28	TI random* or AB random*	97,366
29	TI controlled or AB controlled	56,119
30	TI ( “control* N1 clinical” or “control* N1 group*” or “control* N1 trial*” or “control* N1 study” or “control* N1 studies” or “control* N1 design*” or “control* N1 method*” ) or AB ( “control* N1 clinical” or “control* N1 group*” or “control* N1 trial*” or “control* N1 study” or “control* N1 studies” or “control* N1 design*” or “control* N1 method*” )	1
31	#26 or #27 or #28 or #29 or #30	131,204
32	(MH "Waiting Lists")	2,403
33	TI wait list or wait lists or wait time or wait times	232
34	TI wait target* OR AB wait target*	8
35	AB (reduc* or eliminat* or decreas* or short*) n3 (wait or waiting)	606
36	S32 OR S33 OR S34 OR S35	2,934
37	(MH "Transplantation+")	20,147
38	(MH "Transplant Donors")	2,795
39	(MH "Waiting Rooms")	308
40	TI ( (waiting room* or waiting area or waiting areas ) ) OR AB ( (waiting room* or waiting area or waiting areas ) )	621
41	(MH "Emergencies")	3,961
42	(MH "Emergency Medical Services+") OR (MH "Emergency Service")	45,999
43	(MH "Emergency Nursing") OR (MH "Emergency Nurses Association") OR (MH "Emergency Nurse Practitioners") OR (MH "Emergency Care")	23,948
44	TI ( emergency or emergencies ) OR AB ( emergency or emergencies )	42,206
45	TI ( transplant* or donor or donors or (organ* n3 donat*) ) OR AB ( transplant* or donor or donors or (organ n3 donat*) )	21,682
46	S37 OR S38 OR S39 OR S40 OR S41 OR S42 OR S43 OR S44 OR S45	115,127
47	S36 NOT S46	1,798
48	S31 and S47	105
49	( S25 and S47 ) NOT s48	359

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Hofstetter 2010.

Methods	Study design: re‐analysed ITS study Aim: to determine the relationship between use of telemedicine consultations and changes in patient waiting times, access to care and travel‐related costs Timing: 1992‐2007; before intervention period: from 1992 to December 2001; after intervention period: from January 2002 to 2007 Data collection: retrospective data collected from routine records
Participants	Providers: 2 units, 1 audiologist at Norton Sound Health Corporation in Nome and 1 consulting ear nose throat (ENT) specialist at Alaska Native Medical Centre Participants: All 1690 new patients referred to ENT from 1992 to 2007 (people not previously seen by ENT but for whom the opinion or care of the ENT specialist is requested); unknown number of participants in preintervention and postintervention periods Participant baseline characteristics: Age: no information Gender: no information Ethnicity: no information Clinical problem: audiology patients with a medical need (treatment or surgery) who required an ENT specialist consultation Setting: rural area of Alaska Country: USA (state of Alaska)
Interventions	Type of intervention: intervention aimed at restructuring the referral process Intervention: telemedicine: audiology‐to‐ENT store and forward telemedicine consultation (from village audiology centre to main city hospital). Store and forward technology allows an image, video clip, scanned documents or specific test results to be captured in electronic format and then forwarded on to a provider. This differs from real‐time consultations wherein the consultant actually sees and talks to the patient Control: ENT face‐to‐face visit in main city hospital  Duration of intervention: 5 years (2002‐2007)
Outcomes	Waiting time (months) to see an ENT specialist (outcome included in this review) Percentage of participants seen within 1‐2‐3‐4 months Costs (airfare, and costs for travel escorts if needed)
Notes	Only participants requiring medical treatment or surgery were referred to ENT specialist consultations, whereas the audiologist took care of diagnosing and rehabilitating participants with ear, hearing and vestibular disorders. Even if the intervention had a beneficial effect on waiting time to ENT specialist consultation/examination, it remains unclear whether participants who received telehealth consultations underwent more timely treatment/surgery than those receiving face‐to‐face consultations Study authors provided no information on the number of unsuccessful telehealth appointments for which a face‐to‐face appointment was required
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent of other changes (ITS)?	Low risk	Number of available slots stable during the study period; no change in supply of in‐person appointments
Analysed appropriately (ITS)?	Low risk	Data reanalysed by review authors
Shape of the intervention effect pre‐specified (ITS)?	Low risk	Data reanalysed by review authors
Intervention unlikely to affect data collection (ITS)?	Low risk	Data retrospectively collected from routine records; therefore low risk of bias
Knowledge of the allocated interventions adequately prevented during the study (ITS)?	Low risk	Outcomes objective in nature; thus unlikely to be affected by possible unblinded assessment
Incomplete outcome data addressed (attrition bias) (ITS)?	Unclear risk	No information provided; not clear whether a truncation had occurred at the end of 2007
Free of selective outcome reporting (reporting bias) (ITS)?	High risk	Study protocol not available ‐ so outcomes reported in the paper cannot be checked against any prespecified outcomes. However, it appears that different details were measured during the preintervention and postintervention periods. Before intervention, the waiting time for the participant to see the consultant face‐to‐face, and presumably get treatment, was measured. After intervention, the waiting time until the consultant sees the store and forward participant data sent by telemedicine was measured. But this is not when the participant gets treatment because no information is available on when the consultant diagnosis and treatment are given to the participant
Free of other bias (ITS)?	Unclear risk	No information about whether or not all telehealth consultations were successful, that is, whether all participants who received a specialist consultation through telemedicine could be diagnosed, or if some had to see a specialist face‐to‐face. Telemedicine techniques and equipment may have improved over the study years, thus improving the accuracy of diagnoses made by the consultant

Leach 2004.

Methods	Study design: re‐analysed ITS study Aim: to evaluate effects of implementation of 2 systems for managing generic outpatient waiting list on meeting national targets (3 months for routine outpatient appointment; 6 months for inpatient treatment) Timing: from June 2001 to November 2002; before intervention period: from June 2001 to mid September 2001; after intervention period: from mid September 2001 to November 2002 Data collection: data collected before and after the intervention; not further described
Participants	Providers: consultants and secretariat for integration of MRI appointment; number of providers not reported Participants: outpatients referred to neurosurgical services of Hope Hospital site in Salford for elective non‐complex spinal surgery (clear‐cut signs or symptoms of cervical or lumbar neural compromise, and no obvious underlying disease that might require spinal fixation, e.g. rheumatoid arthritis). Total number of participants not provided Participant baseline characteristics: Age: no information Gender: no information Ethnicity: no information Type of spinal surgery: no information Clinical problem: non‐complex (elective) spinal surgery Setting: all neurosurgical services within Greater Manchester Country: UK
Interventions	Type of intervention: intervention aimed at restructuring the referral process Intervention: managed generic (pooled) waiting lists for both initial outpatient appointments and dates for surgery + a computerised MRI booking system integrated with outpatient review appointments The managed generic outpatient waiting list begins with a consultant screening all new GP‐referred spinal cases to assess their suitability for inclusion in a pooled waiting list. Participants are then allocated to the next available appointment, irrespective of who the consultant might be. The managed generic surgical waiting list works through a similar process. When consultants list a patient for elective non‐complex spinal surgery, they indicate whether the patient should remain under their care or be entered onto a pooled waiting list. Pooled patients are then allocated dates for surgery sequentially vs Control: Each consultant managed his or her own waiting list Duration of intervention: approximately 15 months (mid September 2001‐November 2002)
Outcomes	Time from referral to first outpatient appointment Time from scan to outpatient review Time on waiting list for surgery Number of participants waiting < 9 months; between 9 and 18 months; longer than 18 months (outcomes included in the review
Notes	Waiting time thresholds appear arbitrary and not based on national recommendations for maximum waiting times as described in the introduction of the paper In our reanalysis, we considered as an intervention only the introduction of the generic waiting list for spinal surgery because data collection points related to implementation of a computerised MRI booking system integrated with outpatient review appointments in the preintervention period were insufficient to perform the analysis (< 3)
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent of other changes (ITS)?	Unclear risk	Quote: "To reduce waiting times, we employed two strategies. First, managed generic waiting lists were introduced for both initial outpatient appointments and dates for surgery. Subsequently the computerized MRI booking system was integrated with outpatient review appointments" The 2 interventions were implemented at different times; it is unclear whether there would be an impact on the waiting times on the surgical list. No information was given on other possible concurrent interventions
Analysed appropriately (ITS)?	Low risk	Reanalysed as ITS by review authors
Shape of the intervention effect pre‐specified (ITS)?	Low risk	Data reanalysed by review authors
Intervention unlikely to affect data collection (ITS)?	Unclear risk	No information given
Knowledge of the allocated interventions adequately prevented during the study (ITS)?	Low risk	Outcomes objective in nature; thus unlikely to be affected by a possible unblinded assessment
Incomplete outcome data addressed (attrition bias) (ITS)?	Unclear risk	No information given
Free of selective outcome reporting (reporting bias) (ITS)?	Unclear risk	Study protocol not available ‐ so outcomes reported in the paper cannot be checked against any prespecified outcomes
Free of other bias (ITS)?	Unclear risk	Only 7 out of 10 consultants participated. Not clear whether preintervention and/or postintervention data relate to waiting times for all 10 surgeons, or only for the 7 who agreed to the intervention. In addition, a problem was observed with discrepancies between numbers reported in the figures ‐ with a greater number of participants undergoing surgery than was reported among those referred by the GP

Leggett 2004.

Methods	Study design: RCT Aim: to establish whether instant photography allows a correct dermatological diagnosis and reduces the number of patients needing an outpatient appointment with a dermatologist Unit of allocation: participants Unit of analysis: participants Unit of analysis issue: no (participants were the unit of both allocation and analysis) Stratification: not done Timing: not reported Data collection: prospective recording by investigators
Participants	Providers: 10 GPs participating from 5 practices (but 20 agreed to participate) Patients: 136 patients referred to a GP for a dermatological problem Clinical problem: dermatology referrals Setting: general practices and a teaching hospital Country: UK
Interventions	Type of intervention: intervention aimed at restructuring the referral process Intervention: Instant photography was taken by GP and was inserted with the referral letter into a sealed numbered envelope. Study group letters were sent directly to the dermatologist: If diagnosis was possible, a letter was sent to the GP with management and appointment for further management if needed; if diagnosis was not possible, an appointment with a dermatologist was booked Control: Instant photography was taken by GP and was inserted with the referral letter into a sealed numbered envelope. Control group envelopes had photographs removed and appointments made as usual 1 camera was placed in each practice, and GPs were trained for 15 minutes in its use Duration of intervention: not reported
Outcomes	Primary: Number of participants needing an initial outpatient appointment with a dermatologist Secondary:   Number of participants with a photo‐diagnosis who did not need to be seen by a dermatologist Number of participants with a photo‐diagnosis who needed to be seen by a dermatologist Number of participants for whom a photo‐diagnosis was not possible Waiting times for an appointment with a dermatologist
Notes	This is a feasibility study conducted to assess possible adverse effects; waiting time monitored to assess whether participants initially assessed via photography but for whom diagnosis was not possible suffered from longer waiting times for an appointment than participants not assessed via photography
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: "The GP took photograph(s) of the skin condition and sent them with a referral letter to the dermatologist in a numbered, sealed envelope. The numbers previously were allocated randomly to study and control groups using a computer program"
Allocation concealment (selection bias)	Low risk	Quote: "Group allocations were only revealed at hospital where photographs were removed from control group letters and appointments were made as usual"
Blinding of participants and personnel (performance bias) Waiting time and number of visits to GPs before and after operation	Low risk	Blinding was not possible; however, waiting time is an objective outcome
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Not specified whether investigators were blinded, but given the objective nature of the assessed outcomes, whether the investigators were blinded was not likely to affect study results
Incomplete outcome data (attrition bias) All outcomes	High risk	Only 18 of 71 (25.4%) intervention participants received a photo‐diagnosis and did not need to be seen by a dermatologist; 27 participants (38.0%) needed to be seen face‐to‐face for further management; for 26 intervention participants (36..6%), photo‐diagnosis was not possible
Selective reporting (reporting bias)	Unclear risk	Impossible to check against protocol (protocol not published)
Baseline characteristics similar?	Low risk	Quote: "Study and control groups were similar in age (range: 5 months–94 years; mean 38.5 years, SD 23.2), gender [55 (40%) male; 81 (60%) female], numbers of patients not attending appointments and range of diagnoses (Table 1)"
Baseline outcomes similar?	Low risk	Not possible to provide baseline data for the outcome of interest
Free of contamination?	Low risk	Participants were the unit of randomisation, but despite this, It is unlikely that control group participants received the intervention
Other bias	Low risk	No obvious other risk of bias was identified

Lowthian 2011.

Methods	Study design: re‐analysed ITS study Aim: to show whether streamlining perioperative services reduces hospital‐initiated postponement (HIP), decreases numbers of patients waiting for elective surgery beyond nationally recommended waiting periods or increases hospital surgical treatment capacity Timing: before intervention period: February 2005 to January 2007; intervention introduction: February 2007; after intervention period: February 2007 to February 2010 Data collection: retrospective data collected from an administrative database: de‐identified patient data drawn from a computerised patient management system tracking people from admission to discharge
Participants	Providers: 1 public hospital Participants: patients requiring elective surgery; total number of participants not given Participant baseline characteristics: Age: no information Gender: no information Ethnicity: no information Clinical problem: elective surgery Setting: tertiary hospital Country: Australia
Interventions	Type of intervention: intervention aimed at prioritising demand (and expanding capacity) Intervention: Intervention 1: May 2006 ‐ initial redesign process centred on clinical leadership and a dedicated management structure to co‐ordinate all components of the new service (the streamlining procedure) initiated, with appointment of the perioperative services manager and co‐ordinators for each surgical unit Intervention 2: Final separation of the 3 elective surgery streams, which began in February 2007 with the opening of the Alfred Centre and the new short‐stay beds (< 3 days) Intervention 3: Main Alfred Hospital short‐stay beds (> 3 days and < 5 days) were available from mid 2008 Control: no streamlining, with unplanned emergency surgery competing with scheduled elective surgery Duration of intervention: 3 years (February 2007‐February 2010)
Outcomes	% of participants waiting longer than 30 days, 90 days and 365 days, respectively Hospital‐initiated postponement Staff satisfaction Productivity
Notes	During the intervention period, 3 different interventions were implemented over time, of which streamlining of services was one, which complicates reanalysis and interpretation of results
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent of other changes (ITS)?	High risk	Initial redesign process started in May 2006, i.e. during the period used as the control period in the reanalysis. Additional short‐stay beds were made available from mid 2008; also, from April 2008 to January 2010, elective surgery throughout the Alfred Centre was reduced during further building works, which may have impacted outcomes
Analysed appropriately (ITS)?	Low risk	Quote: "Comparing data from February 2010 with February 2005, there was a 45% decrease in the numbers of Category 2 patients (semi urgent) waiting longer for surgery than the recommended time of < 90 days" Comment: reanalysed as ITS
Shape of the intervention effect pre‐specified (ITS)?	Low risk	Data reanalysed by review authors
Intervention unlikely to affect data collection (ITS)?	Low risk	Retrospective data collection Quote: "Data comprising aggregated monthly figures and patient information were extracted and de‐identified by the Clinical Performance Unit from the computerised patient‐management system (HOMER), which tracks patients from admission to discharge. [...] Data from 12 months before (February 2005 – February 2006) and after (February 2009 – February 2010) the process redesign were analysed"
Knowledge of the allocated interventions adequately prevented during the study (ITS)?	Low risk	Outcomes objective; thus unlikely to be affected by possible unblinded assessment
Incomplete outcome data addressed (attrition bias) (ITS)?	Low risk	Quote: "Data comprising aggregated monthly figures and patient information were extracted and de‐identified by the Clinical Performance Unit from the computerised patient‐management system (HOMER), which tracks patients from admission to discharge. Aggregated monthly data included: summaries of all elective surgery procedures performed; [...]; numbers of elective surgery patients waiting longer than nationally recommended maximum waiting times (including patients ready and not ready for care)" Comment: Study authors looked at all operations ‐ so data should be complete
Free of selective outcome reporting (reporting bias) (ITS)?	Unclear risk	Study protocol not available ‐ so outcomes reported in the paper cannot be checked against any prespecified outcomes
Free of other bias (ITS)?	Unclear risk	Discrepancies noted between figures ‐ Figure 2 plots the number of participants waiting too long for elective surgery. Figure 6 plots the number of elective surgery admissions. According to these figures, the number of admissions is lower than the number of participants who had to wait too long, which does not make sense, It is unclear whether study authors each month added up the number of people who were currently waiting too long and were still waiting. So a person may appear on the graph for several consecutive months, that is, from the time they went over the recommended time to the time they underwent surgery. We contacted the study authors for clarification but received no response

Lukman 2004.

Methods	Study design: reanalysed ITS study Aim: to review results for the 2 years following introduction of direct booking for colposcopy Timing: before intervention period: December 2000 to August 2001; intervention period: September 2001; after intervention period: September 2001 to August 2003 Data collection: Participant information is collected and stored using extensively used regional database; data from this are used to produce the returns required nationally
Participants	Provider: laboratory Participants: women with abnormal cervical cytology needing colposcopy; 2501 women with abnormal cytology referred through direct booking Participant baseline characteristics: Age: no information Gender: 100% female Ethnicity: no information Clinical problem: moderate or severe abnormal cervical cytology Setting: public health setting, in Portsmouth and South East Hampshire Country: UK
Interventions	Type of intervention: intervention aimed at restructuring the referral process Intervention: direct booking for colposcopy clinic: Participant was informed directly by the laboratory of an abnormal result and was given information about the need for colposcopy and a telephone number to call to set an appointment. If the participant did not contact the clinic within 7 days, she received a second letter in max 14 days; if the participant did not contact the clinic within 14 days, letter to GP + monitoring of participant by clinic; co‐intervention: national guidelines, patient information leaflet, extra clinics organised to manage initial extra demand, GP informed 7 days earlier. A special colposcopy nurse was appointed in September 2002 Control: GPs received lab results from cervical cytology and contacted the participant; participant saw the GP who explained the need for a colposcopy appointment; GP referred the participant to the colposcopy service, and the appointment for the colposcopy clinic was then sent to the participant Duration of intervention: 24 months
Outcomes	Percentage of women seen within recommended time (8 weeks or 4 weeks according to the severity of the lesion) Percentage of women adhering to the direct booking programme
Notes	‐
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent of other changes (ITS)?	High risk	Quote: "We organized extra clinics to meet the extra demand" Also, a second intervention (introduction of a dedicated colposcopy nurse) was put in place during the intervention period, which may have affected the results
Analysed appropriately (ITS)?	Low risk	Reanalysed as ITS
Shape of the intervention effect pre‐specified (ITS)?	Low risk	Data reanalysed by review authors
Intervention unlikely to affect data collection (ITS)?	Low risk	Quote: "Patient information is collected and stored using an approved database used extensively within the region"
Knowledge of the allocated interventions adequately prevented during the study (ITS)?	Low risk	Outcomes objective; thus unlikely to be affected by possible unblinded assessment
Incomplete outcome data addressed (attrition bias) (ITS)?	Unclear risk	No information given
Free of selective outcome reporting (reporting bias) (ITS)?	Unclear risk	Study protocol not available ‐ so outcomes reported in the paper cannot be checked against any prespecified outcomes
Free of other bias (ITS)?	Unclear risk	Figure 2 (showing the percentage of participants meeting guidelines on standards of waiting time for patients with abnormal cytology and source of data for analysis) seems to include data for all types of referrals (inadequate smears, abnormal cytology or other referrals), but the study aims to assess the impact of direct booking on waiting time from abnormal cytology smear referral to colposcopy clinic

Mallard 2004.

Methods	Study design: reanalysed ITS study Aim: to test the following propositions in support of same‐day scheduling, using actual data from a public health clinic: Same‐day scheduling will decrease patient waiting time to see a provider Same‐day scheduling will decrease the number of no‐shows at the clinic Same‐day scheduling will increase the number of new patients seeking services at the clinic Same‐day scheduling will increase provider productivity Timing: preintervention period: January 2001 to June 2001; intervention period: July 2001 to August 2001; postintervention period: September 2001 to February 2002 Data collection: not reported
Participants	Providers: paediatricians, nurses; number of providers not given Participants: outpatients calling for routine visits at a public paediatric health clinic; preintervention period: 4063 appointments and 78 new patients/mo; postintervention period: 3531 appointments and 95 new patients/mo Baseline characteristics of participants: Age: no information Gender: no information Ethnicity: no information Clinical problem: all conditions treated at the outpatient public paediatric health clinic (primary health care) Setting: outpatient paediatric clinic, urban Country: USA (state of Alabama)
Interventions	Type of intervention: intervention aimed at restructuring the referral process Intervention: open access/same‐day scheduling: 30% of participants on prescheduled appointments, 70% booked on same day of telephone call; only 3 types of appointments (routine exam, ill patient, recheck); same length of time for all appointments; cap on total number of participants to be seen in a day; no double booking allowed; appointment clerks started an hour earlier; participants calling were asked whether they were willing for same‐day appointment or should call another day; 3531 appointments Control: complex appointment guidelines to differentiate need for appointment scheduling; many participants walking in despite a no‐today‐appointment answer. Appointments made on the basis of next place available; 4063 appointments Duration of intervention: 8 months
Outcomes	Mean waiting time from call to visit (outcome included in review) Attendance rates Number of new patients Provider productivity
Notes	‐
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent of other changes (ITS)?	Low risk	Quote: "Since the first two months following the initiation of the project were already booked according to the prior guideline, the clinicians doubled up and saw both the existing appointments and the same‐day scheduled patients" Comment: Nevertheless, as analysis of data excluded the period of extra activity, risk of bias is presumed to be low
Analysed appropriately (ITS)?	Low risk	Reanalysed as ITS
Shape of the intervention effect pre‐specified (ITS)?	Low risk	Data reanalysed by review authors
Intervention unlikely to affect data collection (ITS)?	Unclear risk	No information given on data collection
Knowledge of the allocated interventions adequately prevented during the study (ITS)?	Low risk	Outcomes objective; thus unlikely to be affected by a possible unblinded assessment
Incomplete outcome data addressed (attrition bias) (ITS)?	Unclear risk	No information given
Free of selective outcome reporting (reporting bias) (ITS)?	Unclear risk	Study protocol not available ‐ so outcomes reported in the paper cannot be checked against any prespecified outcomes
Free of other bias (ITS)?	Unclear risk	Unclear definition on how waiting time was calculated; unclear also how waiting time for prescheduled appointments contributed, i.e. appointments for which shorter wait time presumably was not desired

McKessock 2001.

Methods	Study design: cluster‐RCT Aim: to establish and evaluate a new referral service for women referred to laparoscopic sterilisation, and to report on some methodological issues Unit of allocation: GP practices Unit of analysis: participants Unit of analysis issue: yes, as practices were randomly assigned and participants analysed; unclear also if effects of clustering were taken into account in the analysis Stratification: practices randomly assigned from prestratified lists according to size, location, etc Timing: 1 June 1996 to 31 March 1997 Data collection: prospective data collection through specific questionnaires and hospital records
Participants	Providers: gynaecologists, 230 general practitioners from 57 general practices, and nurses Participants randomly assigned: n = 232; intervention: 75, control: 157 referred for laparoscopic sterilisation Participants withdrawn or lost to follow‐up: intervention: n = 65, control: n = 57 (35 participants later crossed over from control to intervention group again) Baseline characteristics of participants: Age: no information Gender: 100% female Ethnicity: no information Clinical problem: referral for laparoscopic sterilisation Setting: general practices and hospital in the Grampian region, Aberdeen Royal Infirmary Country: Scotland, UK
Interventions	Type of intervention: intervention aimed at restructuring the referral process Intervention: direct referral for laparoscopic sterilisation: GP refers participant directly to clinic ‐ Gynaecology Outpatient Department, GOPD ‐ for laparoscopic sterilisation, bypassing referral to gynaecologist Control: routine referral through GOPD (GP refers to clinic’s gynaecologist, who then refers to sterilisation) GPs randomly assigned to direct referral (intervention) were supplied with a referral pack, which included: Referral criteria (drawn by a multi‐disciplinary team including gynaecologists, GPs and health service researchers, and based on knowledge and expert opinion of those involved and evidence of risk factors for regret following sterilisation from the literature; widespread consultation with local gynaecologists and GPs was carried out before the referral criteria were finalised) Detailed referral sheet Structured referral pro forma (concerning relevant participant history, examination details and counselling provided, which were subsequently sent to the research nurse) Patient information booklet (adapted from Document of Royal College of Obstetricians and Gynaecologists) to be given to the participant by the GP A newsletter kept practices up‐to‐date with the study's progress and encouraged continuing participation; a "theatre list" of two consultants was dedicated to direct referral sterilisation (separate waiting list) Duration of intervention: 10 months
Outcomes	Primary: Participant satisfaction Operative complication rate Participant and NHS costs Secondary:   Short‐term regret GP and gynaecologist satisfaction Waiting time from referral to operation (outcome included in this review) GP adherence to direct referral criteria
Notes	This study was terminated before the expected date after discussions with the funding body, as lower than expected recruitment rates made timely completion impossible. Findings of this trial were reported despite the small numbers, as important lessons were learnt
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Quote: "Randomization was by referring practice to encourage consistency of referral behaviour and to avoid administrative complications within practices. Participating practices were randomised into intervention and control groups stratified by list size, fund holding status and rural or urban location. The practices in the eight blocks were placed into different coloured envelopes (one colour for each block). A coin was flipped to decide to commence with intervention or control allocation. From the first block of small list practices an envelope was selected and at the same time an envelope from a large list block was selected. These practices were randomised to the same arm of the trial and this process continued, allocating control or intervention alternately until all practices in those two groups were allocated into one of the two arms of the trial: one receiving and implementing the guidelines (intervention arm) and one maintaining the status quo (control arm). This process was carried out for each of the eight blocks" Comment: Method of randomisation is obscure: unclear whether envelopes were opaque and numbered; also unclear whether the envelopes were shuffled
Allocation concealment (selection bias)	Unclear risk	See quote above. Comment: Actual method of allocation is obscure. Unclear whether envelopes were opaque and numbered; also unclear whether the envelopes were shuffled
Blinding of participants and personnel (performance bias) Waiting time and number of visits to GPs before and after operation	Low risk	No masking possible but unlikely to affect results (objective nature of the outcomes of interest of the review)
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Unclear whether assessors were blinded, but results unlikely to be affected (objective nature of the outcomes of interest of the review)
Incomplete outcome data (attrition bias) All outcomes	High risk	In the control arm, 57/157 (36.3%) participants withdrew from the trial; among participants randomly assigned to the intervention group, only 10 of 75 received the intervention
Selective reporting (reporting bias)	Unclear risk	Protocol not available; not possible to check whether all prespecified outcomes have been evaluated
Baseline characteristics similar?	Unclear risk	Baseline data only partially reported Quote: "There were no significant differences found in patients' characteristics between control and intervention groups" See Table 1. Referral criteria ‐ suitability for direct referral Comment: This was according to study authors; however, they reported mainly on different clinical criteria, and age was the only non‐clinical characteristic reported
Baseline outcomes similar?	Low risk	Not possible to provide baseline outcome data for the outcome of interest
Free of contamination?	High risk	A large proportion (65/75, 86.7%) of participants assigned to the experimental/intervention group were treated by the standard referral procedure (not eligible for or refused direct referral)
Other bias	Low risk	No obvious other risk of bias identified

Thomas 2003.

Methods	Study design: cluster‐RCT (2 × 2 balanced incomplete block design) Aim: to establish whether guideline‐based open access reduces outpatient waiting times, provides a management decision earlier, completes hospital care sooner and reduces hospital management cost Unit of allocation: primary general practices (Grampian, Scotland) Unit of analysis: participants Unit of analysis issue: adjustment for preintervention data and clustering of participants by practice Stratification: by location and fund holding status Timing: preintervention: February to July 1995 (intervention introduced in August 1995); post‐intervention: August 1995 to May 1996 Data collection: Data were collected from hospital medical records 12 months after referral to determine waiting time to initial appointment and dates of management decision and discharge from hospital care; routine data on waiting times for all new referrals to urology were obtained from GUHT
Participants	Providers: 66 general practices from Grampian region, Scotland, UK, referring for LUTS and MH Participants randomly assigned: n = 959 participants with suspected LUTS or MH: intervention: n = 479; control: n = 480 Participants withdrawn or lost to follow‐up: intervention: no retrievable information; control: no retrievable information Baseline characteristics of participants: Age: 60 years old Gender: 25% female Ethnicity: no retrievable information Clinical condition: lower urinary tract symptoms (LUTS) and microscopic haematuria (MH) Setting: general practices and Grampian Uniiversity Hospital NHS Trust (GUHT) Country: Scotland, UK
Interventions	Type of intervention: intervention aimed at restructuring the referral process Intervention: guideline‐based open access investigation service for LUTS or MH (GPs to refer participants directly from primary care for the day case investigations service at GUHT); participating GPs were offered a 2‐hour educational meeting and were mailed a guideline package (including a guideline booklet for management and referral to the new direct access service for patients with MH/LUTS, quick reference flowchart and structured referral checklists) Control: standard referral system (participants attended an initial outpatient appointment, either at GUHT or at 1 of 3 peripheral outpatient clinics, and at least 1 further appointment for routine day case investigations at GUHT, before a management decision) Duration of intervention: 10 months (August 1995‐May 1996)
Outcomes	Waiting time from referral to initial hospital appointment (outcome included in the review) Number of participants with a management decision reached at initial appointment and discharged by 12 months after referral Compliance with guidelines (number of recommended investigations completed) Number of general practice consultations Number and case mix of referrals Costs
Notes	Satisfaction of GPs also investigated for possible side effects (e.g. increased GP workload).
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: "[General practices] were randomised by a statistician independent of the research team using computer‐generated numbers (stratified by location and fund holding status)"
Allocation concealment (selection bias)	Low risk	Quote: "[General practices] were randomised by a statistician independent of the research team using computer‐generated numbers (stratified by location and fund holding status)"
Blinding of participants and personnel (performance bias) Waiting time and number of visits to GPs before and after operation	Low risk	Unfeasible but not likely to affect study results as the outcome of interest (waiting time) is objective
Blinding of outcome assessment (detection bias) All outcomes	Low risk	During data collection and entry, researchers were blind to the intervention status of the general practices. The outcome of interest (waiting time) is objective
Incomplete outcome data (attrition bias) All outcomes	Low risk	In total, 10/76 practices (7 from LUTS intervention and 3 from MH intervention) were excluded from analysis, as possibly picked up guideline‐based open access referrals only. Intention‐to‐treat analysis declared as carried out
Selective reporting (reporting bias)	Unclear risk	Protocol not available; not possible to check whether all prespecified outcomes have been evaluated
Baseline characteristics similar?	Low risk	Baseline characteristics similar (information provided in supplementary Table 2)
Baseline outcomes similar?	Low risk	Baseline outcome measures for outcome of interest similar (information provided in supplementary Table 2)
Free of contamination?	Low risk	As this was a cluster‐RCT, the risk of the control group receiving the intervention must be considered low
Other bias	Low risk	Low risk of unit of analysis error; quote: "All outcome measures except the number of referrals, costs and waiting time for all urology referrals were analysed using the patient as the unit of analysis and multilevel modelling using MLWiN version 1.01 to account for the clustering of patients within practices" Comment: Waiting times for LUTS and MH were calculated while adjusting for the cluster effect Quote: "However, control patients also experienced a reduction in waiting time. This was partly because of the increase in the available number of new out‐patient slots as intervention group patients referred to the guideline‐based open access service bypassed the initial outpatient appointment. This dilutes the effect of the intervention. Thus the effects found in this study are likely to be underestimates of the true effect of the intervention" Comment: The described phenomenon could have reduced the effect of the intervention&&

GOPD: Gynaecology Outpatient Department.

GP: general practitioner.

GUHT: Grampian University Hospital NHS Trust.

HIP: hospital‐initiated postponement.

ITS: interrupted time series.

LUTS: lower urinary tract symptoms.

MH: microscopic haematuria.

MRI: Magnetic Resonance Imaging.

NHS: National Health Service.

RCT: randomised controlled trial.

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Ahlburg 2005	Ineligible study design
Arnaout 2012	Uncontrolled before‐and‐after (BA) study. No graphs. Conference abstract only
Bassi 2004	Controlled before‐and‐after study (CBA) with only 1 intervention and 1 control site. No graphs
Bellan 2004	Case study. Inadequate outcome measure
Bergin 2009	Ineligible study design
Bibi 2007	Uncontrolled BA study. No graphs
Blick 2010	Uncontrolled BA study. No graphs
Boisjoly 2010	Ineligible study design
Borg 1991	Ineligible study design
Borugian 2001	Not about reducing waiting lists, but about reducing within‐clinic waiting times between diagnosis and treatment
Brook 2010	Not about reducing waiting lists, but about reducing within‐clinic waiting times between diagnosis and treatment
Bungard 2009	Uncontrolled BA study. No graphs
Carrington 1991	Ineligible study design
Chandler 2005	Ineligible study design. Poster abstract
Ciardullo 2003	Uncontrolled BA study, with no baseline data
Clark 1999	Uncontrolled BA study, with no baseline data
Clemente 2006	CBA study with only 1 intervention and 1 control site. No graphs
Cootauco 2007	Ineligible study design
Dennis 1994	Ineligible study design (simplified time series)
Dewson 2001	Ineligible study design
Droulers 1995	Ineligible study design
Elkhuizen 2007	Uncontrolled BA study, with only 1 data point before the intervention. Also already short waiting times (14 days)
Fitzgerald 2006	Uncontrolled BA. No graphs
Fortune 2012	Ineligible study design
Garfield 1976	1‐site CBA. No graphs
Graves 2006	Ineligible study design
Gustafson 2013	Comparison of 3 different implementation strategies for the same intervention
Haggarty 2012	CBA with 1 intervention and 4 control sites. No graphs
Hanning 1996	Uncontrolled BA study with fewer than 3 data points before the intervention
Hanning 2007	Testing the suspension of an intervention instead of testing its introduction
Harding 2012	Ineligible study design
HMT 2012	Ineleigible study design. Descriptive
Hobday 2003	Ineligible study design
Jibawi 2005	Ineigible study design. Abstract
Jones 2000	Ineligible study design
Keller 1997	Uncontrolled BA study with no baseline data
Kendall 2009	Ineligible study design
Kew 2001	Inappropriately analysed interrupted time series (ITS) study. No graphs
Khawaja 2000	Ineligible study design
Kielar 2010	Ineligible study design. Descriptive study
Kirkwood 2006	Ineligible study design
Kumari 2001	Ineligible study design
Lal 2011	Inappropriately analysed ITS study. No graph
Lim 2012	CBA with 1 intervention and 1 control site and no baseline
Lizan‐Garcia 2001	Inappropriately analysed ITS study. No graphs. Main aim with intervention was to assess its acceptability by patients
Magnusson 2010	Inappropriately analysed ITS study with no graph. Conference abstract
Marden 2001	Ineligible study design
Marquez 1994	Ineligible study design. Intervention sites volunteered to receive the intervention
Martin 2005	Inappropriately analysed ITS study. No graphs
Maruthachalam 2005	Ineligible study design
May 2008	Ineligible study design
McLeod 2003	Ineligible study design
Menzies 2001	Ineligible study design
Mitchell 2002	Ineligible study design
Montoro 2002	Ineligible study design. Non‐randomised study comparing medical specialities (intervention group) vs surgical specialities (control group)
Newey 2006	Ineligible study design
Newman 2011	Ineligible study design
Nichols 2011	Ineligible study design. Abstract only
Ogunbamise 2005	Inappropriately analysed ITS study. No graphs
Old 2001	Ineligible study design
Perez 2005	Ineligible study design
Phillips 2001	Inappropriately analysed ITS study. No graphs
Pomerantz 2008	Inappropriately analysed ITS study. No graphs
Poot 2011	Ineligible study design
Proenca 2003	Inappropriately analysed ITS study. No graphs
Rayner 2008	Ineligible study design
Reece 2001	Ineligible study design. Abstract only
Reid 2009	Ineligible study design
Rochester 2008	Ineligible study design
Ross 2010	Ineligible study design. Abstract only
Salam 2006	Inappropriately analysed ITS study. No graphs
Sanderson 2003	Ineligible study design. Descriptive
Scheurmier 2001	Ineligible study design
Seagger 2011	Ineligible study design. Non‐randomised, 1‐site, 2‐group comparison
Shetty 2004	Ineligible study design
Siofradh 2000	Neither abstract nor full text available, impossible to contact study author
Smigorowsky 2007	Inappropriately analysed ITS study. No graphs
Snow 2009	Ineligible study design
Sri‐Ram 2006	Ineligible study design. Non‐randomised, 1‐site, 2‐group comparison
Sulaiman 2004	CBA study with 1 intervention site only. No graphs
Tavakol 2011	Ineligible study design
Tebe 2012	Ineligible study design. Descriptive
Tinkler 2004	Ineligible study design
Toustrup 2011	Inappropriately analysed ITS study. No graphs
Unknown 1950	Ineligible study design
Unknown 1975	Ineligible study design
Unknown 1995	Ineligible study design
West 2001	Ineligible study design
White 2010	Ineligible study design. Abstract only
Wijesekara 2011	Ineligible study design. Case study. Abstract only
Wong 2000	Ineligible study design
Wynn‐Williams 1950	Ineligible study design

Characteristics of studies awaiting assessment [ordered by study ID]

Heptulla 2013.

Methods	Time series analysis
Participants	Children needing a consultation for endocrine problems
Interventions	Intervention: 3 strategies New participant appointments were protected from conversion to follow‐up appointments All physicians, including senior faculty, were scheduled to see 3 to 4 new participants per session Sessions devoted exclusively to follow‐up appointments were added on the basis of demand Control: statistical process control (SPC) charts
Outcomes	Waiting times for new and follow‐up appointments Monthly visit volume Per‐provider visit volume Differences in proportion of new visits Clinic arrival rates
Notes

Salisbury 2013.

Methods	Ransomised controlled trial
Participants	Adults (aged ≥ 18 years) who were referred by general practitioners, or who referred themselves, for physiotherapy for a musculoskeletal problem
Interventions	Intervention: PhysioDirect service, which consists of the possibility for patients to make a phone call to a physiotherapist for initial assessment and advice without waiting for a face‐to‐face appointment; physiotherapists may give advice about self‐management and exercises over the telephone, and the need and priority for seeing them face‐to‐face can be established Control: usual care consisting of participants joining a waiting list for physiotherapy and eventually receiving face‐to‐face care
Outcomes	Primary: Clinical outcome at 6 months, assessed by the physical component summary measure from the Short Form (SF)‐36 v2 questionnaire Secondary: Clinical outcome at 6 months, assessed by the Measure Yourself Medical Outcomes Profile, EQ‐5D, 1 question about overall improvement in the main problem for which the participant was referred to physiotherapy (global improvement score ‐ a 7‐point scale from "very much better" to "very much worse"), A composite measure of response to treatment including pain, function and overall improvement as recommended by the Outcomes Measures in Rheumatology Clinical Trials‐Osteoarthritis Research Society International initiative Mental component summary score and individual scales from the SF‐36 v2 questionnaire Time lost from work Participant satisfaction (questions adapted from the General Practice Assessment Questionnaire) Participant preference for future care Number, type and duration of consultations with physiotherapists Waiting times for treatment (defined as first physiotherapy contact) Rates of non‐attended appointments
Notes

Sobolev 2012.

Methods	Cohort study
Participants	Patients needing elective coronary artery bypass grafting (CABG)
Interventions	Intervention: supplementary funding to tertiary care hospitals providing cardiac surgical care Control: no additional funding for cardiac surgery
Outcomes	Time between decision to operate and surgical revascularisation
Notes	Could be considered inappropriate interrupted time series (ITS) study ‐ study authors to be contacted for additional data

Stewart 2013.

Methods	Controlled before‐and‐after study
Participants	Community‐based alcohol and other drug (AOD) services providing treatment programmes for detoxification, inpatient, residential and outpatient treatment services
Interventions	Intervention: performance contracts and quality improvement initiatives for outpatient substance abuse treatment programmes Control: global payment (fixed monthly installment, one‐twelfth of the annual amount, determined prospectively through negotiations between the programme and the state)
Outcomes	Participant waiting time defined as number of days between the day the individual first contacted the programme and the admission day Participant length of stay (LOS; calculated by subtracting admission date from discharge date and adding 1 day, so that participants admitted and discharged on the same day have an LOS of 1 day)
Notes

Characteristics of ongoing studies [ordered by study ID]

Augestad 2008.

Trial name or title	1‐Stop trial
Methods	Randomised controlled trial
Participants	Patients diagnosed by general practitioners for an inguinal hernia, gallstone disease or pilonidal sinus requiring surgical treatment
Interventions	Intervention: direct electronic referral and booking for outpatient surgery (1 stop) Control: traditional patient pathway: Every participant was seen at the outpatient clinic several weeks ahead of surgery
Outcomes	Primary: Waiting time for outpatient surgery Costs Secondary: General practitioner (GP) satisfaction
Starting date	October 2010
Contact information	Knut Magne Augestad (knut.magne.augestad@telemed.no), Norwegian Centre for Telemedicine, Norway, and Departement of Gastrointestinal Surgery, University Hospital of North Norway, Norway
Notes	Trial registration number in ClinicalTrials.gov: NCT00692497

Wahlberg 2013.

Trial name or title	Practical health co‐operation ‐ the impact of a referral template on quality of care and health care co‐operation: study protocol for a cluster randomized controlled trial
Methods	Cluster‐randomised controlled trial
Participants	Unit of randomisation: 14 general practitioner offices (in an area primarily served by the same University Hospital) Unit of analysis: patients referred from general practices to hospital care for dyspepsia or suspected colonic malignancy or chest pain or chronic obstructive pulmonary disease or suspected chronic obstructive pulmonary disease
Interventions	Intervention: implementation of a referral template for the referral pathway from general practice to hospital care Control: standard referral practice
Outcomes	Primary: Quality indicator score (generated from previous international quality assessment tools and national and international treatment guidelines and adapted to locally accepted practice) Secondary: Quality of the referrals Health process outcomes (waiting time from referral to appointment, number of appointments before a diagnosis is reached, time before treatment is initiated, application or not of appointment prioritisation and the outcome of any given referral (appointment, return information or referral rejected)
Starting date	September 2011.
Contact information	Henrik Wåhlberg (henrik.wahlberg2@unn.no; henrik.wahlberg@uit.no), Department of Community Medicine, Faculty of Health Sciences, University of Tromsø, Tromsø, and University Hospital of North Norway Harstad, St. Olavsgate 70, 9480, Harstad, Norway
Notes	Trial registration number in ClinicalTrials.gov: NCT01470963

Differences between protocol and review

The original protocol included "number of patients in a waiting list" among the outcomes. Considerations of the clear inadequacy of this measure of outcome (Siciliani 2008; Siciliani 2013) led the review authors to exclude it from the inclusion criteria of the review.

Except for one (RG), all authors of this review are different from those of the protocol.

Contributions of authors

RG wrote the protocol for the original review. LB led this review. GF and AH (Alison Hirst) assessed studies for inclusion. LB, LV, SM, AN, GF, IS and JH participated in data extraction and contributed to data analysis. AN and JH carried out the statistical analyses. LB, LV, AN and SM drafted the review, and all review authors commented on this.

Sources of support

Internal sources

• No sources of support supplied

External sources

• Research grant by the Italian Ministry of Health (grant number: GR ‐ 2010 ‐ 2317133), Italy.

Declarations of interest

LB is editor for the Cochrane Effective Practice and Organisation of Care Review Group.

New