Training interventions for improving telephone consultation skills in clinicians

Abstract

Background

Since 1879, the year of the first documented medical telephone consultation, the ability to consult by telephone has become an integral part of modern patient‐centred healthcare systems. Nowadays, up to a quarter of all care consultations are conducted by telephone. Studies have quantified the impact of medical telephone consultation on clinicians' workload and detected the need for quality improvement. While doctors routinely receive training in communication and consultation skills, this does not necessarily include the specificities of telephone communication and consultation. Several studies assessed the short‐term effect of interventions aimed at improving clinicians' telephone consultation skills, but there is no systematic review reporting patient‐oriented outcomes or outcomes of interest to clinicians.

Objectives

To assess the effects of training interventions for clinicians' telephone consultation skills and patient outcomes.

Search methods

We searched CENTRAL, MEDLINE, Embase, five other electronic databases and two trial registers up to 19 May 2016, and we handsearched references, checked citations and contacted study authors to identify additional studies and data.

Selection criteria

We considered randomised controlled trials, non‐randomised controlled trials, controlled before‐after studies and interrupted time series studies evaluating training interventions compared with any control intervention, including no intervention, for improving clinicians' telephone consultation skills with patients and their impact on patient outcomes.

Data collection and analysis

Two review authors independently selected studies for inclusion, extracted data and assessed the risk of bias of eligible studies using standard Cochrane and EPOC guidance and the certainty of evidence using GRADE. We contacted study authors where additional information was needed. We used standard methodological procedures expected by Cochrane for data analysis.

Main results

We identified one very small controlled before‐after study performed in 1989: this study used a validated tool to assess the effects of a training intervention on paediatric residents' history‐taking and case management skills. It reported no difference compared to no intervention, but authors did not report any quantitative analyses and could not supply additional data. We rated this study as being at high risk of bias. Based on GRADE, we assessed the certainty of the evidence as very low, and consequently it is uncertain whether this intervention improves clinicians' telephone skills.

We did not find any study assessing the effect of training interventions for improving clinicians' telephone communication skills on patient primary outcomes (health outcomes measured by validated tools or biomedical markers or patient behaviours, patient morbidity or mortality, patient satisfaction, urgency assessment accuracy or adverse events).

Authors' conclusions

Telephone consultation skills are part of a wider set of remote consulting skills whose importance is growing as more and more medical care is delivered from a distance with the support of information technology. Nevertheless, no evidence specifically coming from telephone consultation studies is available, and the training of clinicians at the moment has to be guided by studies and models based on face‐to‐face communication, which do not consider the differences between these two communicative dimensions. There is an urgent need for more research assessing the effect of different training interventions on clinicians' telephone consultation skills and their effect on patient outcomes.

Plain language summary

Training interventions for improving telephone consultation skills in clinicians

What is the aim of this review?

The aim of this Cochrane review was to find out whether some training interventions are more effective than others for improving clinicians' telephone communication skills and patient outcomes.

Key messages

One very small study reported no difference between a training intervention and no intervention on paediatric residents' telephone skills, but the certainty of the evidence was very low, so it is uncertain if this intervention makes any difference in clinicians' telephone communication skills.

We did not find any study assessing the effect of training intervention for improving clinicians telephone communication skills on patient outcomes.

There is no evidence to inform clinician training in telephone communication skills, so currently it has to be guided by studies and models based on face‐to‐face communication that do not consider the differences between these two communicative dimensions.

What was studied in this review?

Since its invention, the telephone has been increasingly used by clinicians and patients to manage health problems. Nowadays, up to a quarter of all clinical consultations happen by phone. What we know from previous research is that clinicians' telephone skills are of low quality, and clinicians receive poor specific training in their education. These important and delicate professional skills are left to individual clinicians' own intuition and judgement to cultivate and improve. Overall research in telephone medicine clearly shows that telephone consultations can have an important role in the delivery of healthcare with quantitative impact on clinicians' workload, hospital care, patient outcomes and job satisfaction levels.

We searched for studies where doctors, nurses and other healthcare professionals underwent educational interventions for developing and improving telephone consultation skills with patients. We included studies from all settings and excluded studies dealing just with communication between clinicians.

What are the main results of the review?

This review found no studies assessing the effect of training intervention for improving clinicians telephone communication skills on patient outcomes measured by validated tools or biomedical markers, on patient behaviours, patient morbidity or mortality, patient satisfaction, urgency assessment accuracy or adverse events. We found one very small study performed more than 25 years ago reporting no difference between a training intervention and no intervention on paediatric residents' history‐taking and case management skills. This study provided no quantitative data.

This review found no specific evidence to inform clinicians' telephone consultation skills training; high quality studies in this field are urgently needed.

How up‐to‐date is this review?

The review authors searched for studies that had been published up to May 2016.

Summary of findings

for the main comparison.

Training intervention compared with no intervention for improving telephone consultation skills in clinicians
Patient or population: health professionals Settings: primary care Intervention: training intervention Comparison: no intervention
Outcomes	Impact	No of studies (participants)	Certainty of the evidence (GRADE)
Patient health outcomes	Not assessed	0	No evidence available
Patients mortality	Not assessed	0	No evidence available
Patients morbidity	Not assessed	0	No evidence available
Patient satisfaction	Not assessed	0	No evidence available
Urgency assessment accuracy	Not assessed	0	No evidence available
Adverse events	Not assessed	0	No evidence available
Clinicians' telephone consulting skills	It is uncertain whether the intervention improves clinicians' history‐taking and management skills	1 (11)	⊕⊝⊝⊝ Very lowa
GRADE Working Group grades of evidence High quality: further research is very unlikely to change our confidence in the estimate of effect. Moderate quality: further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. Low quality: further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. Very low quality: we are very uncertain about the estimate.

^aThe overall quality/certainty of the evidence was judged to be very low because the initial level of confidence about the only included study was low (non‐randomised evidence), and we downgraded the confidence further due to serious risk of bias (incomplete outcome data management and selective outcome reporting) and because of the impossibility to assess consistency of effect, imprecision, indirectness and publication bias due to lack of other studies.

Background

Description of the condition

Since 1879, the year of the first documented medical telephone consultation, the ability to consult by telephone has become an integral part of modern, patient‐centred healthcare systems (No author 1879, Evans 2003). In the USA, doctors have spent at least one eighth of their professional time assessing clinical cases on the telephone for decades (Bergman 1966), with upwards of a quarter of all care consultations being conducted by this method today (Mendenhall 1981; Patel 1997). More recently, the British Medical Association (BMA) has provided guidance for general practitioners (GPs), entitled Consulting in the Modern World: Guidance for GPs (BMA 2001), advising that "telephone consultations, when correctly conducted can be considered to be safe and acceptable practice". Reisman 2005 described how telephone communication is the primary mode of communication between physicians and patients outside of an office visit. Car 2004 and Patel 2005 argued that telephone consulting is both a feasible and effective form of clinical intervention.

Bunn 2004 described telephone consulting as a process whereby patients receive medical advice by one or more qualified healthcare professionals via the telephone. The authors concluded that telephone consultations appear to be safe and that healthcare users were just as satisfied with them as with face‐to‐face consultations. They also suggested that telephone consultations appear to decrease the number of immediate visits to doctors without increasing attendance to emergency departments. A recent large study found otherwise, reporting that telephone triage merely redistributes GP workload away from face‐to‐face consultations and towards more telephone consultations or nurse‐led care (Campbell 2014).

Clinicians and patients use telephones for a range of healthcare services, including routine and emergency care, prescription renewals, laboratory investigations and health promotion (Car 2003, Giesen 2011). Examples of telephone consultations include the management of conditions such as heart failure (Clark 2007; Riegel 2002), asthma (Gruffydd‐Jones 2005; Patel 2009; Pinnock 2003), and palliative care (Pimentel 2015; Zhou 2012).

Telephone consultations may reduce doctors' face‐to face workloads and enhance access to care without the inconvenience and cost associated with physically attending a consultation, thus increasing the flexibility and availability of service (Campbell 2014; Hallam 1992; Patel 2005).

Katz 2008 and Car 2008 highlighted some of the existing safety concerns in relation to telephone consultations, such as the vulnerability of patients to errors in management and of clinicians to malpractice claims. The authors suggested that the most effective risk‐management strategy is to improve the quality of telephone care and service to patients. They also suggested that prevention should include a more disciplined approach to documentation, improved workload systems, increased skills training and an upfront commitment to evaluation.

Bunn 2004 pointed to remaining questions about the effect of telephone consultations on service use. Since telephone consultations play a role in patient management, it is essential that when consulting via the telephone, healthcare professionals feel confident with their skills to conduct and document the interview with accuracy and clinical competency. It is therefore important that they receive adequate training to enable them to carry out their clinical roles effectively.

Purc‐Stephenson 2012 found that overall patient adherence with triage advice provided by tele‐nurses was 62%, and adherence was influenced by the interplay between patient perceptions and the quality of provider communication. The authors highlight the need for communication skills training in a telephone consultation context that is patient centred and specifically addresses active listening and advising skills, providing guidance on how to structure the call.

Description of the intervention

As with face‐to‐face consulting skills, the ability of consulting via the telephone requires adequate training. This training may occur at any stage of a professional's career. We used the term training as defined within the Medical Subject Heading (MesH) of the US National Library of Medicine vocabulary thesaurus, under the MesH term of education (www.ncbi.nlm.nih.gov/mesh/68004493). Essentially, we used the term within the context of medical education.

Training can employ varied educational interventions such as traditional one‐way teaching methods (e.g. lectures) as well as more interactive techniques (e.g. simulation software).

One‐way methods

One‐way training usually consists of lectures, reading materials, or both. A survey found that only a few internal medicine programmes offer teaching in telephone management, and when it is part of the curriculum, it is delivered via a single and informal lecture (Flannery 1995).

Interactive methods

Medical education increasingly uses simulated patients. As early as 1983, Evens and colleagues described patient simulators to teach telephone communication skills without testing it in a controlled study (Evens 1983). In a controlled study, Greenberg 1999a used a non‐validated tool to assess a telephone management educational programme with standardised patients to train paediatric residents. A cross‐sectional analysis by Derkx 2009a reported that the telephone incognito standardised patient (TISPs) method can be valuable both for training and assessment of performance in telephone consultation carried out by doctors, trainees and other personnel involved in medical services.

Computerised training programmes use specially designed software to simulate calls and provide pertinent feedback. These programmes can be delivered either online or via CD‐ROM. Ottolini 1998 designed a randomised controlled trial where healthcare professionals used an interactive CD‐ROM programme with scripts representing the 10 most common complaints to simulate telephone conversations with the parents of paediatric patients. Unfortunately the tool used to assess the results had not been previously validated.Kosower 1991 described a non‐experimental programme called TALK to teach telephone communication skills by allowing residents to analyse recorded calls in group and to participate in individual feedback sessions. In a controlled before‐after study, Wood 1989 used several testing tools to assess a specially developed role play telephone management curriculum on history‐taking and management skills.

Structured tools

Marshall 2009 described a randomised controlled trial assessing a communication tool for improving the quality of telephone clinical referrals between health personnel. This study used final year medical students to measure the effect of the intervention on the content and clarity of telephone referrals.

Multifaceted approach       

Interventions may often incorporate different types of training, including both one‐way and interactive methods. In a before‐after study without control, King 2007 developed and evaluated a continued educational programme called the Effective Patient Teaching and Problem Solving (EPT‐PS) course. The intervention consisted of several sessions incorporating didactic presentations with modelling, demonstrations of taught skills and interactive group exercises.

The purpose of this review is to identify and measure the effectiveness of these types of interventions on clinicians' telephone skills.

How the intervention might work

The intervention should aim ultimately to improve patient care. We hypothesise that this can be achieved by successfully changing clinicians' behaviour, which would affect specific areas of practice.

It is vital that any training intervention changes clinicians' behaviour in the desired manner. Grimshaw 2002 described some of the factors that may influence change.

• Implementation of effective change strategies through understanding the determinants of physician behaviour and identification of barriers.

• More evidence‐based strategies of implementation and dissemination.

• Emphasis on population‐based improvement in clinical outcomes.

This review looked at the various determinants of clinicians' behaviour as described in the previous section to help understand how training programmes can ensure the desired outcomes. Results may inform the appropriate programme implementation and dissemination.

It is also worth applying relevant theoretical considerations in efforts to change clinicians' behaviour. Many theories of behavioural change and learning theory exist. Slotnick 2002 described some of the ways in which various theories may be applied. Grol 2002 described the following six elements of effective change.

• The complex reality of clinical practice needs to be considered: this may relate to clinician workload, resources and experience in relation to telephone consulting.

• Specific attention must be directed toward the designed change/improvement: the same intervention may lead to different outcomes for different learners. This review aimed to address the reasons for this through the variety of variables it assessed (e.g. age, sex, professional specialty, etc.).

• A diagnostic analysis of the target group and setting: this review aimed to help understand learning needs and environments that are conducive to effective training.

• A mix of actions including training, rewards, feedback and organisational measures addressing the needs and problems of the target groups and the barriers is needed: this review aimed to inform the development of any multifaceted training programme that incorporates more than one of these elements.

• It is necessary to develop a plan indicating which actions will be taken when, by whom and in what order: the results of this review aimed to inform organisations (such as educational institutions) on how to develop, design and implement effective training programmes for telephone consulting.

• Continuous monitoring, feedback and adaptation of strategies should be implemented as needed: an effective training programme for teaching telephone consulting skills must include appropriate evaluation and feedback methods to ensure learning objectives are being achieved. This review aimed to identify and analyse the existing evidence in this area.

Why it is important to do this review

There is an important role for telephone consultations within healthcare, so it is important to know which is the best way to provide the adequate skills to the relevant healthcare professionals.

The need for effective training

Although the need for telephone skills training was first documented in the 1970s (Brown 1974), there is still no systematic review reporting which interventions are most effective in training clinicians.

As BMA 2001 noted, during a telephone consultation the doctor "cannot see, touch, examine, investigate, smell or, in the strictest terms, even hear the caller/patient." We believe that the differences in telephone consulting compared to face‐to‐face warrant specific, evidence‐based training. This review was conducted to provide the evidence base for effective clinicians' training in telephone consulting and to develop telephone‐specific consulting models.

In the same way as in face‐to‐face consulting, we argue that there should be robust studies to investigate the best ways to teach telephone consulting skills. Modern face‐to‐face consultation models have been developed with an emphasis on informed information provision; exploration of patients' concerns, ideas and expectations; patient centeredness; and patient satisfaction. An increasing body of evidence supports the use of a patient‐centred approach to healthcare (Hayden 2003). Models of face‐to‐face consulting include Balint's model (Balint 1964), Berne's transactional analysis (Berne 1968), Byrne and Long model (Byrne 1976), Middleton agenda model (Middleton 1989), Neighbour's inner consultation (Neighbour 1987), Pendleton's consultation model (Pendleton 1984), and Stott and Davis model (Stott 1979). Developing an evidence base on teaching telephone consulting skills can contribute to the achievement of a similar patient‐centred consulting approach or model for this context.

Ultimately, we think that improved clinician skills can lead to improved clinical outcomes. This can be achieved through identifying an evidence base, transferring appropriate skills and providing consistent delivery of services.

Evidence base

Ideally, training interventions should be evidence‐based. Car 2003 argued that training targeted at telephone consultations, protocols for managing common scenarios, dedicated time for telephone contacts, documentation of all consultations and a low threshold for organising a face‐to‐face consultation may help to ensure quality and safety of telephone consultations. The telephone is a communication tool that poses several disadvantages for patient consultations from the clinician perspective, including an absence of visual clues and non‐verbal communication. Toon 2003 highlighted how, despite this, there has been little study of telephone consulting skills and little critical thinking about how best to work on its limitations or what background and training users need. Developing and utilising this evidence base for training in telephone consulting skills will be necessary for their improvement.

Transfer of appropriate skills

A healthcare professional trained in telephone consultation skills is expected to have a more refined appreciation of verbal cues and focused history‐taking to compensate for their inability to examine the patient (Car 2004), and training can improve these skills (Foster 1999).

Improving outcomes

Reisman 2005 argued that increasing familiarity with common challenges when consulting with patients over the telephone may help healthcare professionals decrease the likelihood of negative outcomes. A randomised, prospective, controlled comparison of residents' management of two telephone calls concluded that the use of a CD‐ROM telephone management programme was associated with better postintervention telephone management (Ottolini 1998). Marklund 1989 evaluated the effect of a teaching programme on telephone advice, finding that the educational programme resulted in improved quality of advice, confidence and satisfaction among participating nurses. Lattimer 1998 evaluated the safety and effectiveness of nurse telephone consultation in out‐of‐hours primary care. A key recommendation was that further testing in the selection and training of nurses may improve outcomes.

Consistency

O'Cathain 2003 examined the consistency of triage outcomes when nurses used telephone communication supported by computerised decision support software in out‐of‐hours emergency services by NHS Direct in the UK. The study found that there was variability in the ways nurses dealt with the calls, in particular, the triage outcomes such as recommending self‐care versus advising attendance to accident and emergency services. The authors claimed that effective training on telephone consultation skills in the specific setting can enable nurses to answer calls in a more standardised manner.

Potential effects of changing clinicians' behaviour

Training may lead to several potential outcomes through modified clinician behaviour. Grimshaw 2001 described how multifaceted interventions targeting different barriers are more likely to effectively change provider behaviour. Importantly, the authors also concluded that future educational activities should be informed by the findings of systematic reviews of professional behaviour change interventions. Current evidence suggests that training in telephone consultation skills could help to overcome a variety of perceived barriers.

Addressing the perceived lack of training

Elnicki 2000 reveals that practicing physicians and residents can benefit from regular review of telephone cases, both for educational purposes and for making practice policies. Interestingly, Patel 2009 showed that primary care physicians do not feel a need for specific training, as they perceive telephone consultations as just another form of history‐taking. Another study, Reisman 2005, reported that only 6% of residency programmes in the USA teach any aspect of telephone communication. The authors suggest the paucity of training in telephone medicine in residency programmes may be a significant contributor to telephone communication errors.

Improving clinician satisfaction and confidence

Hannis 1996 found that primary care physicians are often dissatisfied with telephone encounters and that their level of confidence is lower when consulting via a telephone than when seeing patients face–to‐face.

Improving consultation techniques

Innes 2006 highlighted that when attending via the telephone, physicians adopt a more dominant approach than with face‐to‐face consultations. Telephone exchanges also tend to be rich in biomedical and poor in psychosocial aspects. Derkx 2009b assessed the quality of staff communication skills using the RICE telephone communication rating list described in Derkx 2007. The authors found the mean overall score for communication skills was 35% of the maximum feasible; staff usually asked questions about the clinical situation correctly but elicited little information about the patients' personal situation, their perception of the problem or their expectations. Clinicians usually gave advice about the outcome of triage and self‐care without checking for patients' understanding and acceptance; calls were often handled in an unstructured way, without summarising or clarifying the different steps within the consultation.

Appropriate length of telephone consultation

Innes 2006 reported that the length of interaction accounts for much of the variation seen between consultations in the domains of rapport, data‐gathering, patient education, counselling and partnership. Derkx 2009b concluded that apart from adequate communication skills, staff needed sufficient time for telephone consultation to enable high‐quality performance.

Improving clinical accuracy (history‐taking, urgency assessment, management)

Isaacman 1992 reported that advice given via the telephone within paediatric care by emergency departments revealed inadequate histories, variable advice and insufficient follow‐up care. Hallam 1989 considered the use of the telephones within primary care and found 20% of the calls to be incorrect on follow‐up as well as inappropriate in triage decisions for common problems. Derkx 2008 considered that 21% of all questions that trialists asked were obligatory, while just 54% of the obligatory questions were carried out, and required care advice was given in the 58% of the cases. Pasini 2015 found similar results.

Improving documentation

A series of case reviews of telephone‐related claims by Katz 2008 found that absent or poor documentation was present in almost all cases, highlighting the need to document all calls of significant relevance. Derkx 2010 found 22% of telephone consultations at Dutch out‐of‐hours centres were not reported at all, and in the 78% reported, reports almost always contained information about the medical reason for calling but little information about details of the clinical history. Patients' expectation, personal situation or perception of the care advice were also seldom documented.

This review assesses the impact of training and educational programmes on clinician skills and relevant care. The results of this review aim to inform training and evaluation of programmes to provide effective telephone consultations skills that could result in better clinician behaviour and ultimately improve patient outcomes.

Objectives

To assess the effects of training interventions for clinicians' telephone consultation skills and patient outcomes.

Methods

Criteria for considering studies for this review

Types of studies

We considered the following types of studies meeting the minimum criteria used by the Cochrane Effective Practice and Organisation of Care Group (EPOC 2013). 

• Randomised controlled trials (RCTs).

• Non‐randomised controlled trials (NRCTs).

• Controlled before‐after (CBA) studies with a minimum of two intervention and two control sites.

• Interrupted time series studies (ITS) of interventions with a clearly defined point in time when the intervention occurred and at least three data points before and three after the intervention.

We based this decision on our initial evidence searches that identified few randomised controlled trials in this area.

Types of participants

We included clinicians (a broad term that encompasses all doctors, nurses and other health professionals) who underwent educational interventions for developing and improving telephone consultation skills with patients. We included studies from all settings including primary care, outpatient, inpatient and public health. We excluded studies regarding communication between clinicians.

Types of interventions

We considered any kind of intervention aiming at improving the clinicians' telephone consultation skills regardless of the means and the way they were delivered (computerised, written, face‐to‐face training programmes or decision support software).

The eligible comparators were any control intervention with a possible effect on the same outcomes set or no intervention.

Consequently the comparison could be, for example, an interactive e‐learning programme on telephone consultation structure versus a classroom intervention on the same topic or no intervention; or computerised decision support software versus written management algorithms or no intervention.

Types of outcome measures

We considered the following types of outcomes, as assessed through a validated tool. We use the definition of 'validated tool' provided by the Joint Commission: "an instrument that has been psychometrically tested for reliability (the ability of the instrument to produce consistent results), validity (the ability of the instrument to produce true results), [and] sensitivity (the probability of correctly identifying a patient with the condition)" (manual.jointcommission.org/Manual/Questions/UserQuestionId03Sub0015).

Primary outcomes

• Patient outcomes

• • Health outcomes (e.g. biomedical markers and patient behaviour)

  • Effect on morbidity/mortality

  • Patient satisfaction

  • Urgency assessment accuracy

  • Adverse events

• Clinicians' telephone consulting skills (e.g. RICE tool, Derkx 2007)

Secondary outcomes

• Clinician knowledge gain

• Attitudes to telephone consultation (e.g. confidence, satisfaction)

• Time effectiveness (length and frequency of consultations, avoidance of face‐to‐face contact, effect on further clinical contact)

• Referral patterns

• Economic evaluation (litigation issues, resource issues, time effectiveness)

We only included studies if they assessed primary outcomes (e.g. not those with just secondary outcomes).

Search methods for identification of studies

T Rader, the EPOC Information Specialist (IS), developed the search strategies in consultation with the review authors and ran the searches of the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects (DARE) for related systematic reviews and the databases listed below for primary studies. The most recent search was conducted on 19 May 2016.

We searched the following databases.

• MEDLINE Ovid (1946 to 19 May 2016).

• Embase Ovid (1947 to 19 May 2016).

• CINAHL EbscoHost (1980 to 19 May 2016).

• Cochrane Central Register of Controlled Trials (CENTRAL) via OVID (19 May 2016).

• Cochrane Library via Wiley (19 May 2016), including Cochrane Database of Systematic Reviews, Cochrane Methodology Register, Cochrane NHS Economic Evaluation Database, Cochrane Database of Abstracts of Reviews of Effects, Cochrane HTA Database.

We did not apply language nor date restrictions to the searches. We used two methodological search filters to limit retrieval to appropriate study designs: the Cochrane Highly Sensitive Search Strategy (sensitivity‐ and precision‐maximising version, 2015 revision) to identify randomised trials (cf. Cochrane Handbook for Systematic Reviews of Interventions 6.4d, Lefebvre 2011). To retrieve non‐randomised controlled trials, controlled before/after studies (CBAs) and interrupted time series (ITS), we applied the EPOC Group Methods Filter 2.6 (January 2013 version). For other databases, where no filter exists, we identified study designs at the screening stage (see Types of studies). We provide detailed search strategies used for searches in Appendix 1.

We devised the search strategy for the MEDLINE Ovid interface and then adapted it for the other databases. We consulted relevant individuals and organisations or information about unpublished or ongoing studies.

Searching other resources

In addition to selecting grey literature and searching Google Scholar (we screened the first 500 items retrieved), we searched the following trial registries and additional thesis resources.

• Trial registries.

  • WHO International Clinical Trials Registry Platform (ICTRP) (www.who.int/ictrp/en).

  • ClinicalTrials.gov (clinicaltrials.gov).

We also:

• screened individual journals and conference proceedings (e.g. handsearching);

• reviewed reference lists of relevant systematic reviews or other publications;

• contacted authors of relevant studies or reviews to clarify reported published information or seek unpublished results/data (when necessary);

• contacted researchers with expertise relevant to the review topic or EPOC interventions; and

• conducted cited reference searches in ISI Web of Science/Web of Knowledge.

Data collection and analysis

Selection of studies

Two pairs of review authors (AV and YP; RG and MA) independently assessed the eligibility of all titles and abstracts identified from electronic searches. We retrieved full text copies of all articles judged to be potentially eligible. The same review authors independently assessed the retrieved articles to determine whether they met the inclusion criteria.

We resolved all disagreements through discussion, involving an additional review author if necessary, and we agreed on the final list of included and excluded studies. Where there was insufficient detail about the study to decide whether it met the inclusion criteria, we contacted the study authors to enable a more informed decision.

Data extraction and management

Two review authors (AV, YP) independently extracted data from all included studies using a standard data recording form derived from the data extraction template provided by the Cochrane EPOC Group EPOC 2013a. We compared results and resolved disagreements by discussion and, when necessary, through the involvement of an third review author. We contacted study authors to obtain or clarify data from included studies. We planned to use Review Manager 5 (RevMan) to manage the study data (RevMan 2014).

Assessment of risk of bias in included studies

Two review authors (AV, MA) independently assessed the risk of bias of the included study using the nine standard criteria for RCTs and the seven standard criteria for ITS as outlined by the Cochrane EPOC Group (EPOC 2015). We planned to use a template to guide our assessment of risk of bias, judging each item as having a low risk of bias, a high risk of bias or unclear risk of bias, providing a description to explain the decision using the guidance outlined by the Cochrane EPOC Group and in section 8.3 of the Cochrane Handbook for Systematic Reviews of Interventions (EPOC 2015; Higgins 2008). We compared judgments and resolved any disagreements by discussion and consensus, consulting a third review author where necessary.

If we considered that all the above‐mentioned elements were at low risk of bias, we assigned a low risk of bias to the study in question. Conversely, if we found that one or more of these key elements were at high risk of bias, we planned to classify the selected study as being at high risk.

We present the results of the 'Risk of bias' assessment for the included study in table format and incorporate the results of the assessment into the review through systematic narrative description and commentary about each of the quality items.

When necessary, we contacted study authors for additional information about the included studies.

Measures of treatment effect

We reported the findings of the included study in narrative form as described by the study authors. When further studies are identified and included in this review, we plan to analyse effect measures in relation to the primary outcome measures to assess whether there are definable and significant changes in a variety of outcomes after the training intervention. We anticipate that with additional studies, the primary outcomes will reveal data that can be assessed by measures such as mean difference (MD), standardised mean difference (SMD) and proportions where appropriate.

For dichotomous data: where feasible, we plan to analyse outcomes with dichotomous data (such as confidence rating scales) with relative effect.

For continuous data: we plan to report the mean difference (MD) or standardised mean difference (SMD) (if there was a difference in measurement of scales across trials), using 95% confidence intervals (CI) as measures of the amount of random errors influencing the outcome estimates.

Future included studies could use standardised assessment tools of consulting (such as Pendleton's Consultation Rating Scale). These again could be measured using MD and standard deviation (SD) or SMD if different tools were used. If medians are used, then we will measure interquartile ranges (IQR). Where total numbers and effect sizes are not recorded then we will describe results narratively.

Unit of analysis issues

We did not identify unit of analysis issues, as we included only one study. When the review includes future studies, including cluster trials, we plan to analyse the data according to recommendations in the Cochrane Collaboration Open Learning Module on issues related to the unit of analysis (Alderson 2002).

Dealing with missing data

We contacted the authors of the included study for missing data.

Assessment of heterogeneity

We did not assess heterogeneity, as we included only one study. When future studies are included, we will evaluate heterogeneity using tables and box plots to compare effect sizes of studies grouped according to potential effect modifiers. These include:

1. type of health professional;

2. type of intervention;

3. duration of education/intervention;

4. outcomes of intervention;

5. setting and contextual factors: primary/secondary care, face‐to‐face/e‐Learning;

6. study design (e.g. RCT, CCT, CBA, ITS);

7. methodological quality of studies.

With additional included studies, we expect to find substantial variation in the study results due to differences in types of interventions, the type of healthcare professional (targeted population), the design of the intervention, duration of the intervention and the context in which the intervention was implemented. We will conduct subgroup analyses based on type of intervention, type of health professional and study setting when two or more studies considering the same outcomes or using the same intervention in a similar population.

Assessment of reporting biases

We did not assess reporting bias, as we included only one study. When future studies are included, we plan to use funnel plots to assess the potential existence of small study bias. As there are a number of explanations for asymmetry in a funnel plot (Sterne 2001), we plan to carefully interpret results (Lau 2006).

Data synthesis

We did not perform quantitative analysis, as we included only one study.

When future studies are included, we plan to begin the data synthesis with a narrative overview of the findings and a table systematically summarising the extracted results. We will assess the participants, interventions and outcomes for comparability, which is necessary for statistical pooling. We will look for studies sufficiently similar in terms of study design, setting, intervention, follow‐up and outcome measures in order to combine the study data in a meta‐analysis. We plan to review the appropriateness to carry out a meta‐analysis collectively as a review team.

The choice of model will depend on the heterogeneity of the studies included in the meta‐analysis. We plan to conduct the analysis according to guidance in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2008). If meta‐analysis is feasible, we plan to use a random‐effects model, which provides a more conservative estimate of effect and can be used where there is moderate heterogeneity.

With additional included studies, we will measure median effect sizes across groups as originally described by Grimshaw 2004 and used by several subsequent authors (Jamtvedt 2006; Shojania 2004; Shojania 2009; Steinman 2006; Walsh 2006).This method is considered to help measure the median effect of each outcome within a study and subsequently measure the overall single effect size for that study. It is from these single effect sizes for each study that we can calculate the median effect size and interquartile range across all studies. This type of analysis is still subject to limitations, as it would assume that studies have equal weight. However, Grimshaw 2004 argued that using the median as opposed to the mean would limit the impact of outliers on the summary estimate of effect.

We will synthesise data through specific analysis of outcome measures previously described. Where possible, we plan to separately present results of studies, comparing:

• the intervention versus no intervention (e.g. telephone training programmes alone);

• the intervention versus other forms of intervention (e.g. telephone consulting training versus face‐to‐face consulting training).

Subgroup analysis and investigation of heterogeneity

We did not perform subgroup analysis. When future studies are included and where data are sufficient and it is appropriate in the context of the study, we plan to conduct subgroup analyses according to several factors (type of participants, patient characteristics, location of the study, year of publication, type of intervention, disease specific training interventions and development of protocols). This will allow the examination of the effect of certain studies on the pooled effects of the intervention.

Sensitivity analysis

We did not perform sensitivity analyses. When future studies are included, we plan to remove studies deemed to be at high risk of bias after examination of individual study characteristics from the analysis in order to examine the effect on the pooled effects of the intervention.

Summary of findings table

We summarised the findings for each primary outcome in a 'Summary of findings' table to draw conclusions about the certainty of the evidence for the main comparison within the text of the review. Two review authors independently assessed the certainty of the evidence (high, moderate, low, and very low) using the five GRADE considerations (study limitations, consistency of effect, imprecision, indirectness and publication bias) reported in the specific guidance developed by EPOC (EPOC 2013b). We resolved disagreements on certainty ratings by discussion and provide justification for decisions to down‐ or upgrade the ratings using footnotes in the table.

Results

Description of studies

Results of the search

We identified 12,209 articles through the search strategy and one additional article from experts' suggestion. We excluded 12,168 articles on the basis of abstracts considered not pertinent. We retrieved a total of 42 articles that potentially fulfilled the inclusion criteria for full text assessment. We excluded 41 with reasons: 18 did not use an experimental design, 7 were before‐after studies without control, 2 used a non‐validated tool to measure the clinicians' telephone skills change, and 14 were ineligible for other reasons. We described these studies in the Characteristics of excluded studies. Only one study fulfilled our inclusion criteria (see Characteristics of included studies). We summarise the study selection process in Figure 1.

1.

Study flow diagram.

Included studies

One controlled before‐after study met the inclusion criteria for this review (Wood 1989). This study assessed the effect of a role play telephone management curriculum for paediatric residents on history‐taking and case management skills. Six residents participated in three, half‐hour group sessions with a role play curriculum stressing a structured approach to telephone management of two paediatric problems; each resident had the opportunity to participate in several mock telephone conversations in which one resident played the parent of a sick child and the other resident played the clinic doctor. A seven‐resident control group received no formal instruction in telephone case management. Residents received pre‐test and post‐test calls (at three months) to a particular resident assigned to function as "telephone doctor" in the residents continuity clinic. Using standardised scripts, a simulated mother played the role of a mother calling the clinic for advice concerning her sick child. The standardised rating form was composed of three scales, each consisting of weighted items that were summed and then expressed as a percentage of possible perfect score; however, we could only take into account one previously validated scale (Specific History taking Scale (SHS) and its subset Specific History Triage (SHT)).

Unfortunately, authors did not report quantitative data on specific history scale (and specific history triage scale) and could not provide additional data: the only results they stated were, "There were no differences between intervention and control groups on the Specific History taking Scale. Even when specific history‐taking scores were calculated using a subset of heavily weighted items (Specific History Triage), there were no differences between groups".

We provide more details of this study in the Characteristics of included studies table.

Excluded studies

We describe these 41 studies under Characteristics of excluded studies. We contacted the authors to be sure that there was no additional unreported information available whenever the characteristics of the study were not completely clear. We excluded Greenberg 1999a and Ottolini 1998 because, despite their agreement with all the other inclusion criteria, we found no evidence of previous validation for the tools they used in order to detect clinicians' telephone consulting skills changes (see Table 2).

1. Studies comparison for validated tool use before exclusion.

Original study	Tool	What it is known about this tool validation			Final decision	Final reason for final exclusion
		From the original paper	From the authors (we explicitly asked them, "Was this instrument previously validated?")	From related publications
Wood 1989	GHS	Nothing	Nothing	No related publication	Data excluded	No clear evidence of tool validation
	SHS/SHT	see Perrin 1978	We used the rating skills developed by Dr Ellen Perrin and colleagues (see Methods section of NEJM article). She describes inter‐rater reliability. The scoring systems were developed using expert panels, which addresses in part the issue of validity Wood 2015 [pers comm].	Perrin 1978: "Inter‐rater reliability was 86%" (p 131, left column)	Data included	—
	GMS	Nothing	Nothing	No related publication	Data excluded	No clear evidence of tool validation
Greenberg 1999a	No name	Nothing, we just know the tool maximum score is 26, so this tool cannot be PPQ used by Ottolini 1998 because PPQ maximum score is 35 (p 396, right column)	"In those days editors didn't request the names of instruments. I went through my hard copies and could not find this information. If you check my paper on giving bad news to residents in Greenberg 1999b both of the communications skills instruments [the instruments used in Greenberg 1999b and Ottolini 1998] are published in that article." Greenberg 2015 [pers comm]	Greenberg 1999b speaks about 2 valid, reliable instruments used to assess interpersonal skills (p 1210, right column): GTS and PPQ. GTS has a maximum score of 20 points, while PPQ has a maximum score of 35 points. So none of them can be the score used in Greenberg original study. About PPQ, the paper refers to see Schnabl 1991 In the result section the paper says, "The interpersonal or counselling skills checklist was not examined for reliability" (p 1211, right column)	Data excluded	No clear evidence of tool validation
Ottolini 1998	PPQ	see Schnabl 1991	"The tool was based on Barbara Korsch's work but not psychometrically validated" Ottolini M [pers comm]	The tool used is not PPQ (7 items with a maximum score of 35 points) but IPS (13 items with a maximum score of 91 points)	Data excluded	No clear evidence of tool validation

GHS: general history scale; GMS: general management scale; PPQ: patient perception questionnaire; SHS: specific history scale; SHT (subset of items of SHS): specific history triage.

Risk of bias in included studies

We assessed the risk of bias of the included study using the nine standard criteria suggested by the Cochrane EPOC Group (EPOC 2015). We assessed the overall risk of bias for this study to be high. See Figure 2 and Figure 3.

2.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

3.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Allocation

For generation and concealment of allocation sequence, EPOC considers that a controlled before‐after study – as the study we found – is at high risk by default according to suggested risk of bias criteria for EPOC reviews (EPOC 2015). Regarding baseline outcome measurements and characteristics, authors provided no data, and the risk of selection bias was unclear.

Blinding

A single trained rater who was unaware of the resident group subsequently transcribed and rated audiotapes, so the detection bias was low. Authors reported nothing about contamination, but the study design protected against it.

Incomplete outcome data

The authors excluded the unpaired audiotapes from the analysis, determining a high risk of bias for incomplete outcome data management.

Selective reporting

Many outcomes in the Results section did not appear in the Methods, so we rated the risk of reporting bias to be high.

Other potential sources of bias

No other source of bias was identified. The overall risk of bias was high because of a high risk of bias in at least one key domain.

Effects of interventions

See: Table 1

We present the available evidence in the Table 1 for the main comparisons.

One study reported effects of training intervention on clinicians telephone consulting skills and found no difference between intervention and control among 11 paediatric residents, but authors did not provide quantitative data. We did not find any eligible study that reported effects of the intervention on primary patient outcomes (health outcomes, mortality, morbidity, satisfaction, urgency assessment accuracy or adverse events).

The included study only assessed primary outcomes, so we did not evaluate any secondary outcomes.

Certainty of the evidence

We judged the overall quality/certainty of the evidence for the outcomes reported to be very low (Summary of findings table 1) because the initial level of confidence about the only included study was low (non‐randomised evidence) and we downgraded the confidence due to a high risk of bias (incomplete outcome data management and selective outcome reporting) and because of the impossibility to assess consistency of effect, imprecision, indirectness or publication bias due to lack of other studies. Based on our GRADE assessment, it is uncertain whether the intervention improves clinicians' history‐taking or management skills (very low certainty).

Discussion

Summary of main results

We identified one controlled before‐after study at high risk of bias evaluating the effect of a training intervention on clinicians' telephone consulting skills. Authors found no difference between intervention and control on history‐taking and case management skills. We did not identify any studies that assessed the effect of training interventions for clinicians on patient primary outcomes (health outcomes measured by validated tools, biomedical markers or patient behaviours, patient morbidity or mortality, patient satisfaction, urgency assessment accuracy or adverse events).

Overall completeness and applicability of evidence

We excluded several studies in the telephone medicine literature because they did not have an experimental design, and we excluded two studies that fulfilled all other inclusion criteria because they used non‐validated evaluation tools: it seems no reliable evidence is available for this research topic.

Quality of the evidence

The overall quality/certainty of the evidence was very low because the initial level of confidence about the only included study was low (non‐randomised evidence), and we downgraded the confidence further after rating the risk of bias to be serious (incomplete outcome data management and selective outcome reporting) and because of the impossibility to assess consistency of effect, imprecision, indirectness or publication bias due to lack of other studies.

Potential biases in the review process

This review did not formally explore publication bias because we only included one study.

Agreements and disagreements with other studies or reviews

To our knowledge, this is the only systematic review on this topic.

Authors' conclusions

Implications for practice.

The paucity of studies assessing the effect of interventions aiming to improve clinicians' telephone skills, the limited size and the low quality of the only study found, do not allow robust evidence‐based conclusions. Essentially, this review cannot provide any guidance on effective methods to train healthcare professionals in telephone skills. Given the established use of telephone consultations within the medical field, this apparent lack of evidence is surprising. We have described in the Background section some of the important roles that telephone communication has within healthcare: this lack of knowledge is even more severe because telephone consultation is nowadays an important means of initial assessment of clinical cases and management of everyday practice in all clinical specialities.

Our review suggests the existence of a paradox in clinical practice: on the one hand, the widespread use of the telephone as a method of medical consultations plus patient adherence to recommendations that is influenced by the quality of clinicians' communication and on the other, generally low quality of clinicians' communication skills, lack of specific training during the undergraduate and postgraduate education, and the complete absence of specific evidence needed to inform this training.

At the moment the training of clinicians on telephone consultation has to be guided by studies and models set on face‐to‐face communication that do not consider the differences between these two communicative dimensions.

The very limited evidence that this review has identified would suggest that telephone consultation should be employed cautiously and carefully in healthcare provision until proven methods are developed around the training and practice of remote consultation skills.

Implications for research.

High‐quality randomised controlled trials, using validated tools to assess the effect of training interventions on clinicians' telephone skills on both patient‐oriented and clinician‐orientated outcomes, should be undertaken to ensure telephone skills can be taught based on reliable evidence. Similarly to face‐to‐face consultation skills, we need to ensure rigorous and robust methods for teaching telephone consultation skills. A substantial evidence base is lacking, and future studies could focus on a researching the type of teaching methods to be employed and developing telephone consultation models and validated assessment tools.

What's new

Date	Event	Description
8 February 2017	Amended	MInor correction in abstract: 'Nowadays, upwards of a quarter of all care consultations are conducted by telephone', has been changed to 'Nowadays, up to a quarter of all care consultations are conducted by telephone',

Appendices

Appendix 1. Search strategies

MEDLINE (OVID)

Ovid MEDLINE(R) In‐Process & Other Non‐Indexed Citations and Ovid MEDLINE(R) <1946 to Present>

Search date: April 8, 2015

1     (teleconsult$ or tele‐consult$).ti,ab. (862)

2     ((telephon$ or phone or phoning or phones or phoned) adj3 (advice or advise? or advising or consult$ or diagnos$ or evaluat$)).ti,ab. (1972)

3     ((telephon$ or phone? or phoning) adj3 (physician? or GP or nurse or nurses or doctor? or general practitioner? or family doctor? or family practitioner? or consultant?)).ti,ab. (1136)

4     (telephone management or telephone communication or telephone medicine or telephone intervention$ or telephone skill$).ab,ti. (522)

5     ((telephone or telephones or phone or phones) and (care or counselling or diagnos$ or health$ or intervention? or manag$ or therap$ or treat$ or medicine or medical or nursing or nurse? or physician? or doctor? or practitioner?)).ti. (2626)

6     Remote Consultation/ and (telephon$.ti,ab,hw. or (phone or phones).ti,ab.) (781)

7     ((telephone? or phone) adj3 skill?).ab. (51)

8     or/1‐7 [Teleconsultation Keyword Set] (6506)

9     telemedicine/ or telepathology/ or teleradiology/ or Telenursing/ (13644)

10     (teleassist$ or tele‐assist$ or teleaudiolog$ or tele‐audiolog$ or telebased or tele‐based or telecancer or tele‐cardiolo$ or telecardiolog$ or telecounselling or tele‐counselling or teledental or tele‐dental or telederm$ or tele‐derm$ or telediagnos$ or tele‐diagnos$ or teledialysis or tele‐dialysis or teleecho$ or tele‐echo$ or teleemerg$ or tele‐emerg$ or teleepileps$ or tele‐epileps$ or telefollow$ or tele‐follow$ or teleguidance or tele‐guidance or telehealth$ or tele‐health$ or telehome$ or tele‐home$ or teleICU or tele‐ICU or teleintervention$ or tele‐intervention$ or telemanag$ or tele‐manag$ or telemedicine or tele‐medicine or telemental$ or tele‐mental$ or telemonitor$ or tele‐monitor$ or telenurs$ or tele‐nurs$ or teleoncolo$ or tele‐oncolo$ or teleopthalm$ or tele‐opthalm$ or telepalliat$ or tele‐palliat$ or tele‐patholog$ or tele‐patholog$ or teleprocedu$ or tele‐procedu$ or telepsych$ or tele‐psych$ or teleradiol$ or tele‐radiol$ or telerefer$ or tele‐refer$ or telerehab$ or tele‐rehab$ or telesurger$ or tele‐surger$ or telesurgic$ or tele‐surgic$ or teletherap$ or tele‐therap$ or teletreat$ or tele‐treat$ or teletriage or tele‐triage).ti,ab. (11997)

11     (tele$ adj2 (care or counselling or diagnos$ or health$ or intervention? or manag$ or therap$ or treat$ or medicine or medical or nursing or nurse? or physician? or doctor? or practitioner?)).ab. (5276)

12     or/9‐11 [Telemedicine] (21445)

13     Telephone/ or Cellular phone/ (14197)

14     (telephone? or phone or phones or transtelephon$).ti. (8503)

15     (telephone based or phone based).ab. (1165)

16     (telephone? or phone or phones).ab. (51046)

17     or/13‐16 [Telephones] (58467)

18     (remote adj2 (care or consult$ or diagnos$ or evaluat$ or monitor$ or treat$ or therap$)).ti,ab. (2587)

19     (e‐care or ecare or e‐consult$ or econsult$ or e‐diagnos$ or ediagnosis$ or e‐health$ or ehealth$ or e‐medicine or emedicine or e‐nurse? or enurse? or e‐nursing or enursing or e‐physician? or ephysician? or e‐psych$ or epsych$ or e‐therapy or etherapy).ti,ab. [e‐Health] (2631)

20     "referral and consultation"/ (52433)

21     consultation?.ti,ab. (50405)

22     Remote Consultation/ (3672)

23     or/18‐22 [Remote Consultation/Referral/ecare] (100684)

24     Patient Care/ or After Care/ or Ambulatory Care/ or Postoperative Care/ or Preoperative Care/ or exp Nursing Care/ or Palliative Care/ or Perinatal Care/ or Postnatal Care/ or Prenatal Care/ or Preconception Care/ (324030)

25     exp General Practice/ (64523)

26     exp Diagnosis/ (6633188)

27     exp Diagnostic services/ or mass screening/ or anonymous testing/ or mass chest x‐ray/ or multiphasic screening/ or neonatal screening/ (124118)

28     (exp patient care management/ not (exp Telemedicine/ or telenursing/)) or comprehensive health care/ or "delivery of health care"/ or disease management/ or nurse's practice patterns/ or patient care team/ or patient‐centered care/ or physician's practice patterns/ (534776)

29     exp health services/ or community health services/ or emergency medical services/ or triage/ or nursing care/ or nursing services/ (1619235)

30     exp Medical History Taking/ (18472)

31     (history adj2 taking).ti,ab. (4473)

32     (patient? adj2 (assess$ or care or diagnos$ or evaluat$ or screen$)).ti,ab. (313378)

33     Public Health/ or Preventive Medicine/ or Preventive Psychiatry/ or Public Health Nursing/ or exp PUblic Health Practice/ (582470)

34     or/24‐33 [Patient Care/Care Delivery/Diagnosis/Public Health] (8411538)

35     health personnel/ or infection control practitioners/ or medical staff/ or exp nurses/ or exp nursing staff/ or pharmacists/ or exp physicians/ (241128)

36     allied health personnel/ or exp nurses' aides/ or exp physician assistants/ (18414)

37     (doctor? or nurse or nurses or physician? or practitioner?).ti. or ((medical or health$ or nursing or allied health$) adj2 (personnel or staff$)).ti,ab. (256381)

38     or/35‐37 [Health Personnel] (422911)

39     exp Education, Continuing/ or Clinical Clerkship/ or "Internship and Residency"/ or Teaching Rounds/ or Preceptorship/ (96312)

40     Inservice Training/ or Staff Development/ (24622)

41     (inservice or ((staff or physician? or nurse or nurses or doctor? or resident? or residency or intern or interns or practitioner?) adj2 (educational$ or train$ or development?))).ti,ab. (30673)

42     rounds.ti,ab. (13876)

43     ed.fs. (224443)

44     (education$ adj2 (intervention$ or program$ or hospital? or office? or practitioner? or GP or doctor?)).ti,ab. (43405)

45     (skill? adj2 develop$).ti,ab. (5175)

46     (continuing adj2 education$).ti,ab. (15732)

47     CME.ti,ab. and education.ti,ab,hw,fs. (1810)

48     communication? skill?.ti,ab. (6575)

49     (Health Communication/ or *Communication/) and (skill?.ti. or (skill? adj2 develop$).ab.) (1518)

50     ((telephone? or phone) adj3 skill?).ab. (51)

51     or/39‐50 [Education/Training] (352190)

52     Professional‐Patient Relations/ or Physician‐Patient Relations/ or Nurse‐Patient Relations/ (112825)

53     (professional patient or physician patient or nurse patient).ti,ab. (5694)

54     or/52‐53 [Professional Professional Relations] (114936)

55     and/8,51 [Teleconsult Keyword & CME] (669)

56     (and/17,23,51) not 55 [Remote Consult & Telephone & CME] (203)

57     (and/17,34,51) not (or/55‐56) [Telephone & Patient Care & CME] (2224)

58     (and/17,38,51) not (or/55‐57) [Telephone & Health Personnel & CME] (177)

59     (and/17,51,54) not (or/55‐58) [Telephone & Phys‐Patient Relations & CME] (26)

60     (and/17,51) not (or/55‐59) [Telephone & CME] (848)

61     (and/12,51) not (or/55‐60) [Telemed & CME/Training] (1169)

62     (randomized controlled trial or controlled clinical trial).pt. or randomized.ab. or placebo.ab. or clinical trials as topic.sh. or randomly.ab. or trial.ti. (944447)

63     exp animals/ not humans.sh. (4003797)

64     62 not 63 [Cochrane RCT Filter 6.4.d Sens/Precision Maximizing] (871197)

65     or/55‐61 [Results before filters] (5316)

66     intervention?.ti. or (intervention? adj6 (clinician? or collaborat$ or community or complex or DESIGN$ or doctor? or educational or family doctor? or family physician? or family practitioner? or financial or GP or general practice? or hospital? or impact? or improv$ or individuali?e? or individuali?ing or interdisciplin$ or multicomponent or multi‐component or multidisciplin$ or multi‐disciplin$ or multifacet$ or multi‐facet$ or multimodal$ or multi‐modal$ or personali?e? or personali?ing or pharmacies or pharmacist? or pharmacy or physician? or practitioner? or prescrib$ or prescription? or primary care or professional$ or provider? or regulatory or regulatory or tailor$ or target$ or team$ or usual care)).ab. (176295)

67     (pre‐intervention? or preintervention? or "pre intervention?" or post‐intervention? or postintervention? or "post intervention?").ti,ab. [added 2.4] (11527)

68     (hospital$ or patient?).hw. and (study or studies or care or health$ or practitioner? or provider? or physician? or nurse? or nursing or doctor?).ti,hw. (742163)

69     demonstration project?.ti,ab. (2014)

70     (pre‐post or "pre test$" or pretest$ or posttest$ or "post test$" or (pre adj5 post)).ti,ab. (71301)

71     (pre‐workshop or post‐workshop or (before adj3 workshop) or (after adj3 workshop)).ti,ab. (654)

72     trial.ti. or ((study adj3 aim?) or "our study").ab. (689980)

73     (before adj10 (after or during)).ti,ab. (372918)

74     (time points adj3 (over or multiple or three or four or five or six or seven or eight or nine or ten or eleven or twelve or month$ or hour? or day? or "more than")).ab. (10064)

75     pilot.ti. or (pilot adj (project? or study or trial)).ab. (71640)

76     (multicentre or multicenter or multi‐centre or multi‐center).ti. (31299)

77     random$.ti,ab. or controlled.ti. (801571)

78     (control adj3 (area or cohort? or compare? or condition or design or group? or intervention? or participant? or study)).ab. (504579)

79     ((evaluation or prospective or retrospective) adj study).ti,ab. [Added Jan 2013] (189091)

80     (utili?ation or programme or programmes).ti. [Added Jan 2013] (57068)

81     (during adj5 period).ti,ab. [Added Jan 2013] (311957)

82     ((strategy or strategies) adj2 (improv$ or education$)).ti,ab. [Added Jan 2013] (20149)

83     *experimental design/ or *pilot study/ or quasi experimental study/ (24662)

84     ("quasi‐experiment$" or quasiexperiment$ or "quasi random$" or quasirandom$ or "quasi control$" or quasicontrol$ or ((quasi$ or experimental) adj3 (method$ or study or trial or design$))).ti,ab. (107071)

85     ("time series" adj2 interrupt$).ti,ab. (1182)

86     or/66‐85 (3340397)

87     (rat or rats or cow or cows or chicken? or horse or horses or mice or mouse or bovine or animal?).ti. (1369875)

88     (exp animals/ or exp invertebrate/ or animal experiment/ or animal model/ or animal tissue/ or animal cell/ or nonhuman/) and (human/ or normal human/ or human cell/) (13775378)

89     (exp animals/ or exp invertebrate/ or animal experiment/ or animal model/ or animal tissue/ or animal cell/ or nonhuman/) not 88 (4003943)

90     86 not (or/87,89) [EPOC Filter 2.5‐‐Added Lines Jan. 2013] (2878509)

91     (2014$ or 2015$).ep,ed,yr. [2014‐2015 Limits] (1806844)

92     65 and 91 [2015 results before filters] (564)

93     92 and 64 [2015 RCT Results] (134)

Embase (OVID)

Embase Classic+Embase <1947 to 2013 January 04>

Search date: April 8, 2015

1     (teleconsult$ or tele‐consult$).ti,ab. (1013)

2     ((telephon$ or phone or phoning or phones or phoned) adj3 (advice or advise? or advising or consult$ or diagnos$ or evaluat$)).ti,ab. (2911)

3     ((telephon$ or phone? or phoning) adj3 (physician? or GP or nurse or nurses or doctor? or general practitioner? or family doctor? or family practitioner? or consultant?)).ti,ab. (1676)

4     (telephone management or telephone communication or telephone medicine or telephone intervention$ or telephone skill$).ab,ti. (733)

5     ((telephone or telephones or phone or phones) and (care or counselling or diagnos$ or health$ or intervention? or manag$ or therap$ or treat$ or medicine or medical or nursing or nurse? or physician? or doctor? or practitioner?)).ti. (3243)

6     Remote Consultation/ and (telephon$.ti,ab,hw. or (phone or phones).ti,ab.) (3409)

7     ((telephone? or phone) adj3 skill?).ab. (73)

8     or/1‐7 [Teleconsultation Keyword Set] (10733)

9     telemedicine/ (12445)

10     telecardiology/ or teleconsultation/ or teledermatology/ or telemonitoring/ or telepathology/ or telepsychiatry/ or teleradiology/ or teleradiotherapy/ or telesurgery/ or teletherapy/ (10738)

11     telehealth/ (1555)

12     (teleassist$ or tele‐assist$ or teleaudiolog$ or tele‐audiolog$ or telebased or tele‐based or telecancer or tele‐cardiolo$ or telecardiolog$ or telecounselling or tele‐counselling or teledental or tele‐dental or telederm$ or tele‐derm$ or telediagnos$ or tele‐diagnos$ or teledialysis or tele‐dialysis or teleecho$ or tele‐echo$ or teleemerg$ or tele‐emerg$ or teleepileps$ or tele‐epileps$ or telefollow$ or tele‐follow$ or teleguidance or tele‐guidance or telehealth$ or tele‐health$ or telehome$ or tele‐home$ or teleICU or tele‐ICU or teleintervention$ or tele‐intervention$ or telemanag$ or tele‐manag$ or telemedicine or tele‐medicine or telemental$ or tele‐mental$ or telemonitor$ or tele‐monitor$ or telenurs$ or tele‐nurs$ or teleoncolo$ or tele‐oncolo$ or teleopthalm$ or tele‐opthalm$ or telepalliat$ or tele‐palliat$ or tele‐patholog$ or tele‐patholog$ or teleprocedu$ or tele‐procedu$ or telepsych$ or tele‐psych$ or teleradiol$ or tele‐radiol$ or telerefer$ or tele‐refer$ or telerehab$ or tele‐rehab$ or telesurger$ or tele‐surger$ or telesurgic$ or tele‐surgic$ or teletherap$ or tele‐therap$ or teletreat$ or tele‐treat$ or teletriage or tele‐triage).ti,ab. (15206)

13     (tele$ adj2 (care or counselling or diagnos$ or health$ or intervention? or manag$ or therap$ or treat$ or medicine or medical or nursing or nurse? or physician? or doctor? or practitioner?)).ab. (7918)

14     or/9‐13 [Telemedicine] (32656)

15     telephone/ (28070)

16     (telephone? or phone or phones or transtelephon$).ti. (10515)

17     (telephone based or phone based).ab. (1563)

18     (telephone? or phone or phones).ab. (70876)

19     or/15‐18 [Telephones] (80042)

20     (remote adj2 (care or consult$ or diagnos$ or evaluat$ or monitor$ or treat$ or therap$)).ti,ab. (3771)

21     (e‐care or ecare or e‐consult$ or econsult$ or e‐diagnos$ or ediagnosis$ or e‐health$ or ehealth$ or e‐medicine or emedicine or e‐nurse? or enurse? or e‐nursing or enursing or e‐physician? or ephysician? or e‐psych$ or epsych$ or e‐therapy or etherapy).ti,ab. [e‐Health] (3400)

22     *patient referral/ (12688)

23     consultation?.ti,ab. (74016)

24     or/20‐23 [Remote Consultation/Referral/ecare] (90680)

25     exp *patient care/ (119980)

26     *medical care/ or *professional practice/ (25577)

27     exp *general practice/ or general practitioner/ (105079)

28     diagnosis/ or *diagnostic procedure/ or *computer assisted diagnosis/ or *early diagnosis/ or nursing diagnosis/ (1215705)

29     ambulatory care/ or *health care delivery/ or ambulatory care nursing/ or *postoperative care/ or nursing care/ (123598)

30     mass screening/ or *screening/ or anonymous testing/ or auditory screening/ or *cancer screening/ or developmental screening/ or *genetic screening/ or newborn screening/ or prenatal screening/ (118141)

31     *medical information/ (7761)

32     health service/ or *emergency health service/ or exp *health care/ or medical service/ or public health service/ (1312117)

33     anamnesis/ (113277)

34     (history adj2 taking).ti,ab. (7010)

35     (patient? adj2 (assess$ or care or diagnos$ or evaluat$ or screen$)).ti,ab. (477145)

36     preventive health service/ or *public health/ (79559)

37     or/25‐36 [Patient Care/Care Delivery/Diagnosis/Public Health] (3169101)

38     exp *physician/ or exp *nurse/ (181263)

39     exp *paramedical personnel/ (201136)

40     *clinical practice/ (27763)

41     (doctor? or nurse or nurses or physician? or practitioner?).ti. or ((medical or health$ or nursing or allied health$) adj2 (personnel or staff$)).ti,ab. (293018)

42     or/38‐41 [Health Personnel] (540737)

43     continuing education/ or staff training/ or in service training/ (50907)

44     professional development/ (5212)

45     clinical education/ or teaching round/ (10927)

46     (inservice or ((staff or physician? or nurse or nurses or doctor? or resident? or residency or intern or interns or practitioner?) adj2 (educational$ or train$ or development?))).ti,ab. (41489)

47     rounds.ti,ab. (17752)

48     (education$ adj2 (intervention$ or program$ or hospital? or office? or practitioner? or GP or doctor?)).ti,ab. (57052)

49     (skill? adj2 develop$).ti,ab. (6754)

50     (continuing adj2 education$).ti,ab. (19691)

51     CME.ti,ab. and education.ti,ab,hw,fs. (3798)

52     communication? skill?.ti,ab. (8593)

53     communication skill/ (7314)

54     ((telephone? or phone) adj3 skill?).ab. (73)

55     training/ (68443)

56     or/43‐55 [Education/Training] (261066)

57     Professional‐Patient Relations/ or Physician‐Patient Relations/ or Nurse‐Patient Relations/ (186605)

58     (professional patient or physician patient or nurse patient).ti,ab. (6638)

59     or/57‐58 [Professional Professional Relations] (188952)

60     and/8,56 [Teleconsult Keyword & CME] (790)

61     (and/19,24,56) not 60 [Remote Consult & Telephone & CME] (190)

62     (and/19,37,56) not (or/60‐61) [Telephone & Patient Care & CME] (1592)

63     (and/19,42,56) not (or/60‐62) [Telephone & Health Personnel & CME] (228)

64     (and/19,56,59) not (or/60‐63) [Telephone & Phys‐Patient Relations & CME] (59)

65     (and/19,56) not (or/60‐64) [Telephone & CME] (1593)

66     (and/14,56) not (or/60‐65) [Telemed & CME/Training] (1046)

67     intervention?.ti. or (intervention? adj6 (clinician? or collaborat$ or community or complex or DESIGN$ or doctor? or educational or family doctor? or family physician? or family practitioner? or financial or GP or general practice? or hospital? or impact? or improv$ or individuali?e? or individuali?ing or interdisciplin$ or multicomponent or multi‐component or multidisciplin$ or multi‐disciplin$ or multifacet$ or multi‐facet$ or multimodal$ or multi‐modal$ or personali?e? or personali?ing or pharmacies or pharmacist? or pharmacy or physician? or practitioner? or prescrib$ or prescription? or primary care or professional$ or provider? or regulatory or regulatory or tailor$ or target$ or team$ or usual care)).ab. (235415)

68     (pre‐intervention? or preintervention? or "pre intervention?" or post‐intervention? or postintervention? or "post intervention?").ti,ab. [added 2.4] (16141)

69     (hospital$ or patient?).hw. and (study or studies or care or health$ or practitioner? or provider? or physician? or nurse? or nursing or doctor?).ti,hw. (1910971)

70     demonstration project?.ti,ab. (2520)

71     (pre‐post or "pre test$" or pretest$ or posttest$ or "post test$" or (pre adj5 post)).ti,ab. (112782)

72     (pre‐workshop or post‐workshop or (before adj3 workshop) or (after adj3 workshop)).ti,ab. (995)

73     trial.ti. or ((study adj3 aim?) or "our study").ab. (995797)

74     (before adj10 (after or during)).ti,ab. (510978)

75     (time points adj3 (over or multiple or three or four or five or six or seven or eight or nine or ten or eleven or twelve or month$ or hour? or day? or "more than")).ab. (14359)

76     pilot.ti. or (pilot adj (project? or study or trial)).ab. (100816)

77     (multicentre or multicenter or multi‐centre or multi‐center).ti. (45037)

78     random$.ti,ab. or controlled.ti. (1036869)

79     (control adj3 (area or cohort? or compare? or condition or design or group? or intervention? or participant? or study)).ab. (689371)

80     ((evaluation or prospective or retrospective) adj study).ti,ab. [Added Jan 2013] (262721)

81     (utili?ation or programme or programmes).ti. [Added Jan 2013] (76505)

82     (during adj5 period).ti,ab. [Added Jan 2013] (437057)

83     ((strategy or strategies) adj2 (improv$ or education$)).ti,ab. [Added Jan 2013] (25998)

84     *experimental design/ or *pilot study/ or quasi experimental study/ (10264)

85     ("quasi‐experiment$" or quasiexperiment$ or "quasi random$" or quasirandom$ or "quasi control$" or quasicontrol$ or ((quasi$ or experimental) adj3 (method$ or study or trial or design$))).ti,ab. (136224)

86     ("time series" adj2 interrupt$).ti,ab. (1389)

87     or/67‐86 (5043446)

88     (rat or rats or cow or cows or chicken? or horse or horses or mice or mouse or bovine or animal?).ti. (1673506)

89     (exp animals/ or exp invertebrate/ or animal experiment/ or animal model/ or animal tissue/ or animal cell/ or nonhuman/) and (human/ or normal human/ or human cell/) (15909155)

90     (exp animals/ or exp invertebrate/ or animal experiment/ or animal model/ or animal tissue/ or animal cell/ or nonhuman/) not 89 (6012408)

91     87 not (or/88,90) [EPOC Filter 2.5‐‐Added Lines Jan. 2013] (4362434)

92     controlled clinical trial/ or controlled study/ or randomized controlled trial/ [EM] (4629477)

93     randomi?ed.ti. or ((random$ or control) adj3 (group? or cohort? or patient? or hospital$ or department?)).ab. or (controlled adj2 (study or trial)).ti. (778994)

94     (multicenter and (study or trial)).ti. (24038)

95     (random sampl$ or random digit$ or random effect$ or random survey or random regression).ti,ab. not randomized controlled trial/ [Per BMJ Clinical Evidence filter] (63028)

96     (exp animals/ or exp invertebrate/ or animal experiment/ or animal model/ or animal tissue/ or animal cell/ or nonhuman/) and (human/ or normal human/ or human cell/) (15909155)

97     (exp animals/ or exp invertebrate/ or animal experiment/ or animal model/ or animal tissue/ or animal cell/ or nonhuman/) not 96 (6012408)

98     (or/92‐94) not (or/95,97) [RCT Filter for EMBASE] (3180754)

99     or/60‐66 [Results before filters] (5498)

100     (2014$ or 2015$).em,dp,yr. [2014‐2015 EM limits] (2242899)

101     99 and 100 [2015 Results before filters] (918)

102     101 and 98 [2015 RCT Results] (190)

103     101 and 91 [2015 EPOC Results] (737)

Cochrane Central Register of Controlled Trials (OVID)

EBM Reviews ‐ Cochrane Central Register of Controlled Trials <December 2012>

Search date: April 8, 2015

1     (teleconsult$ or tele‐consult$).ti,ab. (45)

2     ((telephon$ or phone or phoning or phones or phoned) adj3 (advice or advise? or advising or consult$ or diagnos$ or evaluat$)).ti,ab. (605)

3     ((telephon$ or phone? or phoning) adj3 (physician? or GP or nurse or nurses or doctor? or general practitioner? or family doctor? or family practitioner? or consultant?)).ti,ab. (496)

4     (telephone management or telephone communication or telephone medicine or telephone intervention$ or telephone skill$).ab,ti. (263)

5     ((telephone or telephones or phone or phones) and (care or counselling or diagnos$ or health$ or intervention? or manag$ or therap$ or treat$ or medicine or medical or nursing or nurse? or physician? or doctor? or practitioner?)).ti. (995)

6     Remote Consultation/ and (telephon$.ti,ab,hw. or (phone or phones).ti,ab.) (129)

7     ((telephone? or phone) adj3 skill?).ab. (25)

8     or/1‐7 [Teleconsultation Keyword Set] (1989)

9     telemedicine/ or telepathology/ or teleradiology/ or Telenursing/ (914)

10     (teleassist$ or tele‐assist$ or teleaudiolog$ or tele‐audiolog$ or telebased or tele‐based or telecancer or tele‐cardiolo$ or telecardiolog$ or telecounselling or tele‐counselling teledental or tele‐dental or telederm$ or tele‐derm$ or telediagnos$ or tele‐diagnos$ or teledialysis or tele‐dialysis or teleecho$ or tele‐echo$ or teleemerg$ or tele‐emerg$ or teleepileps$ or tele‐epileps$ or telefollow$ or tele‐follow$ or teleguidance or tele‐guidance or telehealth$ or tele‐health$ or telehome$ or tele‐home$ or teleICU or tele‐ICU or teleintervention$ or tele‐intervention$ or telemanag$ or tele‐manag$ or telemedicine or tele‐medicine or telemental$ or tele‐mental$ or telemonitor$ or tele‐monitor$ or telenurs$ or tele‐nurs$ or teleoncolo$ or tele‐oncolo$ or teleopthalm$ or tele‐opthalm$ or telepalliat$ or tele‐palliat$ or tele‐patholog$ or tele‐patholog$ or teleprocedu$ or tele‐procedu$ or telepsych$ or tele‐psych$ or teleradiol$ or tele‐radiol$ or telerefer$ or tele‐refer$ or telerehab$ or tele‐rehab$ or telesurger$ or tele‐surger$ or telesurgic$ or tele‐surgic$ or teletherap$ or tele‐therap$ or teletreat$ or tele‐treat$ or teletriage or tele‐triage).ti,ab. (1307)

11     (tele$ adj2 (care or counselling or diagnos$ or health$ or intervention? or manag$ or therap$ or treat$ or medicine or medical or nursing or nurse? or physician? or doctor? or practitioner?)).ab. (1720)

12     or/9‐11 [Telemedicine] (2964)

13     Telephone/ or Cellular phone/ (1357)

14     (telephone? or phone or phones or transtelephon$).ti. (1862)

15     (telephone based or phone based).ab. (396)

16     (telephone? or phone or phones).ab. (7460)

17     or/13‐16 [Telephones] (8189)

18     (remote adj2 (care or consult$ or diagnos$ or evaluat$ or monitor$ or treat$ or therap$)).ti,ab. (273)

19     (e‐care or ecare or e‐consult$ or econsult$ or e‐diagnos$ or ediagnosis$ or e‐health$ or ehealth$ or e‐medicine or emedicine or e‐nurse? or enurse? or e‐nursing or enursing or e‐physician? or ephysician? or e‐psych$ or epsych$ or e‐therapy or etherapy).ti,ab. [e‐Health] (234)

20     "referral and consultation"/ (1415)

21     consultation?.ti,ab. (3054)

22     Remote Consultation/ (316)

23     or/18‐22 [Remote Consultation/Referral/ecare] (4791)

24     Patient Care/ or After Care/ or Ambulatory Care/ or Postoperative Care/ or Preoperative Care/ or exp Nursing Care/ or Palliative Care/ or Perinatal Care/ or Postnatal Care/ or Prenatal Care/ or Preconception Care/ (13725)

25     exp General Practice/ (2208)

26     exp Diagnosis/ (241852)

27     exp Diagnostic services/ or mass screening/ or anonymous testing/ or mass chest x‐ray/ or multiphasic screening/ or neonatal screening/ (4614)

28     (exp patient care management/ not (exp Telemedicine/ or telenursing/)) or comprehensive health care/ or "delivery of health care"/ or disease management/ or nurse's practice patterns/ or patient care team/ or patient‐centered care/ or physician's practice patterns/ (14041)

29     exp health services/ or community health services/ or emergency medical services/ or triage/ or nursing care/ or nursing services/ (66106)

30     exp Medical History Taking/ (263)

31     (history adj2 taking).ti,ab. (212)

32     (patient? adj2 (assess$ or care or diagnos$ or evaluat$ or screen$)).ti,ab. (46676)

33     Public Health/ or Preventive Medicine/ or Preventive Psychiatry/ or Public Health Nursing/ or exp PUblic Health Practice/ (10744)

34     or/24‐33 [Patient Care/Care Delivery/Diagnosis/Public Health] (303554)

35     health personnel/ or infection control practitioners/ or medical staff/ or exp nurses/ or exp nursing staff/ or pharmacists/ or exp physicians/ (3230)

36     allied health personnel/ or exp nurses' aides/ or exp physician assistants/ (253)

37     (doctor? or nurse or nurses or physician? or practitioner?).ti. or ((medical or health$ or nursing or allied health$) adj2 (personnel or staff$)).ti,ab. (6897)

38     or/35‐37 [Health Personnel] (8980)

39     exp Education, Continuing/ or Clinical Clerkship/ or "Internship and Residency"/ or Teaching Rounds/ or Preceptorship/ (1799)

40     Inservice Training/ or Staff Development/ (545)

41     (inservice or ((staff or physician? or nurse or nurses or doctor? or resident? or residency or intern or interns or practitioner?) adj2 (educational$ or train$ or development?))).ti,ab. (2207)

42     rounds.ti,ab. (420)

43     ed.fs. (5144)

44     (education$ adj2 (intervention$ or program$ or hospital? or office? or practitioner? or GP or doctor?)).ti,ab. (6869)

45     (skill? adj2 develop$).ti,ab. (386)

46     (continuing adj2 education$).ti,ab. (386)

47     CME.ti,ab. and education.ti,ab,hw,fs. (107)

48     communication? skill?.ti,ab. (513)

49     (Health Communication/ or Communication/) and (skill?.ti. or (skill? adj2 develop$).ab.) (143)

50     ((telephone? or phone) adj3 skill?).ab. (25)

51     or/39‐50 [Education/Training] (14416)

52     Professional‐Patient Relations/ or Physician‐Patient Relations/ or Nurse‐Patient Relations/ (1802)

53     (professional patient or physician patient or nurse patient).ti,ab. (692)

54     or/52‐53 [Professional Professional Relations] (2380)

55     and/8,51 [Teleconsult Keyword & CME] (225)

56     (and/17,23,51) not 55 [Remote Consult & Telephone & CME] (40)

57     (and/17,34,51) not (or/55‐56) [Telephone & Patient Care & CME] (437)

58     (and/17,38,51) not (or/55‐57) [Telephone & Health Personnel & CME] (17)

59     (and/17,51,54) not (or/55‐58) [Telephone & Phys‐Patient Relations & CME] (2)

60     (and/17,51) not (or/55‐59) [Telephone & CME] (148)

61     (and/12,51) not (or/55‐60) [Telemed & CME/Training] (43)

62     (or/55‐61) and ("2014" or "2015").yr. (55)

Cochrane Database of Systematic Reviews and DARE, NHS EED, HTA, MTH (WILEY)

Search date: April 8, 2015

ID             Search  Hits

#1             (teleconsult* or tele‐consult*):ti,ab       46

#2             ((telephon* or phone or phoning or phones or phoned) near/3 (advice or advise or advising or consult* or diagnos* or evaluat*)):ti,ab                453

#3             ((telephon* or phone or phoning) near/3 (physician or GP or nurse or nurses or doctor or (general practitioner) or (family doctor) or (family practitioner) or consultant)):ti,ab       262

#4             ("telephone management" or "telephone communication" or "telephone medicine" or "telephone intervention*" or "telephone skill*"):ab,ti                 263

#5             ((telephone or telephones or phone or phones) and (care or counselling or diagnos* or health* or intervention or manag* or therap* or treat* or medicine or medical or nursing or nurse or physician or doctor or practitioner)):ti          957

#6             (telephon*:ti,ab,kw or (phone or phones):ti,ab)                 8553

#7             MeSH descriptor: [Remote Consultation] this term only                333

#8             #6 and #7              134

#9             ((telephone? or phone) near/3 skill?):ab                2

#10          #1 or #2 or #3 or #4 or #5 or #8 or #9          1693

#11          MeSH descriptor: [Telemedicine] this term only                1020

#12          MeSH descriptor: [Telepathology] this term only               5

#13          MeSH descriptor: [Teleradiology] this term only                22

#14          MeSH descriptor: [Telenursing] this term only   15

#15          (teleassist* or tele‐assist* or teleaudiolog* or tele‐audiolog* or telebased or tele‐based or telecancer or tele‐cardiolo* or telecardiolog* or telecounselling or tele‐counselling teledental or tele‐dental or telederm* or tele‐derm* or telediagnos* or tele‐diagnos* or teledialysis or tele‐dialysis or teleecho* or tele‐echo* or teleemerg* or tele‐emerg* or teleepileps* or tele‐epileps* or telefollow* or tele‐follow* or teleguidance or tele‐guidance or telehealth* or tele‐health* or telehome* or tele‐home* or teleICU or tele‐ICU or teleintervention* or tele‐intervention* or telemanag* or tele‐manag* or telemedicine or tele‐medicine or telemental* or tele‐mental* or telemonitor* or tele‐monitor* or telenurs* or tele‐nurs* or teleoncolo* or tele‐oncolo* or teleopthalm* or tele‐opthalm* or telepalliat* or tele‐palliat* or tele‐patholog* or tele‐patholog* or teleprocedu* or tele‐procedu* or telepsych* or tele‐psych* or teleradiol* or tele‐radiol* or telerefer* or tele‐refer* or telerehab* or tele‐rehab* or telesurger* or tele‐surger* or telesurgic* or tele‐surgic* or teletherap* or tele‐therap* or teletreat* or tele‐treat* or teletriage or tele‐triage):ti,ab     1339

#16          (tele* near/2 (care or counselling or diagnos* or health* or intervention or manag* or therap* or treat* or medicine or medical or nursing or nurse or physician or doctor or practitioner)):ab                 1466

#17          #11 or #12 or #13 or #14 or #15 or #16    2860

#18          MeSH descriptor: [Telephone] this term only     1413

#19          MeSH descriptor: [Cellular Phone] this term only              327

#20          (telephone? or phone or phones or transtelephon*):ti                   474

#21          ("telephone based" or "phone based"):ab            417

#22          (telephone or phone or phones):ab         7529

#23          #18 or #19 or #20 or #21 or #22                    8023

#24          (remote near/2 (care or consult* or diagnos* or evaluat* or monitor* or treat* or therap*)):ti,ab            231

#25          (e‐care or ecare or e‐consult* or econsult* or e‐diagnos* or ediagnosis* or e‐health* or ehealth* or e‐medicine or emedicine or e‐nurse? or enurse? or e‐nursing or enursing or e‐physician or ephysician or e‐psych* or epsych* or e‐therapy or etherapy):ti,ab               264

#26          MeSH descriptor: [Referral and Consultation] this term only      1515

#27          consultation:ti,ab            2360

#28          #24 or #25 or #26 or #27 or #7     4273

#29          MeSH descriptor: [Patient Care] this term only                   135

#30          MeSH descriptor: [Aftercare] this term only        433

#31          MeSH descriptor: [Ambulatory Care] this term only         3047

#32          MeSH descriptor: [Postoperative Care] this term only   3882

#33          MeSH descriptor: [Preoperative Care] this term only     3559

#34          MeSH descriptor: [Nursing Care] explode all trees           1620

#35          MeSH descriptor: [Palliative Care] this term only              1425

#36          MeSH descriptor: [Perinatal Care] this term only              126

#37          MeSH descriptor: [Postnatal Care] this term only             314

#38          MeSH descriptor: [Prenatal Care] this term only                1078

#39          MeSH descriptor: [Preconception Care] this term only  65

#40          MeSH descriptor: [General Practice] explode all trees  2334

#41          MeSH descriptor: [Diagnosis] explode all trees                  257329

#42          MeSH descriptor: [Diagnostic Services] explode all trees              5205

#43          MeSH descriptor: [Mass Screening] this term only            4349

#44          MeSH descriptor: [Anonymous Testing] this term only   10

#45          MeSH descriptor: [Mass Chest X‐Ray] this term only       32

#46          MeSH descriptor: [Multiphasic Screening] this term only              16

#47          MeSH descriptor: [Neonatal Screening] this term only  274

#48          MeSH descriptor: [Patient Care Management] explode all trees               17004

#49          MeSH descriptor: [Telemedicine] explode all trees         1309

#50          #48 not (#14 or #49)         15688

#51          MeSH descriptor: [Comprehensive Health Care] this term only                  80

#52          MeSH descriptor: [Delivery of Health Care] this term only            751

#53          MeSH descriptor: [Disease Management] this term only               644

#54          MeSH descriptor: [Nurse's Practice Patterns] this term only       62

#55          MeSH descriptor: [Patient Care Team] this term only     1419

#56          MeSH descriptor: [Patient‐Centered Care] this term only             327

#57          MeSH descriptor: [Physician's Practice Patterns] this term only                1104

#58          MeSH descriptor: [Health Services] explode all trees     73049

#59          MeSH descriptor: [Community Health Services] this term only  864

#60          MeSH descriptor: [Emergency Medical Services] this term only                  900

#61          MeSH descriptor: [Triage] this term only                260

#62          MeSH descriptor: [Nursing Care] this term only                  198

#63          MeSH descriptor: [Nursing Services] this term only          13

#64          MeSH descriptor: [Medical History Taking] explode all trees      309

#65          (history near/2 taking):ti,ab        190

#66          (patient? near/2 (assess* or care or diagnos* or evaluat* or screen*)):ti,ab       24745

#67          MeSH descriptor: [Public Health] this term only                 229

#68          MeSH descriptor: [Preventive Medicine] this term only                  138

#69          MeSH descriptor: [Public Health Nursing] this term only                72

#70          MeSH descriptor: [Public Health Practice] explode all trees       11535

#71          #29 or #30 or #31 or #32 or #33 or #34 or #35 or #36 or #37 or #38 or #39 or #40 or #41 or #42 or #43 or #44 or #45 or #46 or #47 or #50 or #51 or #52 or #53 or #54 or #55 or #56 or #57 or #58 or #59 or #60 or #61 or #62 or #63 or #64 or #65 or #66 or #67 or #68 or #69 or #70              311757

#72          MeSH descriptor: [Health Personnel] this term only       533

#73          MeSH descriptor: [Infection Control Practitioners] this term only             8

#74          MeSH descriptor: [Medical Staff] this term only                  32

#75          MeSH descriptor: [Nurses] explode all trees       1004

#76          MeSH descriptor: [Nursing Staff] explode all trees            529

#77          MeSH descriptor: [Pharmacists] this term only                   446

#78          MeSH descriptor: [Physicians] explode all trees                1293

#79          MeSH descriptor: [Allied Health Personnel] this term only           177

#80          MeSH descriptor: [Nurses' Aides] explode all trees          55

#81          MeSH descriptor: [Physician Assistants] explode all trees           47

#82          (doctor or nurse or nurses or physician or practitioner):ti or ((medical or health* or nursing or allied health*) near/2 (personnel or staff*)):ti,ab            5336

#83          #72 or #73 or #74 or #75 or #76 or #77 or #78 or #79 or #80 or #81 or #82                 8050

#84          MeSH descriptor: [Education, Continuing] explode all trees        1008

#85          MeSH descriptor: [Clinical Clerkship] this term only        124

#86          MeSH descriptor: [Internship and Residency] this term only       767

#87          MeSH descriptor: [Teaching Rounds] this term only         6

#88          MeSH descriptor: [Preceptorship] this term only               27

#89          MeSH descriptor: [Inservice Training] this term only       515

#90          MeSH descriptor: [Staff Development] this term only     58

#91          (inservice or ((staff or physician or nurse or nurses or doctor or resident or residency or intern or interns or practitioner) near/2 (educational* or train* or development))):ti,ab      1300

#92          rounds:ti,ab       428

#93          (education* near/2 (intervention* or program* or hospital or office or practitioner or GP or doctor)):ti,ab          6629

#94          (skill near/2 develop*):ti,ab        111

#95          (continuing near/2 education*):ti,ab     378

#96          (CME):ti,ab and (education):ti,ab,kw      104

#97          (communication skill):ti,ab         105

#98          MeSH descriptor: [Health Communication] this term only            46

#99          MeSH descriptor: [Communication] this term only           1453

#100       (skill):ti or (skill? near/2 develop*):ab    644

#101       (#98 or #99) and #100     16

#102       ((telephone or phone) near/3 skill):ab   2

#103       #84 or #85 or #86 or #87 or #88 or #89 or #90 or #91 or #92 or #93 or #94 or #95 or #96 or #97 or #101 or #102     10192

#104       MeSH descriptor: [Professional‐Patient Relations] this term only            601

#105       MeSH descriptor: [Physician‐Patient Relations] this term only 1081

#106       MeSH descriptor: [Nurse‐Patient Relations] this term only          314

#107       (professional patient or physician patient or nurse patient):ti,ab            6910

#108       #104 or #105 or #106 or #107     8437

#109       #10 and #103     125

#110       (#23 and #28 and #103)                   53

#111       (#23 and #71 and #103)                   400

#112       (#23 and #83 and #103)                   89

#113       (#23 and #103 and #108)                115

#114       (#23 and #103) 596

#115       (#17 and #103) 150

#116       #109 or #110 or #111 or #112 or #113 or #114 or #115 Publication Year from 2014 to 2015          57

CINAHL (Ebsco)

Search date: January 9, 2013

#	Query	Results
S97	S95 AND S85 [EPOC Results]	655
S96	S94 AND S95 [RCT Results]	316
S95	( S54 OR S55 OR S56 OR S57 OR S58 OR S59 ) OR ( S12 AND S50 ) [Results before filters]	1,368
S94	S86 OR S87 OR S88 OR S89 OR S90 OR S91 OR S92 OR S93 [RCT Filter]	137,865
S93	TI controlled AND TI ( trial or trials or study or experiment* or intervention )	15,931
S92	AB ( (multicent* n2 design*) or (multicent* n2 study) or (multicent* n2 studies) or (multicent* n2 trial*) ) or AB ( (multi‐cent* n2 design*) or (multi‐cent* n2 study) or (multi‐cent* n2 studies) or (multi‐cent* n2 trial*) )	5,903
S91	TI multicentre or multicenter or multi‐centre or multi‐center	3,900
S90	TI ( cluster N2 trial* or cluster N2 study or cluster N2 group or cluster N2 groups or cluster N2 cohort or cluster N2 design or cluster N2 experiment* ) OR AB ( cluster N2 trial* or cluster N2 study or cluster N2 group or cluster N2 groups or cluster N2 cohort or cluster N2 design or cluster N2 experiment* )	1,454
S89	TI ( control group or control groups OR control* experiment* or control* design or controlled study ) OR AB ( control group OR control groups or control* cohort* or controlled experiment* controlled design or controlled study)	44,895
S88	TI random* or AB random*	98,033
S87	TI ( “clinical study” or “clinical studies” ) or AB ( “clinical study” or “clinical studies” )	6,327
S86	(MM "Clinical Trials+")	7,551
S85	S61 or S62 or S63 or S64 or S65 or S66 or S67 or S68 or S69 or S70 or S71 or S72 or S73 or S74 or S75 or S76 or S77 or S78 or S79 or S80 or S81 or S82 or S83 or S84 [EPOC Filter]	382,111
S84	TI ( (time points n3 over) or (time points n3 multiple) or (time points n3 three) or (time points n3 four) or (time points n3 five) or (time points n3 six) or (time points n3 seven) or (time points n3 eight) or (time points n3 nine) or (time points n3 ten) or (time points n3 eleven) or (time points n3 twelve) or (time points n3 month*) or (time points n3 hour*) or (time points n3 day*) or (time points n3 "more than") ) or AB ( (time points n3 over) or (time points n3 multiple) or (time points n3 three) or (time points n3 four) or (time points n3 five) or (time points n3 six) or (time points n3 seven) or (time points n3 eight) or (time points n3 nine) or (time points n3 ten) or (time points n3 eleven) or (time points n3 twelve) or (time points n3 month*) or (time points n3 hour*) or (time points n3 day*) or (time points n3 "more than") )	1,357
S83	TI ( (control w3 area) or (control w3 cohort*) or (control w3 compar*) or (control w3 condition) or (control w3 group*) or (control w3 intervention*) or (control w3 participant*) or (control w3 study) ) or AB ( (control w3 area) or (control w3 cohort*) or (control w3 compar*) or (control w3 condition) or (control w3 group*) or (control w3 intervention*) or (control w3 participant*) or (control w3 study) )	41,546
S82	TI ( multicentre or multicenter or multi‐centre or multi‐center ) or AB random*	88,922
S81	TI random* OR controlled	30,266
S80	TI ( trial or (study n3 aim) or "our study" ) or AB ( (study n3 aim) or "our study" )	74,120
S79	TI ( pre‐workshop or preworkshop or post‐workshop or postworkshop or (before n3 workshop) or (after n3 workshop) ) or AB ( pre‐workshop or preworkshop or post‐workshop or postworkshop or (before n3 workshop) or (after n3 workshop) )	286
S78	TI ( demonstration project OR demonstration projects OR preimplement* or pre‐implement* or post‐implement* or postimplement* ) or AB ( demonstration project OR demonstration projects OR preimplement* or pre‐implement* or post‐implement* or postimplement* )	1,198
S77	(intervention n6 clinician*) or (intervention n6 community) or (intervention n6 complex) or (intervention n6 design*) or (intervention n6 doctor*) or (intervention n6 educational) or (intervention n6 family doctor*) or (intervention n6 family physician*) or (intervention n6 family practitioner*) or (intervention n6 financial) or (intervention n6 GP) or (intervention n6 general practice*) Or (intervention n6 hospital*) or (intervention n6 impact*) Or (intervention n6 improv*) or (intervention n6 individualize*) Or (intervention n6 individualise*) or (intervention n6 individualizing) or (intervention n6 individualising) or (intervention n6 interdisciplin*) or (intervention n6 multicomponent) or (intervention n6 multi‐component) or (intervention n6 multidisciplin*) or (intervention n6 multi‐disciplin*) or (intervention n6 multifacet*) or (intervention n6 multi‐facet*) or (intervention n6 multimodal*) or (intervention n6 multi‐modal*) or (intervention n6 personalize*) or(intervention n6 personalise*) or (intervention n6 personalizing) or (intervention n6 personalising) or (intervention n6 pharmaci*) or (intervention n6 pharmacist*) or (intervention n6 pharmacy) or (intervention n6 physician*) or (intervention n6 practitioner*) Or (intervention n6 prescrib*) or (intervention n6 prescription*) or (intervention n6 primary care) or (intervention n6 professional*) or (intervention* n6 provider*) or (intervention* n6 regulatory) or (intervention n6 regulatory) or (intervention n6 tailor*) or (intervention n6 target*) or (intervention n6 team*) or (intervention n6 usual care)	36,970
S76	TI ( collaborativ* or collaboration* or tailored or personalised or personalized ) or AB ( collaborativ* or collaboration* or tailored or personalised or personalized )	33,986
S75	TI pilot	10,372
S74	(MH "Pilot Studies")	26,778
S73	AB "before‐and‐after"	15,409
S72	AB time series	1,576
S71	TI time series	219
S70	AB ( before* n10 during or before n10 after ) or AU ( before* n10 during or before n10 after )	29,229
S69	TI ( (time point*) or (period* n4 interrupted) or (period* n4 multiple) or (period* n4 time) or (period* n4 various) or (period* n4 varying) or (period* n4 week*) or (period* n4 month*) or (period* n4 year*) ) or AB ( (time point*) or (period* n4 interrupted) or (period* n4 multiple) or (period* n4 time) or (period* n4 various) or (period* n4 varying) or (period* n4 week*) or (period* n4 month*) or (period* n4 year*) )	44,497
S68	TI ( ( quasi‐experiment* or quasiexperiment* or quasi‐random* or quasirandom* or quasi control* or quasicontrol* or quasi* W3 method* or quasi* W3 study or quasi* W3 studies or quasi* W3 trial or quasi* W3 design* or experimental W3 method* or experimental W3 study or experimental W3 studies or experimental W3 trial or experimental W3 design* ) ) or AB ( ( quasi‐experiment* or quasiexperiment* or quasi‐random* or quasirandom* or quasi control* or quasicontrol* or quasi* W3 method* or quasi* W3 study or quasi* W3 studies or quasi* W3 trial or quasi* W3 design* or experimental W3 method* or experimental W3 study or experimental W3 studies or experimental W3 trial or experimental W3 design* ) )	10,969
S67	TI pre w7 post or AB pre w7 post	8,092
S66	MH "Multiple Time Series" or MH "Time Series"	1,210
S65	TI ( (comparative N2 study) or (comparative N2 studies) or evaluation study or evaluation studies ) or AB ( (comparative N2 study) or (comparative N2 studies) or evaluation study or evaluation studies )	9,471
S64	MH Experimental Studies or Community Trials or Community Trials or Pretest‐Posttest Design + or Quasi‐Experimental Studies + Pilot Studies or Policy Studies + Multicenter Studies	30,981
S63	TI ( pre‐test* or pretest* or posttest* or post‐test* ) or AB ( pre‐test* or pretest* or posttest* or "post test* ) OR TI ( preimplement*" or pre‐implement* ) or AB ( pre‐implement* or preimplement* )	6,241
S62	TI ( intervention* or multiintervention* or multi‐intervention* or postintervention* or post‐intervention* or preintervention* or pre‐intervention* ) or AB ( intervention* or multiintervention* or multi‐intervention* or postintervention* or post‐intervention* or preintervention* or pre‐intervention* )	132,670
S61	(MH "Quasi‐Experimental Studies")	5,300
S60	( S12 AND S50 ) NOT ( S54 or S55 or S56 or S57 or S58 or S59 )	232
S59	( S17 AND S50 ) NOT ( S54 or S55 or S56 or S57 or S58 )	290
S58	( S17 AND S53 AND S50 ) NOT ( S54 or S55 or S56 or S57 )	17
S57	( S17 AND S38 AND S50 ) NOT ( S54 or S55 or S56 )	128
S56	( S17 AND S34 AND S50 ) NOT ( S54 or S55 )	649
S55	( S17 AND S23 AND S50 ) NOT S54	37
S54	S8 AND S50	279
S53	S51 OR S52	73,431
S52	TI ( professional patient or physician patient or nurse patient ) OR AB ( professional patient or physician patient or nurse patient )	32,618
S51	MH Professional‐Patient Relations OR MH Physician‐Patient Relations OR MH Nurse‐Patient Relations	46,421
S50	S39 OR S40 OR S41 OR S42 OR S43 OR S44 OR S45 OR S46 OR S47 OR S48 OR S49	83,710
S49	AB (telephone# or phone) N3 skill#	42
S48	( MM "Communication+" or "health communication" ) AND ( TI skill# OR AB skill# N2 develop* )	1,502
S47	TI communication# skill# OR AB communication# skill#	3,569
S46	( TI CME OR AB CME ) AND ( TI education OR AB education OR MW education )	1,496
S45	TI continuing N2 education* OR AB continuing N2 education*	8,879
S44	TI skill# N2 develop* OR AB skill# N2 develop*	3,924
S43	TI ( education* N2 (intervention* or program* or hospital# or office# or practitioner# or GP or doctor#) ) OR AB ( education* N2 (intervention* or program* or hospital# or office# or practitioner# or GP or doctor#) )	20,254
S42	TI rounds OR AB rounds	3,812
S41	TI ( inservice or ((staff or physician# or nurse or nurses or doctor# or resident# or residency or intern or interns or practitioner#) N2 (educational* or train* or development#)) ) OR AB ( inservice or ((staff or physician# or nurse or nurses or doctor# or resident# or residency or intern or interns or practitioner#) N2 (educational* or train* or development#)) )	14,872
S40	MH Staff Development OR "Inservice Training"	16,936
S39	( MH "Education, Continuing+" or MH "Internship and Residency" or MH "Preceptorship" ) OR ( "Clinical Clerkship" or "Teaching Rounds" )	25,093
S38	S35 OR S36 OR S37	291,252
S37	TI ( doctor# or nurse or nurses or physician# or practitioner# ) OR ( TI ( (medical or health* or nursing or allied health*) N2 (personnel or staff*) ) OR AB ( (medical or health* or nursing or allied health*) N2 (personnel or staff*) ) )	150,524
S36	MH "allied health personnel" or MH "nursing assistants+" or MH "physician assistants+"	9,211
S35	MH "health personnel" or MH "infection control practitioners" or MH "medical staff" or MH "nurses+" or MH "pharmacists" or MH "physicians+" or "nursing staff"	199,788
S34	S24 OR S25 OR S26 OR S27 OR S28 OR S29 OR S30 OR S31 OR S32 OR S33	1,088,777
S33	( MH "Public Health" or MH "Preventive Health Care" or "Community Health Nursing" ) OR ( "Preventive Psychiatry" or "Public Health Practice" )	42,665
S32	TI ( patient# N2 (assess* or care or diagnos* or evaluat* or screen*) ) OR AB ( patient# N2 (assess* or care or diagnos* or evaluat* or screen*) )	80,244
S31	TI history N2 taking OR AB history N2 taking	922
S30	(MH "Patient History Taking+")	10,251
S29	MH "health services+" OR MH community health services OR MH emergency medical services OR MH triage OR MH nursing care OR MH nursing service	504,609
S28	( "patient care management" NOT (MH "telemedicine+" or MH "telenursing") ) OR ( MH health care delivery or MH disease management or MH multidisciplinary care team or MH patient‐centered care or "comprehensive health care" or "nurse's practice patterns" or "physician's practice patterns" )	44,331
S27	( (MH "Diagnostic services+" or MH "neonatal assessment") ) OR ( "mass screening" or "anonymous testing" or "mass chest x‐ray" or "multiphasic screening" )	38,908
S26	(MH "Diagnosis+")	610,376
S25	MH Family Practice	9,663
S24	MH "Nursing Care+" or MH "Patient Care" or MH "After Care" or MH "Ambulatory Care" or MH "Postoperative Care" or MH "Preoperative Care" or MH "Palliative Care" or MH "Perinatal Care" or MH "Postnatal Care" or MH "Prenatal Care" or MH "Prepregnancy Care"	244,491
S23	S18 OR S19 OR S20 OR S21 OR S22	25,892
S22	MH Remote Consultation	499
S21	TI consultation# OR AB consultation#	11,152
S20	MH "referral and consultation"	14,943
S19	TI ( e‐care or ecare or e‐consult* or econsult* or e‐diagnos* or ediagnosis* or e‐health* or ehealth* or e‐medicine or emedicine or e‐nurse# or enurse# or e‐nursing or enursing or e‐physician# or ephysician# or e‐psych* or epsych* or e‐therapy or etherapy ) OR AB ( e‐care or ecare or e‐consult* or econsult* or e‐diagnos* or ediagnosis* or e‐health* or ehealth* or e‐medicine or emedicine or e‐nurse# or enurse# or e‐nursing or enursing or e‐physician# or ephysician# or e‐psych* or epsych* or e‐therapy or etherapy )	942
S18	TI ( remote N2 (care or consult* or diagnos* or evaluat* or monitor* or treat* or therap*) ) OR AB ( remote N2 (care or consult* or diagnos* or evaluat* or monitor* or treat* or therap*) )	465
S17	S13 OR S14 OR S15 OR S16	24,701
S16	AB telephone# or phone or phones	13,198
S15	AB telephone based or phone based	692
S14	TI telephone# or phone or phones or transtelephon*	3,496
S13	MH Telephone or Wireless Communications	14,862
S12	S9 OR S10 OR S11	6,793
S11	AB tele* N2 (care or counselling or diagnos* or health* or intervention# or manag* or therap* or treat* or medicine or medical or nursing or nurse# or physician# or doctor# or practitioner#)	2,096
S10	TI ( teleassist* or tele‐assist* or teleaudiolog* or tele‐audiolog* or telebased or tele‐based or telecancer or tele‐cardiolo* or telecardiolog* or telecounselling or tele‐counselling or teledental or tele‐dental or telederm* or tele‐derm* or telediagnos* or tele‐diagnos* or teledialysis or tele‐dialysis or teleecho* or tele‐echo* or teleemerg* or tele‐emerg* or teleepileps* or tele‐epileps* or telefollow* or tele‐follow* or teleguidance or tele‐guidance or telehealth* or tele‐health* or telehome* or tele‐home* or teleICU or tele‐ICU or teleintervention* or tele‐intervention* or telemanag* or tele‐manag* or telemedicine or tele‐medicine or telemental* or tele‐mental* or telemonitor* or tele‐monitor* or telenurs* or tele‐nurs* or teleoncolo* or tele‐oncolo* or teleopthalm* or tele‐opthalm* or telepalliat* or tele‐palliat* or tele‐patholog* or tele‐patholog* or teleprocedu* or tele‐procedu* or telepsych* or tele‐psych* or teleradiol* or tele‐radiol* or telerefer* or tele‐refer* or telerehab* or tele‐rehab* or telesurger* or tele‐surger* or telesurgic* or tele‐surgic* or teletherap* or tele‐therap* or teletreat* or tele‐treat* or teletriage or tele‐triage ) OR AB ( teleassist* or tele‐assist* or teleaudiolog* or tele‐audiolog* or telebased or tele‐based or telecancer or tele‐cardiolo* or telecardiolog* or telecounselling or tele‐counselling or teledental or tele‐dental or telederm* or tele‐derm* or telediagnos* or tele‐diagnos* or teledialysis or tele‐dialysis or teleecho* or tele‐echo* or teleemerg* or tele‐emerg* or teleepileps* or tele‐epileps* or telefollow* or tele‐follow* or teleguidance or tele‐guidance or telehealth* or tele‐health* or telehome* or tele‐home* or teleICU or tele‐ICU or teleintervention* or tele‐intervention* or telemanag* or tele‐manag* or telemedicine or tele‐medicine or telemental* or tele‐mental* or telemonitor* or tele‐monitor* or telenurs* or tele‐nurs* or teleoncolo* or tele‐oncolo* or teleopthalm* or tele‐opthalm* or telepalliat* or tele‐palliat* or tele‐patholog* or tele‐patholog* or teleprocedu* or tele‐procedu* or telepsych* or tele‐psych* or teleradiol* or tele‐radiol* or telerefer* or tele‐refer* or telerehab* or tele‐rehab* or telesurger* or tele‐surger* or telesurgic* or tele‐surgic* or teletherap* or tele‐therap* or teletreat* or tele‐treat* or teletriage or tele‐triage )	2,782
S9	MH telemedicine or telepathology or teleradiology or Telenursing	4,348
S8	S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7	2,831
S7	AB (telephone# or phone) N3 skill#	42
S6	MH Remote Consultation AND ( (TI telephon* OR AB telephon* OR MW telephon*) or (TI ( phone or phones ) OR AB ( phone or phones )) )	109
S5	TI (telephone or telephones or phone or phones) and (care or counselling or diagnos* or health* or intervention# or manag* or therap* or treat* or medicine or medical or nursing or nurse# or physician# or doctor# or practitioner#)	1,242
S4	TI ( telephone management or telephone communication or telephone medicine or telephone intervention* or telephone skill* ) OR AB ( telephone management or telephone communication or telephone medicine or telephone intervention* or telephone skill* )	971
S3	TI ( (telephon* or phone# or phoning) N3 (physician# or GP or nurse or nurses or doctor# or general practitioner# or family doctor# or family practitioner# or consultant#) ) OR AB ( (telephon* or phone# or phoning) N3 (physician# or GP or nurse or nurses or doctor# or general practitioner# or family doctor# or family practitioner# or consultant#) )	672
S2	TI ( ((telephon* or phone or phoning or phones or phoned) N3 (advice or advise# or advising or consult* or diagnos* or evaluat*)) ) OR AB ( ((telephon* or phone or phoning or phones or phoned) N3 (advice or advise# or advising or consult* or diagnos* or evaluat*)) )	704
S1	TI ( teleconsult* or tele‐consult* ) OR AB ( teleconsult* or tele‐consult* )	78

EPOC Specialised Register, Reference Manager 12

Search date: March 20, 2014

All Fields: {teleconsult} OR {tele‐consult} OR {telephone} or {phone} or {phoning}

AND

{consult} OR {advise} OR {advice} OR {diagnos*} OR {evaluat*} OR {counselling} OR {medicine}

{medical} OR {nurs*} OR {doctor} OR {physician} OR {practioner} (995)

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Wood 1989.

Methods	Study design: controlled before‐after with 2 intervention and 2 control sites Study dates: not reported Unit of allocation: single participant Unit of analysis: single participant Power calculation: not reported Inclusion/exclusion criteria: not reported Ethical approval and informed consent: each resident provided informed consent for recording of telephone calls prior to initiation of the study, and the study was approved by the Institutional Review Board.
Participants	Participating providers Number: 11 Setting: primary care paediatric residency training programme (USA) Profession: doctors, first year residents Clinical speciality: paediatrics Age: not reported Time since graduation (or year of practice): not reported Level of professional training: residents Telephone consultation training previously received: no previous training mentioned Methods of recruitment of participants: not reported Type of targeted behaviour: obligatory questions asked at the phone Identification of recipients of interventions: incognito standardised patients Participating patients Number: simulated mothers calling by phone for a 6‐month‐old baby with diarrhoea and a 1‐year‐old child with fever without providing any information that was not specifically requested by the physician. Sex: not reported (sex of child used by simulating mothers) Age: not reported Ethnicity: not reported Type of patient (primary care, outpatient, inpatient, other): outpatient Socioeconomic profile: not reported Relevant treatment history or previous contact with healthcare professionals: not reported Clinical problem (illness type): diarrhoea and fever in children Concurrent conditions (comorbidity): none declared
Interventions	Aim of the study: to evaluate the effect of a role play telephone management curriculum for paediatric house officers on history‐taking and management skills. Intervention group: intervention group residents (n = 6) participated in a role play curriculum. In the curriculum, a structured approach to telephone management of paediatric problems was stressed; curriculum content was drawn from a variety of sources including textbooks on telephone management and previous studies of telephone management: as a group, the residents generated a list of key elements that are important in history‐taking and management by telephone; after this activity, each resident had the opportunity to participate in several mock telephone conversations in which one resident played the parent of a sick child and the other resident played the clinic doctor; other group members were observers, and all participants were given general instructions about role play. Evaluation of the encounter was followed by role play of a new case, by role reversal, or by replay of specific aspects of the encounter that were problematic. Control group: control group residents (n = 7) received no formal instruction in telephone management Duration of Intervention: the curriculum consisted of 3, half‐hour group sessions. Repetition of intervention: none declared Follow‐up period (postintervention): 2 post‐test calls were recorded for each resident during his or her next rotation through the outpatient department approximately 3 months later. Rationale for chosen follow‐up period: not reported
Outcomes	Primary outcomes: 3 scores on history‐taking and management skills: GHS (15 items of historic information that applied to any telephone encounter: e.g. the chief complaint, duration of symptoms, and overall condition of the child) SHS/SHT (13 for diarrhoea and 15 for fever historic items that were specific to the given case) GMS (6 management items believed to apply across cases; e.g. providing a tentative diagnosis, giving instructions for home treatment or – for those telephone encounters in which the family was told to bring the child to the clinic, the general management score on the basis of only 2 items: providing a diagnosis and making plans for follow‐up). Secondary outcomes: Number of minutes spent answering the call was determined by measuring the length of the recorded conversation Proportion of clinical cases managed just by phone Total number of historic questions asked and total number of management statements Participant satisfaction Methods for measuring outcomes of interest: at the beginning of the rotation, each resident received 2 baseline or pretest calls made by a simulated mother using standardised scripts. After the curriculum administration in the intervention group, residents from both groups were assigned to answer telephone calls during their outpatient rotations. Timing of outcome assessment: after the curriculum (as baseline pretest) and 3 months later (post‐test), obtaining in this way a couple of assessments for each resident. Methods of follow‐up: calls repetition Tools used to measure outcomes: pretest and post‐test calls were made to a particular resident during times when he or she was assigned to function as 'telephone doctor' in the resident continuity clinic. Using standardised scripts, simulated mothers played the role of a mother calling the clinic for advice concerning her sick child. These calls were handled in an identical manner to other calls received during a given clinic session. The standardised rating form was composed of 3 scales, each consisting of weighted items that were summed and then expressed as a percentage of possible perfect score. Tool validation: one the tools (SHS/SHT) was validated (see Table 2)
Notes	No quantitative data were provided in the paper and no answer was received from the authors after specific request. Declaration of interest: not reported Source of funding: this work was supported, in part, by US Department of Health and Human Services grant and by an institutional research grant as part of the National Institutes of Health Biomedical Research Support grant programme.
Risk of bias
Bias	Authors' judgement	Support for judgement
Was the allocation sequence adequately generated?	High risk	CBA studies should be scored 'high risk' according to suggested risk of bias criteria for EPOC reviews. Authors say group assignment (intervention or control) was determined by residents' month of rotation.
Was the allocation adequately concealed?	High risk	CBA studies should be scored 'high risk' according to suggested risk of bias criteria for EPOC reviews
Were baseline outcome measurements similar?	Unclear risk	No data provided
Were baseline characteristics similar?	Unclear risk	No data provided
Were incomplete outcome data adequately addressed?	High risk	Unpaired audio tapes (n = 5) were excluded from analysis.
Was knowledge of the allocated interventions adequately prevented during the study?	Low risk	All calls were audio‐recorded by the 'mothers'. Audiotapes were subsequently transcribed and rated by a single trained rater who was unaware of resident's identity or group status, and of the timing of the call.
Was the study adequately protected against contamination	Low risk	Not mentioned, but the study design protected against contamination
Was the study free from selective outcome reporting?	High risk	Many outcomes are provided in the Results section and not in the Methods one.
Was the study free from other risks of bias?	Low risk	No evident other risk of bias
OVERALL RISK OF BIAS	High risk	High risk of bias in one key domain

CBA: controlled before‐after; GHS: general history scale; GMS: general management scale; SHS: specific history scale; SHT: specific history triage.

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Afzali 2012	Not an experimental design (it is the description of tasks and competencies of medical operators in relation to quick recognition of cardiac arrest)
Banach 2012	Not complying with inclusion criteria (it is a before‐after study without control)
Bediang 2013	Not complying with inclusion criteria (the outcome is not telephone skills improvement)
Bell 2003	Not complying with inclusion criteria (it is a survey on opinions of a website users providing expert recommendations and advice to front‐line telephone staff)a
Berlin 2010	Not complying with inclusion criteria (it is a before‐after study without control)
Billington 2015	Not complying with inclusion criteria (the outcome is self‐management)
Bishop 2013	Not an experimental design (it complements the protocol and main trial results papers – assessing outcomes not matching our criteria – by describing the system used in the trial and how physiotherapists were prepared for using the system and delivering the service)
Boutin 2006	Not complying with inclusion criteria (it is a before after study without control)
Brown 1974	Not an experimental design (participants were asked to list questions that they thought were important in taking a telephone history)
Campbell 2014 (ESTEEM trial)	We discussed a lot the opportunity to include or not this trial: in the protocol we stated we our aim was to "make comparisons between outcomes of clinicians who have received the intervention with those who have not"; we decided to exclude this trial because, while the 2 intervention groups were telephone‐skills training for GPs and nurses, the control group was a mix of non‐trained GPs and non‐trained nurses, and in 78.6% of cases. nobody was allocated to telephone triage; for this reason we decided to exclude the trial; moreover, the training intervention is not described but just cited.
Caralis 2010	Not an experimental design (it is a review of random selected medical calls: 366 patient calls were studied and 42% of these patients were called back interviewed about their satisfaction)
Chandrasekaran 2015	Not an experimental design (survey regarding communication between doctors)
Cunningham 2012	Not complying with inclusion criteria (it is a study about communication between clinicians during hand‐over procedures)
Curry 1978	Not complying with outcome inclusion criteria (percentage of required questions asked)
Delvin 2014	Not an experimental design (it explores the potential pitfalls of giving professional advice over the telephone)
Dougherty 1988	Not an experimental design (it is an article with advice for handling telephone calls)
Evens 1983	Not an experimental design (it is a simple description of a training method and experience)
Gleason 2013	Not an experimental design (it is the description of an initiative to identify the educational needs of nurses and design strategies to educate and improve telephone triage skills)
Greenberg 1999a	Clinicians' telephone consulting skills measured/assessed by a non‐validated tool
Hamilton 2014	The phone is just the means by which the intervention (a role play) is delivered but the skills tested are not exactly telephone skills (and there is no evidence the tool used to test them is previously validated and the authors say it was initially developed for medical students, a different population)
Kaakinen 2016	Not complying with inclusion criteria (it is a before‐after study without control)
King 2007	Not complying with inclusion criteria (it is a before after study without control)
Kosower 1991	Not an experimental design (it is the description of an educational programme)
Lamb 2004	Not an experimental design (it is a survey about experiences and needs with telephone precepting)
Lane 1998	Not an experimental design (it is the description of a peer counselling training programme)
Larsen 1997	Not an experimental design (it is a survey about a training experiences on telephone consultation)
Letizia 2012	Not complying with inclusion criteria (the intervention is aimed to train to advice patients by email)
MacFarlane 2006	Not an experimental design (semi‐structured interviews and focus groups were used to collect data to explore participants' views on the educational aspect of joint teleconsultations at the primary–secondary care interface)
NCT00799461	the intervention is aimed to help long‐term survivors of haematopoietic stem cell transplant cope with late complications
Nestel 2007	Not complying with inclusion criteria (it is a before‐after study without control)
No author 1997	Not an experimental design (it is a newsletter)
Orlov 2003	Not an experimental design (it is the description of telemedicine service)
Ottolini 1998	Clinicians' telephone consulting skills measured/assessed by a non‐validated tool
Record 2015	Not an experimental design (patients' survey about postdischarge experience skimmed on the base of discharging doctor curriculum)
Reisman 2005	Not an experimental design (it is an article with advice for handling telephone calls)
Rothwell 2011	Not an experimental design (it is qualitative content analysis of a group discussion)
Rowe 2014	Not complying with inclusion criteria (nurse experience with telephone triage)
Rudd 2012	Not complying with inclusion criteria (it is a retrospective case series about the teleconsultations between clinicians in order to decide to start or not intravenous thrombolysis for acute ischemic stroke in an emergency department
Santos 2013	Not complying with inclusion criteria (it is a before‐after study without control)
Strasser 1979	Not complying with inclusion criteria (it is a trial comparing telephone management with face‐to‐face management of paediatric complaints)
Whitson 2008	Not complying with inclusion criteria (it is a longitudinal quality improvement study where nurses completed several satisfaction surveys about a programme of individualised training sessions and decision support tools)

^aThe study reporting is not completely clear in terms of PICO framework so we wrote to the authors (krbell@u.washington.edu and Kathleen.Bell@UTSouthwestern.edu) before excluding it

Differences between protocol and review

We did not search the ERIC, ProQuest Dissertations & Theses, and Web of Science databases and these other databases as stated in our protocol: TrialsCentralTM (www.trialscentral.org)
 Current Controlled Trials (www.controlled‐trials.com)
 Australasian Digital Theses Program (adt.caul.edu.au)
 EThOS, Electronic Thesis Online Service, British Library (ethos.bl.uk)
 Networked Digital Library of Theses and Dissertations (www.ndltd.org)
 Index to Theses (www.theses.com) (Great Britain and Ireland)

We integrated the definition of 'validated tool' using the definition proposed by the Joint Commission: "an instrument that has been psychometrically tested for reliability (the ability of the instrument to produce consistent results), validity (the ability of the instrument to produce true results), sensitivity (the probability of correctly identifying a patient with the condition" (manual.jointcommission.org/Manual/Questions/UserQuestionId03Sub0015).

Kazeem A did not contribute to the review.

We slightly modified the section Types of interventions in the Methods in order to distinguish between the intervention itself and the mean/way it was delivered.

In the protocol text we use the concept of 'diagnostic accuracy' referred to an outcome of telephone consultation: thanks to the reviewers' comments we prefer to change this concept 'urgency assessment accuracy'.

Contributions of authors

All listed authors have contributed to this review.

Sources of support

Internal sources

• New Source of support, Other.

• eHealth Unit, Imperial College, UK.

  Support was gained from previous Cochrane Reviews either submitted or in the process of being completed

External sources

• No sources of support supplied

Declarations of interest

Alberto Vaona, Yannis Pappas, Rumant S Grewal, Mubasshir Ajaz, Azeem Majeed, and Josip Car declare that they do not have any affiliation or involvement with any organisation or entity with an interest in the review findings.

Edited (no change to conclusions)