Skip to main content
PMC home page
BMC Health Services Research logoLink to BMC Health Services Research
. 2006 Feb 15;6:10. doi: 10.1186/1472-6963-6-10

Using Ontario's "Telehealth" health telephone helpline as an early-warning system: a study protocol

Elizabeth Rolland 1,2,✉, Kieran M Moore 1,3, Victoria A Robinson 1,4, Don McGuinness 1
PMCID: PMC1431529  PMID: 16480500

Abstract

Background

The science of syndromic surveillance is still very much in its infancy. While a number of syndromic surveillance systems are being evaluated in the US, very few have had success thus far in predicting an infectious disease event. Furthermore, to date, the majority of syndromic surveillance systems have been based primarily in emergency department settings, with varying levels of enhancement from other data sources. While research has been done on the value of telephone helplines on health care use and patient satisfaction, very few projects have looked at using a telephone helpline as a source of data for syndromic surveillance, and none have been attempted in Canada. The notable exception to this statement has been in the UK where research using the national NHS Direct system as a syndromic surveillance tool has been conducted.

Methods/design

The purpose of our proposed study is to evaluate the effectiveness of Ontario's telephone nursing helpline system as a real-time syndromic surveillance system, and how its implementation, if successful, would have an impact on outbreak event detection in Ontario. Using data collected retrospectively, all "reasons for call" and assigned algorithms will be linked to a syndrome category. Using different analytic methods, normal thresholds for the different syndromes will be ascertained. This will allow for the evaluation of the system's sensitivity, specificity and positive predictive value. The next step will include the prospective monitoring of syndromic activity, both temporally and spatially.

Discussion

As this is a study protocol, there are currently no results to report. However, this study has been granted ethical approval, and is now being implemented. It is our hope that this syndromic surveillance system will display high sensitivity and specificity in detecting true outbreaks within Ontario, before they are detected by conventional surveillance systems. Future results will be published in peer-reviewed journals so as to contribute to the growing body of evidence on syndromic surveillance, while also providing an non US-centric perspective.

According to the Oxford Handbook of Public Health Practice[1], two of the principal objectives of an effective surveillance system are to "give early warning changes of incidence," and "detect outbreaks early." Unfortunately, the reality of public health practice is that monitoring agencies such as public health units routinely fall short of these objectives, whether it is due to lags/cuts in the passive surveillance communication between physicians and public health agencies [2-4], or because of delays in laboratory confirmation[5,6]. This is even more of an issue in large and remote areas such as Northern Ontario.

A new era of surveillance research is attempting to address these issues while taking advantage of available data. Electronic data captured at the point of care provides an efficient means for the conversion of clinical data to surveillance information. This has resulted in a new area of research called syndromic surveillance.

Syndromic surveillance is a newly emerging field in the science of epidemiological surveillance. Its growth has been encouraged in large part in the United States as a response to potential bioterrorism threats. In effect, it is a complementary surveillance system, in that it can provide prediagnostic data to rapidly detect infectious disease outbreaks before they are detected through conventional surveillance methods.

The Walker Report[7] highlighted Canadian post-SARS interest in syndromic surveillance as a method of increasing Ontario's capacity to manage communicable diseases adequately. An important feature of syndromic surveillance systems is that they rely on existing data streams. In other words, it does not require the development of new datasets (and the challenges surrounding this), but rather makes use of available data and increases communications within the public health system. Data streams that are being investigated as part of the larger syndromic surveillance picture include:

- Over The Counter (OTC) drug sales [8-12],

- emergency department visits (coded either by ICD code or by chief complaint) [13,14],

- emergency (911) calls[15],

- ambulance dispatch[10],

- patient transfers[16],

- school/work absenteeism records[10],

- telephone medical helpline calls (e.g. NHS Direct in the UK, Ontario Telehealth) [10,17-22],

- insurance/HMO claim data[23].

The science of syndromic surveillance is still very much in its infancy. While a number of syndromic surveillance systems are being evaluated in the US, very few have had success thus far in predicting an infectious disease event.

However, syndromic surveillance can and has been able to assist in "determin[ing] the size, spread and tempo of an outbreak after it has been detected," [24]. It can also theoretically be useful in times of calm, "providing reassurance that a large-scale outbreak is not occurring," [23].

Currently, most systems are primarily based in an emergency department/room setting, where they rely on either chief complaint information recorded at onset of contact with a syndromic surveillance source (e.g. triage nurse), or ICD9/10 codes, following diagnosis by the physician. Although studies have shown that using ICD codes results in better sensitivity, positive predictive value and specificity[25,26], there is often a lag between the contact with the case, and the coding of his/her information into an ICD case. In some settings, this can take up to a week[23,27]. Consequently, the majority of emergency department syndromic surveillance systems rely primarily on chief complaint for the classification of visits into syndromes (symptom categories).

To date, the majority of syndromic surveillance systems have been based primarily in emergency department settings, with varying levels of enhancement from other data sources, such as the ones listed earlier. While research has been done on the value of telephone helplines on health care use and patient satisfaction[28], very few projects have looked at using a telephone helpline as a source of data for syndromic surveillance, and none have been attempted in Canada. The notable exception to this statement has been in the UK where research using the national NHS Direct system as a syndromic surveillance tool has been conducted.

The NHS Direct national telephone helpline was established in the UK in 1998 to provide both health information as well as medical referral to callers. The country has 21 regional calling sites in England and Wales[29]. In order to address calls pertaining to medical referrals, a clinical decision support software package (NHS CAS) with over 200 clinical algorithms is used. Each of these 200+ algorithms has a set of questions attached to them; their purpose is to ascertain the caller's symptoms and provide them with the most appropriate advice and course of action[28].

Since 1999, researchers in the UK have been retrospectively and prospectively investigating the use of NHS Direct as a syndromic surveillance method for a number of syndromes[18,19,28,30], including influenza[21,30,31] and gastrointestinal illness[22]. To date, their results have been optimistic about the long-term use of NHS Direct as an "early warning system," but acknowledge that work continues to be required to fine-tune the system.

Due to its central database structure, Ontario's Telehealth system should theoretically be better suited as a source for a syndromic surveillance system than its UK counterpart. While the UK system relies on regional calling centers with local databases to deal with call volumes from their proprietary calling region, the Ontario Telehealth System has four call centres that manage the call loads for the province as a whole, but decision rules are identical for all four centers, and all data are stored within one central database. This ensures that triage algorithms are the same for all regions, which is not the case in the UK.

Background

The Ontario Teleheath System (henceforth referred to as "Telehealth") is a tollfree telephone helpline provided by the Ontario Ministry of Health and Long-term Care, and is available to all residents of Ontario. The Telehealth program was initiated as a pilot in 2001 in the Greater Toronto (416 and 905 calling areas) and Northern areas (705 calling area). The program became province-wide at the end of 2001. The Telehealth services are provided by a private contractor (currently "Clinidata") hired by the Ministry of Health and Long-Term Care.

Telehealth operates 24 hours a day, 7 days a week and all calls are answered by registered nurses who have at least three years of recent clinical experience[32]. Calls are answered in both official languages, but translators are available via three-way call within 60 seconds for 110 languages[32]. The most commonly requested languages other than English and French are Mandarin, Cantonese, Farsi, Italian and Portuguese[32,33]. An average of 3,100 calls are recorded daily[33], with the highest call volumes occurring during the evenings and holidays. Seasonally, the busiest months are the winter months (January-March), which coincide with cold and influenza seasons[34]. The average call length is approximately 10 minutes[32], and calls are classified in one of five categories[35]:

1) priority (call 911 immediately);

2) emergent (see physician within hours – routinely directed to a hospital emergency department);

3) urgent (contact family physician within 24 hours);

4) referral 72 (contact family physician with 72 hours);

5) self-care.

According to information on the Ontario Ministry of Health and Long-Term Care website, the volume of calls attributed to these different categories is as follows: "about 43 per cent of callers received self-care advice; 35 per cent if callers were advised to visit their physician; 14 per cent were referred to a hospital emergency department; two per cent were considered urgent calls and were connected to 911," [33].

To date, 40% of callers have been parents, primarily mothers, calling about symptoms being displayed by their children. Overall, the top five reported symptoms have been: nausea/vomiting, abdominal pain, fever in children aged 3 months-3 years, cough/cold, and rash[32].

Currently, Ontario has a broad syndromic surveillance agenda, which includes pilot studies of over-the-counter drug sale monitoring[10], and emergency department visits[17]. Telehealth is expected to be a part of this overarching surveillance strategy, supplementing current syndromic surveillance systems, as well as, potentially identifying call volume and type aberrations before they also occur in hospital emergency departments.

Research Methods/design

Using data collected by Clinidata as part of the Ontario Telehealth program since December 2001, the effectiveness of Telehealth as a real-time surveillance system will be investigated. As no research has been done in this area in Canada, and international research to date is very limited, the development of this translational tool is innovative and will add significantly to the limited state of knowledge on syndromic surveillance.

This project will have both a retrospective and a prospective component. In the first stage, our research will be retrospective in nature. Using data collected retrospectively by Clinidata, we will link all "reasons for call" ("chief complaint") and assigned algorithms to one of the following syndromes: gastrointestinal, constitutional, respiratory, rash, hemorrhagic, botulinic, neurological, and other (Table 1). These syndromes are identical to those currently used by the Ontario Syndromic Surveillance Pilot Project (also known as QUESST). This will be done in conjunction with QUESST, using their recent expertise in the mapping of health algorithms and chief complaint to these syndromes.

Table 1.

Syndrome Categories

Syndrome Definition
Gastrointestinal pain or cramps anywhere in the abdomen nausea, vomiting, diarrhea, and abdominal distension and swelling.
Constitutional non-localized, systemic problems including fever, chills, body aches, flu symptoms (viral syndrome), weakness, fatigue, anorexia, malaise, lethargy, sweating (diaphoresis), light headedness, faintness and fussiness.
Respiratory problems of the nose (coryza) and throat (pharyngitis), as well as the lungs. Examples of respiratory include congestion, sore throat, tonsillitis, sinusitis, cold symptoms, bronchitis, cough, shortness of breath, asthma, chronic obstructive pulmonary disease (COPD), and pneumonia. The presence of both cold and flu symptoms is counted in this category, not Constitutional.
Rash any rash, such as macular, papular, vesicular, petechial, purpuric, or hives. Ulcerations are not counted as Rash unless consistent with cutaneous anthrax (an ulcer with a black eschar).
Hemorrhagic bleeding from any site, e.g., vomiting blood (hematemesis), nose bleed (epistaxis), hematuria, gastrointestinal bleeding (site unspecified), rectal bleeding, and vaginal bleeding. Bleeding from a site for which there is a syndrome is counted as Hemorrhagic and as the relevant syndrome (e.g. hematochesia is Gastrointestinal and Hemorrhagic; hemoptysis is Respiratory and Hemorrhagic).
Botulinic ocular abnormalities (diplopia, blurred vision, photophobia), difficulty speaking (dysphonia, dysarthria, slurred speech), and difficulty swallowing (dysphagia).
Neurological non-psychiatric complaints that relate to brain function. Included are headache, head pain, migraine, facial pain or numbness, seizure, tremor, convulsion, loss of consciousness, syncope, fainting, ataxia, confusion, disorientation, altered mental status, vertigo, concussion, meningitis, stiff neck, tingling and numbness. (Dizziness is both Constitutional and Neurological).
Other anything which does not fall into any of the above categories, particularly injuries.
Open in a new tab

Following this step, analyses will be carried out to determine normal thresholds for the different syndromes. There are many different methods documented in the literature, including, but not limited to, cumulative sums[36,37], control charts[38], recursive least squares[39,40], and upper confidence limits, the latter which are the preferred method for NHS Direct research[18]. Using knowledge of past outbreak activity such as SARS and Norwalk-type outbreaks, the validity and reliability of the data, as well as, the sensitivity, specificity and positive predictive values of these different methods will be evaluated in order to ascertain which statistical method best applies to these data.

Once thresholds have been established and evaluated, the prospective monitoring of syndromic activity will take place. This will be done both temporally and spatially. The decision to pursue both types of analyses is due to Ontario's large geographic size, combined with the uneven distribution of population. Temporal trends, usually measured through one-dimensional scan statistics[36,41,42] are of interest to monitor general patterns of symptoms. This is of particular interest for unusual events where a few cases would result in a likely alert (e.g. anthrax). Spatial clustering analysis, using spatial scan statistics, have been adapted for infectious disease surveillance[36,43,44]. This statistical method will be used to ascertain unusual activity in specific geographic areas. For example, this will allow for the monitoring of underserviced areas such as northern Ontario where infectious disease surveillance is limited and where, coincidentally, use of Telehealth is proportionally high. This statistical method relies on a geographic region's expected counts rather than on its population distribution.

It is critical that new and emerging surveillance systems implement process evaluation targets within its logic model to monitor its effectiveness and progress.

In order to meet this need, we plan on implementing the Centers for Disease Control and Prevention "Framework for Evaluating Public Health Surveillance Systems for Early Detection of Outbreaks," [45]. As part of the routine evaluation of this surveillance system, it is our intention to compare syndromic surveillance activity monitored by the Telehealth Project (both retrospectively and prospectively) with emergency department syndromic surveillance activity. At present, the Ontario Emergency Department Syndromic Surveillance System only monitors hospitals located in Kingston, Ontario.

Consequently, this portion of our evaluation is only planned for Telehealth calls made in the Kingston, Frontenac, Lennox and Addington catchment area. As other hospitals are added to the ED Syndromic Surveillance Project, they will be included in the comparative evaluation of Ontario Telehealth and Emergency Department Syndromic Surveillance.

The last step of the evaluation of Telehealth as an early warning system will be an economic analysis that will attempt to ascertain whether Telehealth provides medical and non-medical cost savings, both in terms of monitoring, and public health surge capacity planning.

Discussion

The use of Telehealth as an early-warning syndromic surveillance system will promote communication between the acute care sector and the public health sector of the health care system, by relying on the acute care system as a frontline source of information for the more effective planning and management of public health resources.

This system has the potential of identifying outbreaks (such as pandemic influenza and gastrointestinal illness) and other adverse health events (such as bioterrorist attacks) before the acute care and public health sectors would be made aware of them through established avenues. Furthermore, this would improve the commonly accepted lag between identification of an event and reporting of the event to the proper authorities.

This system, because of its automated and real-time nature would immediately identify aberrations in call types and would alert the proper authorities (including acute care facilities and public health) immediately.

Secondly, with its province-wide focus, it will potentially remedy some of the systemic failures identified by Justice Archie Campbell in his assessment of the Ontario SARS outbreak[46]. Particularly, this province-wide system would address issues surrounding a lack of public health surge capacity. As an early warning system, Telehealth has the potential to identify aberrations long before traditional reporting systems. In the case of the Walkerton, Ontario outbreak[47], if Telehealth had been up and operational as an early warning system, we can speculate that the number of calls for gastrointestinal complaints would have gone up, which would have created an alert for public health to investigate these calls before an increase in cases would have been identified through routine reporting systems (especially RDIS). By acting early, the levels of morbidity and mortality witnessed in Walkerton could have been curtailed by a more timely involvement of Public Health. This would have meant that control measures such as "boil water" orders) could have implemented more quickly, which would have decreased the number of cases and would have allowed public health to work more effectively and within its available resources.

This system would also improve communication and coordination between the different levels of government (especially regional and provincial) by centralizing and streamlining the information flow, and potentially identifying important health events before they are identified by routine surveillance activities. This would also allow for the implementation of emergency plans (e.g. antiviral distribution in the event of pandemic influenza) before the needs placed on public health and the acute care centers exceeds available resources, thereby enhancing the effectiveness of medical practice. This has both cost- and life-saving implications, as the Ontario SARS experience has shown us.

This surveillance system is also unique and innovative in that it does not rely on reporting of cases by physicians. In other words, it allows for the surveillance of underserviced areas such as northern Ontario, as the system relies on individuals calling a toll-free number rather than trying to get access to a physician who then has a reporting responsibility. In effect, for the purpose of surveillance, it cuts out the need for a physician as a "reporting" intermediary. This will potentially allow for a more complete picture of underserviced areas, as well as, a much timelier one as it is available 24 hours a day, seven days a week, 365 days a year. The continuous availability of this service provides a front line that each and every Ontarian can access prior to or instead of seeking help from a family physician or an emergency department. When a call is placed to Telehealth, all callers are given advice based on the type and severity of symptoms they are reporting. Advice can range from staying at home to going to an emergency department immediately. Because of its province-wide coverage, this means that data for areas with limited access to emergency departments and family physicians will be collected, thereby providing new information on an underserviced subset of the population.

Furthermore, areas such as aboriginal communities have health clinics, often staffed with nurse practitioners. However, these clinics are not integrated with routine surveillance systems. Use of Telehealth for surveillance would also allow for isolated communities with few health resources to be included into a more formal surveillance system.

Finally, this project is important and of value because it has the potential to show the different levels of government that province-wide surveillance that relies on the integration of available data sources into the surveillance information flow is feasible, cost-effective, and has the potential to positively impact morbidity and mortality through better planning and the subsequent enhanced effectiveness of public health practice.

List of abbreviations

CAS Clinical Assessment System

ED Emergency Department

HMO Health Management Organization

ICD International Classification of Diseases

NHS National Health Service (United Kingdom)

OTC Over the counter

QUESST Queen's University Emergency Syndromic Surveillance Team

RDIS Reportable Disease Information System (Ontario, Canada)

SARS Severe Acute Respiratory Syndrome

Competing interests

The author(s) declare that they have no competing interests.

Authors' contributions

ER participated in the design of the study and drafted the manuscript.

KMM conceived of the study, and participated in its design and helped to draft the manuscript.

VR and DM helped to draft the manuscript.

All authors read and approved the manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6963/6/10/prepub

Acknowledgments

Acknowledgements

Dr. Elizabeth VanDenKerkhof, Assistant Professor, Department of Anesthesiology, Queen's University, Kingston, ON, Canada

Dr. Punam Mangtani, Senior Lecturer, Infectious Disease Epidemiology Unit, London School of Hygiene and Tropical Medicine, London, UK

Contributor Information

Elizabeth Rolland, Email: erolland@gmail.com.

Kieran M Moore, Email: moorek1@kgh.kari.net.

Victoria A Robinson, Email: victoria.robinson@utoronto.ca.

Don McGuinness, Email: dmcguinness@healthunit.on.ca.

References

  1. Hadden F, O'Brien S. Assessing acute health trends: surveillance. In: Pencheon D, Guest C, Melzer D, Muir Gray JA, editor. Oxford Handbook of Public Health Practice. New York: Oxford UP; 2001. pp. 14–9. [Google Scholar]
  2. Barthell EN, Cordell WH, Moorhead JC, Handler J, Feied C, Smith MS, Cochrane DG, Felton CW, Collins CA. The Frontlines of Medicine Project: A Proposal for the Standardized Communication of Emergency Department Data for Public Health Uses Including Syndromic Surveillance for Biological and Chemical Terrorism. Ann Emerg Med. 2002;39:422–29. doi: 10.1067/mem.2002.123127. [DOI] [PubMed] [Google Scholar]
  3. Espino JU, Wagner MM. Accuracy of ICD-9-coded Chief Complaints and Diagnoses for the Detection of Acute Respiratory Illness [abstract] Proc AMIA. 2001. pp. 164–8. [PMC free article] [PubMed]
  4. Lazarus R, Kleinman KP, Dashevsky I, DeMaria A, Platt R. Using automated medical records for rapid identification of illness syndromes (syndromic surveillance): the example of lower respiratory illness. BMC Public Health. 2001;1:9. doi: 10.1186/1471-2458-1-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Ashford DA, Kaiser RM, Bales ME, Shutt KK, Patrawalla A, McShan A, Tappero JW, Perkins BA, Dannenberg AL. Planning against biological terrorism: lessons from outbreak investigations. Emerg Infect Dis. 2003;9:515–9. doi: 10.3201/eid0905.020388. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Townes JM, Kohn MA, Southwick KL, Bangs CA, Zechnich AD, Magnuson JA, Jui J. Investigation of an Electronic Emergency Department Information System as a Data Source for Respiratory Syndrome Surveillance. J Public Health Mgmt Practice. 2004;10:299–307. doi: 10.1097/00124784-200407000-00006. [DOI] [PubMed] [Google Scholar]
  7. Walker D. For the Public's Health: Initial Report of the Ontario Expert Panel on SARS and Infectious Disease Control. Toronto: Ministry of Health and Long-Term Care; 2003. [Google Scholar]
  8. Goldenberg A, Shmueli G, Caruana RA, Fienberg SE. Early statistical detection of anthrax outbreaks by tracking over-the-counter medication sales. Proc Natl Acad Sci USA. 2002;99:237–40. doi: 10.1073/pnas.042117499. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Magruder SF, Lewis SH, Najmi A, Florio E. Progress in understanding and using over-thecounter pharmaceuticals for syndromic surveillance. MMWR Morb Mortal Wkly Rep. 2004. pp. 117–22. [PubMed]
  10. Edge VL, Pollari F, Lim G, Aramini K, Sockett P, Martin SW, Wilson J, Ellis A. Syndromic surveillance of gastrointestinal illness using pharmacy over-the-counter sales. A retrospective study of waterborne outbreaks in Saskatchewan and Ontario. Can J Pub Health. 2004;95:446–50. doi: 10.1007/BF03403991. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Wagner MM, Robinson JM, Tsui FC, Espino JU, Hogan WR. Design of a national retail data monitor for public health surveillance. J Am Med Inform Assoc. 2003;19:409–18. doi: 10.1197/jamia.M1357. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Lombardo J, Burkom H, Elbert E, Magruder S, Lewis SH, Pavlin J. A systems overview of the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE II) J Urban Health. 2003;80:i32–2. doi: 10.1007/PL00022313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Irvin CB, Nouhan PP, Rice K. Syndromic analysis of computerized emergency department patients' chief complaints: an opportunity for bioterrorism and influenza surveillance. Ann Emerg Med. 2003;41:447–52. doi: 10.1067/mem.2003.104. [DOI] [PubMed] [Google Scholar]
  14. Lober WB, Trigg LJ, Karras BT, Bliss D, Ciliberti J, Duchin JS. Syndromic surveillance using automated collection of computerized discharge diagnosis. J Urb Health. 2003;80:i97–106. doi: 10.1007/PL00022320. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Pavlin JA, Mostashari F, Kortepeter MG, Hynes NA, Chotani RA, Mikol YB, Ryan MA, Neville JS, Gantz DT, Writer JV, Florance JE, Culpepper RC, Henretig FM, Kelley PW. Innovative surveillance methods for rapid detection of disease outbreaks and bioterrorism: results of an interagency workshop on health indicator surveillance. Am J Public Health. 2004;93:1230–5. doi: 10.2105/ajph.93.8.1230. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. MacDonald RD, Farr B, Neill M, Loch J, Sawadsky B, Mazza C, Daya K, Olynyk C, Chad S. An emergency medical services transfer authorization center in response to the Toronto severe acute respiratory syndrome outbreak. Prehosp Emerg Care. 2004;8:223–31. doi: 10.1016/j.prehos.2003.12.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Moore K. Real-time syndrome surveillance in Ontario, Canada: the potential use of emergency departments and Telehealth. Eur J Emerg Med. 2004;11:3–11. doi: 10.1097/00063110-200402000-00002. [DOI] [PubMed] [Google Scholar]
  18. Cooper DL, Smith G, Baker M, Chinemana F, Verlander N, Gerard E, Hollyoak V, Griffiths R. National symptom surveillance using calls to a telephone health advice service – United Kingdom, December 2001-February 2003. MMWR Morb Mortal Wkly Rep. 2004. pp. 179–83. [PubMed]
  19. Baker M, Smith GE, Cooper D, Verlander NQ, Chinemana F, Cotterill S, Hollyoak V, Griffiths R. Early warning and NHS Direct: a role in community surveillance? J Public Health Med. 2003;25:362–8. doi: 10.1093/pubmed/fdg096. [DOI] [PubMed] [Google Scholar]
  20. Rodman J, Frost F, Jabukowski W. Using nurse hotline calls for disease surveillance. Emerg Infect Dis. 1998;4 doi: 10.3201/eid0402.980226. Apr-Jun. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Cooper DL, Smith GE, Hollyoak VA, Joseph CA, Johnson L, Chaloner R. Use of NHS Direct calls for surveillance of influenza – a second year's experience. Commun Dis Public Health. 2002;5:127–31. [PubMed] [Google Scholar]
  22. Cooper DL, Smith GE, O'Brien SJ, Hollyoak VA, Baker M. What can analysis of calls to NHS Direct tell us about the epidemiology of gastrointestinal infections to the community? J Infect. 2003;46:101–5. doi: 10.1053/jinf.2002.1090. [DOI] [PubMed] [Google Scholar]
  23. Miller B, Kassenborg H, Dunsmuir W, Griffith J, Hadidi M, Nordin JD, Danila R. Syndromic surveillance for influenza-like illness in an ambulatory care network. Emerg Infect Dis. 2004;10:1806–11. doi: 10.3201/eid1010.030789. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Anonymous Syndromic surveillance. Healthcare Hazard Management Monitor. 2003;16:1–6. [PubMed] [Google Scholar]
  25. Begier EM, Sockwell D, Branch LM, Davies-Cole JO, Jones LH, Edwards L, Casani JA, Blythe D. The National Capitol Region's Emergency Department syndromic surveillance system: do chief complaint an discharge diagnosis yield different results? Emerg Infect Dis. 2003;9:393–6. doi: 10.3201/eid0903.020363. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Fleishauer AT, Silk BJ, Schumacher M, Komatsu K, Santana S, Vaz V, Wolfe M, Hutwagner L, Cono K, Berkelman R, Treadwell T. The validity of chief complaint and discharge diagnosis in emergency department-based syndromic surveillance. Acad Emerg Med. 2004;11:1262–7. doi: 10.1197/j.aem.2004.07.013. [DOI] [PubMed] [Google Scholar]
  27. Bio-terrorism early warning syndromic surveillance http://www.health-infosys-dir.com/wphcemg1.htm
  28. Bunn F, Byrne G, Kendall S. Telephone consultation and triage: effects on health care use and patient satisfaction. Cochrane Library. 2005;2 doi: 10.1002/14651858.CD004180.pub2. [DOI] [PubMed] [Google Scholar]
  29. Cooper D, Chinemana F. NHS Direct derived data: an exciting new opportunity or an epidemiological headache? J Pub Health. 2004;26:158–60. doi: 10.1093/pubmed/fdh133. [DOI] [PubMed] [Google Scholar]
  30. Harcourt SE, Smith GE, Hollyoak V, Joseph CA, Chaloner R, Rehman Y, Warburton F, Ejidokun OO, Watson JM, Griffiths RK. Can calls to NHS Direct be used for syndromic surveillance? Commun Dis Public Health. 2001;4:178–82. [PubMed] [Google Scholar]
  31. Crofts JP, Joseph CA, Zambon M, Ellis J, Fleming DM, Watson JM. Influenza surveillance in the United Kingdom: October 2002 to May 2003. Commun Dis Rep CDR Wkly. 2004;14:1–9. [Google Scholar]
  32. Career Connection http://torontosun.jobboom.com/News/2005/04/13/1225887-sun.html
  33. Ontario Ministry of Health and Long-Term Care Public Information: Telehealth http://www.health.gov.on.ca/english/public/updates/archives/hu_03/docnurse/telehealth_f
  34. Telehealth Nursing: Lois Scott is one of the minds behind this innovative and successful form of nursing http://www.medhunters.com/articles/telehealthNursing.html
  35. Lightstone S. Health-care-by-phone services spreading across country. JAMC. 2002;166:80. [PMC free article] [PubMed] [Google Scholar]
  36. Heffernan R, Mostashari F, Das D, Karpati A, Kulldorff M, Weiss D. Syndromic Surveillance in Public Health Practice, New York City. Emerg Infect Dis. 2004;10:858–64. doi: 10.3201/eid1005.030646. [DOI] [PubMed] [Google Scholar]
  37. McKenna VB, Gunn JE, Auerbach J, Brinsfield KH, Dyer KS, Barry MA. Local Collaborations: Development and Implementation of Boston's Bioterrorism Surveillance System. J Pub Health Mgmt Practice. 2003;9:384–93. doi: 10.1097/00124784-200309000-00009. [DOI] [PubMed] [Google Scholar]
  38. Hanslik T, Boelle PY, Flahault A. The control chart: an epidemiological tool for public health monitoring. Public Health. 2001;115:277–81. doi: 10.1038/sj/ph/1900782. [DOI] [PubMed] [Google Scholar]
  39. Tsui FC, Espino JU, Dato VM, et al. Technical description of RODS: a real-time public health surveillance system. J Am Med Inform Assoc. 2003;10:399–408. doi: 10.1197/jamia.M1345. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Gesteland PH, Gardner RM, Tsui FC, Espino JU, Rolfs RT, James BC, Chapman WW, Moore AW, Wagner MW. Automated Syndromic Surveillance for the 2002 Winter Olympics. J Am Med Inform Assoc. 2003;10:547–54. doi: 10.1197/jamia.M1352. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Wallenstein S. A test for detection of clustering over time. Am J Epidemiol. 1980;111:367–72. doi: 10.1093/oxfordjournals.aje.a112908. [DOI] [PubMed] [Google Scholar]
  42. Weinstock MA. A generalized scan statistic test for the detection of clusters. Int J Epidemiol. 1981;10:289–93. doi: 10.1093/ije/10.3.289. [DOI] [PubMed] [Google Scholar]
  43. Kulldorff M, Rand K, Gherman G, Williams W, DeFrancesco D. SaTScan v21: software for the spatial and space-time scan statistics. Bethesda, MD: National Cancer Institute; 1998. [Google Scholar]
  44. Mostashari F, Kulldorff M, Hartman JJ, Miller JR, Kilasekera V. Dead bird clusters as an early warning system for West Nile virus activity. Emerg Infect Dis. 2003;9:641–6. doi: 10.3201/eid0906.020794. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Centers for Disease Control and Prevention Framework for Evaluating Public Health Surveillance Systems for Early Detection of Outbreaks: Recommendations from the CDC Working Group. MMWR. 2004;53 [inclusive page numbers] [PubMed] [Google Scholar]
  46. Campbell A. The SARS Commission Interim Report: SARS and Public Health in Ontario. Toronto: Government of Ontario; 2004. [DOI] [PubMed] [Google Scholar]
  47. Clark CG, Price L, Ahmed R, Woodward DL, Melito Pl, Rodgers FG, Jamieson F, Cieben B, Li A, Ellis A. Characterization of waterborne outbreak-associated Campylobacter jejeuni, Walkerton, Ontario. Emerg Infect Dis. 2003;9:1232–41. doi: 10.3201/eid0910.020584. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from BMC Health Services Research are provided here courtesy of BMC

RESOURCES