Abstract
Background
Public Health England (PHE) coordinates a suite of real-time national syndromic surveillance systems monitoring general practice, emergency department and remote health advice data. We describe the development and informal evaluation of a new syndromic surveillance system using NHS 111 remote health advice data.
Methods
NHS 111 syndromic indicators were monitored daily at national and local level. Statistical models were applied to daily data to identify significant exceedances; statistical baselines were developed for each syndrome and area using a multi-level hierarchical mixed effects model.
Results
Between November 2013 and October 2014, there were on average 19 095 NHS 111 calls each weekday and 43 084 each weekend day in the PHE dataset. There was a predominance of females using the service (57%); highest percentage of calls received was in the age group 1–4 years (14%). This system was used to monitor respiratory and gastrointestinal infections over the winter of 2013–14, the potential public health impact of severe flooding across parts of southern England and poor air quality episodes across England in April 2014.
Conclusions
This new system complements and supplements the existing PHE syndromic surveillance systems and is now integrated into the routine daily processes that form this national syndromic surveillance service.
Keywords: epidemiology, primary care, public health
Introduction
Syndromic surveillance is the real-time (or near real-time) collection, analysis, interpretation and dissemination of health-related data.1 The Public Health England (PHE) Real-time Syndromic Surveillance Team (ReSST) coordinates a suite of national syndromic surveillance systems monitoring consultation data from general practitioners (GPs), attendances at sentinel emergency departments and calls to a telephone triage, healthcare advice and information line for the early identification of the impact (or absence of impact) of potential public health threats that require effective public health action.2
There are several examples of syndromic surveillance systems based on telephone triage, healthcare advice and information lines;3–6 however, the use of these systems is not as commonplace for syndromic surveillance as other healthcare sources, e.g. emergency departments. Initially developed in 2001, the National Health Service (NHS) Direct syndromic surveillance system4,7 formed a cornerstone of syndromic surveillance in England, supporting daily health protection surveillance and forming an integral part of the national surveillance of a wide number of local and national outbreaks, incidents and events, including the 2009 influenza pandemic;8 widespread flooding in 2007; the Eyjafjallajökull volcanic eruption in Iceland in 20109 and the London 2012 Olympic and Paralympic Games.10
During 2012–13, NHS Direct was gradually decommissioned and replaced with a new national telephone triage, healthcare advice and information service, NHS 111.11 The new NHS 111 service replaced NHS Direct but was also designed to provide a gateway to access out-of-hours GP and primary care services to the entire English population. Using a three-digit telephone number (111) accessible across the whole of England, 24 hours a day, 365 days a year, patients can access help with urgent, but non-life-threatening health problems.12 Both NHS Direct and NHS 111 used/use symptom-based Clinical Decision Support Software (CDSS) to assess the urgency of need and the most appropriate course of action for the patient and had/have the capability to deal with emergency situations and calls that are assessed as requiring ambulance response.
From the announcement of the implementation of the new NHS 111 service, PHE recognized a requirement to maintain the flow of telephone triage data for remote health advice syndromic surveillance. This paper describes the development and informal evaluation of a new remote health advice-based syndromic surveillance system based upon aggregated call data from the NHS 111 service and illustrates some key outputs generated during the first year of its operation.
Methods
Data extraction, transfer and structure of syndromic calls
PHE received a fully anonymized extract of NHS 111 call data from the NHS 111 Repeat Caller Service (RCS) database on a daily basis. These syndromic call data represented users of the NHS 111 service, who were able to identify and prioritize a main symptom or problem during the triage process, were assessed by the call handler or the NHS 111 clinician and entered into the system. Daily data were transferred to PHE via Secure File Transfer Protocol (SFTP), providing a secure and robust transfer method. The anonymized call data were stored within a secure SQL database at PHE for daily analysis as part of the routine daily surveillance process that included monitoring of epidemiological trends and the identification of significant statistical exceedances.13
NHS 111 syndromic surveillance indicators
Syndromic indicators are groupings of symptoms/chief complaints that enable monitoring of key infections or environmental issues for health protection purposes. All NHS 111 call handlers use the NHS Pathways clinical decision support system14 to assess callers during the triage process and identify the main symptom or chief complaint for each call. The information on the NHS Pathway selected during the triage process for each call was used to create the syndromic indicators for the NHS 111 syndromic surveillance system. These syndromic indicators were selected to be as similar as possible to those previously validated for the NHS Direct syndromic surveillance system and were monitored over the winter of 2013–14. The NHS 111 syndromic indicators included cold/flu, fever, cough, difficulty breathing, diarrhoea, vomiting and eye problems. A further indicator, heat/sunstroke, was developed for use during summer 2014 as part of the Heatwave Plan for England.15 Syndromic call data for each indicator were plotted as a percentage of total syndromic calls to compensate for any changes in the denominator as the NHS 111 service was rolled out nationally and to adjust for fluctuations in daily call numbers.
NHS 111 total call data were analysed by age and gender. Data were obtained from a similar national syndromic surveillance system in Scotland (NHS 24),5 to compare the relative age and gender burden of NHS 111 calls. Data were compared using patient gender and age analysed by quintile age bands. Trends in NHS 111 syndromic indicator data were also analysed by age (age groups <1, 1–4, 5–14, 15–44, 45–64, 65–74, 75+ years and all ages).
Epidemiological trends were monitored for each indicator on a daily basis. To assess the expected seasonal trends for each indicator, NHS Direct indicator data for the period 2010 to 2013 were used to create historical baselines. This was to provide a proxy measure for expected seasonal activity for each NHS 111 indicator, which also contributed to the validation of the new system.
Statistical analysis
Statistical models were applied to the daily data to identify significant exceedances, either compared with recent, or historically expected levels.16 Statistical baselines were estimated for each syndromic indicator and area using a multi-level hierarchical mixed effects model incorporating variables such as day of the week and public holidays.16 An upper threshold for expected activity each day, equivalent to a 99% upper prediction interval, was set using estimated baselines and the estimated variation in the data. ‘Historical’ alarms indicated current activity significantly higher than previous years; ‘spike’ alarms revealed sudden increases in activity.13 The mixed effects model included a number of parameters to account for changes in the underlying proportion of calls coded to syndromic indicators, including the change from NHS Direct to NHS 111 and the decline in NHS Direct call numbers during 2013 as the NHS Direct service was phased out. Spike thresholds were created by scaling the historic thresholds up or down depending on whether recent activity had been above or below the historic baseline.
Results
Total syndromic calls
Daily trend analysis of the total number of syndromic calls revealed that the number of NHS 111 syndromic calls were highest over weekends and during public holidays. During the first year of operation of the new syndromic surveillance system weekday syndromic calls averaged 19 095; weekend syndromic calls were higher (Saturdays averaged 46 135; Sundays averaged 40 033). The highest number of daily syndromic calls was 58 547, recorded on 19 April 2014 (Easter Saturday). An increase was observed in the trend in total syndromic calls. This may have been due to provider sites coming on stream towards the end of 2013 during the final stages of the NHS 111 service implementation.
Data quality for syndromic surveillance was continually monitored: between 20 and 30% of syndromic calls did not have symptom information recorded and <2% did not have a postcode district. All calls had an outcome or disposition recorded.
Age/gender distribution
The age/gender distribution of syndromic calls received by PHE was assessed using gender of caller and quintile age group. There was a predominance of females using the NHS 111 service (57% female). However, in children aged under 5 years, there was an excess of male calls. Young children aged 1–4 years accounted for 14% of calls: there was less use in the 10–19 years age groups, with increased use in 20–29 years and older age groups (Fig. 1). When compared with the NHS 24 syndromic surveillance system, the overall gender use between the systems was very similar (NHS 111 male 42.9%; NHS 24 male 41.8%). There was also a very close fit between the quintile age groups between the two systems (Fig. 1).
Fig. 1.
Age/gender distribution of NHS 111 syndromic calls compared with NHS 24 syndromic call data. Call analysis based upon data collected during December 2013.
NHS 111 public health outputs
A weekly public health surveillance bulletin was launched in November 2013 providing a weekly summary of the key national epidemiological trends reported on the NHS 111 call data and interpretation of those trends.2
The NHS 111 syndromic surveillance system was used to supplement existing national surveillance programmes. During winter 2013–14, indicators for respiratory illness were monitored daily as part of the seasonal influenza and respiratory syncytial virus (RSV) monitoring programme. NHS 111 cold/flu calls showed little variation during the winter 2013–14 with the percentage of calls below historical NHS Direct baselines (Fig. 2a). There was some degree of seasonal variation within fever calls; however, compared with the historical NHS Direct baseline, the overall level of fever calls was much lower (Fig. 2b).
Fig. 2.
Daily syndromic indicator calls, shown as a percentage of the total number of syndromic calls (7 day moving average adjusted for public holidays) monitored through the NHS 111 syndromic surveillance system. (a) Cold/flu calls and (b) fever calls.
There was an increase in the percentage of cough calls to NHS 111 during November and December 2013 with a peak in calls initially on 1 December 2013, with calls maintained until 26 December 2013, and then decreasing after the Christmas period (Fig. 3a). Overall, the level of cough calls received in NHS 111 was higher than the NHS Direct baseline. A number of spike alarms were recorded during November and December 2013 indicating increased significant cough activity, with another group of spike alarms in February (Fig. 3a). An age breakdown of cough calls revealed that the predominant burden of calls was in children aged under 5 years (Fig. 3b).
Fig. 3.
Daily cough calls shown as (a) the percentage of the total number of syndromic calls monitored through the NHS 111 syndromic surveillance system and (b) calls presented by age group (presented as 7-day moving averages adjusted for public holidays).
Calls for difficulty breathing peaked on 1 December 2013, coinciding with the observed peak of cough calls (Fig. 4a). There were a number of spike alarms recorded throughout the winter period; however, there was a period of significant activity that occurred during February 2014. The age breakdown of difficulty breathing calls was predominantly children aged under 5 years (Fig. 4b).
Fig. 4.
Daily difficulty breathing calls shown as (a) the percentage of the total number of syndromic calls monitored through the NHS 111 syndromic surveillance system and (b) calls presented by age group (presented as 7-day moving averages adjusted for public holidays).
Record rainfall levels during December 2013 to March 2014 resulted in widespread flooding across parts of southern and south west England. The NHS 111 syndromic surveillance system formed part of the PHE surveillance response to the flooding by contributing to the daily surveillance of specific public health indicators (diarrhoea and vomiting) for those areas most at risk from the floods. The NHS 111 indicators were routinely monitored to upper-tier local authority (LA) level and to postcode district level for those areas most affected. Reassurance was provided to national incident response teams that there were no unusual increases in these indicators in the flood-affected areas.17
In March and April 2014 episodes of poor air quality across England prompted enhanced monitoring of all respiratory, cardiac and eye problem indicators across all of the PHE syndromic surveillance systems and any significant changes in these indicators were noted during the period 3 April to 8 April 2014. The NHS 111 cough, eye problem (data not shown) and difficulty breathing indicators were included in the enhanced surveillance. Spike and historical alarms were recorded for cough and difficulty breathing calls during the air pollution incident, but calls quickly returned to expected levels when air quality returned to normal levels.18
Discussion
Main finding of this study
This report describes the development of a new remote health advice syndromic surveillance system using health data generated from a telephone triage, healthcare advice and information service operating across England. Data extraction and transfer systems proved reliable and resilient with a 1% failure rate of the automated daily data transfer. Syndromic call data quality was continually assessed: 98% of the calls included a postcode district enabling the majority of calls to be included in spatial analyses facilitating daily surveillance at local health boundary level.
It is important to evaluate a surveillance system following its development and launch. Guidelines for evaluating surveillance systems have been previously published, to identify key tasks that are critical for describing a functioning surveillance system.19 This report does not represent a formal evaluation of the NHS 111 syndromic surveillance system; however, a number of key evaluation tasks have been considered and described, including the components and attributes of the NHS 111 syndromic surveillance system, some of the health events under surveillance and how the data are analysed.
What is already known on this topic?
The new NHS 111 syndromic surveillance system was developed to replace an existing syndromic surveillance system based on call data from NHS Direct. The NHS Direct syndromic surveillance system had formed a cornerstone of the syndromic surveillance service in England since 2001, supporting daily health protection surveillance and forming an integral part of the national surveillance of a number of local and national outbreaks, incidents and events.4,7 The planned decommissioning of the NHS Direct service created the opportunity for the development of a new, national syndromic surveillance system.11
What this study adds?
Internationally, this work is of interest as the number of remote health advice syndromic surveillance systems (also referred to as ‘telehealth’) used for public health surveillance increases.3,5,20,21 It is important to share expertise and experience of developing and maintaining this type of surveillance system in order that countries who have such systems can see the opportunities in developing surveillance around these data. In the event of major epidemics, or mass gatherings, these systems can provide valuable health information as a result of good representative coverage of the population, supplementing health intelligence that can be used to estimate the total burden of disease or health impact in the community.
The epidemiological assessment of syndromic indicators, particularly during winter 2013–14 raised some interesting questions around the differences between the NHS 111 call levels and the NHS Direct baselines. The NHS Direct cold/flu and fever calls had previously been validated as early warning indicators for community influenza activity.22 The differences observed here between the equivalent NHS 111 indicators and the NHS Direct baselines (Fig. 2a) may be due to a number of factors. The winter of 2013–14 witnessed very low and late influenza activity compared with previous years.23 This may explain the lower level of NHS 111 cold/flu calls compared with the NHS Direct baselines, which also included the winter of 2010–11 when influenza activity reached a peak not seen in subsequent winters. Other syndromic surveillance systems also reported little influenza-like illness activity over the winter of 2013–14, and laboratory confirmed influenza cases were also lower than in previous years.23 The NHS 111 cold/flu syndromic indicator will be monitored over future influenza seasons to further assess its usefulness for providing early warning of influenza activity.
A comparison of the NHS 111 age distribution to that of the NHS Direct system (data not shown) revealed a higher proportion of elderly patients using the NHS 111 service.24 NHS 111 acts as a gateway to accessing primary care out-of-hours and other urgent care services, which may explain why elderly patients were better represented in the NHS 111 system. The comparison made with the NHS 24 syndromic surveillance system revealed that despite the volume of calls handled by the NHS 111 system approaching 10 times that of NHS 24 (reflecting the differing populations of England and Scotland), the age/gender breakdown was similar between the systems. This wider representation of the elderly population by NHS 111 is an advantage for the surveillance system enabling better provision of situational awareness during winter seasons of the impact of respiratory pathogens on the elderly.25,26
During its first year, the national NHS 111 syndromic surveillance system contributed to the public health response to incidents including seasonal influenza and RSV outbreaks, seasonal gastrointestinal infections, flooding and episodes of poor air quality. The availability of routine surveillance data, available in near real time, and to a local geography was a valuable asset to the national surveillance response. These data complemented existing syndromic surveillance systems utilizing data from general practitioners and emergency departments enabling signals and messages to be triangulated to deliver strong public health messages. A key attribute of the NHS 111 syndromic surveillance system is the flexibility and simplicity of the system enabling the system to respond to different public health incidents in near real time at national and local level and to contribute to the monitoring of seasonal respiratory and gastrointestinal infections. Remote health advice data have previously been used as a benchmark for providing early warning of seasonal pathogens including influenza and norovirus.22,27 The utility of the new NHS 111 surveillance system for providing early warning will be explored through the development of activity-based early warning thresholds.
Limitations
The different CDSS used by the NHS 111 and NHS Direct services may have resulted in some variation in the recording and management of symptoms and possible differences in outcomes between the two services. Although the symptom titles of the protocol/Pathways are similar, the way the services deal with different symptoms/conditions may vary, making the two systems not directly comparable. This variation in the recording of main symptom has implications, as syndromic surveillance has previously been based on which symptom protocol/Pathway was used for each call. Most notably, the main impact appears to have been in the recording of ‘fever’ symptoms. In NHS 111, ‘fever’ is not recorded as the main presenting symptom if there is another predominant symptom, e.g. cough. In NHS Direct, ‘fever’ would have been recorded as a main complaint and as such used as the symptom indicator for syndromic surveillance purposes. However, despite the differences observed between the two systems for levels of certain indicators, historic NHS Direct baselines were included as a proxy for expected seasonal activity, which were, in most cases, similar. Since Week 47 of 2014, new baselines have been introduced for each NHS 111 syndromic indicator for comparison with previous years. These use historical data from the NHS Direct surveillance system to estimate seasonal trend but with levels adjusted to reflect changes since the switch to using NHS 111 data in September 2013.
Between 20 and 30% of the NHS 111 calls in the PHE dataset excluded information about the patient's main symptom or chief complaint (‘Pathway Selected’) and therefore could not be allocated to a syndromic indicator. There are exceptions where a symptomatic call may not result in ‘Pathway Selected’ being identified and recorded, examples including immediate threats to life (including fighting for breath), heavy bleeding, major trauma, callers with a pre-determined management plan (for a pre-existing condition), ‘frequent callers' (as identified by the service), ‘probable’ heart attack, stroke or allergic reaction (including ‘hayfever’). Non-symptomatic calls are not triaged using symptomatic pathways and therefore do not generate an entry in the ‘Pathway Selected’ field. The underlying principle of a successful syndromic surveillance system is to refrain from requesting additional coding or information from data providers.1 It was therefore not possible to improve the completeness of the symptom recording; however, the consistency of this gap was reassurance that this would have a minimal impact on the interpretation of syndromic trends.
A further limitation of this system was the lack of historical data for generating NHS 111 baselines at a time when the system was required to replace the former NHS Direct syndromic surveillance system. A solution was to use NHS Direct baselines until there were sufficient NHS 111 data available to create NHS 111 historical baselines. While there were some differences between the NHS 111 and NHS Direct data, in general, the overall seasonal trends were similar. New baselines introduced after Week 47 of 2014 improved the interpretation of the differences between the two syndromic surveillance systems.
Further work
Work is needed to further strengthen and expand the public health utility of the NHS 111 syndromic system: this will include an analysis of the health advice outcome, of which information is available for each call. Health advice provided to the patient ranges from advice to visit a local pharmacy, to the urgent need to visit an emergency department. These data might provide an indicator of changing severity of calls if individual outcomes are grouped into general categories, e.g. ‘emergency’ (ambulance/emergency department), general practitioner visit, home/self-care and other (e.g. dental services). Future work will include developing baselines for the percentage of calls for each indicator falling into these groups which can be monitored to provide an indicator of increasing or decreasing severity in the community.
Funding
This work was undertaken as part of the national surveillance function of Public Health England and received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. G.E.S. and A.J.E. receive support from the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Emergency Preparedness and Response.
Acknowledgements
We acknowledge the support and help of the following individuals and organisations who have contributed to the development and implementation of the PHE Remote Health Advice Syndromic Surveillance System (NHS 111). Ian Smith (HSCIC); Linda Ralph, Lead Analyst and the NHS 24 Information Services Team; the PHE Real-time Syndromic Surveillance Team.
References
- 1. Triple S. Assessment of syndromic surveillance in Europe. Lancet 2011;378:1833–4. [DOI] [PubMed] [Google Scholar]
- 2. Public Health England. Syndromic Surveillance: Systems and Analysis. 2014. https://www.gov.uk/government/collections/syndromic-surveillance-systems-and-analyses (14 October 2014, date last accessed).
- 3. Andersson T, Bjelkmar P, Hulth A et al. . Syndromic surveillance for local outbreak detection and awareness: evaluating outbreak signals of acute gastroenteritis in telephone triage, web-based queries and over-the-counter pharmacy sales. Epidemiol Infect 2014;142:303–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Baker M, Smith GE, Cooper D et al. . Early warning and NHS Direct: a role in community surveillance? J Public Health Med 2003;25:362–8. [DOI] [PubMed] [Google Scholar]
- 5. Meyer N, McMenamin J, Robertson C et al. . A multi-data source surveillance system to detect a bioterrorism attack during the G8 Summit in Scotland. Epidemiol Infect 2008;136:876–85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Rolland E, Moore KM, Robinson VA et al. . Using Ontarios “Telehealth” health telephone helpline as an early-warning system: a study protocol. BMC Health Serv Res 2006;6:10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Smith GE, Cooper DL, Loveridge P et al. . A national syndromic surveillance system for England and Wales using calls to a telephone helpline. Euro Surveill 2006;11:220–4. [PubMed] [Google Scholar]
- 8. Smith S, Smith GE, Olowokure B et al. . Early spread of the 2009 influenza A(H1N1) pandemic in the United Kingdom—use of local syndromic data, May-August 2009. Euro Surveill 2011;16:pii:19771. [PubMed] [Google Scholar]
- 9. Elliot AJ, Singh N, Loveridge P et al. . Syndromic surveillance to assess the potential public health impact of the Icelandic volcanic ash plume across the United Kingdom, April 2010. Euro Surveill 2010;15:pii:19583. [PubMed] [Google Scholar]
- 10. Elliot AJ, Morbey RA, Hughes HE et al. . Syndromic surveillance - a public health legacy of the London 2012 Olympic and Paralympic Games. Public Health 2013;127:777–81. [DOI] [PubMed] [Google Scholar]
- 11. Cook EJ, Randhawa G, Guppy A et al. . A study of urgent and emergency referrals from NHS Direct within England. BMJ Open 2015;5:e007533. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. NHS Choices. Emergency and Urgent Care Services. 2014. http://www.nhs.uk/NHSEngland/AboutNHSservices/Emergencyandurgentcareservices/Pages/NHS-111.aspx (29 September 2014, date last accessed).
- 13. Public Health England. Statistical Methodologies Used by the Real-Time Syndromic Surveillance Team in Their Daily Surveillance. 2014. http://webarchive.nationalarchives.gov.uk/20140722091854/http://www.hpa.org.uk/Topics/InfectiousDiseases/InfectionsAZ/RealtimeSyndromicSurveillance/ (22 October 2015, date last accessed).
- 14. Health and Social Care Information Centre. NHS Pathways. 2015. http://systems.hscic.gov.uk/pathways (22 October 2015, date last accessed).
- 15. Public Health England. Heatwave Plan for England. 2014. https://www.gov.uk/government/publications/heatwave-plan-for-england (22 October 2015, date last accessed).
- 16. Morbey RA, Elliot AJ, Charlett A et al. . Development and refinement of new statistical methods for enhanced syndromic surveillance during the 2012 Olympic and Paralympic Games. Health Informatics J 2014;21:159–69. [DOI] [PubMed] [Google Scholar]
- 17. Public Health England. 2014 Floods: PHE Surveillance Report Summaries. 2014. http://data.parliament.uk/DepositedPapers/Files/DEP2014-0382/PQ191065_part_4.pdf (2 October 2014, date last accessed).
- 18. Smith GE, Bawa Z, Macklin Y et al. . Using real-time syndromic surveillance systems to help explore the acute impact of the air pollution incident of March/April 2014 in England. Environ Res 2014;136:500–4. [DOI] [PubMed] [Google Scholar]
- 19. Klaucke DN, Buehler JW, Thacker SB et al. . Guidelines for evaluating surveillance systems. MMWR Morb Mortal Wkly Rep 1988;37:1–18.3122001 [Google Scholar]
- 20. Perry AG, Moore KM, Levesque LE et al. . A comparison of methods for forecasting emergency department visits for respiratory illness using telehealth Ontario calls. Can J Public Health 2010;101:464–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. van-Dijk A, Aramini J, Edge G et al. . Real-time surveillance for respiratory disease outbreaks, Ontario, Canada. Emerg Infect Dis 2009;15:799–801. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Cooper DL, Verlander NQ, Elliot AJ et al. . Can syndromic thresholds provide early warning of national influenza outbreaks? J Public Health 2009;31:17–25. [DOI] [PubMed] [Google Scholar]
- 23. Public Health England. Surveillance of Influenza and Other Respiratory Viruses in the UK: Winter 2013 to 2014. 2014. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/325203/Flu_annual_report_June_2014.pdf (13 October 2014, date last accessed).
- 24. Cooper D, Arnold E, Smith G et al. . The effect of deprivation, age and sex on NHS Direct call rates. Br J Gen Pract 2005;55:287–91. [PMC free article] [PubMed] [Google Scholar]
- 25. Elliot AJ, Cross KW, Fleming DM. Acute respiratory infections and winter pressures on hospital admissions in England and Wales 1990–2005. J Public Health 2008;30:91–8. [DOI] [PubMed] [Google Scholar]
- 26. Molbak K, Espenhain L, Nielsen J et al. . Excess mortality among the elderly in European countries, December 2014 to February 2015. Euro Surveill 2015;20:pii:21065. [DOI] [PubMed] [Google Scholar]
- 27. Loveridge P, Cooper D, Elliot AJ et al. . Vomiting calls to NHS Direct provide an early warning of norovirus outbreaks in hospitals. J Hosp Infect 2010;74:385–93. [DOI] [PubMed] [Google Scholar]