Identifying problems with data collection at a local level: survey of NHS maternity units in England
Natalie Kenney, Alison Macfarlane
National Perinatal Epidemiology Unit, Institute of Health Sciences, Headington, Oxford OX3 7LF
Natalie Kenney
survey researcher
Alison Macfarlane
reader in perinatal and public health statistics
Correspondence to: Alison Macfarlane alison.macfarlane{at}perinat.ox.ac.uk
Abstract
Objectives
To document the extent to which maternity data are collected and how they are recorded, and to identify problems that may affect their availability throughout the NHS.Methods
Postal survey in September 1997 with structured questionnaires.Setting 207 NHS trusts with maternity units in England.
Participants
Heads of midwifery in NHS trusts with maternity units.Main outcome measures
Extent to which maternity data were routinely recorded, how they were recorded and evaluated, and to whom they were made available.Results
167 (81%) of questionnaires were returned, representing 166 trusts. Of these trusts, 165 collected ³ 17 of the 19 data items in HES maternity tail, and 158 collected ³ 40 of the 45 items selected from Körner dataset. Only 18 collected all five items selected from the "indicators of success," and 17 did not collect any. In 58 trusts data were primarily recorded on paper. A computerised maternity information system was used by 106 trusts, but many recorded data on paper first. Thirty four trusts did not audit data for accuracy. Most trusts analysed data not routinely collected at national level, but 18 did not analyse HES maternity tail and 17 did not analyse Körner data.Conclusions
Improvement is needed in quality, completeness, and availability of maternity data at a national level, particularly if the NHS information strategy is to be successfully implemented. Although most of the data items in national datasets are recorded locally, variations in the way data are defined, recorded, and analysed and lack of linkage between computer systems restrict their access, availability, and use at local, district, and national levels.Introduction
The NHS information strategy for England, described in Information for Health, (1) reiterated the need to improve the accuracy, completeness, and availability of NHS data. (2) Data about maternity care are needed at all levels of the NHS to monitor the health of women and their babies and the services provided. (3) (4) These data are also required to audit, monitor, and evaluate changes in the provision of care. (5) (6) Women also need this information to inform decisions about their care. (1) (7) (8) It is widely acknowledged that national maternity data for England are incomplete and that some are inaccurate or unavailable (9) (10) (11) (12) and that they need to be improved. (9) (10) (11) (12) (13) (14) (15)
At a national level, the Department of Healthâs hospital episode statistics (HES) system collects a core set of data about each episode of "admitted patient care." For episodes in which a baby is born, a supplementary set of data should be appended in the HES "maternity tail." (16) These data items include the place of delivery, care provided during labour and delivery, and information about the babyâs condition. The items contained in the HES maternity tail were specified in the early 1980s as part of a wider process of defining minimum datasets for the NHS and are commonly referred to as "Körner" datasets. (17) In maternity, the term Körner is applied to the core HES record as well as the additional data items in the maternity tail. Although maternity units are expected to send HES maternity tail data to the Department of Health, this is not mandatory, and these data are lacking from about a third of delivery records. (16)
The Changing Childbirth Implementation Team, set up to implement the recommendations of the Expert Maternity Group, (8) commissioned a project to investigate the reasons for deficiencies in national maternity data. The aims of the project were to document the extent of maternity data collection and to identify problems that may affect the quality or availability of this information for use elsewhere in the NHS and to women using the maternity services. NHS maternity units, health authorities and representatives of users of the maternity services in England were surveyed. This paper reports on the survey of NHS trusts with maternity units in England.
Methods
With advice from people responsible for maternity data in several NHS trusts and from the Department of Health, we designed a structured questionnaire that asked NHS trusts which maternity data items they routinely collected. It also asked about how the data were recorded, whether they were audited for accuracy, the extent to which they were analysed, how accessible they were, how they were used, and to whom they were made available. Different sections were included for trusts that recorded data on a computerised maternity information system and for those that recorded data in paper format. The questionnaire was piloted in Northern Ireland.
In September 1997 we sent questionnaires to heads of midwifery in all 207 NHS trusts with maternity units in England. Reminder letters were sent to non-respondents three and six months later.
The replies were checked for anomalies and analysed with SPSS 7.5 for Windows.
Results
We received replies from 167 (81%) of maternity units, representing 166 NHS trusts (table 1). Some of the respondents did not provide answers for all of the questions. Midwifery managers and midwives usually completed the questionnaires.
Table 1
Response rate by former NHS region of NHS trusts with maternity units in England to postal surveyNHS region | No of trusts with maternity units | No (%) of replies received* |
Northern and Yorkshire | 30 | 26 (87) |
| 19 |
|
Anglia and Oxford | 19 | 18 (95) |
North Thames | 29 | 24 (83) |
South Thames | 31 | 23 (74) |
South and West | 28 | 22 (79) |
West Midlands | 20 | 16 (80) |
North West | 31 | 22 (71) |
Total | 207 | 167 (81) |
*One NHS trust requested separate questionnaires for both of its maternity units as each was responsible for its own data. These questionnaires were analysed separately.
Extent of data collection
Of the 165 NHS trusts that replied to the question about data collection, all routinely collected =17 of the 19 data items contained in the HES maternity tail (16) and 158 (96%) routinely collected = 40 of the 45 data items selected from the Körner maternity dataset. (17) The trusts also collected data about other aspects of maternity care (table 2). Of the 10 "indicators of success" listed in Changing Childbirth, (8) five are easily quantified, but only 18 trusts routinely collected all five items, while 17 collected none (table 3).
Table 2
Additional maternity data items collected in maternity units of 166 NHS trusts in EnglandData item | No of responses to question | No (%) of NHS trusts collecting data |
Mother's country of birth | 164 | 151 (92) |
Mother's ethnic origin according to 1991 census definition | 165 | 153 (93) |
Mother's smoking history | 165 | 161 (98) |
Mother smoker during pregnancy | 164 | 158 (96) |
Mother's medical conditions | 163 | 163 (100) |
Assisted conception | 162 | 155 (96) |
Use of antenatal corticosteroids | 157 | 146 (93) |
No of antenatal clinic visits | 160 | 146 (91) |
Ultrasound scans | 163 | 161 (99) |
Other antenatal tests | 163 | 160 (98) |
Length of third stage | 165 | 164 (99) |
Management of third stage | 165 | 164 (99) |
Perineal tears | 165 | 165 (100) |
Episiotomy | 165 | 165 (100) |
Suturing perineal damage | 165 | 164 (99) |
Obstetric complications | 165 | 165 (100) |
Length of postnatal stay | 161 | 159 (99) |
Abnormalities detected | 164 | 164 (100) |
Transfer of baby to NICU or SCBU | 164 | 164 (100) |
NICU=neonatal intensive care unit. SCBU=special care baby unit.
Table 3
Extent to maternity units of 166 NHS trusts in England collected five "indicators of success" listed in Changing Childbirth6Indicator | No of responses | No (%) of trusts collecting data |
Women who have midwife as lead professional | 145 | 51 (35) |
Women who know the person who cares for them during their delivery | 142 | 47 (33) |
Women admitted under management of a midwife | 143 | 55 (38) |
Average No of antenatal visits for women with uncomplicated pregnancies | 145 | 59 (41) |
Women carrying their own case notes* | 155 | 118 (76) |
*Trusts may have responded positively if women usually carried their case notes, irrespective of whether the trust kept data about this practice.
How data were recorded
In 58 of the 166 trusts data were primarily recorded on paper. Some data may have been recorded on the patient administration system, but we did not investigate the extent to which this was done. Of these trusts, 48 stated that they experienced problems, including the time taken to record data manually, concerns about accuracy, difficulties in retrieving data, and records being incomplete.
In 106 (63%) of the trusts data were recorded on a computerised maternity information system. In only 35 of all trusts did midwives enter data directly on computer systems. In 50 trusts data were entered separately on both the maternity information system and the patient administration system, and only 14 trusts were able to transfer data from the maternity system to the patient administration system automatically.
Overall, 113 (68%) of the trusts used both paper and computerised recording methods, with many recording some data on paper and other data on computers. Two trusts did not indicate how data were recorded.
Audit of data
One hundred and twenty six (76%) of the trusts stated that data were routinely audited for accuracy, with 63 doing so monthly or more frequently. Resources did not allow us to check the extent or quality of these audits. Data were more likely to be audited if midwives collected data on paper and subsequently entered them onto computers.
Analysis of data
Data from the HES maternity tail and Körner dataset were analysed by 150 (90%) of the trusts, and most trusts produced at least some statistical analyses for the additional data that they collected. Table 4 shows the data items that were most likely to be used in statistical analyses. We did not ascertain the extent to which data were analysed. Most trusts analysed data with computer software, but 28% did so manually.
Table 4
Data items that were most likely to be used in statistical analyses by maternity units of 166 NHS trusts in EnglandData item | No of responses | No (%) of trusts producing statistical analysis |
HES maternity tail and Körner datasets | ||
Whether baby live or stillborn | 142 | 137 (96) |
Multiple births or No of babies | 140 | 134 (96) |
Method of delivery | 142 | 135 (95) |
Pain relief in labour | 136 | 115 (85) |
Method of onset of labour | 141 | 118 (84) |
Presentation of baby at delivery | 136 | 114 (84) |
Baby's birth weight | 141 | 116 (82) |
Feeding method at discharge | 137 | 113 (82) |
Place of delivery | 137 | 107 (78) |
Status of person conducting delivery | 132 | 86 (65) |
Parity | 133 | 85 (64) |
Date and time of delivery | 134 | 84 (63) |
Original intention for place of birth | 129 | 77 (60) |
Gestational age at onset of labour by last menstrual period | 123 | 71 (58) |
Sex of baby | 132 | 76 (58) |
Apgar score at 1 and 5 minutes | 127 | 70 (55) |
Gestational age at onset of labour by ultrasound scan | 116 | 63 (54) |
Birth order | 125 | 62 (50) |
Mother's date of birth | 128 | 53 (41) |
Additional maternity data items | ||
Episiotomy | 143 | 134 (94) |
Perineal tears | 142 | 125 (88) |
Baby transferred to NICU or SCBU | 134 | 109 (81) |
Length of postnatal stay | 131 | 102 (78) |
Infant abnormalities | 128 | 99 (77) |
Obstetric complications for mother | 130 | 97 (75) |
Mother's smoking history | 137 | 94 (69) |
Mother smoker during pregnancy | 136 | 92 (68) |
Suturing of perineal tears | 137 | 90 (66) |
Mother's ethnic origin by census definition | 124 | 74 (60) |
Ultrasound scans | 125 | 62 (50) |
NICU=neonatal intensive care unit. SCBU=special care baby unit.
Access to data
e/B>We asked the trusts to provide aggregated figures for six data categories for the year 1996 (table 5). Four of these categories included items in the HES maternity tail and Körner dataset, while the other two required data about perineal damage and the total number of births. Only 15 trusts could provide figures for all the data items. Although a further 151 trusts were able to provide some of the data, many could not provide them in the form requested --either because aggregated statistics were not routinely produced for those items or there were difficulties accessing paper records. Some trusts provided aggregated statistics for the whole data category.
Table 5
Maternity units: access to maternity data in 166 NHS trusts in EnglandAggregated data item requested | No of responses | No (%) of trusts providing data |
Total No of women delivering in trust | 158 | 156 (99) |
Onset of labour: |
|
|
Spontaneous | 142 | 99 (70) |
Elective caesarean section | 153 | 143 (93) |
Induction of labour*: |
|
|
Prostaglandins | 150 | 86 (57) |
Artificial rupture of membranes or amniotomy | 145 | 73 (50) |
Oxytocics | 140 | 65 (46) |
Oxytocics and artificial rupture of membranes | 142 | 55 (39) |
Method of delivery: |
|
|
Spontaneous vaginal | 154 | 150 (97) |
Ventouse | 156 | 151 (97) |
Forceps | 156 | 150 (96) |
Breech (vaginal)* | 148 | 113 (76) |
Breech (assisted)* | 129 | 66 (51) |
Caesarean section (elective) | 156 | 147 (94) |
Caesarean section (emergency) | 156 | 147 (94) |
State of perineum*: |
|
|
Episiotomy | 154 | 134 (87) |
1st degree tears | 148 | 74 (50) |
2nd degree tears | 149 | 77 (52) |
3rd degree tears | 151 | 97 (64) |
Perineum intact | 134 | 79 (59) |
Gestational age at delivery (weeks)*: |
|
|
<24 | 141 | 54 (38) |
24-27 | 144 | 53 (37) |
28-36 | 143 | 56 (39) |
37-41 | 140 | 55 (39) |
ï³ 42 | 140 | 53 (38) |
*Differences in data definitions were encountered.
Many NHS trusts could not provide the required figures because they used different definitions or groupings of data. Nearly a quarter of trusts used different definitions for induction of labour, which has been found in other surveys. (6) (12) Differences in definition also occurred with descriptions of breech deliveries and the degree of perineal tearing. The questionnaire asked for gestational age at delivery based on the age groups recommended for collection, (18) but some trusts used different groupings.
Ninety six of the 151 trusts that could not provide some data claimed that the data could be accessed if required urgently. Of these trusts, 42 could access data in less than a day, but 11 stated that it would take two weeks or more to obtain the information.
Availability and use of data
Table 6 shows how the trusts used their maternity data and to whom the data were made available. The type of data made routinely available was not specified. Although 91% of trusts provided data to their local health authority, our parallel survey of health authorities showed that these were more likely to relate to contracting than to clinical practice. (19)
Table 6
Availability and use of data by maternity units of 166 NHS trusts in England
| No of responses | No (%) of trusts disseminating data |
Availability of maternity data | ||
Data presented at local clinical meetings | 152 | 141 (93) |
Data provided to all clinicians routinely | 146 | 97 (66) |
Data provided to managers on request | 145 | 141 (97) |
Data provided to CESDI | 160 | 153 (96) |
Data provided to general practitioners routinely | 108 | 57 (53) |
Data provided to local health authority | 140 | 128 (91) |
Data provided in leaflet format to women using maternity services | 118 | 32 (27) |
Data provided to maternity service liaison committee | 133 | 89 (67) |
Use of maternity data at local level | ||
Annual report | 136 | 96 (71) |
Research | 72 | 48 (67) |
Audit | 118 | 106 (90) |
Monitoring Changing Childbirth | 96 | 58 (60) |
Contracting | 141 | 125 (89) |
Bed or ward plans | 132 | 89 (67) |
Staff planning | 135 | 97 (72) |
CESDI=confidential enquiry into stillbirths and deaths in infancy.
Discussion
Problems in the collection of maternity data in English NHS trusts lead to inadequacies in the quality, completeness, and availability of the information. These problems restrict the use of the data by the NHS and their availability to women using the maternity services. The problems may also impede the implementation of the NHS information strategy. (1)
We found that many trusts used different definitions for various data items, which has also been found in other studies. (6) (12) This may affect the usefulness of data for the NHS as it is difficult to aggregate or compare data that have been collected using different definitions. This may restrict comparisons between NHS trusts or districts as like would not be compared with like. (20)
Although nearly all trusts collected data about planned home births, the Department of Health received data for only 28% of home births in England in 1994-5. (16) It may be that some trusts did not record all home births or that records were not completed correctly. In addition, some women have home births outside the NHS with independent midwives, who may not submit their data to the NHS in the way originally envisaged. (17)
If national maternity policies are to be successfully implemented they need to be monitored. Many trusts monitored the implementation of policies set out in Changing Childbirth, but few collected data about its "indicators of success." This could be because of local differences in interpretation of the policy's recommendations, insufficient resources for implementing them, or difficulties in compiling and measuring the indicators, (21) (22) but the lack of these data makes it difficult to assess the policy's overall impact. This information may be collected on a sample or ad hoc basis for health authorities, but our survey of health authorities found that only 58% monitored the implementation of the policy. (19)
The combination of computerised and paper methods commonly used by NHS trusts to record maternity data can make linkage and management of data difficult, restrict their use, and affect the flow of data through the NHS and to the Department of Health. The duplication of data recording is wasteful of time and NHS resources. (20) Improved linkage is needed between computerised maternity information systems and patient administration systems to ensure that data are recorded only once.
Although computerised maternity information systems are supposed to make it easier to record and use data (23) (24) there are many problems associated with them, including incompatibility with the patient administration system, inconsistencies in coding, and difficulties in retrieving information. (25) (26) (27) Eleven of the NHS trusts in our survey reported that it would take two or more weeks to obtain data that were not immediately accessible. Unless information is readily available it cannot be used effectively to inform NHS decision making. (1) (8) (13) (20)
It is of concern that 28% of the trusts analysed their data manually. This process is laborious, time consuming, and limits the extent of analyses. This can impede the use of data locally and the forwarding of information to NHS systems. If the NHS information strategy is to be successfully implemented, these issues must be addressed.
Evaluating the quality of data will soon become mandatory in the NHS, (1) but 24% of the trusts did not routinely audit their maternity data for accuracy. Conducting regular audit of data and educating clinicians about how to audit their maternity records has been shown to improve data quality. (14) (28) (29) Feeding back data to those who collect them is also associated with improving data quality and clinical practice, (28) (29) (30) but 33% of the trusts did not routinely provide these data to clinicians.
Most trusts produced statistical analyses of data not routinely collected by the Department of Health. This suggests that there is a local need for these analyses, but as there are no national figures available for comparison this limits their use to a local level. Less than half of NHS trusts stated that they routinely analysed data collected about mother's date of birth. If the outcomes for women and their babies are not analysed by mothers' age there may be serious implications for monitoring the care provided to them.
Although maternity data were often made available within NHS trusts, the availability of data to others was limited. With the emphasis on sharing information in the NHS and the provision of information to users of the health services, (1) (2) availability must be improved considerably if NHS objectives are to be met.
Conclusions
This survey has shown that most of the maternity data missing at a national level are widely collected at a local level. Variations in the ways data are recorded, and many other problems contribute to inadequacies in the quality, completeness and availability of this information. Lack of computer systems for recording data in some trusts and lack of linkage between 'stand alone' computer systems and hospitals' patient administration systems in others are major problems. These problems restrict data use at all levels of the NHS and may impede the implementation of the NHS information strategy. (1)
At a local level, NHS trusts can do much to improve the quality of their maternity data. This can be achieved by conducting regular audits of data, informing clinicians of the need for accurate and complete data, and feeding data back to those collecting them. Now that the Department of Health has started to publish comparative HES maternity data, (16) trusts will need to ensure that their data are accurate, complete, and up to date so that their services are accurately represented in comparisons with other trusts.
At a national level, the Department of Health has committed itself to a revised minimum set of maternity data. (1) (31) Once implemented, this should ensure that data are collected according to agreed definitions, making it easier to aggregate and use this information throughout the NHS. The results of our survey have already been used to inform discussions about the revised dataset. Once a maternity dataset is agreed, the Department of Health should allocate resources to maternity units for computer systems that will enable them to collect data in a consistent way and which can be linked to other NHS computer systems.
It has been suggested that contributing maternity data to national systems has a low priority at a local level. (12) Now that it has begun to publish maternity data, (16) the Department of Health should consider making it mandatory for maternity units to contribute data to national systems. When new policies such as those recommended in Changing Childbirth are implemented, it is essential that the relevant data are collected locally and aggregated nationally so that the impact of policies on the maternity services and their benefit to women and their babies can be assessed.
Key messages
· Most NHS maternity data missing at a national level are collected at a local level
· Variations in the way in which maternity data are defined, recorded, audited, and analysed can affect the quality, completeness, and availability of this information throughout the NHS
· Considerable work and investment will be required to implement the plans set out in the NHS information strategy Information for Health
We thank all those who completed questionnaires for this project.
Contributors: AM initiated the research (at the suggestion of Kate Jackson, director of the Changing Childbirth Implementation Team), cowrote the protocol, assisted with questionnaire design and data analysis, and participated in writing and editing the paper. NK cowrote the protocol, assisted with questionnaire design, and participated in data collection and analysis and writing and editing the paper. Pam Dobson (Kings College Healthcare NHS Trust), Cathy Winter (Southmead NHS Trust), Maggie Jeffries (Oxford Radcliffe NHS Trust), Fiona Alderdice (Queens University Belfast), Lesz Lancucki (Department of Health), Jane Cowl (Changing Childbirth Implementation Team), and Meg Goodman (Maternity Alliance) commented on and discussed core ideas about the content and design of the questionnaire. Sarah Ayers (National Perinatal Epidemiology Unit) designed the computer databases and dealt with all computing issues. Jo Garcia, Rona McCandlish, Sally Marchant, Valerie King, and Kirstie McKenzie-McHharg commented on the questionnaire design and drafts of the paper. Suzanne Williams, Sarah Bowler, and Hedwig Van Asten entered the data. AM and NK are guarantors for the paper.
Funding: AM was funded through the Department of Health's policy research programme. NK was funded by the Changing Childbirth Implementation Team, Department of Health.
Competing interests: None declared.
(Accepted 29 July 1999)
Read all Rapid Responses