Accuracy of caregivers’ recall of hospital admissions: implications for research

Abstract

Aim

To determine the accuracy of caregivers’ recall of hospital admissions in early childhood.

Methods

Prospective cohort study of babies born at risk of neonatal hypoglycaemia at Waikato Hospital, New Zealand, a regional public hospital and sole provider of acute inpatient care to over 100,000 children.

Caregivers’ recall of children’s hospital admissions up to 4.5 years were compared with medical records. Accuracy of recall was related to neonatal and socio-demographic characteristics.

Results

Out of 267 children, 179 (67%) visited hospital and 106 (40%) were admitted at least once. The most frequent reasons for admission were for respiratory (29%) and gastrointestinal (18%) problems. Of 106 children admitted to hospital, 27 (25%) caregivers did not recall the admission and only 37 (35%) accurately recalled the number of admissions. The accuracy of recall was lower for gastrointestinal (38%) and surgical (40%) problems, while recall of respiratory (64%) and ear, nose and throat (60%) admissions was more accurate. Low socio-economic status and multiple admissions were associated with less accurate recall of number of admissions.

Conclusion

Caregivers do not accurately report hospital admissions. Questionnaire data about use of hospital facilities should be interpreted cautiously, and may not be sufficiently accurate for use in research studies.

INTRODUCTION

The use of and access to health care services, especially hospital facilities, is an important indicator of childhood health and is often used as an outcome in research. It is particularly important for low-income families and those living in rural areas who might have difficulty accessing medical care (1,2). History of healthcare visits is often collected in research, audits and surveys via extraction from medical records or self-report in questionnaires. Although extracting data from healthcare provider files is considered the most accurate method (3,4), it is problematic in large research studies because of the time and cost involved (5). Medical events of infants and children in younger age groups are often recorded from recall by a caregiver. Therefore, the accuracy of caregivers’ recall is an important factor to consider when choosing study methodology.

Previous studies of caregiver recall have been inconsistent, with some studies showing reasonably accurate recall of a child’s medical history (6) and others showing poor recall (7). Furthermore, accuracy of caregiver recall has been both positively (8) and negatively (7) related to the number of illness episodes.

There are no recent studies on the recall of hospital visits and factors associated with accuracy of recall in preschool children. Therefore, we aimed to assess the agreement between caregivers’ reports of hospital admissions and hospital medical records in a cohort of preschool children born at risk of poor health outcomes and enrolled in a prospective cohort study from birth.

METHODS

This study was part of a larger prospective cohort study of babies born at risk of neonatal hypoglycaemia, the CHYLD Study, which is investigating the impact of neonatal hypoglycaemia on later neurodevelopment. All babies in the cohort were born at Waikato Women’s Hospital, Hamilton, New Zealand, and recruited to one of two studies, BABIES (9) and Sugar Babies (10). Eligible babies were born late preterm (32-36 completed weeks’ gestation), small (≤ 2500g or ≤10^th percentile), large (≥4500g or ≥90^th percentile), of diabetic mothers, or with other conditions potentially increasing the risk of hypoglycaemia. Babies were excluded from these studies if they had congenital or life-threatening disorders, had been previously treated for hypoglycaemia or had other medical conditions that would interfere with the study protocol. Children included in the analysis were born between December 2006 and February 2010.

Follow-up assessment was completed at 4.5 years ± 2 months. Children were examined by the research team according to standardised protocols. Assessment included developmental, vision examination, neurologic status and general health assessment. A questionnaire was also completed by caregivers that included questions on ethnicity, household income, parental education, and hospital admissions (age at admission, reasons for and duration of each admission and name of the hospital). Socio-economic status was assessed using household income and New Zealand Deprivation Index decile (11), where 1 indicates the least deprived and 10 the most deprived population decile. Details were collected from Waikato District Health Board medical records from birth up to 4.5 years of age, including outpatient and Emergency Department (ED) visits, hospital admissions (admission to inpatient ward of any duration), number of nights in hospital (both inpatient admissions and ED overnight stays), and date and reason for visit or admission. Waikato Hospital services a population of 400,000 in the upper central North Island of New Zealand and is the sole provider of secondary and tertiary acute medical services for children in the region.

For children whose caregivers indicated that there had been hospitalisations outside Waikato District Health Board, medical records were obtained from the hospital indicated.

Data were analysed using JMP Statistical Software, version 10.0.2, SAS Institute Inc., Cary, NC, 2012, and are presented as number (percent) or median (range). Differences between risk groups and associated socio-demographic factors were analysed using Chi-squared test. Agreement between caregivers’ recall and confirmed admissions in hospital records were analysed using kappa coefficients (95% Confidence Interval) and interpreted as described by Landis and Koch (12). The study was approved by the Northern Y Health and Disability Ethics committee (approval number NTY/10/03/021). Parents provided written consent to the assessment, and also to the study team accessing the medical records of their children.

RESULTS

Medical records were extracted for 267 children who were assessed at 4.5 years ± 2 months. Over a third (101/267, 38%) were born pre-term and about a third (91/267, 34%) were born to diabetic mothers (Table 1). Approximately one half of the cohort were New Zealand European (139, 54%) and a third were Maori (83, 32%). More children (97, 37%) in this cohort lived in high deprivation areas (worst three deciles) when compared to national data.

Table 1.

Characteristics of the cohort

		≥1 Hospital admission by hospital records		Number of admissions accurately recalled by caregiver
Characteristic †	Total cohort N=267	Yes N=106	No N=161	Yes N=37	No N=63
Neonatal risk factors, prioritised
IDM	91(34)	31(29)	60(37)	13(35)	17(27)
Pre-term	101(38)	41(39)	60(37)	12(32)	28(44)
Small	37(14)	18(17)	19(12)	7(19)	7(11)
Large	24(9)	7(7)	17(11)	4(11)	3(5)
Other	14(5)	9(8)	5(3)	1(3)	8(13)
Boys	136(51)	55(52)	81(50)	19(51)	31(49)
Ethnicity
Maori
Other	83(32)	34(33)	49(31)	11(30)	22(35)
New Zealand	37(14)	9(9)	28(18)	3(8)	5(8)
European	139(54)	60(58)	79(51)	23(62)	36(57)
Household income
>$70,000	91(42)	33(39)	58(45)	16(53)	17(33)
$40,001 – 70,000	62(29)	24(28)	38(29)	7(23)	16(31)
< $40,000	62(29)	28(33)	34(26)	7(23)	19(37)
NZ Deprivation index
Most deprived (8- 10)	97(37)	41(39)	56(35)	9(24)*	29(46)
Less deprived (<8)	168(63)	65(61)	103(65)	28(76)	34(54)
Mother’s education, highest level
School	73(29)	35(35)	38(25)	11(31)	24(39)
Tertiary	178(71)	66(65)	112(75)	25(69)	38(61)
Number of siblings in the household
0-1	145(57)	57(56)	88(56)	23(64)	32(52)
2-3	95(37)	35(35)	60(38)	11(31)	24(39)
≥4	17(7)	9(9)	8(5)	2(6)	6(10)
Admissions in hospital records
0	161(60)	0(0)	161(100)	0(0)§	0(0)
1	58(22)	58(55)	0(0)	25(68)	31(49)
2-3	34(13)	34(32)	0(0)	10(27)	21(33)
≥4	14(5)	14(13)	0(0)	2(5)	11(17)

Data are number (percent); IDM, infant of a diabetic mother;

^†Total number of children with and without hospital visits or hospital admissions differ for each demographic factor due to missing data: ethnicity, 8; household income, 52; deprivation index, 2; mother’s education, 16; father’s education, 44; number of siblings, 10;

^*p=0.03;

^§p=0.04 using Chi-squared test comparing children whose caregivers accurately vs. inaccurately recalled the number of admissions.

Two thirds of children (179/267, 67%) had at least one hospital visit and over a third (106/267, 40%) had at least one hospital admission confirmed in hospital records by 4.5 years of age. For children who had at least one admission, the median (interquartile range) number of overnight stays up to 4.5 years was 2 (1; 5). Neonatal and socio-demographic factors were not significantly different between children who had visited the hospital or were admitted and those who had not (Table 1).

Of 106 children who had been admitted according to hospital records, caregivers of 27 (27/106, 25%) reported no admissions. Caregivers of children who lived in more deprived areas (deprivation index 8 to 10 vs <8) were less accurate in recall of their children ever being admitted (Table 1). The accuracy of caregiver recall for admissions lasting ≥ 2 nights was not significantly different compared to admissions of only one night (67% of caregivers were accurate vs 42%, P=0.74). Overall, there was a total of 945 visits to the hospital and 208 hospital admissions for the entire cohort (Table 2). Most hospital admissions were for respiratory (60/208, 29%) and gastrointestinal (GIT) (38/208, 18%) problems, followed by ear, nose and throat (ENT) (32/208, 15%) and surgical (20/208, 10%) problems.

Table 2.

Reasons for hospital admissions

Reason	Hospital admissions N=208	Children whose caregiver accurately reported reasons for admission† (total N of admissions = 87) N=68	Children with ≥1 admission whose caregiver completed questionnaireⱡ (total N of admissions = 193)
			Confirmed by hospital records N=100	Recalled by a caregiver N=100	Kappa coefficient (95% CI)
Respiratory	60(29)	60(88)	33(33)	21(21)	0.53(0.38; 0.67)
GIT, feeding problems	38(18)	56(82)	29(29)	11(11)	0.40(0.21; 0.58)
ENT	32(15)	59(87)	25(25)	15(15)	0.58(0.40; 0.76)
Surgical	20(10)	65(96)	15(15)	6(6)	0.48(0.23; 0.73)
Other	58(28)	50(74)	38(38)	26(26)	0.38(0.23; 0.54)

GIT, gastro-intestinal tract; ENT, ear nose and throat; N, number; Data are number (percent) and kappa coefficient (95% Confidence Interval).

^†For children whose caregivers accurately reported number of hospital admissions or who had only one admission.

^ⱡOf 106 children who had ≥1 admission confirmed by hospital record.

Number of hospital admissions

Complete questionnaire data were available for 100 of the 106 children who were admitted to hospital. Of these, only 37 (37%) caregivers were accurate in their recall of the number of hospital admissions (Table 1), indicating only slight agreement with hospital records (kappa coefficient [95% CI] 0.13[0.02; 0.25]). Caregivers who lived in more deprived areas were less accurate in their recall of number of hospital admissions. Recall was also less accurate with increasing number of hospital admissions. Fifty-six children had one hospital admission confirmed in medical records; caregivers of 25 (45%) of them recalled it accurately. Of 13 children with four or more admissions, only two (15%) caregivers were accurate in their recall. Other socio-demographic factors were not significantly different between children whose caregivers had accurate and inaccurate recall of the number of admissions (Table 1). The proportion of caregivers who accurately recalled the number of hospital admissions was similar for admissions that occurred before the age of 2 years and for admissions from 2 - 4.5 years (57% vs 52%, P=0.42), and for admissions that lasted 1 night and ≥ 2 nights (50% vs 53%, P=0.55).

Reasons for hospital admissions

Since the number of admissions was often inaccurately recalled, it was difficult to match the reported reason for admission with the relevant hospital record. We therefore assessed recall of reason for admission in two ways. First, we compared caregivers’ recall of the reason for admission with the hospital record for those children with accurate report of the number of admissions (n=37) and for all other children with only one hospital admission (n=31) (Table 2). The accuracy of caregivers’ recall of the reasons for hospital admission ranged from 82% for gastrointestinal problems to 96% for surgical admissions (Table 2).

Second, we included all children in the analysis, and determined how accurately caregivers reported that their children had been admitted for common health problems at least once. Gastrointestinal (11/29, 38%; kappa 0.40 [0.21; 0.58]) and surgical (6/15, 40%; kappa 0.48 [0.23; 0.73]) problems were less likely to be reported than respiratory (21/33, 64%; kappa 0.53 [0.38; 0.67]) and ENT (15/25, 60%; kappa 0.58 [0.40; 0.76]) problems (Table 2).

DISCUSSION

We aimed to determine if caregivers accurately recalled hospital admissions of their children from primary neonatal discharge up to 4.5 years when using a questionnaire, and the factors that influence this recall.

Of concern, we found that a quarter of caregivers did not recall their children ever being admitted to hospital and only a third accurately recalled the number of admissions, with lower socio-economic status and higher number of admissions associated with poorer recall. Similar results were shown by D’Souza-Vazirani et al. (13) who reported that mothers with higher income reported recent admissions more accurately than those with lower incomes. This suggests that researchers should carefully consider the method of collecting data about use of hospital facilities, especially in low socio-economic settings.

Although accuracy of recall was positively related to socio-economic status, we did not find any association with parental education level. In other studies, the relationship between accuracy of recall and parental education has not been consistent. For example, Pleas et al. found no relation between recall accuracy and education of parents (6). Conversely, Hoekelman et al. found that maternal education was correlated with accuracy of recall of immunisations, but not the recall of clinic visits (14). It is possible that low socio-economic status might be associated with poorer recall because of its association with poorer health and higher admission rates. However, we did not find evidence that this applied in our cohort, as there were no differences in socio-economic status of children who had at least one admission and those who did not. We also found no association between the number of children in the household and the accuracy of recall. Reports in the literature have variously shown that having other children in the family was associated with poor recall (7), improved recall (13), or no effect on recall (6,14).

We also aimed to investigate if caregivers could accurately identify reasons for being admitted. Most hospital admissions were for respiratory, GIT, surgical and ENT problems, which is consistent with other reports (15,16). For the subgroup of children where the reason for admission could be matched with caregivers’ report, reasons for admissions were reported reasonably accurately. However, this might be because children who had no admissions for a specific reason and no report of that problem by a caregiver would be counted as agreement for this analysis. Thus, surgical problems, which contributed to the least number of admissions (20/208, 10%), were associated with the highest recall rate (96%). However, when considering any admission for a particular problem up to 4.5 years, only 38% and 40% of caregivers whose children had been admitted for GIT and surgical reasons recalled this, although recall was better for respiratory (64%) and ENT (60%) admissions. Other studies have also shown that accuracy of parental recall depends on the reason for the visit, with respiratory problems being reported more accurately than ENT problems in a Canadian study of 1 to 13 year olds (6).

We analysed only hospital admissions, which we expected would be more likely to be remembered, as they would be perceived as serious events. Others have reported that hospitalisations were better recalled than ED visits when mothers were interviewed by telephone at 2 to 4 and 30 to 33 months after the birth of their children (13). However, in our study, caregivers of only 6 of 15 children who were admitted for surgical problems, which are most likely to be perceived as serious event, accurately reported this. Similarly, poor agreement has been reported when comparing maternal reports and medical records for other relatively severe conditions such as acute asthma (3).

One possible factor that could influence our results was that the recall period was relatively long. Participants enrolled in a study with relatively short intervals between recall questionnaires or interviews may be more likely to pay attention and remember hospital visits, as they expect to be approached by the research team. Some previous studies used relatively short recall periods, but a longer time interval is advised for collection of hospital admission data as admissions are relatively rare events (17). Although the CHYLD Study team examined children at 2 years, and some caregivers would expect to be contacted later with similar questions about hospitalisation details, it is highly unlikely to have had an effect on the recall results due to the long time interval between assessments and the fact that recall was not the main focus of the study. Indeed, we found no differences in accuracy of recall of early hospital visits (before 2 years) and those that occurred more recently (2-4.5 years). However, others have reported that recall is poor even over short time periods. Low-income mothers could not accurately identify the reason for seeing a doctor when they were interviewed three times at 4 month intervals (18). In addition, Grover et al. found that parents could not accurately identify the reasons for an ED visit, even within a few minutes after discharge (19). Asthma and otitis media were recalled more accurately than GIT or skin conditions. Many parents could not recall the diagnosis, but stated the complaints children presented to ED with.

A potential limitation of our study is that we may have missed some hospitalisations if children were admitted outside the Waikato Hospital area or to private hospitals and parents did not recall that admission. However, the New Zealand health care system is mainly public and few private hospitals admit children, especially in the Waikato area. Moreover, we compared parental recall with known admissions and reasons, so the under-reporting that we found is likely to be a minimum estimate, with any missed admissions only increasing the extent of parental under-report.

Our data show that hospital visits and admissions are very common in children born at risk of neonatal hypoglycaemia. Previous reports from both Australia (15) and New Zealand (20) found that up to 20% of children were admitted to hospital during the preschool years. Thus, there was a two-fold greater rate of admission in our cohort. This may relate to the long-term health effects of risk factors for neonatal hypoglycaemia, such as prematurity and fetal growth restriction, and also to socio-demographic factors. Indeed, there was greater social deprivation in this cohort compared with the general New Zealand population.

Researchers should carefully choose methods for data collection on use of hospital facilities. This includes recall period, administration approach and data source. A suitable approach will depend on the cohort characteristics, including literacy level, but also study research questions. Recall bias may be lower when reporting events in an interview than in a self-administered format, but sensitive information may be more accurately collected via a self- administered questionnaire (21). Diaries completed by parents can provide accurate information on visits to medical specialists, but are most useful in a cohort with high literacy levels (22). Therefore, if a study requires accurate data on health care utilisation, medical records are likely to provide the most complete information.

CONCLUSIONS

Caregivers often do not accurately recall details of hospital admission of their pre-school children. Data collected on use of hospital facilities obtained from caregiver questionnaires should be interpreted cautiously, especially in low socio-economic environments and when use of hospital facilities is high. For accurate assessment of hospital admissions, researchers should consult medical records.

Key notes.

• Reasons for hospital admissions were not recalled accurately at 4.5 years for a cohort of preschool children born at risk.

• Low socio-economic status and multiple admissions were associated with inaccurate recall.

• Data on hospital admissions in preschool children collected from caregiver reports should be interpreted cautiously.

Abbreviations

ED

Emergency Department

ENT

ear nose and throat

GIT

gastrointestinal