Pneumonia case-finding in the RESPIRE Guatemala indoor air pollution trial: standardizing methods for resource-poor settings

Abstract

Objective

Trials of environmental risk factors and acute lower respiratory infections (ALRI) face a double challenge: implementing sufficiently sensitive and specific outcome assessments, and blinding. We evaluate methods used in the first randomized exposure study of pollution indoors and respiratory effects (RESPIRE): a controlled trial testing the impact of reduced indoor air pollution on ALRI, conducted among children ≤ 18 months in rural Guatemala.

Methods

Case-finding used weekly home visits by fieldworkers trained in integrated management of childhood illness methods to detect ALRI signs such as fast breathing. Blindness was maintained by referring cases to study physicians working from community centres. Investigations included oxygen saturation (SaO₂), respiratory syncytial virus (RSV) antigen test and chest X-ray (CXR).

Findings

Fieldworkers referred > 90% of children meeting ALRI criteria, of whom about 70% attended a physician. Referrals for cough without respiratory signs and self-referrals contributed 19.0% and 17.9% of physician-diagnosed ALRI cases respectively. Intervention group attendance following ALRI referral was 7% higher than controls, a trend also seen in compliance with RSV tests and CXR. There was no evidence of bias by intervention status in fieldworker classification or physician diagnosis. Incidence of fieldworker ALRI (1.12 episodes/child/year) is consistent with high sensitivity and low specificity; incidence of physician-diagnosed ALRI (0.44 episodes/child/year) is consistent with comparable studies.

Conclusion

The combination of case-finding methods achieved good sensitivity and specificity, but intervention cases had greater likelihood of reaching the physician and being investigated. There was no evidence of bias in fieldworkers’ classifications despite lack of concealment at home visits. Pulse oximetry offers practical, objective severity assessment for field studies of ALRI.

Introduction

Acute lower respiratory infections (ALRI) are the single most important cause of death of children under 5 years, responsible annually for approximately 20% of the 10 million under-5 deaths globally.^1,2 Prevention strategies are required urgently, including control of risk factors. A growing body of evidence links household indoor air pollution from solid fuels with ALRI in developing countries: recent estimates suggest this may be responsible for nearly one million child ALRI deaths.³ However, these figures are based on relatively few observational studies with considerable variation in ALRI case-finding methods, indirect exposure assessment (using proxies such as fuel type) and risk of residual confounding.⁴ To address these limitations we conducted a community-based randomized controlled trial with improved chimney stoves in rural Guatemala.

Weaknesses in previous ALRI field studies, and the methodological issues common to trials of environmental interventions, highlighted three particular challenges for this study:

1. To ensure that few cases are missed. Frequent home visits by staff trained to recognize signs such as fast breathing can achieve high sensitivity for ALRI.⁵ It has been suggested that early treatment associated with more frequent visits may reduce cases of severe ALRI, but a recent review found no evidence of association between surveillance interval (less than 2 weeks) and incidence.^1,5

2. To ensure high specificity, as ALRI constitutes a minority (~10%) of all acute respiratory infections. Any impact of reduced exposure on ALRI incidence may be missed if ALRI cases are classified mistakenly with larger numbers of acute upper respiratory infections (AURI). To achieve specificity, all cases identified by fieldworkers should undergo physician examination and preferably chest X-ray (CXR).⁵

3. To take measures to make physicians’ assessments blind and incorporate objective outcome assessments, as it was not possible for subjects or staff visiting homes to be blind to their intervention status.

This report’s objectives are to describe these methods, evaluate the effectiveness of case-finding and identify any evidence of bias by intervention status. Analysis was carried out using Stata version 9.1.⁶ An annual ethical review was conducted by the Centers for Disease Control and Prevention (CDC) and the institutional review boards of the Universities of California (Berkeley), del Valle de Guatemala and Liverpool (UK).

Methods

Study area and population

Following extensive feasibility studies,^7–13 a rural area of San Marcos in western Guatemala was selected. The indigenous population speaks mainly a Mayan language, Mam, and some Spanish. Wood is the main household fuel, burned indoors on open fires. Key features of the study area are presented in Table 1.

Table 1. Study site and population information^a.

Altitude	Mean 2600 m (range 2200 to 3000)
Seasons	Warm and wet: May to October
	Dry and cold: November to February
	Dry and warmer: March to April
Temperature	Average daily temperatures 10.3–12.7 °C. Occasionally below freezing at night during coldest months, producing demand for space-heating
Rainfall	Up to 36 mm/day during wet season; almost none December–April
Other major diseases that may be confused with ALRI	Malaria is not endemic; few reports of identified cases of HIV and TB
Low birth weight	Reported to be 14% prevalence in Guatemala,30 but estimated at 20% in neighbouring Quetzaltenango31
Breastfeeding	Almost universal. Exclusive breastfeeding of children < 4 months reported by 94% of study sample
Nutritional status	Area characterized by severe undernutrition. By 17 months, average height-for-age and weight-for-age z-scores for study children were 3 standard deviations below normalb
Vaccination	Of 95 study children between 2 and 4 months at recruitment, 77 (81%) had received Bacille Calmette–Guérin, 45 (47%) had received DPT/Polio-1

ALRI, acute lower respiratory infections; TB, tuberculosis
^a This information conforms with the minimum data set proposed by Lanata et al.⁵ Little reliable information is available from external sources, and data from the current study are provided where relevant.
^b Thompson L. Unpublished data, 2006.

Study design and ALRI case-finding

The study design was a randomized controlled trial comparing ALRI incidence in children ≤ 18 months using the traditional three-stone fire (controls), with intervention homes using a flued wood stove (plancha):^7,14 534 homes with either children under 4 months or a pregnant woman were randomized, and planchas constructed in 269. Sample size was determined to detect a 25% change in ALRI incidence of 0.5 episodes per child per year, at 5% significance, 80% power. Surveillance began after 5 weeks when the planchas were ready, from which time 518 children were followed until the age of 18 months, withdrawal or death. ALRI case-finding was carried out at four levels:

1. Weekly household visits by fieldworkers trained in WHO integrated management of childhood illness (IMCI) methods.¹⁵

2. Study physicians, working in local community centres to maintain blindness, undertook clinical assessments of children referred by fieldworkers, or self-referred.

3. Extraction of information from hospital records.

4. Verbal autopsy to investigate all deaths.¹⁶

Household visits

Weekly home visits ran from December 2002 to December 2004. Each home was visited on the same day every week. If unavailable, one repeat visit was made on the Friday of the same week. From a total of 21 fieldworkers, 16 were recruited from local community leaders, midwives and health promoters; 14 of these were bilingual (Mam-Spanish) and eight (50%) were female. Fieldworkers were allocated equal numbers of plancha and control homes to visit each day.

The fieldworkers’ questionnaire was based closely on IMCI criteria¹⁵ and focus groups identified appropriate Mam terms. All fieldworkers underwent one-week IMCI training in symptom and sign recognition (including video for wheeze and stridor), and classification of children as well, sick but suitable for home treatment or requiring referral to study physician. Respiratory rate was measured over one minute using a timer (UNICEF) and repeated when over 60 in children < 2 months (repeat used).

One supervisor carried out repeat home visit assessments in 10% of homes, while a second supervisor directly observed 10% of home visits. Most repeat assessments agreed with the original findings but the child was re-examined when there were disagreements. All forms were reviewed after each day of fieldwork to identify and correct errors.

Not all weekly scheduled visits could be realized, mostly due to internal migration (Table 2). Although there was a slightly lower rate of completed visits to plancha homes, there were similar numbers of realized visits per child, children with no missed visits and dropouts.

Table 2. Possible and completed weekly visits for intervention and control groups.

Weekly visits		Intervention	Control
Number of participating children		265	253
Weekly visits	Total possible in follow-up period	16 446	15 664
	Completed	14 756	14 369
Possible weekly visits completed (%)		89.7%	91.7%a
Mean (SD, range) weekly visits per child		55.7 (17.8; 1–80)	56.8 (17.3; 2–81)b
Children with no missed visits (%)		17 (6.4%)	19 (7.5%)b
Withdrawals		19 (7.2%)	14 (5.5%)b

NS, non-significant; SD, standard deviation.
^a P < 0.001
^b NS.

A critical indicator of potential bias was how well fieldworkers adhered to the referral algorithm, by intervention status. No child with specific respiratory signs (raised respiratory rate, chest wall indrawing, stridor or wheezing) was classified as well, but approximately 8% (7.9% plancha, 8.8% control) of cases with raised respiratory rates were not referred, with similar findings for other respiratory signs. All non-referrals with fieldworker-assessed fast breathing had respiratory rates in the ranges considered normal for the next youngest age group (Table 3),¹⁵ suggesting that non-referral resulted from uncertainty about ages and thresholds for rapid breathing. Most of these referrals (81%) occurred in the first half of the study.

Table 3. Mean respiratory rate per minute (and distributions) for children with respiratory illness with fieldworker-measured raised respiratory rate.

Age group	n	Referred to doctor	Mean (SD)	Geometric mean	Range	25th–75th percentile
2 to < 12 months	202	Yes	56.3 (6.5)	55.9	50–88	52–59	MW: P = 0.09 t-testa: P = 0.11
	10	No	53.1 (3.2)	53.0	50–58	50–56
≥ 12 months	291	Yes	48.7 (7.0)	48.3	40–89	44–52	MW: P < 0.0001 t-testa: P < 0.0001
	32	No	43.9 (2.9)	43.8	40–49	42–46.5

MW, Mann–Whitney test; SD, standard deviation.
^a t-test on log transformed respiratory rate.

Fieldworker-assessed symptoms and signs were combined to produce four definitions of new ALRI cases (Table 4): 668 cases met the criteria for lower respiratory illness, plus stridor, a rate of 1.12 (95% confidence interval, CI: 1.03–1.20) episodes per child per year. No cases were classified as well, but 8.9% and 9.3% were classified as unwell and suitable for home treatment in plancha and control groups respectively.

Table 4. Compliance with classification algorithms for fieldworker-assessed categories of new episodes of ALRI, and with referral to the study physicians^a.

Stage in referral process	Possible pneumoniab (n = 540)				Wheezing illnessc (n = 114)
	Plancha	Control	P-value		Plancha	Control	P-value
Of those meeting outcome criteria:
• Classified by FW as “well”	0	0	0.61		0	0	0.61
• Classified by FW as “unwell” but for home treatment	9.3	8.1			14.6	18.2
• Classified by FW as “unwell” and referred to physician	90.7	91.9			85.4	81.8
Attended physician before next home visit, as % of cases meeting criteria	72.5	70.1	0.53		71.2	58.3	0.14
Attended physician before next home visit, as % of cases referred by FW to physician	79.4	75.9	0.33		82.2	72.2	0.24
Physician visit same day or later same week, as % of cases meeting criteria	69.0	64.3	0.25		60.4	43.9	0.08
Stage in referral process	Lower respiratory illnessd (n = 637)				Lower respiratory illness, plus stridore (n = 668)
	Plancha	Control	P-value		Plancha	Control	P-value
Of those meeting outcome criteria:
• Classified by FW as “well”	0	0	0.97		0	0	0.88
• Classified by FW as “unwell” but for home treatment	9.6	9.5			8.9	9.3
• Classified by FW as “unwell” and referred to physician	90.4	90.5			91.1	90.7
Attended physician before next home visit, as % of cases meeting criteria	72.3	67.8	0.2		71.6	67.1	0.18
Attended physician before next home visit, as % of cases referred by FW to physician	79.8	75.3	0.17		78.7	74.3	0.17
Physician visit same day or later same week, as % of cases meeting criteria	68.6	61.9	0.07		68.8	62.2	0.07

ALRI, acute lower respiratory infections; FW, fieldworker.
^a All figures (other than total numbers of cases meeting criteria and P-values) are percentages within study arms. Definitions of ALRI health outcomes are based on fieldworker assessment at home visits.
^b Children less than 2 months of age with any of: raised respiratory rate (> 60/minute), nasal flaring, grunting or chest wall indrawing. Children aged 2 months or more with cough or difficulty breathing and any of: increased respiratory rate (> 50 for children 2 months to less than 12 months; > 40 for children 12 months or over) or lower chest wall indrawing. Stridor is not included. These children may or may not have wheeze as heard without auscultation by the fieldworker.
^c For child less than 2 months, FW reported audible wheeze on examination. For child aged 2 months or older wheezing disease was defined as a child having a cough or difficulty breathing and FW reported audible wheeze. These children do not have any of the above signs meeting criteria for possible pneumonia.
^d Possible pneumonia and/or wheezing illness, using definitions above.
^e Using definition (d) above, but including fieldworker reported stridor.

Severe fieldworker-assessed cases (severe WHO pneumonia) were defined as new cases (including nine with non-severe ALRI the previous week) of lower respiratory illness, plus stridor, with chest indrawing and/or inability to drink or breastfeed. There were 72 of these cases, a rate of 0.12 (95% CI: 0.09–0.15) episodes per child per year.

It was expected that not all referred sick children would be taken to the study physician. Almost 80% of referrals for possible ALRI attended the physician before the next weekly visit (Table 4), consistently 3–5% higher in the plancha group (NS). About 70% of all children meeting referral criteria attended the physician before the next weekly visit, 5% more in the plancha group (NS).

Approximately two-thirds of all children with ALRI criteria completed consultations on the day of referral if referred or later that week (Table 4), 5–7% higher in the plancha group (16% for cases with wheeze only), 0.05 < P < 0.1 for outcomes including wheeze. Nearly all (96%) of plancha children referred with severe WHO pneumonia attended the physician, compared to 73% of controls (Fisher’s exact P = 0.02). For 295 visits where a child had ALRI signs but did not see the physician that week, 48 (16.3%) had signs of ALRI at the following weekly visit and 7 of these had severe WHO pneumonia; all 7 were in the control group (P = 0.03).

Fieldworkers also referred 1212 episodes of cough or difficulty breathing, but no specific respiratory sign: 842 (69.5%) attended, and 49 (5.8%) were diagnosed with pneumonia.

Clinical assessment by study physicians

Study physicians assessed children in community centres located up to 1 km from their homes. A standardized history and examination was developed from earlier studies.^17,18 Training sessions were held every one to two months at San Marcos Hospital in order to maintain consistent interpretation of clinical signs – each physician assessed children independently, then compared findings with other physicians and the resident paediatrician.

Six Guatemalan physicians were employed during the study: one (SO) worked almost throughout and four carried out the majority (94.6%) of consultations. A total of 1991 consultations among 467 study children were completed for illnesses other than minor skin and eye conditions (recorded separately). Five respiratory diagnoses were used: AURI; otitis media; laryngo-tracheo-bronchitis; pneumonia; and wheezing illness. Pneumonia was categorized as none, possible or definite, based on the physician’s clinical judgment, and new cases were defined as those following at least one weekly home visit without ALRI signs. With the exception of wheezing illness, respiratory diagnoses usually were applied exclusively (Table 5).

Table 5. Co-morbidity in terms of respiratory diagnoses by study physicians (for 263 cases diagnosed).

Other diagnoses	Presence or absence		Physician-diagnosed pneumonia per diagnostic category: number (%)
			None		Possible		Definite
AURI	Present		885	(51.3)	1	(0.5)	0	–
	Absent		841	(48.7)	207	(99.5)	55	(100)
Otitis media	Present		105	(6.1)	3	(1.4)	0	–
	Absent		1621	(93.9)	205	(98.6)	55	(100)
LTB (croup)	Present		6	(0.3)	0	–	0	–
	Absent		1720	(99.7)	208	(100)	55	(100)
Wheezing illness	Present		23	(1.3)	48	(23.1)	10	(18.2)
	Absent		1703	(98.7)	160	(76.9)	45	(81.8)

AURI, acute upper respiratory infections; LTB, laryngo-tracheo bronchitis.

Physicians recorded 263 new episodes of pneumonia, a rate of 0.44 (95% CI: 0.39–0.49) episodes per child per year: 216 (82%) from fieldworker referrals, 47 (18%) from self-referrals. The incidence of fieldworker-assessed AURI was 3.44 episodes per child per year based on current respiratory symptoms, and 5.96 when “symptoms since last visit but now resolved” are included, thus between 7.8 and 13.5 times the physician-diagnosed pneumonia rate.

Intervention and control groups reported similar signs used in pneumonia diagnosis, apart from crepitations (Table 6). Using physician-diagnosed pneumonia as the reference, a raised respiratory rate had moderate sensitivity, but poor specificity and positive predictive value (PPV). Chest wall indrawing was recorded in 36% of pneumonia cases but few other cases so while sensitivity was low, specificity and PPV were high. Crepitations were recorded in well over 80% of pneumonias, and in almost no other diagnoses, thus having high sensitivity, specificity and PPV. A lower proportion of intervention pneumonia cases had crepitations recorded (P = 0.02).

Table 6. Total cases seen by study physicians, and recording of key respiratory signs in making diagnoses of (a) pneumonia and (b) all other cases for intervention (I) and control (C) groups^a.

Physician	Total cases seen by physicianb	Cases with rapid breathing (%)					Cases with chest indrawing (%)					Cases with crepitations (%)
		Pneumonia		Other illness			Pneumonia		Other illness			Pneumonia		Other illness
		I	C	I	C		I	C	I	C		I	C	I	C
SO	837 (42.1%)	73	74	22	17		34	18	–	–		94	97	–	–
EO	476 (23.9%)	60	32	9	11		14	9	–	0.5		86	94	–	0.5
AJ	378 (19.0%)	61	81	18	18		48	72	–	0.7		76	84	–	–
DJ	192 (9.7%)	74	92	55	39		52	50	6.5	–		74	83	–	–
Other	107 (5.4%)	83	80	32	22		41	40	2.7	–		67	90	2.7	3.1
All	1990	70	69	22	17		36	39	0.7	0.4		81	91	0.1	0.4
P-valuec		0.88		0.02			0.65		0.4			0.02		0.27

FW, fieldworker; AJ, DJ, EO, and SO, initials of physicians.
^a A total of 1991 cases were seen, but MD identification is missing for one. Apart from total cases seen (column 2), all figures are percentages of cases within study groups.
^b Total cases refers to all potentially serious and respiratory conditions. All minor conditions such as rashes and conjunctivitis in an otherwise well child were recorded separately.
^c χ² test for intervention versus control comparisons, for all cases (last row).

Fieldworker classifications were compared against physician diagnoses: only 5.8% of referrals with cough or difficulty breathing but no specific lower respiratory signs were diagnosed with pneumonia, compared to 28.1% of non-severe ALRI referrals and 47.3% of severe ALRI referrals (P < 0.001).

Pulse oximetry

Low oxygen saturation is a complication of ALRI associated with increased mortality.^19,20 Study physicians used pulse oximetry to measure oxygen saturation (SaO₂) in all consultations. A Sims BCI Mini Corr was used initially, left on the foot for two to three minutes until the reading was stable. From November 2003, Nellcor N-20 oximeters were used. These were more reliable with distressed children, taking six readings on the foot at 30-second intervals (mean used).

To obtain typical SaO₂ levels at the study altitude (Table 1), 55 randomly selected study children were investigated by using the Nellcor N-20 and recording whether or not the child was well (reported by mother). Well children had mean SaO₂ of 93.2% (standard deviation, SD 3.0; 95% CI: 92.7–93.7), median 93.5% and range 82% to 98% (Table 7), consistent with other studies.¹⁹ Hypoxaemic pneumonia cases will be defined as SaO₂ > 2 SD below the mean for well children¹⁹: for this study, 93.2% – (2×3.0) = 87.2% (rounded to 87%).

Table 7. SaO₂ data from pulse oximetry study of well children (n = 55).

Diagnostic group	Number of measurements	Mean (SD)	Median	Minimum	Maximum	25th–75th centile
Children reported “well” by mother/carer	128	93.2 (3.0)	93.5	82	98	91.4–95.4

SaO₂, oxygen saturation; SD, standard deviation.

Table 8 and Fig. 1 illustrate SaO₂ values for children with non-respiratory diagnoses, AURI, wheezing illness (without pneumonia), and pneumonia (possible, definite). Distributions for the diagnostic categories differ overall (Kruskal-Wallis P = 0.0001). Similar statistically significant differences are seen for readings from the Sims and Nellcor oximeters. Since SaO₂ was measured during the consultation, it is possible that diagnosis was influenced by the reading, although a substantial number of non-serious diagnoses (AURI, non-respiratory) had low values recorded, and vice-versa.

Table 8. SaO₂ data from physician examination of study children: all cases^a.

Diagnostic group	Number of episodes	Mean (SD)	Median	Minimum	Maximum	25th–75th centile
No respiratory condition	654	90.8 (4.7)	92.0	72	99	89–94
Acute upper respiratory infection (without wheeze)	877	91.6 (3.8)	92.0	71	100	90–94
Wheeze (without pneumonia)	22	88.6 (5.4)	89.0	74	96	85–92
Possible pneumonia	205	84.8 (9.0)	88.0	43	97	80–90
Definite pneumonia	55	78.2 (11.6)	80	45	95	73–86

SaO₂, oxygen saturation; SD, standard deviation.
^a Kruskal–Wallis test of differences between diagnostic groups (df = 4): P = 0.0001.

Fig. 1.

Distributions of oxygen saturation readings from children examined by the study physicians, by diagnostic category

AURI, acute upper respiratory infections.

Direct antigen test for RSV

A substantial but widely varying proportion of ALRI cases are reported to be viral,²¹ although mixed viral and bacterial etiology also occurs.^21,22 Respiratory syncytial virus (RSV) is the predominant viral agent.^21,23 We are aware of only one study reporting on the risk of RSV ALRI associated with IAP finding a protective effect of exposure for severe hospitalized cases (OR = 0.31 for mothers’ cooking at least once daily, 95% CI: 0.14–0.7).²⁴ If reducing IAP does not reduce risk of RSV-related ALRI, failure to distinguish between viral and non-viral cases would mask any impact on bacterial cases. This has added significance because most ALRI deaths result from bacterial infection, probably due to their invasive nature.^21,25–27

An enzyme immunoassay test (Becton-Dickinson Directigen RSV) was performed on all children diagnosed with pneumonia (with parental consent), using naso-pharyngeal aspirate following infusion of 1 ml of normal saline. Kits include positive and negative test samples, used at the start and end of each batch. The product manual reports the sensitivity and specificity of this test to be 93–97% and 90–97% respectively.

RSV tests were completed for 236 (89.7%) of the 263 cases of physician-diagnosed pneumonia, but fewer control (87.1%) than intervention (93.6%) cases (0.05 < P < 0.1). Mean SaO₂ was 1.2% lower in cases receiving the test (NS).

Referral for chest X-ray

Initially, children diagnosed with pneumonia were referred to San Marcos Hospital for chest X-ray; from April 2004 a private clinic with better quality control was used. Due to apprehension about hospital, and barriers of travel time and cost, initially a substantial proportion of referrals did not attend. During the first six months, the study team worked to build trust and provided transport twice per week. There was less anxiety about attending the private clinic.

All CXRs used the antero-posterior view, and were conducted in 208 (79.1%) pneumonia cases, more intervention (82.3%) than control (76.3%) cases (NS). Median SaO₂ was similar in those who did and did not have a CXR. Acceptance varied over the study: lowest during the first year (70%) but rising to 90% during the second. Following training in standardized methods for pneumonia interpretation developed by WHO, two readers (MG, JB) assessed CXRs.^28,29 These were reported as: (1) normal; (2) minor changes; (3) major changes, pneumonia or bronchopneumonia, excluding 4 and 5 below; (4) lobar pneumonia or effusion; and (5) overlap between 4 and 5. Good agreement was achieved on five-way classification (Cohen’s kappa = 0.83), and 16 films with disagreement were resolved through consensus reading by two experienced WHO staff. A combination of categories 3, 4 and 5 are taken as radiological evidence of pneumonia. In an attempt to define outcome more strictly (specifically) as primary end-point pneumonia,²⁸ two additional readers reviewed all films (blind). However, agreement was unsatisfactory – all kappa values for pairs with new readers were < 0.4.

Hospital referral

Children requiring hospital admission were referred to San Marcos Hospital, with transport provided. Following discharge or death, hospital clinical information was added to study records. Two additional pneumonia cases admitted by self-referral were obtained from hospital records. CXRs were available, but neither SaO₂ nor RSV status.

Deaths

Deaths were investigated by verbal autopsy¹⁶ approximately six weeks after death. Interviews were conducted by the field project coordinator (AD) assisted by a bilingual field supervisor, and analysed blind by four WHO readers. During surveillance, 23 deaths occurred (37 per 1000 child years), of which 9 (39%) were due to pneumonia (14.9 per 1000 child years, 95% CI: 7.7–28.6). Verbal autopsies provided 6 additional pneumonia cases, yielding 271 cases in total (0.45 episodes/child/year, 95% CI: 0.40–0.50) and case fatality of 3.3%.

Discussion

The aim of the first stage of case-finding was high sensitivity. Although this cannot be calculated directly, the 47 self-referrals that resulted in a diagnosis of pneumonia provide some indication of false negatives. This must be viewed alongside the large number of cases of respiratory illness without lower respiratory signs referred by fieldworkers, which contributed 49 physician-diagnosed pneumonias. Overall, the weekly visits, referrals (ALRI and AURI) and self-referrals together probably delivered high sensitivity, a conclusion consistent with observed pneumonia rates. The finding that one-third of fieldworker-defined ALRI cases were diagnosed with pneumonia is consistent with the positive predictive value (PPV) expected from the incidence rates of upper and lower respiratory infections in this study and reported sensitivity and specificity of community case-finding.¹⁸ It was reassuring to find no evidence of bias by intervention status in the classification of children for referral.

Self-referrals to the physician did not compensate for the differential compliance with fieldworker referral by intervention status. There were more self-referrals for respiratory illness among plancha children, and slightly more of these cases were diagnosed with pneumonia in the intervention group (14.6% versus 13.0%, NS). This, combined with the higher referral compliance for respiratory illness in the plancha group, indicates that intervention cases had a consistently higher likelihood of attending the physician.

Our assessment of physician examinations focused on evidence of bias in the use of clinical signs for diagnosis. The lower proportion of pneumonia cases among intervention children with crepitations was notable. This may be an intervention outcome rather than bias, as diagnosing pneumonia more frequently among plancha children without crepitations seems counterintuitive if blinding of physicians had not been successful.

The investigations (pulse oximetry, RSV test, chest X-ray) were important for increasing specificity and objectivity in outcome assessment. Pulse oximetry was consistent with diagnosis and severity, and with other studies.¹⁹ RSV tests were carried out on almost 90% of pneumonias, but control cases had twice as many missed tests as plancha cases. The trend towards higher intervention group compliance was also seen for CXRs.

The nested case-finding allows a range of outcome definitions for analysis of risk estimates, the degree of blindness and objectivity increasing as more specific criteria are introduced (Table 9). Despite incomplete compliance with referral, the rate of physician-diagnosed pneumonia was close to that assumed for power calculation.

Table 9. Outcome definitions and rates for ALRI available from fieldworker assessment, physician examination and investigations.

	ALRI outcome definition	Blindness and objectivity of assessment	Number of cases	Rate/1000/year (95% CI)a
1	FW-assessed fast breathing, chest wall indrawing, etc. meeting criteria for referral to study physician as possible ALRI	Assessed in homes so not blinded. Assessment by trained FW, with supervision	668	1115 (1033–1203)
2	FW-assessed “severe WHO pneumonia” defined as (1) above, with chest wall indrawing and/or inability to breastfeed	As above	72	120.2 (95.4–151.4)
3	Physician-diagnosed pneumonia based on history and clinical examination. CXR not available to physician at time of diagnosis	Assessed away from homes, so theoretically blinded. Physicians attended training sessions every 1–2 months to maintain standardized assessment	263	439.0 (389.1–495.4)
4	Physician-diagnosed pneumonia, hospital referral recommended	As (3) above	64	106.8 (83.6–136.5)
5	Physician-diagnosed pneumonia with hypoxia measured by pulse oximeter	As (3) above, with additional objectivity of pulse oximeter measurement	136	227.0 (191.9–268.6)
6	Physician-diagnosed pneumonia with RSV negative	As (3) above, with additional objectivity of RSV test	150	250.4 (213.4–293.9)
7	Physician diagnosed pneumonia with RSV positive	As (3) above, with additional objectivity of RSV test	86	143.6 (116.2–177.3)
8	Physician-diagnosed pneumonia with hypoxia measured by pulse oximeter, stratified by RSV status	As (3) above, with additional objectivity of pulse oximeter measurement and RSV test	RSV neg, hypox: 69	115.2 (91.0–145.8)
			RSV pos, hypox: 57	95.1 (73.4–123.4)
9	Physician-diagnosed pneumonia with pneumonia on CXR	As (3) above, with additional objectivity of CXRs read blind and independently	88	145.6 (118.1–179.4)
10	Physician-diagnosed pneumonia with hypoxia and pneumonia on CXR	As (9) above, with additional objectivity of pulse oximetry readings	55	91.0 (69.8–118.5)
11	Physician-diagnosed pneumonia with hypoxia and pneumonia on CXR, stratified by RSV status	As (9) above, with additional objectivity of pulse oximetry readings and RSV antigen test result	RSV neg: 32	52.9 (37.4–74.8)
			RSV pos: 22	36.4 (23.9–55.3)
12	Pneumonia deaths based on verbal autopsy (VA) approximately 6 weeks after death	VA data collection in homes not blinded. Independent blind interpretation by WHO	9	14.9 (7.7–28.6)

ALRI, acute lower respiratory infections; CXR, chest x-ray; FW, fieldworker; RSV, respiratory syncytial virus.
^a Rates for physician-diagnosed pneumonia (and definitions 4–11) are calculated after exclusion of ≥ 3 successive missed weekly home visits from the denominator. This cut-off was selected as children who missed up to 2 successive weeks did contribute to physician-diagnosed cases, and are therefore considered “at risk”. Children missing three or more successive weeks were more likely to have migrated (temporarily) out of the study area, and therefore could not be at risk of a diagnosis of pneumonia by the study physician. Although some pneumonia cases did not receive RSV and/or CXR (see text) the denominator has not been adjusted, therefore these rates are likely to be underestimates.

Conclusions

Although case-finding met the stated aims with reasonable success, there are several issues for analysis, in particular the differential compliance with referral and investigations. Although not large at any one level, this occurs at each stage of the process and will impact most on outcomes using RSV and CXR information. To address this, analyses will include risk estimates adjusted for missing data. The problem of differential compliance, combined with no apparent evidence of bias by intervention group in fieldworker classification in this study, might suggest that referral for physician assessment is not useful. However, it would seem wise to retain blinded clinical and objective (investigation-based) assessment in any trial where intervention concealment at home is not possible. Adding pulse oximetry to home visit ALRI assessment may be worthy of further investigation. ■