Evaluation of a clinical scoring system and directed laboratory testing for respiratory virus infection in hematopoietic stem cell transplant recipients

Abstract

P.E. Ferguson, N.M. Gilroy, T.P. Sloots, M.D. Nissen, D.E. Dwyer, T.C. Sorrell. Evaluation of a clinical scoring system and directed laboratory testing for respiratory virus infection in hematopoietic stem cell transplant recipients.
Transpl Infect Dis 2011: 13: 448–455. xx: 000–000. All rights reserved

Abstract: A simple clinical screening (CS) tool for respiratory virus (RV) infection was introduced and evaluated in a single hematology ward, as part of a strategy to reduce nosocomial RV infection. Up to 6 clinical symptoms or signs were scored and a predefined threshold score of ≥2 prompted paired nose/throat swab (NTS) collection for RV testing. The criterion standard for RV infection was positive immunofluorescence (IF) or polymerase chain reaction (PCR) for 7 and 15 viruses, respectively. The tool was shown to be most beneficial at excluding infection at a threshold score of 1 (negative predictive value [NPV] 89%, [95% confidence interval 78–96%], sensitivity 85% [70–94%], specificity 35% [27–43%]), compared with a score of 2 (NPV 85% [76–91%], sensitivity 63% [46–77%], specificity 57% [48–65%]) at a prevalence of 22%. The tool's ability to diagnose infection was limited (positive predictive value 27% and 29% at thresholds 1 and 2). The sensitivity of IF compared with PCR was 45% for the 7 viruses common to both, and 23% for the extended virus panel detected by PCR. An algorithm incorporating CS, paired NTS collection at a threshold of 1 symptom or sign, and sensitive testing including PCR can guide infection control measures in hospitalized hematopoietic stem cell transplant recipients.

Nosocomial respiratory virus (RV) infections after hematopoietic stem cell transplantation (HSCT) are an avoidable complication with severe clinical outcomes. For example, pneumonia has been reported in up to 75% of HSCT recipients acquiring parainfluenza virus (PIV) nosocomially, with an associated mortality of 47% (1). Nosocomial RV outbreaks have been reported in multiple HSCT units (1, 2, 3, 4), with 8 outbreaks in 13 European centers over 10 years; in 6, transplant programs were halted or closed in order to contain these infections (5).

Recommended methods to prevent nosocomial RV infection include a daily checklist for the presence of signs and symptoms of RV infection in patients beginning on the day of admission (6), placing all patients with respiratory symptoms in isolation and using respiratory and contact precautions when entering the room, excluding symptomatic staff or visitors from the unit, and deploying staff with a respiratory illness to non‐patient areas (6, 7). Isolation precautions can be modified when the etiology of the illness is known (7). Although recommended, the diagnostic accuracy of using a daily checklist to prompt viral testing and infection control measures has not been assessed.

In this study, we developed a clinical case definition for RV infection based on scoring a suite of signs and symptoms of respiratory infection. A threshold score of 2 or more triggered the collection of respiratory tract specimens for viral diagnosis. The diagnostic accuracy of the case definition was evaluated in adult HSCT recipients.

Methods

All patients admitted to the hematology ward, Westmead Hospital (a university teaching hospital) in Sydney, Australia, between July 1, 2005, and September 30, 2007, were eligible for clinical screening (CS) and per‐protocol testing for RV infection. This ward manages general hematology and HSCT patients, and performs 40–55 allogeneic and 15–25 autologous HSCTs annually on patients aged 16 years and above. Approval for the study was granted by the Sydney West Area Health Service Ethics Committee.

A CS score was derived from a checklist of 6 features: cough; fever >38°C in the last 24 h; sneezing, runny nose, or nasal stuffiness; shortness of breath; oxygen saturation <95% on room air; and crackles on chest examination documented by clinician in the patient record. One point was assigned to each positive finding. Nurses were requested to perform the screen daily, and enter the CS score into a log located in the bedside chart. Paired nose and throat swabs (NTS) for viral testing were recommended for patients with scores ≥2.

The screening tool was piloted from April 2005. Hematology nurses were educated by the researcher (P. E. F.), the nurse educator, and senior nursing staff on the use of the screening tool, sample collection techniques, and transport of swabs to the laboratory. Screening logs were placed in the patient's bedside charts on admission.

Clinical characteristics of episodes of respiratory illness were ascertained prospectively by direct questioning of patients and caregivers by the researcher, and observations of temperature, oxygen saturation, and physical examination findings provided by treating clinicians in the case notes, and laboratory and radiology results in electronic hospital databases. Demographic data and HSCT type and date were retrieved.

The following definitions were used:

Upper respiratory tract infection (URTI): Rhinorrhea, sneezing, cough, coryza, with or without fever, with a normal chest examination and absence of pulmonary infiltrates on radiological imaging (chest x‐ray or computed tomography).

Viral URTI: URTI with detection of RV from URT secretions collected by NTS (8, 9, 10, 11).

Lower respiratory tract infection (LRTI, or pneumonia): Fever and hypoxia (oxygen saturations<95% on room air) or pulmonary infiltrates on radiological imaging (chest x‐ray or computed tomography).

Viral LRTI: LRTI with the detection of RV in bronchoalveolar lavage (BAL) specimen (bronchoscopic or non‐bronchoscopic), endotracheal aspirate, or URT secretions (8, 9, 10).

Upper and lower respiratory tract infection (U&LRTI): LRTI with concurrent rhinorrhea, sneezing, or coryza.

Viral U&LRTI: U&LRTI with detection of RV in BAL (bronchoscopic or non‐bronchoscopic), endotracheal aspirate, or URT secretions (8, 9, 10).

Combined NTS were collected using sterile swabs moistened with viral transport medium (Copan Diagnostics, Corona, California, USA). BAL was performed on patients with adequate respiratory reserve at the discretion of hematology and respiratory physicians. Intubated and ventilated patients frequently had BAL or non‐bronchoscopic–BAL samples collected during intubation.

Initial virological testing was performed in the Respiratory Virology Laboratory of the Centre for Infectious Diseases and Microbiology Laboratory Services, the Institute of Clinical Pathology and Medical Research, Westmead Hospital. Samples were processed for indirect immunofluorescence (IF) using monoclonal antibodies (Chemicon Inc., Temecula, California, USA) against influenza A and B, PIV 1–3, respiratory syncytial virus (RSV), and adenovirus (ADV). Human metapneumovirus antisera for IF (D3DFA, Diagnostic Hybrids, Athens, Ohio, USA) were available from November 2006. IF‐negative NTS samples and all BAL samples were cultured for viruses in monkey kidney cell lines (LLC‐MK2\BGM) and human embryonic lung fibroblasts (MRC5). Cultures were observed twice weekly (days 1–21) for cytopathic effect. IF tests were performed during normal working hours, with results available the same day for samples reaching the laboratory by 2 pm.

Residual fluid from clinical samples was frozen (−30°C). An aliquot was subsequently collected and stored at −80°C until transferred to the Queensland Paediatric Infectious Diseases Laboratory for polymerase chain reaction (PCR) for PIV 1–3; influenza A and B; RSV; human metapneumovirus (12, 13); ADV; rhinoviruses; coronaviruses OC43, 229E, NL63, and HKU1 (14, 15); polyomaviruses WU and KI (16, 17); and human bocavirus (18).

The number of respiratory tract infection (RTI) episodes in HSCT recipients was determined by review of all clinical data. To confirm the validity of the CS score, a maximum possible (MP) score was calculated by reviewing the clinical information obtained on the same day that the CS score was logged. This information was gathered prospectively, with the MP score tallied retrospectively without knowledge of viral test results.

Statistical analysis was performed using SPSS version 15.0 and DAG___Stat (19). Medians were reported with the 25th and 75th interquartile values and compared using the Wilcoxon signed rank test. Chi‐squared tests were used to compare proportions. Performance of the CS score and MP score was calculated using the κ measurement of agreement. Diagnostic accuracy was assessed using sensitivity, specificity, positive, and negative predictive values (NPV) and likelihood ratios with corresponding 95% confidence intervals (95% CI). The CS score was compared with combined IF and PCR testing as the criterion standard, and IF testing with PCR as the criterion standard. Two‐tailed P‐values are reported.

Results

A total of 1181 respiratory specimens (1078 NTS, 100 BAL samples, and 3 sputa) were collected over the 3‐year study period from 377 patients (183 HSCT and 194 general hematology patients). HSCT recipients were predominantly male (123/183, 67%) with a median age at first sampling of 46 years (25th quartile 34, 75th quartile 55). A median of 3 NTS (1, 5) was collected from each HSCT recipient.

The screening protocol was assessed in NTS specimens collected from HSCT patients on the hematology ward (498/1181 samples, 42.2%): 302 (60.6%) from allogeneic recipients, 71 (14.3%) from autologous recipients, and 125 (25.1%) specimens from patients scheduled to receive HSCT within 7 days. A CS score was recorded concurrently with 36.9% (184/498) of samples, of which 87 (17.5%) yielded a score ≥2, 97 (19.5%) yielded a score of 0–1, and 314 (63.1%) had no score recorded (Table 1). Both IF and PCR were performed on 448 of 498 (90%) NTS samples, IF alone on 39 (7.8%), PCR alone on 10 (2%), and neither test on 1 sample. PCR was not performed on 40 (8.0%) samples owing to an insufficient residual sample volume or sample loss before storage.

Table 1.


 Virus detection in nose and throat swabs collected from hematopoietic stem cell transplant patients, stratified by clinical screening scores and presence of respiratory tract infection (RTI) (n=498)

	N samples (n, % positive)
	No score recorded	Score 0–1	Score ≥2	P *
All samples	317 (54, 17.2)	97 (15, 15.5)	87 (25, 28.7)	0.03
All RTI samples	197 (50, 25.4)	53 (12, 22.6)	83 (24, 28.9)	0.7
URTI samples	56 (17, 30.4)	22 (5, 22.7)	19 (6, 31.6)	0.8
LRTI samples	99 (19, 19.2)	26 (6, 23.1)	59 (14, 23.7)	0.8
U&LRTI samples	42 (14, 33.3)	5 (2, 40)	5 (4, 80)	0.1

^* Test of significance is the chi‐squared test.

URTI, upper respiratory tract infection; LRTI, lower respiratory tract infection; U&LRTI, upper and lower respiratory tract infection.

There was moderate agreement between CS and MP scores (Cohen's κ=0.59 [0.48–0.70], P<0.0001), confirming the validity of the CS score. Both scores placed a patient in the same category to trigger RV testing (or not) for 146/184 (79.3%) samples.

A higher proportion of samples from HSCT recipients with a clinical score ≥2 had laboratory confirmation of RVs (25/87, 28.7%) compared with those having scores of 0–1 (15/97, 15.5%) or no score recorded (54/314, 17.2%) (Table 1, P=0.03). RVs were detected twice as often in patients with a score ≥2 as those with a score of 0–1 (odds ratio [OR]=2.15, 95% CI 1.05–4.42, P=0.04). Median CS scores were highest during episodes of LRTI and U&LRTI (Table 2, P=0.003), and in RV‐positive episodes (P=0.02).

Table 2.


 Median clinical screening scores corresponding to samples taken during episodes of respiratory tract infection (RTI) in hematopoietic stem cell transplant recipients on the hematology ward, stratified for grade of RTI, and virus detection (n=136)

	Clinical screen score
	All samples	Virus‐positive RTI samples	Virus‐negative RTI samples	P *
All episodes RTI	2.0 (1.0, 2.0)
URTI	1.0 (1.0, 2.0)			0.003
LRTI	2.0 (1.0, 2.0)
U&LRTI	2.0 (0, 2.0)
All episodes RTI		2.0 (1.0, 2.0)	1.0 (0, 2.0)	0.02
URTI		2.0 (1.0, 2.0)	1.0 (0.75, 2.0)	0.7
LRTI		2.0 (1.0, 2.0)	2.0 (1.0, 2.0)	0.9
U&LRTI		2.0 (1.0, 2.0)	0.0 (0.0, 1.5)	0.2

^* Test of significance is the Wilcoxon signed rank test.

URTI, upper respiratory tract infection; LRTI, lower respiratory tract infection; U&LRTI, upper and lower respiratory tract infection.

The sensitivity and specificity of the CS score in samples from HSCT recipients using a threshold of 2 were 62.5% (45.8–77.3%) and 56.9% (48.4–65.2%), respectively (Table 3). Using a threshold of 1, the corresponding values were 85.0% (70.2–94.3%) and 34.7% (27.0–43.1%). Results were similar in patients sampled during the admission for HSCT. The diagnostic accuracy was comparable during the high influenza/RSV winter season in Australia (June–September) and the remainder of the year (data not shown). The post‐test probability of RV infection in HSCT recipients was calculated for varying prevalence rates (Table 4). Compared with the observed prevalence of 22%, a score ≥2 increased probability of infection to 29% (23–35%), and a score 0–1 reduced this probability to 15% (11–22%). The corresponding post‐test probabilities using a threshold score ≥1 were 26% (23–30%) and 11% (5–20%).

Table 3.


 Diagnostic accuracy of the clinical screening scores compared with combined viral immunofluorescence and polymerase chain reaction results in nose and throat swab samples collected from all hematopoietic stem cell transplant (HSCT) recipients, and during admission HSCT was received

	Laboratory testing		% (95% CI)			% (95% CI)
Threshold	Positive N	Negative N	Sensitivity	Specificity	Prevalence	Likelihood ratio positive test	Likelihood ratio negative test	Positive predictive value	Negative predictive value
HSCT recipients (n=184)					21.7%
Score ≥1	34	94	85.0	34.7		1.30	0.43	26.6	89.3
Score <1	6	50	(70.2–94.3)	(27.0–43.1)		(1.09–1.55)	(0.20–0.93)	(19.2–35.1)	(78.1–96.0)
Score ≥2	25	62	62.5	56.9		1.45	0.66	28.7	84.5
Score <2	15	82	(45.8–77.3)	(48.4–65.2)		(1.07–1.97)	(0.43–1.01)	(19.5–39.4)	(75.8–91.1)
Patients during admission HSCT was received (n=174)					13.5%
Score ≥1	15	72	83.3	37.4		1.33	0.45	17.2	93.5
Score <1	3	43	(58.6–96.4)	(28.6–46.9)		(1.04–1.71)	(0.15–1.29)	(10.0–26.8)	(82.1–98.6)
Score ≥2	10	47	55.6	59.1		1.36	0.75	17.5	89.5
Score <2	8	68	(30.8–78.5)	(49.6–68.2)		(0.85–2.17)	(0.44–1.29)	(8.8–29.9)	(80.3–95.3)

CI, confidence interval.

Table 4.


 Post‐test probability of respiratory virus infection in hematopoietic stem cell transplant recipients using the clinical screening score at different prevalence rates, and number needed to test for a positive result

	% (95% CI)
	Threshold score ≥1				Threshold score ≥2
	≥ 1		0		≥ 2		0–1
Pre‐test probability (prevalence)	Post‐test probability	Number needed to test for positive result	Post‐test probability	Number needed to test for positive result	Post‐test probability	Number needed to test for positive result	Post‐test probability	Number needed to test for positive result
22%	26 (23–30)	3.8 (3.3–4.3)	11 (5–20)	9.1 (5–20)	29 (23–35)	3.4 (2.9–4.3)	15 (11–22)	6 (4.5–9.1)
10%	13 (12–14)	7.7 (7.1–8.3)	5 (3–7)	20 (14.3–33.3)	14 (12–16)	7.1 (6.3–8.3)	7 (5–9)	14 (12–16)
35%	41 (39–43)	2.4 (2.3–2.6)	19 (15–23)	5.3 (4.3–6.7)	44 (41–47)	2.3 (2.1–2.4)	26 (23–29)	3.8 (3.4–4.3)
50%	57 (55–58)	1.8 (1.7–1.8)	30 (25–35)	3.3 (2.9–4)	59 (56–62)	1.7 (1.6–1.8)	40 (37–43)	2.5 (2.3–2.7)

CI, confidence interval.

The diagnostic accuracy of viral testing procedures was calculated from 1035 respiratory samples from all hematology patients tested by both IF and PCR (87.6% of all 1181 samples). The sensitivity and specificity of IF alone compared with combined IF and PCR was 36.2% and 100%, respectively. The sensitivity of IF compared with PCR alone was 22.4% with a specificity of 96.2%. The sensitivity and specificity of IF for each virus compared with PCR alone are shown in Table 5.

Table 5.


 Diagnostic accuracy of immunofluorescence (IF) testing compared with polymerase chain reaction (PCR) by respiratory virus tested, those detectable by IF, and any positive result (all available samples, n=1035)

		PCR		% (95% CI)		% (95% CI)			% (95% CI)
	IF	Positive	Negative	Sensitivity	Specificity	Positive predictive value	Negative predictive value	Prevalence (%)	Likelihood ratio positive	Likelihood ratio negative
Influenza A	Positive	13	11	65.0	98.9	54.2	99.3%	1.9	60	0.35
	Negative	7	1004	(40.8–84.6)	(98.1–99.5)	(32.8–74.5)	(98.6–99.7)		(31–117)	(0.19–0.64)
Influenza B	Positive	2	2	40.0	99.8	50.0	99.7	0.5	206	0.60
	Negative	3	1028	(5.3–85.3)	(99.3–99.9)	(6.8–93.2)	(99.2–99.9)		(36–1188)	(0.29–1.23)
RSV	Positive	6	18	75.0	98.3	25.0	99.8	0.8	43	0.25
	Negative	2	1009	(34.9–96.8)	(97.2–99.0)	(9.8–46.7)	(99.3–99.9)		(23–79)	(0.08–0.85)
Adenovirus	Positive	0	0	NA	99.0	NA	NA	0.8	43	0.25
	Negative	10	1025		(98.2–99.5)				NA	NA
PIV1	Positive	7	0	46.7	NA	99.2	NA	1.4	Infinity	0.53
	Negative	8	1020	(21.3–73.4)		(98.5–99.7)			57–infinity	0.34–0.84
PIV3	Positive	2	1	20.0	99.9	66.7	99.2	1.0	205	0.80
	Negative	8	1024	(2.5–55.6)	(99.5–99.9)	(9.4–99.2)	(98.5–99.7)		(20–2082)	(0.59–1.09)
hMPV	Positive	4	0	NA	NA	NA	NA	0.4	Infinity	0.00
	Negative	0	1031						114–infinity	0.01–1.39
Influenza A and B, RSV, PIV, and adenovirus (viruses detectable by IF)	PositiveNegative	3138	35931	44.9(32.9–57.4)	96.4(95.0–97.5)	47.0(34.6–60.0)	96.1(94.7–97.2)	6.7	12(8.17–19)	0.57(0.46–0.71)
Detection of any respiratory virus	PositiveNegative	35120	31849	22.6(16.3–30.0)	96.5(95.0–97.6)	53.0(40.3–65.4)	87.6(85.4–89.6)	15.0	6.41(4.08–10)	0.80(0.74–0.87)

CI, confidence interval; RSV, respiratory syncytial virus; PIV, parainfluenza virus; hMPV, human metapneumovirus; NA, not applicable.

One or more RVs were detected either by IF or PCR in 205 (17.2%) samples collected from 60 HSCT and 35 general hematology patients. IF gave a positive result in 8.5% (59/698) and PCR in 18.0% (115/640) samples from HSCT recipients (OR 2.37, 95% CI 1.70–3.32, P<0.0001). A total of 107 RV infections were detected: 84 episodes in HSCT recipients (including 12 with >1 RV) and 23 episodes in general hematology patients (including 3 with 2 RVs). Nosocomial acquisition, with onset after 4 or more days of hospitalization, occurred in 24 of 84 (28.6%) episodes in HSCT recipients, with an incidence of 5.8% (24/412 admissions). Seven episodes in HSCT recipients were fatal, with LRTI the principle cause of death in 5 episodes. All deaths from LRTI followed nosocomial RV infection.

NTS collected during episodes of URTI were more likely to have RV detected by IF or PCR than those during LRTI episodes (41/117, 35%, and 46/194, 23.7%, respectively, P=0.03, OR 1.48, 95% CI 1.04–2.10). During episodes of LRTI, the proportion of samples with RV detected was similar for NTS and BAL specimens (46/194, 23.7%, and 11/40, 27.5%, respectively, P=0.6).

Following the introduction of the CS tool, collection of NTS from all hematology patients increased 4‐fold (293 in years 2002–2004, 1220 in years 2005–2007, P<0.0001).

Discussion

This is the first study to our knowledge to confirm the validity and value of using a simple clinical score to trigger microbiological testing for RV infections in hematology or HSCT patients. Daily CS of HSCT recipients for symptoms or signs of viral URTI or LRTI has been recommended previously (6), but guidelines for when sampling and testing should be performed have not been developed.

The CS tool was shown to be the most useful at detecting patients without an RV infection, and limited in its ability to diagnose those with such an infection. Such a tool is beneficial to stratify an approach to testing for readily transmissible organisms in a highly vulnerable patient setting. It performed best at excluding RV infection with a threshold score of 1 symptom or sign at the given prevalences with an NPV of 89% (78–96%). Fewer samples were collected at a threshold of 1, and verification bias may have skewed these results. It should be noted that the screening tool quantified the number of characteristics, rather than gave weighting to specific symptoms or signs, in order to detect a variety of RVs that may have different clinical features and severity. Weighting of specific clinical features may improve the performance of the CS tool; this could not be further assessed in this study.

The likelihood ratios for scores above and below the screening thresholds corresponded with small, but potentially important, changes in the post‐test probability of infection (20). The CS tool did not exclude infection when prevalence rates were high. In an outbreak setting with a potential prevalence of 50%, testing and infection control measures are warranted in all patients.

This study, which was designed to evaluate the CS tool in a working clinical context, revealed a significant limitation, namely, the low compliance observed with the screening tool. A score was recorded with only 36% of all respiratory samples collected and in half of these the score exceeded 2. Although the majority of all samples were taken without a logged score, NTS sampling increased 4‐fold following the introduction of the screening tool, with RV detection in 25% of samples without a logged score during episodes of RTI. This is comparable with viral detection from samples scoring ≥2 and may suggest that CS was performed but scores were not recorded.

For practical reasons, paired NTS was chosen as the sample for diagnostic testing rather than nasopharyngeal aspiration (NPA). While NPA obtains the highest viral burden from the URT (21), paired NTS have a sensitivity equivalent to NPA in children (22). While this may not be the case in adults, owing to lower viral loads (23), sensitive molecular methods were used to optimize RV detection. NTS cause less pain than NPA in children (24) and adults (25, 26). NTS were used in this study because of their greater tolerability in adults (especially HSCT recipients who may have complicating factors such as mucositis), particularly as multiple specimens were collected from each patient. Additional benefits of NTS include greater ease of collection by nurses, and consequent earlier collection than with NPA, which requires additional equipment and expertise (24). RVs were detected in a greater proportion of NTS during episodes of URTI than LRTI. As URT specimens are less sensitive for RV detection than BAL during LRTI (27), it remains essential to consider BAL in the absence of a positive RV in this setting (7).

Based on our data, molecular methods of identification should be included in a diagnostic algorithm to detect clinically important RVs in addition to IF and viral culture. We noted 23% of PCR‐positive RVs were also positive by IF. While PCR can detect 15 individual viruses and IF only 7 of these, IF was positive for only 45% of the 7 viruses positive by PCR. The superior performance of PCR for these 7 viruses is well known, with sensitivity of IF in symptomatic children of 63–70% (28). IF for ADV is known to be insensitive (15–30% in symptomatic children [28]), and 0 of 10 PCR‐positive samples in the current study were detected by IF. In adult HSCT recipients, the poor sensitivity of IF compared with combined IF, PCR, and viral culture in nasal wash specimens has been documented previously for influenza (1/3 samples), RSV (3/6), and PIV (2/18) (29). We have confirmed these results with a larger sample and paired NTS specimens. In addition, more than half of all positive specimens were viruses not detectable by IF kits, showing the importance of broadening the panel for RV testing. While multiplex real‐time PCR for 13 RVs is now available at a relatively low cost (30), its use is dependent on laboratory resources and practicalities.

The diagnostic accuracy of a clinical scoring system and viral testing procedures as implemented in this study can guide infection control recommendations for RV infection in HSCT and general hematology units. We recommend that with a score of 1 or more, infection control measures should be implemented and timely, sensitive, and specific laboratory testing undertaken, including molecular testing. Although IF testing is potentially rapid (within a working day), it is insensitive, hence infection control measures should be maintained pending results of PCR. However, the rapid turnaround time of IF and point‐of‐care testing remains important to guide treatment for influenza and RSV.

In summary, a simple CS tool for RV infection was developed and implemented in an HSCT and general hematology ward. The tool was most effective at excluding RV infection using a threshold of 1 clinical symptom or sign, and we recommend that this should trigger sampling for viral testing. We recommend that PCR be included with point‐of‐care testing, IF, and viral culture for comprehensive virological surveillance.