Diagnostic performance and clinical feasibility of a point-of-care test for respiratory viral infections in primary health care

Abstract

Background

Inappropriately high levels of antibiotics are still prescribed in primary health care for respiratory tract infections (RTIs). Access to diagnostic point-of-care tests (POCTs) for RTIs might reduce this over-prescription.

Objective

The purpose of our study was to determine the diagnostic performance and clinical feasibility of a recently developed diagnostic POCT for respiratory viruses, the mariPOC^®, in a Dutch primary healthcare setting.

Methods

In patients with RTI symptoms presenting to a family practice during the 2015–2016 winter season, we determined the test’s sensitivity and specificity relative to polymerase chain reaction (PCR) testing performed in a laboratory. The clinical feasibility of the POCT was evaluated by interviewing general practitioners (GPs).

Results

One or more respiratory viruses were detected in 54.9% of the patients (n = 204). For influenza A virus (n = 24), sensitivity of the POCT was 54.2% and specificity was 98.9%; for influenza B virus (n = 18), sensitivity was 72.2% and specificity 99.5%; and for respiratory syncytial virus (RSV) (n = 12), sensitivity was 50.0% and specificity 100%. In samples with higher viral load, sensitivity was 85.7% for influenza A, 78.6% for influenza B and 85.7% for RSV. The availability of a diagnostic test for respiratory viruses was appreciated by both patients and GPs.

Conclusions

Our study shows that diagnostic POCTs for respiratory viruses might contribute to a precise and evidence-based diagnosis of RTIs and could positively influence prescription of antibiotics by GPs. However, before implementation in primary healthcare, diagnostic accuracy of the POCT needs improvement and it is impact on clinical decision making should be further assessed.

Introduction

Inappropriate use of antibiotics contributes to antimicrobial resistance, increases health care costs and exposes patients to unnecessary risks of adverse drug events (1). The vast majority of antibiotics are prescribed in primary care (2–4). Respiratory tract infections (RTIs) are a frequent indication for prescription although many clinical guidelines recommend restrictive use of antibiotics as RTIs are mainly of viral origin. Despite interventions to reduce overuse of antibiotics, such as the introduction of clinical guidelines, C-reactive protein (CRP) testing and patient and doctor education, inappropriately high levels of antibiotics are still being prescribed (5–7).

Access to point-of-care tests (POCTs) to guide prescription might reduce over-prescription of antibiotics in primary care. POCTs are rapid, easy-to-use tests carried out near the patient by non- laboratory-trained personnel. A benefit of POCTs is their use in helping physicians to manage patients’ expectations for antibiotics and to encourage patients to self-care when suffering from a self-limiting condition (8). At the moment, general practitioners (GPs) in many countries only have access to POCTs for nonspecific biomarkers, such as CRP and cell count. Although these tests have been demonstrated to have an effect on antibiotic prescriptions, recent studies suggest that testing for both biomarkers and for viruses might further reduce these prescription rates (9,10).

Therefore, new POCTs that can accurately detect pathogens associated with an RTI are required (11). To date, the mariPOC^® (produced by ArcDia International Oy Ltd in Turku, Finland) is the only multi-analyte rapid diagnostic test available that is suitable as a point-of-care assay in primary care. While this POCT has been previously evaluated (12,13), it has not yet been tested as a near-patient assay in a family practice. The purpose of our study was therefore to determine the diagnostic performance and clinical feasibility of this POCT in a Dutch primary healthcare setting.

Our research questions were as follows: How does mariPOC^® perform in detecting respiratory viruses in patients with RTI symptoms? Using mariPOC^® as a POCT, are GPs and their patients satisfied with the performance and ease-of-use of the test?

In order to answer these questions, we prospectively analysed the clinical characteristics of all patients, both children and adults, who presented with RTI symptoms to GPs at a single practice during the 2015–2016 winter season. We determined whether or not patients had a respiratory virus using both the POCT and a reference test, the laboratory operated multiplexed polymerase chain reaction (PCR) test. We calculated sensitivity, specificity, positive and negative predictive value (PPV, NPV) of the POCT and interviewed doctors about the added value of this POCT in primary health care.

Methods

Study population and sample collection

This prospective study was conducted in a family practice in the neighbourhood of the Academic Medical Center (AMC), Amsterdam, The Netherlands, from November 2015 until March 2016. This family practice serves approximately 10000 patients and employs seven GPs. The family practice is one of the larger practices in the Netherlands and has been cooperating as a partner within our scientific network for more than fifteen years allowing optimal patient inclusion and study logistics.

All patients of any age with any underlying illness or medical condition presenting to the GP with RTI symptoms during regular opening hours of the family practice were eligible for inclusion. Patients needed to have at least two respiratory symptoms, e.g. cough, rhinorrhoea, headache, myalgia, wheeze, or fever (defined for purposes of this study as >37.5°C measured by ear thermometer), for less than 7 days.

After giving written informed consent, patients were asked to fill in a questionnaire about their demographical and clinical background and symptoms, and to undergo posterior nasopharyngeal swab sampling. The swab was immediately tested at the primary health care practice using the POCT by a member of the study team. Prior to the start of the study, study team members had received 1 hour of training from the manufacturer in how to use the test. The remaining sample solution was transferred on the same day to the Laboratory of Clinical Virology at the AMC for reference testing with a multiplex PCR (14). PCR is considered the gold standard for respiratory virus detection (15).

The study was approved by the Medical Ethical Committee of the AMC. Informed consent was obtained from patients or their parents or caregivers before enrolment, with children providing assent if age appropriate.

mariPOC^® test system

The mariPOC^® is an automated, point-of-care compatible antigen test for the detection of nine respiratory viruses (influenza virus types A and B; respiratory syncytial virus (RSV); parainfluenza virus (PIV) types 1, 2 and 3; human metapneumovirus (hMPV); human bocavirus (HBoV); and adenovirus), and Streptococcus pneumoniae in a single nasopharyngeal sample (ArcDia International Ltd, Turku, Finland). Detection of antigens is based on separation-free two photon excitation fluorometry. The signal response in this technique is directly proportional to the analyte concentration in the sample (16). After adding sample buffer to the swab sample, the sample can be inserted into the analyser. Preliminary results are automatically reported after 20 minutes for medium and high positive samples. After 2 hours, the final results are automatically displayed on the computer. New samples can be inserted for analysis any given time. In our study, the final mariPOC^® results were compared with the reference method. S. pneumoniae results were not included in this study.

Reference method for detection of respiratory viruses

A previously described multiplex PCR (14) was used to test all respiratory samples for the presence of respiratory viruses (influenza virus types A and B; RSV; PIV types 1, 2, 3 and 4; hMPV; HBoV; adenovirus; rhinovirus; human coronavirus (HCoV); enterovirus (EV); and human parechovirus (HPeV)). A threshold cycle (Ct) value of 40 or more was considered to be negative.

Evaluation of the POCT in primary health care

Study participants were asked their opinion on the availability of a rapid test for respiratory viruses in primary care on a scale with the following four options: ‘useless, no opinion, valuable, very valuable’. After the end of the study period the GPs were interviewed on the feasibility of this POCT in primary care. This structured interview was based on a standardized questionnaire covering the following topics: advantages and disadvantages of the POCT, additional value of the POCT in family practice, influence of the POCT on: prescription of antiviral medication, antibiotics, additional testing and patient satisfaction.

Sample size estimation

To calculate the minimal number of patients needed for inclusion, we performed a sample size calculation. We assumed in this sample size calculation that sensitivity of the test was at least 85% for the two most important viruses, influenza and RSV. We decided, because of the limited information available and the different population in which the test was previously evaluated, to choose a precision (expressed as the maximum width of the 95% confidence interval for sensitivity) of 0.20. With an estimated prevalence of 25% for influenza and for RSV at least 196 patients had to be included.

Data analysis

Data were analysed using SPSS statistical software (version 22.0, SPPS Inc, Chicago, IL). Sensitivities, specificities, and positive and negative predictive values with 95% confidence intervals (95% CI) were calculated for each virus separately. Readers of both the POCT and the reference standard were blind to the results of the other test and clinical information was not available for assessors of the reference standard. A two-sided p-value <0.05 was considered statistically significant. For data analysis, all infections were considered statistically independent.

Results

Patient characteristics

From 11 November 2015 through 30 March 2016, a total of 204 patients were enrolled in the study. Patient characteristics are summarized in Table 1. The median age of the patients was 33 years (interquartile range (IQR) 13.8–55 years; range 0–83 years) and 78 were younger than 18 years of age (38.2%); 123 were female (60.3%). Sixty-four patients (31.4%) reported an underlying condition, the most frequent being an illness of the respiratory tract, e.g. asthma or COPD. The vaccination status was available for 200 patients, of whom 41 had received the annual influenza vaccination. Cough was the most frequently reported symptom, followed by rhinorrhoea and headache.

Table 1.

Patient characteristics of 204 patients with respiratory tract infection symptoms visiting the family practice from November 2015 until March 2016

Characteristic	Number of patients, n (%)
Median age in years (IQR)	33, 13.8–55 years
Gender
Male	81 (39.7)
Female	123 (60.3)
Underlying condition	64 (31.4)
Respiratory	19 (29.7)
Cardiac	13 (20.3)
Influenza vaccine received
Yes	41 (20.1)
No	159 (77.9)
Unknown	4 (2.0)
Clinical symptoms
Cough	177 (86.8)
Rhinorrhea	168 (82.4)
Headache	120 (58.8)
Wheezing	83 (40.7)
Fever (>37.5°C)	68 (33.3)
Other, of which sore throat	64 (31.4)

Respiratory viruses detected in samples

In 54.9% of the samples one or more respiratory viruses were detected by PCR. By POCT one or more respiratory viruses were detected in 18.6% of the samples. The distribution of all respiratory viruses detected by PCR and POCT is shown in Table 2. A total of two samples were positive for two viruses by POCT and eight samples were positive for two viruses by PCR. None of the samples were positive for more than two viruses.

Table 2.

Respiratory viruses detected by PCR and POCT

Virus	Number of samples positive by PCR, n (%)	Number of samples positive by POCT, n (%)
RV	25 (12.3)	n/a
INFA	25 (12.3)	15 (7.4)
HCoV	22 (10.8)	n/a
INFB	18 (8.8)	14 (6.9)
RSV	12 (5.9)	6 (2.9)
hMPV	7 (3.4)	1 (0.5)
HBoV	3 (1.5)	1 (0.5)
ADV	2 (1.0)	0 (0.0)
PIV-1	2 (1.0)	1 (0.5)
PIV-2	2 (1.0)	2 (1.0)
PIV-4	1 (0.5)	n/a
EV	1 (0.5)	n/a
Total	112 (54.9)	38 (18.6)

Respiratory viruses included in the PCR, but not in the POCT are shown in italic.

ADV, adenovirus; EV, enterovirus; HBoV, human bocavirus; HCoV, human coronavirus; hMPV, human metapneumovirus; INFA, influenza A virus; INFB, influenza B virus; n/a, not applicable; PIV, parainfluenzavirus; RSV, respiratory syncytial virus; RV, rhinovirus.

Diagnostic performance of the mariPOC^® test

A total of 202 samples were available for evaluation of the diagnostic performance of the POCT as two samples were excluded due to an invalid POCT analysis. Overall, the POCT had a sensitivity of 47.1% (95% CI 35.2–59.4), a specificity of 99.7% (95% CI 99.2–99.9), a PPV of 84.6% (95% CI 68.8–93.6), and a NPV of 97.9% (95% CI 97.1–98.5) for the panel of 9 viruses that it tests for. Sensitivities, specificities, and positive and negative predictive values for the three most frequently detected viruses are shown in Table 3. For influenza A virus (n = 24), sensitivity of the POCT was 54.2% (95% CI 33.2–73.8) and specificity was 98.9% (95% CI 95.6–99.8); for influenza B virus (n = 18), sensitivity was 72.2% (95% CI 46.4–89.3) and specificity 99.5% (95% CI 96.5–100); and for RSV (n = 12), sensitivity was 50.0% (95% CI 22.3–77.7) and specificity 100% (95% CI 97.5–100). Due to the small number of infections with PIV types 1–3, hMPV, HBoV and ADV it was not possible to determine the sensitivity of the POCT for these viruses. Specificity calculations for these viruses resulted in a specificity of 99.0% (95% CI 96.0–99.8), 100% (95% CI 97.6–100), 99.5% (95% CI 96.8–100) and 100% (95% CI 97.7–100) for PIV 1–3, hMPV, HBoV and ADV, respectively.

Table 3.

Sensitivities, specificities, and predictive values of mariPOC^® compared to PCR

Virus	mariPOC®	RT-PCR		% (95% confidence interval)
		Positive	Negative	Sensitivity	Specificity	PPV	NPV
INFA	Positive	13	2	54.2 (33.2–73.8)	98.9 (95.6–99.8)	86.7 (58.4–97.7)	94.1 (89.4–96.9)
	Negative	11	176
INFB	Positive	13	1	72.2 (46.4–89.3)	99.5 (96.5–100)	92.9 (64.2–99.6)	97.3 (93.6–99.0)
	Negative	5	183
INFA or INFB	Positive	26	3	61.9 (45.7–76.0)	99.2 (97.4–99.8)	89.7 (71.5–97.3)	95.7 (93.0–97.5)
	Negative	16	359
RSV	Positive	6	0	50.0 (22.3–77.7)	100 (97.5–100)	100 (51.7–100)	96.9 (93.1–98.7)
	Negative	6	190

The PCR technique generates results in terms of cycle threshold (Ct) values, which is a well-established semi-quantitative estimation of viral load (17), with lower Ct values representing higher amounts of virus. We examined whether the accuracy of the POCT depended on the viral load, i.e. whether inclusion of only those samples with a high viral load (defined as having a Ct value <30) could improve sensitivity. Indeed, the sensitivity of the test improved resulting in sensitivities for influenza A virus, influenza B virus and RSV of respectively 85.7% (95% CI 42.0–99.2), 78.6% (95% CI 48.8–94.3) and 85.7% (95% CI 42.0–99.2).

Clinical feasibility

All patients were asked their opinion on the availability of a rapid diagnostic test for respiratory viruses when visiting the GP with respiratory symptoms. Results were available for 202 patients. Most patients (n = 151, 74.8%) were positive about the test, with 56.4% (n = 114) considering it to be a valuable addition for primary care and 18.3% (n = 37) considering it a very valuable addition. In cases where a respiratory virus was detected by the POCT, some patients reported being more confident about the self-limiting aspect of the illness. During the interviews, five out of 7 GPs considered the POCT helpful as they now had a tool to convince patients about the viral diagnosis. Five thought it would have influence on their antibiotic prescription behaviour, not only during regular opening hours, but also at the out-of-hours department. According to the GPs, referral to a hospital would not be influenced by this POCT as this is often based on clinical symptoms and not on etiological diagnosis. All GPs thought implementing this POCT in primary health care would increase patient satisfaction, which might contribute to patients’ expectations and self-management, possibly influencing inappropriate referrals, a decrease in additional testing and extra visits. Disadvantages of the POCT are the extra costs and the time interval before final test results are reported, which requires a change in work flow. According to the personnel who performed the POCT, the test was rapid and easy-to-use.

Discussion

In our study we evaluated for the first time the diagnostic performance and clinical feasibility of an automated and rapid test for respiratory viruses at the point-of-care in a primary health care setting in the Netherlands. The results of our study suggest that in the setting of Dutch family practice, the POCT is specific for detecting respiratory viruses in patients with respiratory tract symptoms, but sensitivity is lower than expected.

The fact that one or more respiratory viruses were detected by PCR in 54.9% of the patients in our study indicates that more than half of patients presenting to their GP with respiratory tract symptoms indeed have a viral infection. Our finding that rhinovirus and influenza A virus were the most frequently detected viruses is in line with the epidemiology of respiratory viruses in primary care reported in other studies (18,19).

In terms of the diagnostic performance of the POCT, here we report sensitivities for influenza A virus, influenza B virus, and RSV that are lower than those reported in previous studies that evaluated this POCT (12,13). This is likely due to the patients in our sample having a lower viral load than the patients in the other studies, a factor that—as demonstrated here—reduces the sensitivity of the POCT. Three main factors might have influenced the lower viral load in most of our samples and thus the lower sensitivity. Firstly, our study population consisted mainly of adults, while other studies of this POCT were mainly conducted in children. Children often have higher viral loads which can explain better results in antigen detection tests (20). Secondly, while the previous studies were done in patients who were hospitalized or presented at an outpatient department, here we evaluated the POCT in primary care where patients tend to have less severe illness than those in hospital. Some studies suggest that disease severity correlates positively with viral load (21). Thirdly, many patients in our study visited the GP almost a week after symptoms had started (data not shown). Antigen tests are designed to detect pathogens in the acute stage of the infection, during the first 3–4 days after symptom onset, since this is when viral load is highest (22).

Despite the sensitivity of the mariPOC^® test being lower than that found in the previous studies and compared to PCR, we have demonstrated that a substantial proportion of patients with a virus infection can be diagnosed correctly within two hours. The fact that specificity of the POCT ranged from 98.9–100% means that in almost all cases a positive test result correctly rules in a respiratory viral infection. However, correct identification of a viral infection does not rule out a bacterial infection. A limitation of our study is that it did not address the detection of concurrent infections with bacterial pathogens. The diagnosis of bacterial RTIs is notoriously difficult as the presence of respiratory bacteria in nasopharyngeal swabs makes it difficult for a test to distinguish between infection and colonization (23).

Several other limitations of our study need to be addressed. The small number of positive findings for certain viruses (i.e. hMPV, HBoV, PIV types 1–3 and adenovirus) meant we could not calculate the sensitivity of the POCT for each virus included in the panel, although we included more patients than the number suggested by our initial sample size calculation (see Methods). In our study, respiratory virus prevalence was unfortunately lower explaining the wide confidence intervals in the diagnostic accuracy calculations.

Although mariPOC^® is the only multianalyte rapid test available that is suitable as a point-of-care assay for RTIs in primary care, the POCT would have had added value if the viral panel that it tests also included rhinoviruses, the most frequently detected virus. Another limitation might be a discrepancy between the number of eligible patients and the number of included patients. The research team continuously motivated GPs to refer patients for study participation, but some eligible patients might not have been recruited. However, compared to other primary care studies, the number of patients that we included was quite high, probably because of the fact that this study was performed in only one family practice and during just one respiratory season.

Our study shows that both GPs and patients consider diagnostic POCTs for respiratory viruses to be a valuable contribution to primary care. The implementation of such POCTs could help in confirming the presence of a viral infection and positively influence the rate of antibiotic prescription by GPs, in the Netherlands, but possibly also in other countries. This is in agreement with the results of an international survey among GPs which indicated an unmet clinical need for a more widely accessible range of POCTs to assist clinicians’ decisions, especially for acute conditions (24). A first explorative interview amongst the GPs revealed positive feedback on the usefulness of the POCT, but a more appropriate qualitative survey needs to increase insight in the potential additional value of POCTs in primary health care. Nevertheless, GPs should consider use of the POCT with care as the diagnostic performance of antigen tests depends on the viral load of the sample, which is in turn influenced by the severity of the illness and duration of symptoms. We have shown with the diagnostic test sensitivities estimated in our study that false-negative results are common. For clinical decision making, the test result should therefore always be interpreted in the light of the patient’s medical history and physical exam.

Our study paves the road to assess new diagnostic opportunities for RTIs in primary health care. To confirm our findings, assess the effect of the POCT on clinical decision making and determine whether using this POCT might positively affect antibiotic prescription rates additional studies are needed, preferably in the form of randomized controlled trials. Besides, it is important to note that multifaceted interventions to reduce overuse of antibiotics are more effective than single initiatives (25). We therefore emphasize that strategies to guide antibiotic prescription should be combined and we encourage the development of improved POCTs, preferably diagnostic POCTs that can detect both viruses and bacteria in combination with biomarkers such as CRP.

Conclusion

In summary, diagnostic POCTs for respiratory viruses might contribute to a precise and evidence-based diagnosis of RTIs. In this prospective study we determined the diagnostic performance and clinical feasibility of the mariPOC^®, a POCT for the detection of respiratory viruses, in a Dutch family practice. Although the availability of a POCT was appreciated by both patients and GPs, sensitivity of the test was lower than expected. Before implementation of diagnostic POCTs in primary health care, diagnostic accuracy of the POCT need to improve and the impact of POCTs on clinical decision making should be further assessed.

Declaration

Funding: the Seventh Framework Programme of the European Union Marie Curie IAPP under contract PIAPP-GA-2013–612308. ArcDia International Oy Ltd kindly provided the mariPOC^® test system and consumables.

Ethical approval: the Medical Ethical Committee of the AMC.

Conflict of interest: Academic Medical Center Amsterdam and ArcDia International Oy Ltd are both part of AIROPico, a Seventh Framework Programme of the European Union.