Perceptions on Point-of-Care Tests for Sexually Transmitted Infections – Comparison between Frontline Clinicians and Professionals in Industry

Abstract

Objectives

To determine if a gap exists between sexually transmitted infection (STI) clinicians and industry professionals regarding perceptions of the ideal types and characteristics of STI point-of-care tests (POCTs).

Methods

Our online survey design contained sections on demographics; barriers of use for available STI POCTs; characteristics of an ideal POCT, including prioritizing pathogens for targets; and “building your own POCT”. Practicing clinicians and academic experts from two venues, STI-related international conference attendees and U.S. STD clinic clinicians, were invited to participate in the clinician survey. Professionals from industry in the STI diagnostic field were invited to participate in the industry survey. Chi-square test and conditional logistical regression were used for data analysis.

Results

Clinician survey participants (n=218) identified “the time frame required” (39.9%), “complexity” (31.2%), and “interruption of work flow” (30.3%) as the top three barriers making it difficult to use STI POCTs, while the industry survey participants (n=107) identified “complexity” (65.4%), “unreliability” (53.3%), and “difficulty in reading results” (34.6%) as the top three barriers (all p values <0.05). Sensitivity was always the most important attribute to be considered for a new STI POCT by both participant groups. Participants of the clinician group chose cost as the second priority attribute, while those of the industry group chose specificity as the second priority.

Conclusion

We identified differences in the perceptions regarding barriers and ideal attributes for STI POCTs between frontline clinical providers and industry personnel. Tailored training is warranted to inform scientists, biomedical engineers, and other industry experts about characteristics that clinicians desire for STI POCTs.

Introduction

Even though the World Health Organization has identified benchmarks for an ideal sexually transmitted infection (STI) point-of-care test (POCT), many currently developed or available STI POCTs are not accurate and/or are not feasible for use in clinical settings.^{1, 2} This level of unsatisfactory performance results in limited or nonuse of STI POCTs by end users. Consequently, this discourages further investment and development of STI POCTs by industry. Unsatisfactory performance is, in part, due to what appears to be a gap between clinicians’ (users) and industry’s view of what makes an ideal STI POCT. Therefore, we conducted a study to determine if such a gap exists. We surveyed STI clinicians/academic experts and industry professionals regarding perceptions of the ideal types and characteristics of STI POCTs.

Materials and Methods

The online survey based on a large-scale focus group study among STI professionals³ was conducted in two groups of STI professionals. The online survey questionnaire collected information regarding demographics, including gender, country of practice, and profession; barriers to use for currently available STI POCTs; characteristics of an ideal POCT; prioritization of pathogens for POCT development; and “building your own POCT” - preference of POCT for STI(s) with different levels of sensitivity (70–79%, 80–89%, ≥ 90%), specificity (90%, 95%, 99%), turn-around time (5 minutes, 15 minutes, 25 minutes), and cost ($20, $35, $50). In order to identify preferred STI POCT attributes among hypothetical prospective POCTs, we utilized a methodology Choice Experiment,⁴ which is frequently used in the field of Economics, in the “building your own POCT”. We randomly created 16 choice questions each of which contained a pair of STI POCTs with different sets of attributes described above. The participants were asked to select their preferred POCT from a pair of POCTs in each choice question.

Practicing clinicians and academic experts from three venues, STI-related international conference attendees, an international conference for obstetrician-gynecologists and U.S. STD clinic clinicians, were invited to participate in the clinician survey from June 2009 through August 2009. Managers of all U.S. STD clinics (n= ~ 700) throughout the ten federally funded regions were contacted and all of their clinicians were invited to participate in the online survey. Participants from two conference venues, the 18^th Meeting of the International Society for Sexually Transmitted Diseases Research (ISSTDR) and the 39th Annual Scientific Meeting of Infectious Disease Society for Obstetrics and Gynecology (IDSOG), were recruited via in-person outreach, flyers, and pamphlets among attendees. Subjects were excluded from the study if they were not practicing clinicians or ISSTDR and IDSOG conference attendees. The results of the clinician survey have been published elsewhere.⁵ Professionals from industry in the STI diagnostic field, including biomedical researchers and business professionals, were invited via emails, or in-person outreach, flyers, and pamphlets at biomedical-related conferences to participate in the industry survey from June 2010 through November 2010 which used the same format as the clinician survey. Survey respondents who were not professionals from industry in the STI diagnostic field were excluded.

Descriptive analyses were performed followed by chi-square tests in order to compare the responses between the results of the clinician survey and those of the industry survey. Choice modeling, a type of conditional logistic regression modeling, was employed to determine the probability of individuals making a particular choice from presented options⁴ for the “build your own POCT” section (SAS version 9.2 and JMP version 8, SAS Institute Inc., Cary, NC). Subgroup analyses of choice modeling were performed for each of the top two prioritized pathogens chosen for STI POCT development by the participants as well as subgroup of profession (MD clinicians and non-MD professionals for clinician survey; industry business professionals and industry biomedical professionals for industry survey). All p values were 2-sided, with p<0.05 considered to be significant.

Results

Overall, 256 subjects participated in clinician survey and 218 completed the survey. In the industry survey, 149 subjects took the online survey and 107 completed the survey. The demographic characteristics of participants who completed the survey are summarized in Table 1. The majority of clinician survey participants were female while the majority of industry survey participants were male. The majority of participants were from U.S. (clinician: 78%; industry: 86%).

Table 1.

Demographic Characteristics of Survey Participants.

Characteristics	Categories	Clinician Survey N=218	Industry Survey N=107
Gender	Male	48	67
	Female	170	40
Country	United States	169	92
	Other	49	15
Profession a	Medical Doctor (MD)	67	NA b
	Non-MD Professional	151	NA
	Business Professional	NA	35
	Biomedical Professional	NA	72

^aNon-Medical Doctor (MD) professional included nurses, nurse practitioner, physician assistant, academic researchers

Biomedical professional included scientists, biomedical engineers.

^bNA: not applicable

Clinician survey participants (n=218) identified “time frame required”, “complexity”, and “interruption of work flow” as the top three barriers to using currently available STI POCTs; while the industry survey participants (n=107) identified “complexity”, “unreliability”, and “difficulty in reading results” as the top three barriers. Significant differences in barriers named in the two surveys among clinicians and industry included “complexity” (31.2% versus 65.4%, p<0.05), “time frame required” (39.9% versus 27.1%, p<0.05), “laboratory driven” (29.8% versus 19.6%, p<0.05), “difficulty in reading results” (9.6% versus 34.6%, p<0.05), and “unreliability” (23.4% versus 53.3%, p<0.05) (Figure 1). The majority (78.4%) of clinician survey participants perceived that the cost of the test from the manufacturer was a more important economic factor for a health care provider to use an STI POCT than the amount of reimbursement received for performing the test, while only 44.9% of industry survey participants agreed (p<0.05).

Figure 1. Perceived Barriers to Use Sexually Transmitted Infection Point-of-Care Tests (STI POCTs) by Clinician and Industry Professional Participants.

* p<0.05

Participants from both surveys ranked C. trachomatis as the top priority organism chosen for a new POCT (clinician: 62%, industry: 39%, p<0.05), followed by a test that would diagnose early seroconversion for HIV (clinician: 14%, industry: 32%, p<0.05). Ideal attributes for a new STI POCT were perceived differently by the groups. Sensitivity was always the most important attribute to be considered for a new STI POCT by both participant groups. However, participants from the clinician group chose cost as the second priority attribute, while those of the industry group chose specificity as the second priority (Table 2A). This finding mainly came from the significantly different choice between 2 groups in choosing between a test with sensitivity of 90–99%, specificity of 90%, cost of $20, and turn-around-time of 25 minutes and a test with sensitivity of 90–99%, specificity of 99%, cost of $35, and turn-around-time of 5 minutes. Only 5% of industry professional picked the 1^st test, however, 21% of clinicians preferred it (p<0.001).

Table 2.

Comparison of Preferences in Attributes of a New Point-of-Care Test for Sexually Transmitted Infections between Clinician and Industry Survey Participants.

2A: Rank of Attributes
Ranking	Clinician Survey	Industry Survey
A. Overall	n=218	n=107
1st	Sensitivity	Sensitivity
2nd	Cost a	Specificity a
3rd	Specificity a	Time a
4th	Time a	Cost a
B. Chlamydia as Priority	n=136	n=42
1st	Sensitivity	Sensitivity
2nd	Cost a	Specificity a
3rd	Specificity a	Cost a
4th	Time	Time
C. HIV Seroconversion as Priority	n=30	n=34
1st	Sensitivity	Sensitivity
2nd	Specificity	Specificity
3rd	Cost a	Cost a
4th	Time a	Time a

2B: Preferred Level of Value of Attributes b
Attributes	Clinician Survey	Industry Survey
A. Overall	n=218	n=107
Sensitivity	90–99%	90–99%
Specificity	99%	99%
Cost	$20 c	$35 c
Time	5 min	5 min
B. Chlamydia as Priority	n=136	n=42
Sensitivity	90–99%	90–99%
Specificity	99% c	95% c
Cost	$20 c	$35 c
Time	5 min c	15 min c
C. HIV Seroconversion as Priority	n=30	n=34
Sensitivity	90–99%	90–99%
Specificity	99%	99%
Cost	$35	$35
Time	15 min c	25 min c

^aDifference in ranking of attributes between clinicians and industry participants

^bValues of attributes investigated in this study included 3 levels of sensitivity (70–79%, 80–89%, $90%), 3 levels of specificity (90%, 95%, 99%), 3 levels of turn-around time (5 minutes, 15 minutes, 25 minutes), and 3 levels of cost ($20, $35, $50).

^cDifference in preferred level of value of attributes between clinicians and industry participants

For those ranking chlamydia as the top priority for STI POCT development, clinician survey participants preferred the one with highest specificity (99%), lowest cost ($20), and the fastest turnaround time (5 min), while industry survey participants chose a specificity of 95%, a cost of $35, and 15 minutes of turnaround time (Table 2B). Further subgroup analysis on profession group, cost was ranked differently by subgroup of profession. Non-MD professionals ranked cost as second preferred attribute after sensitivity, MD and biomedical professionals ranked cost as third, and business professionals ranked cost last. In addition, non-MD professionals preferred cost of $20 for an ideal STI POCT while the rest preferred $35. Noticeably, business professionals preferred a turn-around time of 15 minutes while the rest preferred a shorter turn-around time, 5 minutes.

Discussion

Significant differences in perceptions of barriers to use of currently available STI POCTs between frontline clinicians and industry professionals exist. This disagreement further extends to the perception of an ideal POCT in terms of preferred qualities, even though they agreed that test performance, cost, and turn-around time were important attributes. Huppert et al. have reviewed all currently available STI POCTs and found that POCTs for chlamydia and gonorrhea currently on the market for clinical use received poor scores in the authors’ scoring system according to WHO benchmark criteria for STI POCTs.¹ The availability of unsatisfactory STI POCT products on the market may increase the probability of clinician’s distrust of STI POCTs and further discourage the use of POCTs as diagnostics for patients. At the same time, industry may be discouraged by the low acceptance of STI POCTs, leading to less investment for development of new and better STI POCTs. This cycle could hamper overall development and use of STI POCTs, thereby lessening adequate point of care testing and immediate treatment efforts required for curbing STI transmission in communities.

One of the major different opinions between clinicians and industry professional participants is their preferred view on cost when building an ideal STI POCT. Among 4 attributes that we investigated in this study, cost was ranked as the second preferred attribute among clinicians. However, it was ranked last among industry professionals. In addition, clinicians preferred the lowest cost in our survey, $20 per test, for general STI POCT and chlamydia POCT while industry professionals settled in $35. One possible explanation is that industry professionals concern cost less because cost is not one of key elements for U.S. Food Drug Administration diagnostics approval process.⁶ On the other hand, practicing clinicians in the public sector – cost is always going to be a top factor and the purchasing decisions are often made by non-medical persons who are in the purchasing department. They look for “low bid”, and they take the clinician’s desires into consideration but have other competing priorities. In the private sector, however, very often the physician makes business decisions and determines which diagnostics are to be purchased. A physician’s determination of what are important test characteristics will often affect the purchase. The mid-level (non-MD) clinicians can and will influence the purchaser by making complaints when the diagnostic doesn’t meet their perceived needs.

For industry, developing POCTs represents a long road “from bench to bedside” with enormous investment in time, money, and resources. Periodic needs assessment by the STI academic researchers to guide development of tailored training is warranted to increase awareness of clinicians’ perceptions of ideal STI POCTs among scientists, biomedical engineers, and other industry experts. An interactive, rapid communication approach between the academic researchers and industry, such as that employed by Hess et al.⁷ could also decrease the differences between the perceptions of industry leaders and those of clinicians, the end users.

In summary, we demonstrated significant differences in perceptions of barriers to currently available STI POCTs and ideal POCT in preferred qualities in this study. This will let industry (the developer and the seller) understands what clinicians (the buyers) really want in STI POCTs. It is imperative for industry to recognize the preferences because industry needs to be able to sell the product to an end user.