Overcoming Challenges with the Adoption of Point-of-Care Testing: From Technology Push and Clinical Needs to Value Propositions

Abstract

Major technical challenges often prevent developers from producing new point-of-care technologies that deliver the required clinical performance in the intended settings of use. But even when devices meet clinical requirements, they can fail to be adopted and successfully implemented. Adoption barriers occur when decision makers do not understand the “value proposition” of new technologies. Current discussions of value in the context of point-of-care testing focus predominantly on the intended use and performance of the device from the manufacturer’s point-of-view. However, the perspective of potential adopters in determining whether new devices provide value is also important, as is the opinion of all stakeholders who will be impacted. Incorporating value concepts into decisions made across the full development-to-adoption continuum can increase the likelihood that point-of-care testing will have the desired impact on health care delivery and patient outcomes. This article discusses how various approaches to technology development impact adoption and compares the characteristics of these approaches to emerging value concepts. It also provides an overview of value initiatives and tools that are being developed to support the evaluation of value propositions. These are presented for a range of technology adoption decision contexts, with particular applicability to point-of-care testing. Expanding the focus of research to address gaps in both the creation and evaluation of value propositions is imperative in order for value concepts to positively influence the adoption of point-of-care testing.

Introduction

The field of point-of-care (POC) testing for health care continues to attract technology developers, who are eager to build on decades of investment and significant advances in underlying technologies to produce clinically useful POC devices. This interest is due in part to the belief that POC testing can play an important role in supporting new models of health care delivery, where primary care practitioners and healthcare workers at the frontline can achieve diagnostic certainty and expedite targeted treatment and advice. In some cases, the characteristics of POC tests are well matched to clinical and user needs, with seemingly obvious catalysts to adoption. For example, the ability to diagnose and treat patients for sexually transmitted infections (STIs) within a single visit can prevent the loss to follow-up and significant delay associated with laboratory testing, which is one of the main challenges associated with preventing the transmission of STIs.^1,2 If information about antibiotic resistance can be obtained at the same time, precision therapy can be offered rather than empiric algorithmic care, which has contributed to gonococcal resistance globally.³

To encourage a clinical needs-driven approach to the development of POC devices, the National Institute of Biomedical Engineering and Bioengineering (National Institutes of Health) created the Point-of-Care Technologies Research Network (POCTRN) to provide guidance to academic and small business developers on pressing clinical needs in POC testing.^4,5 Using the Center for Point-of-Care Technologies Research for Sexually Transmitted Diseases (STDs) at Johns Hopkins University, investigators studied the perceptions of STI professionals, clinicians, healthcare workers, patients, and technology developers through clinical needs assessments (CNAs). The Center communicated CNA results (e.g., desirable characteristics of POC STD tests, target analytes, key device performance criteria and barriers to use of POC STD tests)^6–10 through Center-supported funding initiatives to guide developers in designing new technologies that meet identified clinical needs and thus promote their adoption.¹¹

Despite the sustained investment in development of new devices and widely accepted perceptions of clinical needs, recent studies show that challenges with adoption and access through successful implementation of POC testing are limiting the impact of new devices.^12,13 Unfortunately, developers rarely consider the details of clinical barriers or how adopters might perceive the value of new technologies when making design decisions at the earlier stages of development, with technical considerations their main focus. In this review, we compared technology-driven and clinical needs-driven development to emerging value-based concepts that have the potential to improve technology adoption decision making. We provided examples of value frameworks and tools that can be used to guide technology assessments and offered recommendations to encourage increased use of value concepts across the development-to-adoption continuum to improve the impact of POC tests on patient outcomes.

Technology-Driven Development

When the desire to advance technological capabilities drives device development (“technology push”), widespread adoption can be significantly delayed or unrealized. Table 1 summarizes the key characteristics of the technology-driven approach that impact the adoption process. This approach to development is typically guided by a general understanding of basic requirements for functional POC devices with a focus on overcoming technical challenges related to miniaturization, microfluidics, signal detection, multiplexing, sample preparation, and device integration.¹⁴ Without a highly defined clinical need, the developer prioritizes innovation and the potential to impact a range of applications, independent of clear clinical performance targets or a particular context of use. The focus is on demonstrating analytical performance, often with the use of contrived or surrogate samples rather than clinically obtained samples. With the technology-driven approach, the developer’s understanding of a potential clinical problem informs design choices, with early adoption of resulting technologies relying on a clinical champion whose perspectives might not be widely shared.¹⁵ Input from a broader range of clinical end users later in the development process can ultimately force costly changes.¹⁶ For technology-driven innovations, formal health technology assessments (HTAs) are often required to support policy-level coverage, reimbursement and utilization decisions.¹⁷ HTA process are time-consuming and can delay adoption decisions, especially when HTA methodologies are lacking or when unclear definitions of clinical utility among payers create uncertainty for manufacturers working to fulfill evidence requirements.^18,19

Table 1.

Characteristics of Technology-Driven Development of POC Technologies

Key Question		Technology-Driven Development
What is the motivation?		Innovate
What guides development?		Technological capabilities
What are the metrics?		Analytical Performance
What perspective matters?		Developer interests
How important is context?		Context-free
How is the technology promoted?		Innovation potential
What are the cost considerations?		R&D costs
What is the adoption challenge?		Finding clinical champions
What informs purchasing?		Health Technology Assessment

Clinical Needs-Driven Development

Using the information obtained through clinical needs assessments to guide developers’ decision making is the second approach to technology development, a paradigm that is becoming an integral part of recommended development pathways.^14,20 Table 2 summarizes the key characteristics of the clinical needs-driven approach that impact the adoption process. Developers using this approach are motivated by a desire to solve a clinical problem, with development decisions guided by user-provided technical specifications and clinical performance targets. Often the clinical needs are shared across many settings, providing many contexts of use for the device. Adoption of the device is promoted by clinicians making the business case within their organizations for why the device or test would be a good investment, as part of a traditional procurement process.²¹ For adoption of POC testing in which the clinical need is the main focus, the cost of the test, level of reimbursement, and impact on the clinical workflow and staff workload are major considerations.

Table 2.

Characteristics of Clinical Needs-Driven Development of POC Technologies

Key Question		Clinical Needs-Driven Development
What is the motivation?		Solve clinical problems
What guides development?		Technical specifications
What are the metrics?		Clinical Performance
What perspective matters?		Clinical perceptions
How important is context?		Cross-context
How is the technology promoted?		Business Case
What are the cost considerations?		Cost of test vs. reimbursement
What is the adoption challenge?		Integrating test into clinical workflow
What informs purchasing?		Traditional Procurement

An older POCTRN best practices article²² described the CNA process, covering both qualitative and quantitative approaches. The process is intended to gather information from all stakeholders who would benefit from the ultimate solution; but in practice, many assessments focus on the perceptions of clinical experts with the purpose of identifying high priority tests and determining product requirements and specifications. The recent creation by the World Health Organization (WHO) of target product profiles for high priority STI tests intended for use in low- and middle-income countries²⁰ provides an excellent example of the CNA process and the types of information that developers need to guide decision making. Additionally, early HTA methods (combining health economic modeling and systems engineering) can provide guidance on the potential commercial viability and value of new technologies to build on CNAs.^23,24 Developers can unknowingly hinder adoption by focusing on recovery of R&D and operational costs in the pricing of new devices without consideration of the likely reimbursement or perceived value of the technology to adopters. Early HTA methods can guide manufacturers by providing estimates of maximum reimbursable price.

Beyond Clinical Needs

While using a clinical needs-driven approach to development improves the likelihood of adoption of new technologies, it does not guarantee success. For example, the OSOM^® Trichomonas Rapid Test (Sekisui Diagnostics, Burlington, MA, USA) has been FDA-approved since 2004 and has a sensitivity using vaginal swabs of over 85% and specificity of 99% for TV compared to sensitivity of wet preparation examination of vaginal fluid of 30–70%.^25,26 A 2016 survey among 251 obstetricians and gynecologists reported that 79% were still using wet mount testing; none reported using the OSOM Trichomonas Rapid Test.¹⁰ This is despite STI professionals suggesting a clinical need for rapid and sensitive point-of-care tests for the diagnosis of trichomoniasis²⁷ and recommending minimum clinical performance requirements.²⁰

CNAs are inherently limited in that they reflect the perspectives of those completing them, which can impact how effectively the results inform the development process. If CNAs focus narrowly on the perspectives of clinicians as users, the expressed needs may not fully reflect the complete care pathway or system-level needs. CNAs are often performed with a technology solution in mind, with the goal being to define technical specifications and clinical performance targets to assist developers in meeting regulatory requirements. Recent guides to needs-based innovation recommend evaluating clinical problems independent of proposed solutions²⁸ or considering alternative solutions in verifying unmet needs.²⁹ A consequence of limiting the focus of CNAs to clinical experts is that they fail to capture the perspectives and evidence needs of diverse stakeholders who will influence the adoption decision and implementation process.³⁰ For example, with POC testing, lack of reimbursement is a major barrier, requiring that developers produce evidence of clinical utility (e.g., more rapid throughput in an Emergency Department) in addition to clinical performance to address the information needs of payers. And provider organizations increasingly want to know that use of a new device or test produces beneficial outcomes and value to the organization and its patients.

The Value Landscape

The concept of value is becoming increasingly important in healthcare as the system moves toward improving the quality of care while reducing costs. At the level of care delivery, value-based concepts relate to new payment models that compensate providers based on patient outcomes. This value assessment is increasingly being applied to health care technologies to manage costs associated with the adoption of new technologies. Included in this is consideration of the increased utilization of existing technologies and the assessment of whether previously adopted technologies have delivered desired outcomes.³¹ Value concepts are impacting decisions related to pricing, coverage and reimbursement, purchasing and utilization, and are even being considered relative to manufacturers’ marketing of new technologies.

Decision makers who are applying value-based judgments are seeking to achieve desired outcomes at an acceptable cost. The outcomes they view as important are specific to the context in which value is being determined, although patient-related outcomes are typically a core element of value with some consideration of secondary outcomes for other stakeholders (e.g., caregivers, health care systems, and society).¹⁸ Historically, decision makers have tended to favor single measures of benefit and well-defined quantitative measures of value such as cost-effectiveness for coverage decisions³² and technical performance and price for procurement decisions. There is growing acknowledgement that these historical measures do not capture all of the important aspects of value and, in particular, can miss key elements that are qualitative and subjective in nature.^33–36 Ultimately, each decision maker may prioritize different elements of value and seek different types of evidence. In making value-based adoption decisions, each may use a unique set of criteria for evaluating the evidence and assign different relative weights to those criteria. Decision makers can use different methods to arrive at an overall estimate of value and then apply different thresholds in determining whether a new technology is worth the investment given their budget constraints.

At the heart of value considerations for technology adoption are new value frameworks that contain measures of value for specific decision contexts.^37,38 Frameworks exist to inform health plan coverage and reimbursement decisions, development of clinical guidelines, shared decision making for selection of cancer drug therapies and pricing of drugs. Recent guidance offers key characteristics that value frameworks should possess before being used in practice.³⁹

Provider organizations also apply the concept of value when making decisions to adopt new technologies using two main approaches: hospital-based health technology assessment (HBHTA) and value analysis or value-based procurement. Hospital-based HTA is similar to national-level health technology assessment, having been adapted from traditional HTA methodologies to emphasize the perspective of the individual organization with the use of contextually relevant information and local evidence. Value analysis methods improve on traditional procurement processes and move beyond a focus on technical characteristics and vendor capabilities in the evaluation of new technologies. Combining elements of both approaches can facilitate comprehensive assessment of new technologies while meeting the needs of organizational decision makers.⁴⁰ See Appendix 1 in the Supplementary Materials for additional information on both approaches.

Multicriteria decision analysis (MCDA) can guide health care decision makers in comparing alternative technologies, programs, services, or interventions at the policy, organizational, clinical, or patient level to determine which provides the best value.⁴¹ The components of an MCDA model include the decision options being considered, the criteria against which the options will be compared, scores that represent how each decision option performs on each of the criteria, and criteria weights that indicate the relative importance of each criterion.⁴² MCDA methods can be qualitative, quantitative or involve the use of decision rules, providing flexibility in implementation.⁴³ In qualitative MCDA, decision makers use deliberation to consider the relative overall performance of each technology on the criteria given the evidence; whereas in quantitative MCDA, decision makers use a value measurement model to obtain numerical estimates of the value of each technology which are then considered in a deliberative process. MCDA with decision rules allows decision makers to consider trade-offs between criteria according to a set of rules, often applied to cost-effectiveness thresholds. Recent reviews of MCDA methodologies⁴⁴ and good practices^45,46 offer guidance on choosing the right methods and avoiding the pitfalls often encountered when applying MCDA methods.⁴⁷ Additionally, numerous examples of MCDA models applied to technology adoption decisions in different contexts are available as examples.^48–51

The Value of POC Testing

The value concepts described in the previous section are important for developers and innovators to understand and consider early in the development process given that adopters are applying value concepts when evaluating new technologies, with or without formal processes.⁵² Table 3 summarizes the key characteristics of value concepts that impact adoption decisions for POC testing. Developers must shift their focus from solving clinical problems to achieving value outcomes when developing new POC technologies. Value-based outcomes can be clinical, economic, or workflow-oriented and should be important to all stakeholders (i.e., patients, clinicians, provider organizations, and payers). Developers must consider how introduction of the POC test will impact the full clinical care pathway in addition to how it would impact the workflow and workload of the staff using the test. Developers must identify the needs of adopters at the system level, requiring that they gather the perspectives of all stakeholders who will be involved in the adoption decision and impacted by use of the new technology. Obtaining value-based information is challenging, given that the perceived value of a new technology is ultimately context-specific. Recent studies offer guidance on early HTA methods to estimate the health economic impacts of POC and laboratory diagnostic tests to inform developers early in the development process of potential costs and benefits of new POC technologies.⁵³ These methods can support value-based development by providing guidance on product specifications and test uses that might provide value, as well as uncovering potential implementation barriers. A recently developed framework offers guidance on developing a value proposition for new POC tests.⁵⁴ A major challenge is the lack of evidence to support key elements of the framework, especially with respect to test impact, implementation issues, and cost-effectiveness.⁵⁵ A checklist for evidence generation—the Point-of-Care Key Evidence Tool (POCKET)—suggests 65 different evidence requirements that address the information needs of a range of stakeholders across the spectrum of development, regulatory approval, coverage and reimbursement, and clinical use.⁵⁶ Appendix 2 in the Supplementary Materials offers additional information on tools and resources that developers can use to create value propositions for POC technologies.

Table 3.

Characteristics of Value-Based Development of POC Technologies

Key Question		Value-Based Development
What is the motivation?		Achieve outcomes
What guides development?		System-level needs
What are the metrics?		Clinical/Process/Economic Outcomes
What perspective matters?		Stakeholder perspectives
How important is context?		Context-specific
How is the technology promoted?		Value Proposition
What are the cost considerations?		Total cost
What is the adoption challenge?		Assessing impact of test on care pathway
What informs purchasing?		Value Analysis/Value-Based Procurement

Models that inform on the potential impacts of implementing new POC tests can support the creation of value propositions. In the area of STI testing, a recent model of the population-level impact of POC testing for chlamydia⁵⁷ showed the potential to reduce the prevalence of chlamydia by implementing POC testing using tests with high sensitivity and short turnaround times, allowing for a majority of patients to get immediate treatment. The model additionally showed that the reduction in chlamydia burden can be achieved using lower-sensitivity tests if the frequency of chlamydia screening is increased. This is important given the difficulty developers face with achieving high sensitivity when developing new tests. A better understanding of current patient management and other real-world parameters used in modeling, such as patients’ willingness to wait for test results, can help inform on whether use of a POC test with lower sensitivity (relative to laboratory testing) can provide benefits in certain scenarios.⁵⁸ Additionally, cost-effectiveness modeling can provide information on the costs and benefits of implementing a POC testing program relative to laboratory testing. For example, a study of the costs and benefits of replacing the standard pathway for managing chlamydia and gonorrhea with a clinical pathway that incorporates POC nucleic acid amplification testing showed potential cost savings, especially when considering that over-treatments can be avoided.⁵⁹ As with the previous modeling example, the need for POC tests with high sensitivity as well as a better understanding of patients’ willingness to wait are important elements, emphasizing the need for qualitative studies related to patient management and patient perspectives (including those from “demand generation”) along with studies of test performance.

In addition to requiring evidence from a broad range of sources, decision makers need structured decision processes to assess the value of new POC technologies. One study identified decision criteria viewed as important when comparing a new POC test with a currently used laboratory test, using an MCDA method.⁶⁰ An expert team with clinical, patient, laboratory, technology developer, policy maker, and payer representation validated the criteria and provided their judgments on the relative importance of each. The study acknowledges that additional factors and information are important for making the decision to adopt a POC test at the level of provider organization.

In practice, provider organizations have few tools to assist with the evaluation of POC tests for adoption decisions and implementation planning. The resources that do exist focus on technical features and vendor quality, characteristic of traditional procurement processes. A standardized scorecard for assessing the operational specifications of POC devices⁶¹ offers an objective approach to rating products across six categories: (1) features of equipment, (2) features of consumables, (3) ease of use, (4) quality control, (5) cost, and (6) distribution and service. The Clinical and Laboratory Standards Institute (CLSI) provides guidance on comparing instrumented diagnostic systems for clinics and physician office laboratories,⁶² covering clinical and operational needs assessment, implementation factors, sources for identifying candidate devices, in-house performance evaluation, and cost assessment. The guidance is intended to serve as a starting point for institutions to develop their own comparisons and rating system. Other sources of guidance include published studies that compare the technical characteristics and performance specifications of specific classes of POC devices with the aim of helping healthcare professionals understand the strengths and weaknesses of alternative technologies to guide device selection.⁶³ The World Health Organization provides lists of prequalified in vitro diagnostic products (including POC tests) that can be used by low- and middle-income countries to inform purchasing decisions.⁶⁴

Research Needs in Value Assessment for Point-of-Care Testing

Applying value concepts across the development-to-adoption continuum requires consideration that value propositions are both created (by manufacturers seeking market access) and evaluated (by policy-making groups and provider organizations making adoption decisions). To facilitate communication, creators and evaluators of value should consider shared guiding principles when developing value frameworks and should identify common (high-level) elements of value for which evidence can be generated (Figure 1). In addition, each decision maker must identify context-specific value elements and decision criteria and gather the local evidence needed to support the technology adoption decision process.

Figure 1.

A shared understanding of value is needed between creators and evaluators of value propositions

Much of the recent work to define the value of POC testing has focused on the creation of value propositions and generation of the evidence that is needed to support coverage and reimbursement decisions. But a recent study of the evolving perception of the value of a robotic surgery technology suggests that generating evidence alone isn’t enough–a shared understanding of value for all stakeholders is needed.⁶⁵ Despite generation of evidence over two decades of technology use, differing interpretations of that evidence have sustained controversies about the technology’s value, with disagreements among stakeholders on both informal issues and formal research, and with the generated evidence failing to address many qualitative and implementation-related concerns. This suggests a need to focus more research effort on how payers define value and how they use evidence in making decisions to adopt new technologies.

At the end of the development-to-adoption continuum, provider organizations are faced with difficult decisions when evaluating new technologies given the need to identify evidence that is relevant to their specific decision context. A major gap in research is the development of value frameworks, evidence requirements, and decision processes to guide provider organizations in determining whether available POC tests can provide value. New frameworks intended to assist technology adoption decision makers with implementation planning could provide guidance, although research is needed regarding their applicability to POC testing. Appendix 3 in the Supplementary Materials provides additional details.

Research is also needed to understand the evidence needs of diverse stakeholders who are likely participants in organization-level decisions to adopt POC technologies.⁶⁶ A recent study investigated the use of different sources and types of evidence in management decisions to adopt innovations and found that evidence access and interpretation varied depending on the professional background and role of decision makers as well as organizational culture and external factors.⁶⁷ Development of tools to aid in communicating evidence on POC technologies and testing programs in a format that is accessible, relevant, and practical for primary care professionals can support comprehensive assessment of evidence and facilitate adoption of value-based decision processes in primary care settings.⁶⁸

Conclusions

Applying value concepts to decisions that are made along the development-to-adoption continuum has the potential to improve successful adoption and implementation of POC technologies. This requires development of frameworks that define elements of value for different decision contexts and diverse stakeholders, generation of context-appropriate evidence that supports assessments of value, and adoption of structured decision processes that capture relevant stakeholder perspectives. Encouraging technology developers to think beyond the requirements for regulatory approval (i.e., test accuracy and clinical performance) and produce evidence on outcomes to support value-based decisions is essential. Understanding and refining decision processes at the level of provider organizations is a key research need. Fortunately, recent advances in the many fields that are fundamental to the creation and evaluation of value propositions offer important guidance to researchers and health care professionals interested in applying value concepts to the development and adoption of POC technologies.