Healthcare worker feedback on a prototype smartphone-based point-of-care test platform for use in episodic care

Abstract

With a growing list of new platforms, end-user acceptability is an evolving topic in point-of-care (POC) test development. While technical reports of new experimental POC tests are common, it is rare to find reports which evaluate the end-user acceptability of such innovations. This work illustrates an example of bridging that gap by evaluating the end-user acceptability of an experimental POC test platform with novel technical features. A prototype smartphone-based STI tests was evaluated by ED technicians, followed by a survey of acceptability factors. Our findings suggest that the end-user acceptability of some design features implemented in the prototype.

Introduction

Point-of-care (POC) tests hold much promise for use in sexually transmitted infections (STIs), where immediate diagnosis and treatment of patients can help facilitate new strategies to address the persistent STI public health burden^1,2. Recent POC platforms have successfully incorporated nucleic acid amplification tests (NAAT) to achieve analytical performance on par with laboratory-based reference platforms³. Despite such improvement, barriers that limit the successful implementation of POC tests in clinical settings exist. Insufficient collaboration between developers and end-users can lead to complexity in the use of the device which translate poorly in the clinical setting, resulting in gaps or deficiencies in the usability of the POC device^4,5. Furthermore, these fundamental issues reoccur with new emerging POC devices due to insufficient clinical needs assessments from end-users⁶.

MobiNAAT is a prototype NAAT platform developed for testing chlamydia in episodic care settings using vaginal swabs⁷. The mobiNAAT assay is a cartridge-based isothermal test, which completes automatically following a few minutes of hands-on sample preparation time in 1 hour. Several design features of this platform address portability, ease of use, and affordability. Namely, the cartridge utilizes a battery-operated, mug-sized (3.4′ × 4.6′ × 6′) instrument for process automation. The platform operates using a smartphone application that contains an on-board tutorial video to train first-time users without direct supervision. The cartridge utilizes droplet magnetofluidic technology to minimize reagent consumption and reduces assay cost to $2 per test. In this study, we sought feedback on these design features from the target end-users based on their primary exposure to the experimental prototype.

Materials and Methods

A cross-sectional survey was conducted in an urban academic Emergency Department (ED) serving a socioeconomically disadvantaged inner-city population. This study was approved by the Johns Hopkins Medicine Institutional Review Board, application number IRB 00088884, and all participants provided written informed consent prior to any study-related activities.

A dedicated study coordinator enrolled 30 ED technicians in August 2016 to assess their acceptability of a prototype microfluidic device for POC testing for genital chlamydial infection. ED technicians (e.g., certified nursing assistants, or patient care technicians) were identified as ideal end-users, as they commonly perform CLIA-waived POC testing such as urinalysis and blood glucose test in urgent care settings. The study workflow is presented in Figure 1a. Test setup included the mobiNAAT instrument, a microcentrifuge tube containing a previously prepared test sample, an assay cartridge, and a micropipettor for sample loading. The 6-minute video tutorial on the cell phone for the POC platform was prefaced with a 2-minute introductory section solely for the purpose of this study, outlining the purpose of POC STI testing in acute care settings. Assay cartridges were assembled and delivered to the ED site up to one week prior to use. Participants watched the video tutorial with instructions on how to perform testing, performed a test run using the microfluidics device, and lastly, completed a brief structured survey assessing acceptability (Figure 1b). The survey was designed to identify end user priorities and preferences for various design parameters such as the assay target, instrument size, instruction time, cost per test, and the format of the instruction materials. In order to gauge end user preference for the instrument size, the survey utilized familiar objects with relative differences in size and geometry (e.g. a watch, a mug or a tablet) as opposed to absolute dimensions. A typical session involved about 15 minutes of instruction and a hands-on setup of the test run by the participant, followed by the completion of the survey.

Figure 1.

Flowcharts describing study workflow and structured survey used in the study. (a) Study workflow. Each participant viewed introductory video tutorial, performed a test run on the experimental platform, and completed the acceptability survey. (b) Structured survey flowchart. The survey asked a series of general questions regarding POC testing, preference for POC test features, sample collection approach, and test instruction format. Results obtained from the decision blocks (double-lined diamond) are shown in Figure 2.

Results

Of the 30 technicians enrolled, 26 (86.7%) had previous experience with a POC test, and POC tests were strongly preferred (83.3%) over standard laboratory-based NAAT (16.7%). Results of the survey indicated that turnaround time (70.0 %), sensitivity (36.7 %), and ease of use (23.3 %) were the parameters perceived to be of the highest importance in an STI diagnostic test. Chlamydia (96.7 %) and gonorrhea (96.7 %) were identified as targets that would be the most helpful to test for among a list of the most prevalent STIs, duplicating this preference among clinician end-users⁵. As shown in Figure 2a, participants favored the use of POC tests over laboratory-based tests due to their fast turnaround time (76.7 %), simplicity (43.3 %), and mobility (30.0 %). This finding suggests that the primary value of a POC STI test to end-users in an acute care setting is the potential to save time, expedite patient care decisions, and reduce length of stay. Including the time spent viewing instruction videos, a typical assay took 15 minutes to set up by a first-time user and was completed in 1 hour. Notably, cost appears to be a secondary consideration to the end-users, reflecting some difference in perspective from the clinicians who prioritize cost over time⁸. A tablet-sized device was overwhelmingly preferred (30.0 %) over a mug or a watch, from which we could infer that the technicians preferred the familiarity of handheld devices to miniaturization. As for the cost, most participants preferred low-cost devices ranging from $1 to $15 (38.1%), followed by $16 to $30 (28.6%), and $31 to $50 (23.8%).

Figure 2.

Survey results. (a) Breakdown of reasons for using a POC test. Multiple responses were permitted. (b) Preferred sample collection method. Inset shows the breakdown of reasons for respondents who chose self-collected as the preferred method. (c) Preferred instruction format. Inset shows breakdown for the preferred duration of instructional video material.

Interestingly, most participants preferred provider-collected samples (73.3 %) as shown in Figure 2b. This observation is in contrast to patients and providers, where the majority of respondents preferred self-collected methods⁹. Of note, participants strongly preferred a short video tutorial (90.0 %) with runtime of 5–12 minutes to the conventional text and graphic instructions (6.7 %). Furthermore, most participants favored the use of POC tests in the ED (83.3 %).

Discussion

Our study was primarily designed to evaluate the end-user feedback on prototype POC test platform. While other studies to date focus on clinical parameters such as turnaround time and diagnostic performance, our study included issues regarding technical and design parameters that the platform developers found to be important. Survey results show that strong end-user preferences can be identified in these categories, highlighting the importance of directly engaging end-users for their feedback throughout platform development.

A novel feature of this study was the use of an experimental device as a way to illustrate key design features directly to the participants prior to administering the survey. Implications of new design features such as the format and length of video instruction material would be difficult to convey without incorporating an interactive demonstration as performed in the present study. In that regard, the format of this study is ideally suited for vetting previously untested design features in a POC device directly with potential end users.

In summary, our findings suggest a high level of acceptability for POC tests among ED technicians. In particular, our study encourages further investigation into the format of instructional materials and device dimensions as factors that may affect end user preferences.