Integrating user behavior with engineering design of point-of-care diagnostic devices: Theoretical framework and empirical findings

Abstract

With point-of-care (POC) diagnostic devices becoming increasingly available to untrained users, it will be critical to understand how real-world user behavior can best inform and guide the engineering design process. Social sciences present frameworks for analyzing user behavior, but they have not yet been applied to POC diagnostics in a methodical manner. Here, we develop a framework that synthesizes two models that can collectively account for user behavior and experience with POC diagnostic devices: a social psychological information-motivation-behavior (IMB) model (first described by Fisher and Fisher) for identifying determinants for health-related behavior, and user experience (UX) elements for studying interactions between users and products. Based on studies of 40 naïve users of our smartphone-enabled microfluidics device that can be used for HIV home-testing, we found that untrained participants could perform 90% of steps correctly, with engineering design elements that provided feedback that was either direct (e.g., a light or click) or binary (e.g., a switch) enhancing usability. Interestingly, of the steps performed incorrectly, over 70% were due not to errors in the device or user operation, but user-to-user variability (e.g. time in collecting fingerstick and force applied to initiate vacuum), which could be addressed by further modifications to the device. Overall, this study suggests that microfluidic POC HIV home-testing is likely to benefit from smartphone integration, and that engineering design of POC diagnostic devices can benefit from a structured evaluation of user behavior and experience, as guided by a social-psychological framework, which emphasizes user credibility, accessibility, acceptability, usability, and value.

INTRODUCTION

With microfluidics-based diagnostic devices being increasingly targeted to minimally-trained and untrained users^1–6, it is becoming critical to understand how end users engage with the devices, the device failure points, and the design features best suited to spur adoption. There is increasing recognition of the centrality of analyzing user behavior and experience in informing and guiding the engineering design process. In the engineering and design communities, human-centered design is a process that emphasizes iterations of design, engineering, and evaluation⁷ to fit the user, based on an understanding of users, tasks and environment. From a regulatory perspective, the FDA has outlined the importance of “human factors engineering” and “usability engineering”, stating that medical device developers “should conduct appropriate human factors study, analyses, and tests in the early stages of design process”⁸. Such analysis should examine the details of how a user directly interacts with the system, and include underlying factors that contribute to the interactions that lie beyond the system boundary, such as an understanding of the use-environment (which we have previously delineated for POC diagnostics into use-case settings⁹), user characteristics, and the mechanics of the device interface including hardware and software⁸. The application of human factors engineering principles to device development has also received attention from regulators in the European Union¹⁰, with an emphasis on device failure points and potential risks, particularly for consumer applications.

HIV self-testing, where a serological HIV test is performed by an individual using finger-pricked blood or an oral swab¹¹, is one of the first areas of POC diagnostics (alongside pregnancy and glucose tests) where we are seeing a tremendous increase in interactions between untrained users and diagnostic devices. In the U.S., while overall HIV incidence is declining, men who have sex with men (MSM) accounted for over 67% of total new diagnoses in 2016¹². From 2011 to 2015, there was also a 65% increase in rates of syphilis, with MSM accounting for two-thirds of all reported cases¹³. Syphilis and other sexually transmitted infections (STIs) increase the risk of acquisition and transmission of HIV¹⁴. As a result, the CDC has recommended yearly testing for syphilis and other STIs for sexually active MSM, and even more frequent testing (up to every 3 months) for MSM with multiple or anonymous sex partners. Currently, the only FDA-approved diagnostic product available over-the-counter is the OraQuick In-home HIV Test, an oral-fluid, lateral flow test for detecting the presence of HIV antibodies (HIV-1 and HIV-2), that allows users to self-administer the test at home, obtain results in 20 minutes and interpret results using written instructions¹⁵. Given this clinical need and the dearth of available diagnostic products, it would be helpful to understand how future generations of HIV and STI self-testing devices could best serve populations with the highest risk (such as MSMs, transgender women, and sex workers) as well as the general population for self- and partner-testing to achieve early diagnosis and potentially prevention of infection.

To study user behavior, social science fields, such as economics, psychology and behavioral science, have developed and applied a number of rigorous frameworks. For example, models in behavioral economics¹⁶ attempt to explain why motivation does not always translate into healthy behaviors, and explore additional incentives for achieving healthy outcomes. To achieve behavioral change, the Fogg Behavior Model specifies three key factor: necessary motivation, sufficient ability, and effective triggers¹⁷. In the context of HIV prevention behaviors, the Fisher and Fisher Information-Motivation-Behavioral Skills (IMB) model was first proposed to understand, predict and promote adherence to highly active antiretroviral therapy among HIV-positive individuals^{18, 19}. The IMB model posits that information, motivation and behavioral skills are needed in order for an individual to engage in a given health behavior, and individually serve as determinants of behavior and behavioral change, and was applied to HIV preventative behavior^20–23. Specifically, in cases where “individuals are well informed, motivated to act, possess the behavioral skills required to act effectively, [and are thus] likely to initiate and maintain patterns of HIV-prevention behavior”, the IMB framework has proven useful for studying or developing a particular intervention tool¹⁸. The model also acknowledges that culture, class, economics, environment, and life circumstances may affect specific informational, motivational, and behavioral skills factors. This model has been extensively validated with diverse populations in over 15 years of research^{15, 20, 23–25} and has been applied to motivation to engage in HIV home testing among sexually active, at-risk MSM²⁶, adherence to antiretroviral therapy²⁷, and the development of an HIV prevention smartphone application for high-risk MSM²⁸ as well as different health scenarios, including childhood obesity²⁹, diabetes self-care³⁰ and even curbside recycling behavior³¹. In this study, we incorporate the Fisher-Fisher IMB model due to its demonstrated utility for studying HIV and other health-related behaviors.

In addition, user experience (UX) design is now commonly used in digital technology and design industries to study interactions between users and products. Usability testing was initially developed to analyze users’ interactions with web sites, when limitations were observed in examining only task-related problems without attention to the holistic user experience. In the 1990’s, Morville³² developed the UX Honeycomb model^33,34 as a framework in information architecture to anticipate users’ information needs for internet tools³⁵; since then, the model has been applied to other areas of human-computer interaction that have grown to encompass web and mobile applications³⁶. The UX Honeycomb model outlines an expandable evaluation model where separate facets (e.g. useful, usable, desirable, valuable, findable, accessible, and credible), which are thought to deliver maximum value to the user and can be explored in various permutations to iterate on product design, are considered in order to measure success³⁴. For health, the model has been applied to testing web design for online health resource systems³³, decision boxes for physicians and patients³⁷, and other electronic and mobile health interventions³⁸. The model has also been applied for wearable pain monitors³⁹, wearable activity trackers⁴⁰, and cuff-based blood-pressure telemonitoring devices⁴¹.

Here, we synthesize two frameworks to address user behavior and experience with POC diagnostic devices: the Fisher-Fisher IMB model for understanding user behavior, and the Morville UX Honeycomb model for studying user interactions with devices. We build on our previous work in creating a smartphone-enabled microfluidic HIV and STI test¹, and work with a sample of MSM population in New York City who is at high risk for HIV/STI infection to perform a structured assessment of how naïve users interact with a smartphone-interfaced microfluidic diagnostic device (called “mChip”).

MATERIALS & METHODS

mChip device

Custom-printed circuit boards were designed in Eagle CAD and printed from PCB Universe. Bluefruit LE Bluetooth Low Energy (BLE 4.0) nRF8001 Breakout Board (Adafruit) was connected to the microcontroller and powered by a CR2032 Lithium 3V coin battery (20.0mm). LEDs and photodiodes were precisely aligned with the cassette slot so that testing zones aligned without manual effort. One-mm pinholes made of 1-mm thick black Delrin (McMaster-Carr) were aligned above each photodiode to prevent stray light. The device casing was designed in SolidWorks and printed in-house (Objet 24 3D-Printer, Stratasys). The vacuum chamber was created with a one-way umbrella valve (Minivalve), a rubber bulb from a 60-mL syringe (Becton Dickinson), and a conical spring (Century Spring Corp) inside to aid re-expansion. Silicone rubber o-rings and sheets (McMaster-Carr) were used to connect to outlet and seal the venting port.

Microfluidic test cassettes

Disposable, plastic microfluidic cassettes were functionalized at Columbia University using methods described previously¹. The protruding capillary tip of the sample collector was marked for collecting fingerprick whole blood collection using an orange sticker to indicate directionality and a black fill line for visual metering of 2 μl of blood. An additional sticker labeling the sample collector as “Part A” was attached. Wash buffers (three 2-μl PBS–0.05% Tween 20 and four 2-μl water washes) as well as 50 μl each of silver nitrate (silver A) and silver reducing agent (silver B) were pre-loaded to the reagent cassette by manual pipetting and sealed (OPKO Diagnostics). Plastic guidance pieces were soldered onto functionalized, reagent-filled microfluidic cassettes and labeled with “Part B” stickers as well as matching orange stickers to Part A to indicate directionality of attachment. Microfluidic cassettes and sample collectors (also containing lyophilized antibodies) were stored at 4°C until an hour before use. (While some microfluidic cassette components were obtained from OPKO Diagnostics, the integrated cassettes of this academic study are different from the commercial versions.)

mChip mobile phone application

The mChip mobile app was written in iOS Swift using the XCode IDE for macOS. The app was manually downloaded by research staff to iPhone 5 devices used by participants. Captured results were stored locally on the phone during the study. The app framework consisted of an instructional video followed by step-by-step instructions with pictures and text instructions.

Running a test

The sequence of steps to run the test consisted of preparing materials for the test (unwrapping gauze, removing the lancet cap etc), obtaining a fingerprick blood sample, collecting a 2-μL metered amount of blood in the sample collector (“Part A”), attaching the sample collector to the microfluidic cassette (“Part B”), inserting the cassette into the mChip device, pushing down on the rubber bulb (negative pressure chamber) to initiate fluid flow, waiting 5 minutes for Phase 1 of the test to run (duration of time for sample and initial set of reagents to flow through), pressing the manual switch and pushing down on the rubber bulb again for silver reagent actuation, and waiting another 5 minutes for Phase 2 of the test to run (flow time for silver amplification reagents and signal reading). Full details on associated assay sequence and biochemical reactions on the microfluidic chip surface, as well as performance characteristics such as sensitivity and specificity, are outlined in our prior work¹ (in this study, a final screen showing blank results of HIV and syphilis concluded the test). The rubric used for usability evaluation broke down this process as 16 steps; it should be noted that about half of these are related to fingerprick blood sampling and collection. To mimic a home- or self-testing experience, no training or explanations were provided by the study team. The app features an instructional video and step-by-step directions to run the microfluidic test, during which users could navigate between pages and observe a progress bar, as well as watch and skip instructional videos. As previously published¹, all reagents are kept in cassette and thrown away after, such that the dongle has no contact with blood.

Study design

We conducted the trial in New York City, USA with approval from the New York State Psychiatric Institute (NYSPI) and Columbia University Institutional Review Boards. All study procedures were performed in compliance with the relevant laws and guidelines of the Institutional Review Boards and statements of informed consent were obtained for all participants involved in the study.

Participant recruitment and enrollment:

Compared to quantitative studies where power analysis can be conducted to estimate required sample sizes, qualitative studies in social sciences often propose a set number of interviews and continually assess whether data saturation has been achieved (i.e. recent interviews no longer yield new information or ideas from participants, and no longer require addition of codes to capture new data)^42–44. The number of interviews required to reach data saturation varies based on study design⁴⁴; for example, in a two-site study in Africa, researchers found that they had reached data saturation in 12 sessions and that much of the data gathered was saturated by the sixth interview⁴³.

For this study, a sample of 40 participants were proposed to ensure a sample that was diverse with regards to demographic characteristics (i.e., MSM and transgender women; racial/ethnic diversity) and prior use of HIV self-tests (up to one third of participants were recruited from another HIV self-testing study that did not use the mChip device). Eligibility criteria included: identifying as a man or transgender woman; 18 years of age or older; fluent in English or Spanish (there were no monolingual Spanish speakers); HIV-negative by self-report prior to enrollment; not in a monogamous relationship at the time of enrollment; reports having anal intercourse at least three times per month, on average; reports never or seldom using condoms during anal intercourse (no condom use in last 10 occasions for those with 4 or less partners or in less than 80 % of occasions for those with more than 4 partners in the past year); reports at least three episodes of unprotected anal intercourse (UAI) with serodiscordant or unknown status partners in prior 3 months; aware that UAI may lead to HIV transmission; and owns a smartphone. Our study enrollment criteria included those on PrEP (pre-exposure prophylaxis⁴⁵) if they met remaining criteria to be high risk for syphilis or other STI transmission.

Study procedure and assessments.

After completing the consent process, participants underwent: A) Part 1 of a quantitative computer-assisted self-interview (CASI), B) a skills demonstration using the mChip (mock self-test in front of the interviewer, no results shown to participant), C) Part 2 of the CASI and D) a qualitative in-depth interview. Optional FDA approved rapid tests for HIV (OraQuick or INSTI HIV tests) and syphilis (Syphilis HealthCheck), were offered to participants interested in receiving rapid test results and performed by trained research staff. Post-test counseling procedures included information about test limitations concerning window periods of infection, the need for confirmatory testing, and referrals to facilities for further evaluation and treatment. Participants received $50 in compensation after the study visit. All qualitative interviews and counseling procedures were conducted by trained NYSPI research staff (which included Ph.D.-level clinical psychologists, and a team member with a Masters in Public Health who has extensive experience in qualitative interviewing and data analysis). Protection of patient privacy was addressed in the approved IRB protocols, including the points that in-depth interview audio-recordings did not include any identifying information about the participant, that any identifying information in the transcript about the participant or any other individual was removed, that video recordings of the participant self-testing using the device only captured the participant’s hands, and that participants were offered gloves if they preferred greater privacy. Additional details of study procedures and assessments are outlined in Supplementary Materials.

Data analysis

For qualitative data management and analysis, audio recordings of in-depth interviews were transcribed, reviewed for accuracy and imported into NVivo software for data management and analysis. Coding categories were generated from interview questions and themes identified in participant narratives. An initial set of codes was generated by two research staff, compared, and synthesized. Final coding categories and sub-categories were described with definitions, inclusion and exclusion criteria and examples. Coders discussed discrepancies until they achieved 80% intercoder convergence. As we reached the proposed recruitment target, we assessed the data to confirm that responses from recent participants were not adding new information, ideas, or themes, and as such, we completed recruitment and finalized the qualitative dataset. Analysis of coded material was aimed at progressive identification of regularities in the data under study and to its organization under a conceptual label. Analysis included categorization, abstraction, comparison, integration, iteration and refutation.

For quantitative data management and analysis, CASI (Part 1 and 2) and rubric data (skill demonstration) were compiled for analysis and generation of descriptive statistics using Excel or SPSS software.

RESULTS

A customized UX-IMB framework for POC diagnostic tests

We developed a framework for analyzing user behavior and experience with our HIV/STI POC test by integrating two established models in behavioral science and UX design (Fig. 1). As a basis, we used the IMB model for studying user behavior^{15, 24, 46}, which asserts that informational, motivational, and skill factors are fundamental determinants of behavior, and will vary as a function of culture, class, economics, environment, and life circumstances. This model had been validated with diverse populations, and has been useful for design, implementation and evaluation of preventative interventions²⁰, including HIV prevention and other behavioral health applications^{15, 20, 23–25}. We adapted the IMB framework for HIV self-testing (Fig. 1) by noting that each category (i.e. user knowledge, user motivation, user behavioral skills, and user behavior) can be expanded into additional themes to suit the device or application to be studied. To customize the model to a diagnostics device, we queried UX design elements in the device (as outlined by the UX Honeycomb model) – such as device usability, accessibility, acceptability, and credibility (Fig. 1) – each of which influenced user knowledge, motivation and behavioral skills towards engaging with the HIV/STI self-test as a prevention method. (We did not assess device value in this study, which would be most directly measured by studying user behavior and risk-taking decisions, actual usage of device, and health outcomes after using the device, using methods such as random controlled trials. We also did not include “findability” in the traditional model as applied to searching information on web pages. In the future, categories in the UX model can be further customized depending on the device or application to be studied.) We undertook quantitative and qualitative questionnaires and interviews on how gay and bisexual men at high risk of HIV/STI transmission engaged with the microfluidic device (mChip) for a self- or home-testing application.

Figure 1. UX-IMB framework.

UX-IMB framework for evaluation of microfluidic devices for HIV home-testing application showing how UX components of usability, accessibility, acceptability, and credibility of a microfluidic device, can be mapped onto the major constructs of the IMB model (information, motivation, and behavioral skills). In this feasibility study, key measurements from the UX-IMB framework were used to assess theoretical value of the mChip device as an HIV prevention intervention in a self- or home-testing application.

Design elements of the microfluidic device

We made some changes to the device and smartphone app which were previously run by healthcare workers in resource-constrained health settings¹ (Fig. 2). Since our previous design of a smartphone accessory in the form of a dongle which could physically connect to smartphones through a 3.5 mm audio jack, the audio jack has been removed from iPhone⁴⁷ and recent Pixel phone models⁴⁸. Here, we switched the device communication to Bluetooth and added a button battery for power, which achieved fast data transfer and reliable connection. The small button cell battery used to power the Bluetooth module, which uses 4.6mW (using the Bluetooth Low Energy Mode), could power about 750 assays and can be replaced easily. Furthermore, using the button cell battery decreased the CV in replicate optical density measurements taken on one device to 2.8% compared to 10.3% when using the smartphone powered dongle (Fig. 2C). By increasing the LED intensity, we were able to further decrease CV of signals to 0.23% (Fig. 2C). We also designed a switch which reduced abrasion on the silicone rubber when initiating mixing of silver reagents (Fig. 2D). The overall footprint of the mChip device is almost half (58%) by volume than that of the previous version and untethered from a smartphone for increased portability (Supplementary Fig. 2). We also added a piece to further the guide to the sample collector (Part A) to the microfluidic cassette (Part B) through a slide-and-lock motion, to avoid bending the capillary tube tips out of shape when pushing the two pieces together (Supplementary Fig. 3). The modifications to the app included integration of a short instructional video, and step by step directions with photos and text for naïve users (Fig 2E).

Figure 2. Overview of the mChip.

(A) An image of the mChip kit containing the device with a microfluidic cassette, fingerprick materials of an alcohol swab, lancet, gauze and a bandage, and an iPhone with the mChip app. (B) Schematic diagram of the device highlighting a button-cell battery powered Bluetooth module and power-free vacuum generator for fluidic actuation. Once a microfluidic cassette is inserted into the device, each test zone is sandwiched between paired LEDs and photodiodes. Silver development on the cassette results in a proportional decrease in the light intensity sensed by the photodiode and can be quantified by optical density values. The Bluetooth module is connected to a microcontroller (not shown), which is programmed to wait for a signal from the smartphone, take light intensity readings of each zone and then send the information back to the smartphone. (C) Plot showing optical measurement reproducibility of mChip device. Coefficient of variation (CV %, plotted by dots) is shown for photodiode readings (plotted by grey bars) from devices with smartphone power, button cell battery power and with increased LED intensity. (D) Diagram of the device switch mechanism where an electrical toggle switch was modified such that electrical contacts were removed and a pin placed with silicone rubber below. Previously, the valve opening and associated rubber-o-rings could be jostled out of position with rough handling. The venting port for the one-way valve was moved to the underside of the vacuum bulb chamber, in order to increase device robustness. We also noted that actuation of silver reagents to connect with the microfluidic circuit in our previous design, required users to slide a toggle component that would cover or open a venting port on the reagent cassette. This would initiate the flow and mixing of silver reagents. From our previous field trial, we observed sliding the toggle back and forth for each run, caused the silicone rubber used to seal the port to get worn down easily, and required replacement every couple of days. We designed a manual switch adapted from electrical toggle switches as an alternative to reduce abrasion on the silicone rubber. We removed the electrical contact from the switch and placed a pin in its place with silicone rubber below. When the switch is pushed in the closed direction, a metal level pushes down the pin, sealing the venting port of the microfluidic cassette below. The mechanism can be reversed by pushing the switch back to the open position. (E) Smartphone app user interface showing steps of operation: (1) entering log-in information (ID number); (2) a landing screen giving users the option to watch an instructional video and then follow step by step directions; (3) instructional video providing an overview of the test procedures; and (4) step-by-step directions with images.

Participants in self-testing study in New York City

Forty participants were enrolled in this study (Fig. 3) with 39 completing the in-depth interview (one participant tested HIV positive and did not complete the study, as per eligibility criteria for continuation). Three-quarters (74%) of participants identified as gay, and one-quarter (26%) as bisexual or other (no participants identified as heterosexual). Median income was $30,000, lower than the median income of New York City households⁴⁹. Median age was 33.5. One-quarter of participants (26%) identified as Hispanic/Latino. 87% of participants reported owning a smartphone (Supplemental Table 1), with primary activities being phone calls, texting, email, and photography, followed by streaming music and videos and surfing the internet or social media. Approximately equal numbers used iPhone and Android phones (Supplemental Table 1). Although at high risk for HIV/STI transmission based on self-reported sexual behavior (i.e. mainly from limited or no condom use), 80% of participants reported high motivation (ratings of 8 to 10 out of 10) to remain HIV-uninfected (Supplemental Fig. 1), and about two-thirds of participants (68%) stated high willingness (ratings of 8 to 10 out of 10) to enact actions to avoid HIV, consistent with previous studies showing that gay and bisexual men at high risk of HIV transmission are still motivated to remain HIV-uninfected¹⁵. Between PrEP users (n=23) and non-PrEP users (n=16), there was no statistically significant difference in expressed motivations to remain HIV-uninfected.

Figure 3. Study procedure for testing of naïve users in New York City.

40 participants were recruited based on eligibility criteria focused on MSMs who were considered high risk for HIV/STI transmission based on self-reported sexual behaviors. Participants who were enrolled underwent two quantitative survey and a skill demonstration with our device that was video recorded and scored for analysis. Finally. participants underwent an in-depth qualitative interview, which was audio recorded, transcribed and coded for evaluation. One participant tested HIV positive and did not complete the study, as per eligibility criteria for continuation (qualitative and quantitative interview data were not included in study analysis).

Device credibility

Most participants (over 70%) felt “high” (5) or “moderately high” (4) confidence in results received through the mChip when asked to verbally rate on a 1–5 scale (1 being low and 5 being high confidence) (Fig. 4). Many participants perceived blood as a sample input would be more accurate than an oral saliva swab (Supplemental Table 3), and interestingly, expressed greater confidence in receiving disease-negative results, based on their own perceived risk and behaviors, stating that positive results would cause them to pursue confirmation with a second test or at a clinic (Supplemental Table 3). Those who expressed less confidence in results generated by the mChip device stated this belief stemmed mainly from feeling unsure about performing steps of the test correctly (e.g. if the fingerprick blood collection was acceptable) due to the lack of feedback from the device or app (Supplemental Table 3). Many also expressed that confidence in results would increase if the mChip had regulatory approval (e.g. FDA approval) (Supplemental Table 3).

Figure 4. Credibility of mChip.

Participants’ responses on level of confidence on a scale of 1–5 (high confidence to low confidence), in results generated from the mChip.

Device accessibility

We explored device accessibility by examining training burden as well as willingness to purchase the mChip. In qualitative interviews, participants overwhelmingly described the smartphone app (video and step-by-step directions with photo guides) as easy to use and follow (Supplemental Table 4). The smartphone also provided a familiar platform that users could navigate (with minor comments relating to improving the user-interface of the app through increase in font size and audio options for the step-by-step instructions). Most participants (56%) stated a willingness to pay up to $25 USD for one mChip kit, which would include the reusable mChip device, 2 sets of disposable microfluidic HIV/syphilis test cassettes and fingerprick materials (lancet, gauze, alcohol swab, bandage) (Fig. 5A). Three participants stated they would not purchase the kit and no participant expressed willingness to pay over $100 USD, however overall preference aligned with cheaper price points. The current in-house manufacturing cost of the mChip device is approximately $59 (USD) (Supplemental Table 2). There was a more bimodal distribution in price points that participants were willing to pay for each disposable HIV/syphilis test cassette, around the $3 USD range (~20% willing to pay less than $3 and 31% willing to pay $3-$6) and the $12 USD range (21%) (Fig. 5B). There was no significant correlation between self-reported annual income and willingness to pay for a mChip kit (Spearman’s rho coefficient = 0.084, p =0.637), or for disposable microfluidic cassettes (Spearman’s rho coefficient = 0.25, p = 0.887).

Figure 5. Accessibility of mChip.

(A) Price (in USD) that participants were willing to pay for a full mChip kit (device and 2 sets of disposable microfluidic test cassettes and fingerprick supplies) and (B) Price (in USD) that participants were willing to pay for disposable microfluidic HIV/syphilis test cassettes alone.

Device acceptability

First, we posed questions about participants’ willingness to use the test if it was available. 95% of participants would be likely or definitely likely to use the mChip on themselves, if cost were not an issue (Fig. 6A), and 85% also expressed high likeliness to use with sexual partners. Interestingly, there was no correlation between those who had previously used the Oraquick HIV self-test and willingness to use the mChip, suggesting that even those who had not previously used other STI home-testing products were equally likely to use the microfluidic mChip. Participants expressed that it would be very easy (86%) or easy (15%) to keep the mChip in their homes (Fig. 6B), as they liked the option of doing test at home with follow-up options at a clinic or connecting with a doctor (particularly with potential positive test results), and 74% felt it would be easy to carry it when going out (Fig. 6B). Participants were motivated to use the test in situations with perceived risky sexual behavior or encounters, including the use of drugs or alcohol that might affect judgement (cited 44% of the time), and also as preventative testing to know or confirm one’s status (cited 34% of the time). Participants also liked the ability to get syphilis test results (Supplemental Table 5).

Figure 6. Acceptability of mChip for consumer use.

(A) Participants’ ratings on likeliness to use the mChip for self-use and with sexual partners if cost were not an issue. (B) Participants’ ratings on how easy or hard it would be to keep the mChip at home or carry on the go. (C) Participants’ likeliness to use mChip with different types of sexual partners, including single-encounter partners (“one-night stands”), spouse-equivalents (defined as those participants felt an emotional connection with) and all other partner types, if cost were not an issue. (D) Participants’ responses on approaching partner-testing with the mChip including how hard or easy it would be to show a partner how to use mChip, ask a partner to be tested and raise the idea of using mChip with a partner. (E) Perceived influence of drugs or alcohol on ease of using mChip with partners when participants were under the influence or when partners were under the influence.

Partner-testing is also an important for reducing HIV transmission¹⁵. 92% of participants indicated a higher likelihood to test spouse-equivalent partners (defined here as those with whom participants shared an emotional connection or were repeat partners), compared to 74% who would use it with single-encounter partners (“one-night stands”) and 87% who would use with all other types of partners (Fig. 6C). Interestingly, when asked about perceived use with potential partners, over 70% of participants felt that it would fairly easy or very easy to raise the idea of using the microfluidic mChip with a partner (Fig. 6D), with all participants stating they would find it easy to show a partner how to use mChip (Fig. 6D), and 69% of participants feeling comfortable performing the fingerprick HIV test themselves on a partner. When asked how they imagined typical partners would feel about having a fingerprick HIV test performed on them, participants’ responses were more split, with slightly less than half (41%) envisioning partners feeling uncomfortable. The participants were split on testing partners themselves or having the partner test himself. Another source of concern was usage of test with partners under the influence of alcohol or drugs. Participants expressed it would be slightly easier to use the test while they were under the influence (62%) compared to partners being under the influence (51%), though both cases revealed alcohol/drug use being a potential barrier to using the test (Fig. 6E). Importantly, 85% of participants also expressed they would find it easy to avoid intercourse with partners who refuse to test, suggesting potential impact on sexual behavioral choices could be made through a self-test product. 85% of participants also expressed high confidence in their ability to judge whether partners could become violent over using it and over 87% of participants stated they would find it easy or very easy to avoid violent situations that might arise out of using a test. For disadvantages, 22% of participants stated reservations on how partners might react, and 10% stating concerns on what to do if a partner tested positive (e.g. providing counseling). One participant mentioned that having a self-testing product like this could add to “politicization” of people’s decisions to test or not test, and subsequent stigma or social pressure based on these decisions. Finally, participants supported incorporating a password-protected profile to increase the privacy and confidentiality of the results stored within the app.

Device usability

From initial development of the microfluidic mChip system for HIV testing⁵⁰, we have engineered multiple elements to increase usability – from sample collection via fingerstick whole blood¹, stability of reagents^{1, 50, 51}, and digital interpretation of results^{1, 51} – and demonstrated successful use by healthcare workers previously unfamiliar with our device to achieve sensitivity and specificity of clinical relevance. Here, we apply a methodical study design to further assess usability which could identify further areas for technical refinement, especially for untrained users. During an initial skill demonstration, participants were given the mChip test kit consisting of fingerprick materials, disposable sample collector and microfluidic cassette, the mChip device and a smartphone pre-loaded with an app (Fig. 2). 89% of steps were performed correctly, with an average total time of 15.7 minutes to complete the test (Fig. 7A, 7B). Since this time included a built-in 10 minutes of timers and the instructional video, the average total time to complete all manual steps of the test was around 4.6 minutes (Fig. 7B). We made slight modifications to the instructional video in applying the gauze and bandage the pricked finger, but observed no differences between the two sets of participants in performing each step of the test correctly or overall number of steps performed correctly (87% of steps performed correctly in first set and 89% in the second set) (Fig. 7A). 90% of participants felt confident in their ability to perform the test correctly (Fig. 7C). While there was some variability in collecting blood, all participants successfully performed a fingerprick on themselves, with 75% of participants choosing to perform the fingerprick on their left forefinger, matching the finger shown in the instructional video.

Figure 7. Usability of mChip with naïve users.

(A) Usability rubric for running the mChip device showing number and percentage of participants who performed each step correctly. Slight modifications to the instructional video and app directions were introduced between the first and second set of participants (n=20 in each), with p-value calculations to show potential differences in performance. Two-proportion z-test was applied for p-value calculations. (B) Timing of test use showing a breakdown of time to completion, instructional video length, built-in timers and time of manual steps. (C) Participants’ self-evaluation of confidence in running the test correctly on a scale of 1–5 (not confident to very confident). (D) Comparison of positive control signals after channel priming. A 2μL plug of 0.05% Tween-PBS was flowed through the channel. At time points shown after channel priming, a 2μL sample of disease negative whole blood was introduced, followed by a standard reagent sequence for reconstitution of lyophilized gold-labeled anti-hIgG/anti-hIgM antibodies (three washes of 0.05% Tween-PBS, and four washes of water) and silver amplification. Data are averages ±1 SD (n=3). Asterisk (*) indicates statistical significance (p<0.05) using Student’s t-test. A comparison of signals at time point 0 and 2 hours after channel priming yielded minimal significance (p = 0.058). We observed a decreased signal in positive control zone signals with longer times between channel priming and running the assay. (E) Effect of lag time on fingerprick whole blood flow time through microfluidic channel. Condition 1 consisted of lag time measured in minutes between pricking the finger and subsequent collection and immediate introduction to the channel. Condition 2 consisted of collected the fingerpricked blood sample into the capillary tube of the sample collector, followed by lag time at indicated time points before introduction into microfluidic channel. Time for blood to flow through channel was measured in seconds from inlet to waste pad (end of channel). We saw that uninterrupted flow of the 2μL, unprocessed fingerprick blood sample sample to reach the waste pad, with channel priming was between 2– 2.5 minutes. 90% of participants took more than one minute between fingerprick blood collection and starting the assay (i.e. engaging the vacuum for fluid flow). (F) User activated negative pressure-driven flow showing average time to flow a total sequence of 4 washes: two-2uL washes of 0.05% Tween-PBS and two 2-uL washes of DI water, are shown. All three users were naive users (not enrolled in study, but unfamiliar with device). Data are averages ±1 SD (n=3). Asterisk (*) indicates significance, using one-way ANOVA followed by pair-wise t-test (P = 0.0248).

Attachment of the sample collector was performed correctly by 100% of users (Fig. 7A). Challenging steps including inserting the microfluidic cassette into the mChip device (Fig. 7A), and collecting 2 uL of fingerprick blood in the sample collector (“Part A”), with 20% of participants overfilled past the indicated line and 5% of participants did not collect sufficient blood. The steps cited by participants as relatively difficult to perform were also inserting the microfluidic cassette into the mChip device (cited 18% of the time) and collecting the fingerprick blood in “Part A” (15%). While 80% of participants performed this step correctly, the next most common step cited as challenging was pushing the vacuum bulb (15%), either due to the physical resistance of the bulb or feeling unsure about the step. We investigated possible design improvement changes to increase sample movement through the chip, as shown in previous successful field studies¹, including the effect of (1) channel priming with a wash buffer prior to introduction of whole blood samples and (2) user-induced lag time between fingerprick blood collection and flow through the microfluidic channel (Fig. 7D, 7E). While further investigation would be useful, the results suggest that including a channel-priming step immediately prior to sample introduction, and standardizing the lag time to initiate vacuum, could be useful engineering design steps to facilitate blood flow in the chip and address user-variability in timing of operational steps. Also, while switching to a smaller bulb in vacuum chamber decreased the overall footprint of device (Supplementary Fig. 2), it resulted in more variation in fluidic actuation amongst naïve users. We recruited three additional naïve-users to assess flow times using a standard reagent sequence of washes, and saw a statistically significant difference (P < 0.05 using one-way ANOVA followed by pair-wise t-test, P = 0.0248), between flow times for two out of the three users (Fig. 7F).

DISCUSSION

Untrained users.

In our previous work with a smartphone-enabled microfluidics-based HIV diagnostics device, healthcare workers operated the device under supervision by a research team¹. Here, we found that completely untrained participants at high-risk of HIV transmission and of varying educational background levels were motivated to avoid HIV infection, and found a POC microfluidics diagnostic device to be largely credible, accessible, acceptable, and usable (Fig. 8). Credibility of the device was moderately high, and perceived to increase with more feedback from the device indicating steps were performed correctly. Accessibility was high, as the device had low training burden and was low-cost. Acceptability for home testing was generally high, in that participants saw value in a product that was relatively easy to use, portable, provided both HIV and syphilis test results, and utilized the smartphone as a platform device. These users could navigate the use of a microfluidic HIV-syphilis test via guidance from a smartphone app, with over 85% of steps performed correctly. Nevertheless, testing with naive users pointed to areas for improvement. Most importantly, the study identified failure points in the operation of the device which were not due to user error or device failure, but rather in user-variability in operation, which prevented the generation of a complete test result. For example, of the steps that had less than 90% of participants performing it correctly, most incorrect steps (e.g. steps 6, 7, 10, 11, and 14) were because of the user variability issue. (These findings are consistent with a previous study on oral and fingerstick HIV tests in sub-Saharan Africa, where fewer than 25% of the participants carried out all the steps correctly with most errors in sample collection and transfer⁵².) This feasibility study design allowed us to target high-yield modifications based on real user-data; the device had low tolerance for the variability exhibited by naive users at certain procedural steps (collecting fingerstick blood by visual inspection, fully inserting the chip intro the device, initiating the vacuum), however we were able to identify mechanisms in the design of the device which could reduce the variability and increase usability, such as feedback mechanisms which are binary (such as a switch) or provide direct information (a light or a click) as well as modifications to the assay (e.g. channel priming prior to sample introduction) or device design (e.g. binary vacuum actuation mechanism). Overall, the study illustrated how designing for the intended end-user and use-case is an important factor in POC design.

Figure 8. Findings for mChip according to UX-IMB framework.

Key findings from quantitative and qualitative measurements of the UX-IMB framework in this early stage, feasibility study assessing theoretical device value for HIV home-testing application.

Comparison with previous mChip studies.

In contrast to the now FDA-approved microfluidics test from OPKO Diagnostics⁵³ the smartphone-enabled microfluidics-based HIV mChip test is still in development in an academic setting⁵⁴ Whereas usability analysis is known to be important for late-stage commercialization and regulatory filings, this study demonstrates the opportunities for integrating usability findings (this study took approximately 6 months for recruitment of participants and collection of data) into the engineering design process at an early stage of development when fundamental design choices have not yet been finalized⁹ In our previous studies of the HIV test in sub-Saharan Africa¹ healthcare workers were accustomed to performing fingersticks and running diagnostic tests, in contrast to naïve users in the current study; while previously, patients did not use the device directly themselves, they preferred fingerstick sampling compared to venipuncture, and appreciated short turnaround time to receive test results¹, consistent with the high acceptability found in this study.

HIV self-testing and smartphones.

Integration of smartphones with microfluidics will play a key role in adoption of new consumer diagnostics⁹. This study expanded our understanding of how smartphones could influence device acceptability and usability by general users. Perceived value of the test, which included automation of test interpretation, digitization of results, enabling use with sexual partners, and providing linkage to care options, increased acceptability. Factors that increased usability included low training burden (through the in-app video and step-by-step directions) and familiarity of smartphone interface. An area for improvement in accessibility: whereas the current version of the device costs ~$60 USD under in-house manufacturing protocols (Supplementary Table 2), it is higher than the $25 price point many of the participants in this study stated they would be willing to pay for a full test kit, which includes the disposable fingerprick supplies and microfluidic cassette components. By comparison, an OraQuick home-testing kit retails for ~$45 USD^{55, 56} (in New York City), but it is single-use. It consists of multiple components which could decrease usability: a lateral-flow test, one test tube filled with 1-ml of liquid reagent, written testing directions attached to a plastic box, pre- and post-test booklets, disposal bag and pencil. Additionally, full or partial insurance reimbursement models could be implemented to bridge gaps between final retail costs and out-of-pocket expenses for consumers. Also, while some studies have shown user preference for oral sampling⁵⁷, other studies have shown acceptability of blood-based testing to increase if additional STIs can be detected⁵⁸; in our study, many also perceived that blood based testing was seen as more accurate than saliva-based testing, increasing its acceptability. For comparison, prices of other HIV/STI self-testing products vary from $45 for an at home-STI test⁵⁹, $79-$350 for mail in testing⁵⁹, $219-$349 for STI panel testing⁵⁹, and newer venture-backed funding models based on app purchases⁶⁰.

Integration of social-science models with technology to achieve impact on public health.

To maximize impact of technology on patient care, there exists an opportunity to integrate established social-science models⁶¹ with engineering design processes to design persuasive technologies for consumer health⁹ and influence patient behavior towards positive health outcomes. This study presents a user-centered framework, which incorporates the Fisher-Fisher IMB model for user behavior and the Honeycomb UX model for user experience, for studying user behavior and experience with POC diagnostic devices, especially those targeted towards home- or self-testing. The UX-IMB framework recognizes that achievement of a desired analytical or clinical sensitivity and specificity are necessary but insufficient to motivate consumer use of health devices, and the importance of understanding specific use-settings which differ in constraints and requirements⁹. Specifically, an understanding of user experience and information, motivation, skills, and behavior feedback into design considerations in the engineering design process, and is needed to develop devices that are poised for widespread adoption by target end users in diverse settings. A limitation of the current study is that a survey of high-risk MSM, a population which was relatively knowledgeable about HIV testing. may reveal different results in places from a different or more general population. Finally, to complete the study of user behavior and experience with our POC diagnostic device, we will need to study the device value in the future, by empirically measuring the uptake of the device, changes in health and behavioral outcomes within the framework prescribed in this study.

Table 1.

Participant demographics in self-testing study (New York City, U.S. N=40^a)

Age, mean (range)	38.7 (20– 73)
Income, (USD)
Mean	37,501
SD	41,405
Range	0 to 220,000
Race, n (%)
African-American	19 (48.7)
White	14 (35.9)
More than one / Other	6 (15.4)
Hispanic/Latino, n (%)	10 (25.6)
Education, n (%)
Partial/Completed High School	9 (22.5)
Partial/Completed College	22 (55.0)
Graduate School	9 (22.5)
Sexual Identity, n (%)
Gay	29 (74.4)
Bisexual / Other	10 (25.7)
Previously used commercial, rapid HIV home-test, n (%)
Yes	20 (51.3)
No	19 (48.7)

^aNs may not sum to 40 due to missing data