ELECTRONIC SCREENINGFOR MENTAL HEALTH IN RURAL PRIMARY CARE: FEASIBILITY AND USER TESTING

Abstract

Despite attention to prevention and screening for depression and alcohol use, Healthy People 2010 objectives continue to include goals to increase the detection of depression and decrease the rates of alcohol abuse. These problems remain significant. The overall goal of this study was to develop a computer-based electronic screening tool and to determine the feasibility of implementing computer-based electronic screening technology (eScreening) for rural visitors to a primary care clinic. The study called specifically for an electronic touch screen with voice prompts. This tool, called the eScreening tool, screens for alcohol abuse and depression among rural patients in a primary care setting. The screening was offered to rural adults who are not in acute distress and not at end-of-life, regardless of their stated reason for seeking medical care. Phase one of the pilot was used to determine the perceptions of nurses, other providers and consumers regarding the acceptability and perceived usefulness of an eScreening tool. Phase two involved user testing of the eScreening tool. The longer-term goals of the research program are to work with rural nurses to improve patient outcomes, develop interventions, and for educational, consultation and/or direct clinical care.

Innovative technology offers unlimited opportunities to enhance healthcare delivery through screening, assessment and appropriate treatment of patients in primary care settings. Screening is the first step of secondary prevention, defined as “a defensive posture or set of actions that ward off specific illness conditions or their sequelae that threaten the quality of life or longevity."¹,^p38 Because of its effectiveness in enhancing health outcomes and reducing overall expenditures, screening has been identified as a preventive healthcare intervention that should be promoted so that it will become part of the public consciousness of health care consumers, providers, purchasers, administrators, quality managers and public policy makers. Screening of patients living in rurally-designated areas is especially important because of low levels of health services and fewer medical professionals. ^2–4

Individuals living in rural areas have documented problems accessing health care due to transportation issues, isolation and long waits in clinics.^{3, 5–7} Other challenges facing rural primary care clinics include health-provider shortages, low numbers of specialty providers and high numbers of uninsured and under-insured residents.^8–10 It is the rural resident who may benefit the most from innovative approaches to health information access, screening, and prevention services. Since electronic health or eHealth mechanisms that are culturally appropriate, reliable, and valid may be able to improve delivery of health care services in rural areas, it is important to determine if electronic screening is acceptable and effective.

Screening and prevention efforts have received national attention with the Healthy People 2010 objectives.¹¹ Another United States Administration report, The Decade of Health Information Technology states that information-technology use is suggested as an emerging proactive strategy for prevention and disease management with four targeted goals: a1) “informing clinical practice” to help facilitate decision-making by health professionals, b) “improving population health” to track health problems and facilitate research, c) “inter-connecting clinicians” by making patient records accessible to multiple care sites, and d) “personalizing care” by involving patients in their care and encouraging personal responsibility.¹²

Theoretical Framework

Roger’s Diffusion of Innovations framework ¹³in which typologies for adoption of innovation are identified was used to guide this study. Not all individuals adopt innovations with the same rapidity. Personal characteristics can determine in large part how willing someone may be to adopt an innovation. Electronic screening is considered an innovation in this study in that it is a new way of conducting an intake as patients visit the primary care clinic. In addition to the usual vital signs and weight, nurses would have a new tool to screen for health behaviors and symptoms, in this case specifically depression and use of alcohol. The “early adopters” were readily identified and were key to successful user-testing and subsequent small-scale implementation.

Nurses in primary care already assume the first-line function of triage or intake by completing assessments. Besides measuring basic vital signs, nurses also quickly assess cognitive and physical function. Unfortunately, the time needed to assess mental health issues is too often delayed or deferred until a crisis event occurs.^{14, 15}The American Nurses Association has said that “the strength and promise of technology lie in providing increased access to health care services by augmenting existing services, not replacing them.” ¹⁶ An electronic screening tool that could quickly screen for alcohol and depression might be very useful in supplementing routine intake assessments.

Based on this theoretical framework, the specific aims of this study were to 1) explore the perceptions of consumers and providers concerning the intervention of eScreening (Phase I), 2) test a newly developed eScreen tool (Phase II) and 3) explore consumers’ responses to implementation of the eScreening (Phase III). The first two aims were explored through both a provider focus group and a consumer-testing phase in which consumers were asked to think out loud as they completed the eScreen. The results from Phase I and II will be reported here. A separate article reports on the other aim: results of consumer responses to eScreening. Other outcomes such as effectiveness in identifying depression and alcohol abuse and comparisons to national norms are reported there.

The Use of eScreening for Assessment

Most mobile persons in communities, even if living in rural areas, have used touchscreen technology on automated teller machines (ATMs), automated election/voting ballots or to obtain a ‘made-to-order’ sandwich at convenience stores. Touchscreen technology provides a safe and secure yet easy-to-use system. No typing skills or comprehensive education is required to use this type of technology. While there has been no study reported in the literature regarding patient use of laptops for self-report of key assessment information, the introduction of this technology offers a unique opportunity to dramatically change the way key information is obtained from patients.^{17, 18}

One of the first applications of the computer for assessment was the computerized psychiatric interview.¹⁹Researchers reported that patients not only responded positively to computer interviews but also gave honest answers.²⁰ Subsequently, medical, marketing, personnel and social science researchers have explored computer administration as a means of reducing social-desirability biases and obtaining more-sensitive information from respondents than could be obtained using more-traditional formats. A belief that computer administration encourages self-disclosure has led to the development of important applications, such as computer interviews to detect risk conditions and behaviors of blood donors.²¹ While electronic screening tools have been discussed in the literature, ^{17, 22–26} little is known about their use in rural areas and in the screening of topics such as depression and alcohol abuse. Weisband and Keisler conducted a meta-analysis of the literature from 1969 to 1994 to compare levels of self disclosure on a computer or paper form versus a face-to-face interview.²⁶ They found that across 39 studies using 100 measures, computer administration increased self-disclosure. Recent studies using touchscreen self report support this earlier finding. ^{22, 24, 25}

One example of an electronic screening tool currently being used is the Healthy Town website in the state of Ohio.²⁷ Using innovative computer screening technology and easy-to-use health information, Healthy Town identifies health and injury risks and links seniors and families with children to community prevention services. Another study examined the use of computers to reduce medication misuse of community-based seniors.²⁸ Seniors completed a computerized simple screening for medication misuse and watched short related video clips. Almost all of the seniors found it easy and helpful.

Numerous studies reported using computers with persons having poor vision, lower levels of literacy or when eliciting answers to sensitive issues. To meet the diverse needs of patients, one computer-technology design included an audio computer-assisted, self-interviewing (ACASI) format. ^{29, 30}With ACASI, the questionnaire is programmed on a computer to be displayed on the screen. Each question is voice-recorded along with accompanying instructions to navigate the computer screens. The respondent hears the questions as they are visually displayed on the screen and then answers the question by touching the computer screen. Answering questions via touchscreen does not require keyboard skills allowing for persons to answer questions without difficulty.³⁰Results from ACASI are then immediately available to the professional as part of the health-screening assessment.

One study by Thornberry and colleagues used both ACASI and touchscreen technology to detect alcohol consumption in primary care. ³⁰Their findings supported the feasibility and acceptance of this technology with the respondents who reported that they liked answering questions by computer (90%) and that ACASI was not difficult to use (96%). Another study using ACASI in persons with severe mental illness found this form of data collection to be a valuable tool in improving care.³¹

In order to enhance participation and promote understanding of the questions asked in the eScreening assessment in our study, voice-prompt technology (ACASI) was added to the touch-screen portable computer. This combination of technology provided both visual and auditory prompts to participants in the study. Additional benefits of using technology versus paper and pen include ease of capturing key information and ease of reporting outcomes.^{23, 25}

EScreening provides an opportunity for patients in a primary care setting to participate in their care in an active way by reporting key information about how they are doing using psychometrically-tested instruments. The introduction of eScreening technology changes the way nurses and patients interact. Instead of only face-to-face assessment, the triage nurse and providers have the additional information obtained from eScreening. EScreening has the potential to impact outcomes of care for patients. In a landmark study of patient outcomes used by nurses to evaluate effectiveness of practice, Barrell, Merwin and Poster found the use of patient self-report an important method of outcome evaluation. ³²

Setting

This study was conducted at the University Medical Associates (UMA) at the University of Virginia (UVA) Primary Care Clinic. UMA is a general internal medicine clinic at UVA. With over 25,000 patient visits a year, UMA is the largest provider of care to indigent patients in Central Virginia. More than 76% of patients served by UMA live in rural areas, as determined by the U.S. Bureau of the Census (Source: UVA Health System Clinical Data Repository). The majority of patients qualify for reduced rate care and 40% are totally medically indigent. The patient population is diverse with 45% from minority groups, mostly African-American. Many live in relative isolation, have low literacy skills, lack social support and face substance-abuse problems and other physical and mental co-morbidities. The rate of chronic illnesses is high and many of these are exacerbated by co-occurring major depressive disorder (MDD). Based on ICD-9 codes, 18% of 6300 regular patients (1231) in one primary care clinic had a diagnosis of MDD during 2001.³³ Of these, 486 were newly diagnosed.

Methods

Phase I involved a focus group and the development of the eScreening device for use in Phase II user testing. Participants for the focus group were nurse and physician providers and staff at the primary care clinic of University Medical Associates (UMA) at the University of Virginia. The providers participated in a guided discussion regarding the idea of electronic screening, availability of space and connectivity for hardware, best location for eScreening in the examination rooms of the primary care clinic, and content of the eScreening questionnaire. The Phase I meeting took place in the UMA conference room and staff were interviewed during the lunch hour. One expected outcome of these meetings was that investigators would develop a better understanding of the culture of the rural community that UMA serves. This understanding was used to adapt the innovation and facilitate adoption at all levels of the organization. There were between four to eight members in the focus group, with the session lasting approximately 45 minutes. The PI, a research assistant and a technology expert with content knowledge in the area co-led this focus group, the responses were recorded on a flip chart to help facilitate the discussion and the proceedings were audiotaped for transcription purposes.

Following the focus group, a work group was formed that included the research team and selected stakeholders to adapt the innovation to address the concerns and comments from the data-gathering interviews. These work meetings involved both engineering and technology programmers who helped determine the best technology to adapt the screening. Following these meetings, the study team completed the design of the eScreening and converted it to a touch-screen user interface.

The depression and alcohol instruments were then converted to a touch-screen program. The existing paper and pencil screening instruments were adapted for use with a computer using Microsoft Access to develop a database and software such as Flash® which allowed the creation of sounds and graphic radio buttons in different colors to develop the user screen. As a participant answered each question, the answer automatically went into the computer database. The instruments were adapted, using hypertext markup language (html), so that the participant would also be able to touch the screen on the computer. The computer was a type of screen where the user can touch a large button on the screen instead of using a mouse (see Figure 1).

Figure 1.

Sample Screening Format

In the user tests, nine rural consumers were recruited from UMA, observed and interviewed as they worked through the program. To recruit consumers for the user test, flyers were posted in the waiting area of UMA for participants to test the eScreening tool. Excluded from the study were patients who were 1) under 18 years of age as the screening has been developed for adults, or 2) in acute distress, in order not to interfere with their urgent need to be seen for their medical problem. This stage consisted of three separate tests of three persons each, separately and individually in a private location. This process was iterative so that the last two groups responded to changes made as a result of previous sessions. Suggestions made by the nine participating consumers in the user testing were taken to the programmer in order to finalize the touch-screen conversion of the program before Phase III implementation.

The participants were asked to think aloud about both the navigation and content as they completed the program. They provided feedback about the ease of using the program, how to start the program, how to end the program, how to print out the results of the screening, and whether the buttons were large enough and the instructions clear enough. The PI or research assistant took notes, asked questions of consumers and compiled a list of changes that needed to be made to the program.

Analysis of the user test took place through heuristic evaluation as participants completed the user tests, and were observed and interviewed as they used the eScreening tool. Heuristic evaluation, the most popular of the usability inspection methods, is a systematic inspection of a user-interface design for usability.³⁴ The goal was to find the usability problems in the design so that they could be attended to as part of an iterative design process. In addition, investigators also used heuristic analysis with participants and ordered the themes in priority. The investigators then worked with the primary care staff and other consultants to create the final product: eScreening. The analysis took place in three different iterations, and after each one, results were given to the programmer to improve the innovation before the next group of three.

Findings

Findings from the Focus Group included that the eScreen is valued and needed and that the eScreen would be acceptable the eScreen is valued and needed and that the eScreen would be acceptable if presented in the flow of the regular work appropriately. Also it was believed by members of the Focus Group that the eScreen results would potentially be helpful to the care provider and the patient. Because literacy is sometimes a problem, it was felt that an audio-assisted eScreen would be preferred.

The preliminary version of the eScreening tool underwent user testing in Phase II. An examination room, set aside for the user testing, housed a cart with the eScreening system: a computer with the Flash source file and a touch screen that interfaced with the computer. Nine patients not in acute pain volunteered to take the screening over the course of three days. These users were instrumental in identifying basic difficulties with the interface and provided feedback to help evolve the design. Observation of the users, and thereby identification of other problems inherent to the system the user may not have noticed, also provided important feedback for improving the system. The eScreen was found to be valued and needed and acceptable if a suitable time and location was found to fit in with the workflow of the clinic.

Audio problems identified included difficulties with the volume and a malfunctioning sound toggle button. Also of concern to users was the inconsistency in volume of the voice-over in the movie clips at the start of each question. Also, some of the voice-overs were difficult to hear because of static. The audio problems were addressed by re-recording all of the voice-overs. This resulted in a louder and consistent volume level for the movie clips.

Other issues included uncertainty with the movie clips and the “Next” button. Some users were uncertain whether they must listen to the entire clip before selecting an answer. Also, users did not notice or understand the function of the “Next” button. The latter issue was addressed by flashing the “Next” arrow once it appeared to capture the user’s attention. Furthermore, for both issues, the tutorial was re-designed and re-recorded to eliminate ambiguity.

Administrative issues that arose from the user testing included printing and navigation difficulties. Printing required manual intervention and administrators could not skip the tutorial. To address these issues, an automatic function to print was made and an administrative control was added to skip the tutorial.

This user testing provided important feedback for improving the physical implementation of the system, the amount of time required for the screening, and the presentation of the interface in a way that minimizes security risk and administrator interference. However, to assess how/whether an electronic screening device such as eScreen is useful in addressing the unique needs of the rural population, a much larger scale testing with rural patients is necessary.

Conclusions

The study provided useful pilot information for guiding development of an intervention trial of computer use in this vulnerable population. It also provided an estimate of the efficacy of computer use in this patient population. The eScreen is valued and needed and is acceptable if suitably used in the routine workflow. Needed revisions and adjustments were made to the original eScreen.

There is a need to identify new and innovative applications to improve assessment and possible treatment of depression, especially with the rural adult population. This study explored the combined use of audio computer-assisted touchscreen technology with the use of existing psychometrically-tested screening instruments to screen for alcohol use and depression in a primary care population.

Consistent with the goals of The Decade of Health Information Technology report, ¹² this eScreening research study specifically met the goal of involving patients in their care. Providing a printout of the eScreening results to both the patient and the healthcare provider also increased the power of the intervention to inform clinical practice. Despite the small sample size, this pilot study provides important preliminary findings to help guide a larger study of eScreening in rural primary care.