Provider perceptions of the value of same-day, electronic patient-reported measures for use in clinical HIV care

Abstract

Strong evidence suggests that patient-reported outcomes (PROs) aid in managing chronic conditions, reduce omissions in care, and improve patient-provider communication. However, provider acceptability of PROs and their use in clinical HIV care is not well known. We interviewed providers (n=27) from four geographically diverse HIV and community care clinics in the U.S. that have integrated PROs into routine HIV care, querying perceived value, challenges, and use of PRO data. Perceived benefits included the ability of PROs to identify less-observable behaviors and conditions, particularly suicidal ideation, depression, and substance use; usefulness in agenda-setting prior to a visit; and reduction of social desirability bias in patient-provider communication. Challenges included initial flow integration issues and ease of interpretation of PRO feedback. Providers value same-day, electronic patient-reported measures for use in clinical HIV care with the condition that PROs are 1) tailored to be the most clinically relevant to their population; 2) well-integrated into clinic flow; 3) easy to interpret, highlighting chief patient concerns and changes over time.

INTRODUCTION

Patient-reported measures, also known as patient-reported outcomes (PROs) are assessments of a patient’s health and disability experiences elicited in a structured and standardized format directly from the patient (U.S. Department of Health and Human Services, 2006). PROs are gaining acceptance in clinical practice settings and are increasingly demanded by health care regulators, payers, accreditors, and professional organizations (Jensen et al., 2015; Marshall, Haywood, & Fitzpatrick, 2006; Valderas et al., 2008; E. H. Wagner et al., 2001; E. H. Wagner, Austin, & Von Korff, 1996). It has been suggested that buy-in from providers and clinic leadership is essential to successful integration of PROs into care (Fredericksen et al., 2012; Jensen et al., 2015).

While provider acceptability of PROs has varied by medical discipline and across types of care (Myrvik, 2013; Stover et al., 2015), provider acceptability in clinical HIV care is not well known.

Strong evidence suggests that point-of-care collection and reporting of PRO data aids in managing chronic conditions (Dobscha, Gerrity, & Ward, 2001; Marshall et al., 2006), and reduces errors of omission in care such as under-diagnosis of depression and suicidal ideation(Lowe et al., 2003; Staab et al., 2001), substance use(Conigliaro, Gordon, McGinnis, Rabeneck, & Justice, 2003; Messiah, Loundou, Maslin, Lacarelle, & Moatti, 2001), inadequate adherence to medication regimens(Bangsberg et al., 2001; Gross, Bilker, Friedman, Coyne, & Strom, 2002; Paterson et al., 2000) and HIV transmission risk behaviors (Marks et al., 2002; Morin et al., 2004). In addition, PROs have been found to improve patient-provider communication in identification and discussion of issues related to health-related quality of life (Detmar, Muller, Schornagel, Wever, & Aaronson, 2002; A. K. Wagner et al., 1997), and increase patient satisfaction with care (Wasson et al., 1999). To these ends, our research team developed, deployed, and evaluated a point-of-care PRO web application running on securely-networked touch screen tablet personal computers (iPads) for geographically diverse sites in the Centers for AIDS Research (CFAR) Network of Integrated Clinical Systems (CNICS). The application was designed to 1) collect, assess, and report health related data from HIV-infected patients from domains perceived as valuable and relevant by clinicians and patients; 2) improve care quality by increasing the recognition and treatment of depression, alcohol and other substance use, inadequate antiretroviral medication adherence, HIV transmission risk behavior, and symptoms; and 3) inform patient/provider interactions and clinical decision-making in real time.

This work describes the perceptions of HIV care providers in our network regarding the value, challenges, and use of PRO feedback in clinical HIV care to identify attributes of PROs and PRO collection needed to maximize relevance for providers.

METHODS

Two Ph.D.-level interviewers with expertise in qualitative interviewing methods interviewed providers between 2010 and 2014 at four CNICS clinics that had been administering PROs for two or more years: Fenway Community Health in Boston, MA; the University of Alabama-Birmingham’s 1917 Clinic; Owen Clinic at the University of California-San Diego, and Madison Clinic at Harborview Medical Center/University of Washington in Seattle, WA. At these sites, PROs had been self-administered on-site by patients on touch-screen tablets, generating feedback viewed by their providers immediately prior to their clinic visit. Depending on the site, feedback is viewed in either paper or electronic form, outlining depression level/suicidality (PHQ-9), drug use (ASSIST), alcohol use (AUDIT), adherence to antiretrovirals, HIV transmission risk behavior, and standard review of symptoms.

Providers were recruited in-person by a designated research coordinator at each site. Interviews lasted up to one hour, were digitally recorded, and conducted in private. There was no remuneration. Providers were assured that all responses were confidential. IRB approval for speaking with providers at all sites was obtained through the University of Washington Institutional Review Board.

An outside service transcribed digital recordings of interviews. We used Dedoose software to excerpt transcripts along the following general content areas: benefits of PROs, challenges with using PROs, and ways in which PROs are used in clinical practice. Two trained qualitative researchers that did not conduct interviews then coded the excerpts according to these content areas. Two separate coders then used a set of established codes to identify sub-themes. Although inter-rater reliability was high (89%), a third coder reviewed and reconciled where differences arose. In all of these cases, third coder agreement was established with one of the two coders, comprising the final code.

RESULTS

We reached thematic saturation among a multidisciplinary sample of providers (n= 27) (see Table 1). Results are presented here as perceived benefits and challenges of using PROs.

Table 1.

Providers, by discipline and site.

	Fenway	UAB	UCSD	UW	Total
Physician (MD)	3	6	3	5	17
Physician Assistant	3	-	-	-	3
Nurse Practitioner	2	-	-	-	2
Registered Nurse	-	2	-	-	2
Social Worker	-	-	-	3	3
Total	8	8	3	8	27

Benefits of PROs

The benefits of PROs were described in terms of saving time, promoting clearer communication between patient and provider, and identifying issues not otherwise easily observable. We observed no clear differences in perception of benefits based on provider type or by site.

Helps set agenda for clinic visit

Over half of providers (52%) described PROs as useful in “setting an agenda” for the visit. Several providers commented that PRO data helped target their conversation by identifying the number and severity of problem areas, and allowed providers to avoid inquiry into health domains less relevant to particular patients. Most providers reviewed the same-day results remotely (i.e., from a provider workroom in the clinic) prior to greeting the patient. Many used the PRO data as a tool for initiating and structuring the conversation:

I'll just say something like, you know, “I noticed your PRO results tells us you're kind of depressed. Can you tell me what's going on…?" Or I'll say, "I notice you're missing some of your medicine. So what's happening? "…it's a nice starting point with a conversation as opposed to the more traditional, you know, starting at the bottom and working up (Physician, Birmingham).

Another provider identified PROs as a tool to promote patient-provider collaboration and improve patient involvement:

If my patient's done with [the PROs], I go in and say okay, we're going to review the PROs together. I talk to them about it in front of them, I want them to be part of the discussion. (RN, Birmingham)

Providers also perceived that the PROs promoted more efficient encounters by increasing transparency of providers’ concerns to patients and vice versa, priming both parties for frank discussions of potentially sensitive or embarrassing issues. One provider stated that patients’ repeated exposure to the PROs over time has the effect of “telegraphing” providers’ agendas to patients, giving patients a sense of what might be asked in the appointment:

Patients always have their agenda when they're coming in, but through the PROs they’re also getting a sense of what our agenda is… what I found interesting was that a few of the patients who completed the [PROs], they'll kind of launch and tell me what's going on with their tobacco use - without my even asking…(Physician, Seattle)

This ‘agenda setting’ was viewed as promoting more efficient information exchange, including information regarding ‘sensitive’ issues. Providers also expressed the view that PRO administration helped patients prioritize their own needs.

Identifies less-observable and difficult-to-discuss behaviors and conditions

Two-thirds of providers characterized PROs as a useful tool to identify problems that might have otherwise been minimized or omitted by the patient. Providers especially valued identification of depression and suicidal ideation, and almost universally mentioned this as a benefit of the use of PROs. Several providers described being surprised by the results:

Getting a printout that said “your patient is feeling suicidal” essentially was like “Whoa! I didn't expect this!” –(Physician, Seattle)

Today, the patient sat down [and] said, "Yeah, things are okay, I'm just not sleeping good." But when I looked at the PROs that he had just answered, he was suicidal a couple of times last week. (Physician, San Diego)

One provider echoed many others in the belief that PROs helped circumvent social desirability bias:

…We have found a number of people that were suicidal [in the PROs] that were not being honest with their provider [in person]…there’s a lot of people that don’t want to ‘disappoint’ their provider by telling them what’s really going on. (RN, Birmingham)

Several providers felt that the PROs took an efficient inventory of recent substance use, identifying nature and frequency of use, and acting as a conversation-starter:

Who has the time to ask about all of the types of drugs that people use? [The PROs] right away give you the spectrum of what people are using…from there you can structure the conversation. (Physician, San Diego)

I didn't know the extent of one guy’s alcohol use…we hadn't talked about it for some time. So it's actually quite useful to say “here's what you told the [PROs], let's talk more about that”. –(Physician, Boston)

Some providers noted the time cost of initiating and engaging conversations with patients about adverse health behaviors and mental health symptoms, given that it may take time during the visit for patients to feel comfortable enough to disclose these issues with the provider. One provider described the use of PROs during the visit as “triangulating” the communication dynamic, allowing for reprieve from direct eye contact as both parties review the PRO results on a computer screen or on paper, creating a sense of objectivity and teamwork in discussion of the content.

Challenges of PROs

Providers identified two key challenges to the value of PROs in clinical care: ability to integrate into clinic flow, and interpretation of PROs. These challenges were typically referenced in past-tense, as issues that had already been addressed and resolved in the initial phases of integration of PROs into clinical care. We observed no differences in perception of challenges based on site or provider type.

Initial flow integration issues

Forty-one percent of providers indicated flow integration issues as a challenge. Allocation of private space was a necessary first step for all clinics, to ensure that patients could concentrate and confidentially disclose their symptoms and health behaviors. Providers also identified the need to create a time slot specifically for the purpose of PRO administration approximately 15–20 minutes prior to the visit, or during a time during clinic flow when the patient is likely to be waiting for the provider to enter the room. Patients arriving late or starting the PROs too close to the appointment time disrupted clinic flow. Indeed, some providers pointed out that PROs may not be appropriate for every kind of patient:

Sometimes patients seem a bit overwhelmed by having to answer all of the questions and the broad scope of it. Some of these folks might be better off just skipping it. (Physician, San Diego)

Providers widely expressed that it would be counterproductive to administer PROs to patients with low language or computer literacy, severe mental illness, cognitive problems, or who simply have a low threshold of patience for questionnaires.

Interpretation of Results

Approximately half of providers indicated issues with interpretation of PRO results. A common concern was that PRO feedback made it difficult to tell what was baseline vs. abnormal for particular patients. In the words of one provider:

The scales don’t always help us distinguish between people who are chronically depressed and that's just how they feel all the time versus someone who is truly going to do something. It would be great if the question asks specifically “does this feel worse than normal?”. (Physician Assistant, Boston)

To this point, some providers expressed frustration at measures that excluded the extent to which patients were bothered by symptoms. In addition, one provider echoed the concerns of others when noting that a patient may not feel compelled to discuss all bothersome symptoms at that particular visit:

It would be nice if there was an option for patients to say, do you want to address this with your provider today? Because I as a patient would like that. (Physician, Seattle)

In general, providers reported struggling at times to prioritize large numbers of symptoms and behaviors in a single visit when endorsed in the PROs. In one provider’s words:

… I pay attention to those [PRO scores] that are moderate to severe…. there's a lot to cover in a particular visit and this is helpful, but also can be burdensome if it points out five other things that are moderate to severe. Like, oh my gosh, we just got through [treating] the HIV and the fact that you're failing therapy and you're drinking a lot and …there's six things here that are severe to moderate? (Physician, Seattle)

One provider cautioned against over-reliance of PROs to detect and interpret problems:

…when patients answer a questionnaire, that's not to me as helpful as hearing, seeing, feeling how they're answering that question…to me, it's more the interaction that really gives you the most information. Non-verbal cues…. those things are not captured [in PROs]. I want to experience the patient as they're talking about this. (Physician, San Diego)

In fact, a few providers found the PROs to be intrusive or disruptive of patient-provider communication, and found their use to be at odds with their personal style. Most of these providers expressed a high level of confidence in their relationships with patients, believing their patients could and would directly disclose their needs. This perspective was more prevalent among community health clinic providers in our sample; which, in our network, typically serve higher-functioning patients.

Finally, some providers noted the PROs were incorporated into care with little prior training in how to utilize the results in a clinic visit. These providers desired some form of initial training to help optimize their use.

DISCUSSION

Providers expressed great value in the use of PROs in routine patient care. Benefits of PROs were described and experienced as substantial, with the most profound benefit seen as an enhanced ability to detect behaviors that may not otherwise have been discussed. PROs were described as particularly valuable in detecting suicidal ideation, depression, and substance use. Previous studies have found that the use of an electronic computer interface reduces social desirability bias (Adebajo et al., 2014; Beauclair et al., 2013; Ghanem, Hutton, Zenilman, Zimba, & Erbelding, 2005; Kissinger et al., 1999), suggesting that PROs may be useful in identifying other difficult-to-discuss issues such as inadequate medication adherence or HIV transmission risk behavior.

Another key benefit of PRO use was commonly expressed as its ability to “set the agenda” for the appointment with chief concerns, echoing recent findings elsewhere (Johnston et al., 2016; Stover et al., 2015). Providers believed that PROs administered just prior to the visit helped patients with multiple complaints and concerns to prioritize and articulate their needs. More research is needed to determine whether patients believe the use of PROs improves the quality of communication with their providers.

Challenges and concerns surrounding PRO implementation and use were typically logistical in nature. Providers described initial concerns regarding impact on clinic flow. Several strategies were used to minimize this impact, including appointment reminder calls that encourage patients to arrive 15–20 minutes prior to their scheduled appointment time to complete the assessment; inclusion of PRO collection immediately after vital signs if time allows; and foregoing PRO collection if a patient arrives late. All clinics in this study had resolved initial time, flow, and space-related issues.

Providers also valued some measures more than others, typically driven by the characteristics of their clinic population. For example, providers serving primarily higher-functioning patients found the review of symptoms to be less useful, believing their patients able to communicate their needs. Conversely, some providers desired expansion of certain measures, such as the addition of a relationship context item to a measure of HIV transmission risk, in order to better understand sexual decision-making. Given that the value of specific PRO measures may vary by clinic and even by provider panel, provider participation in PRO domain selection is essential to ensuring the relevance and usefulness of PROs, as has been noted elsewhere (Fredericksen et al., 2012).

A minority of providers expressed concerns with the nature and/or format of the PROs themselves. There was concern that a feedback form reporting multiple symptoms and behaviors may overwhelm the visit with exploration of concerns that may not otherwise have been brought up by, or important to, the patient. A suggested potential solution to this concern was to allow patients to identify what topics they would like to discuss today; a provider could then use these cues to focus the appointment on the chief concerns. Some providers desired a longitudinal view of PROs to identify behavior patterns. One clinic addressed this by including a three-column feedback form indicating patient outcomes for three different visit dates in the recent past.

Provider interviews suggest that the use of PROs is highly valued in detecting adverse health behaviors and symptoms. However, this value depends on the quality of integration into clinic flow, inclusion of only the most clinically relevant content, and ease of interpretation.

Limitations

We used a convenience sample of providers, which may have excluded some of the very busiest providers; this may also have caused oversampling of providers who were more invested in the use of PROs in care. Further, interviews were conducted in the workplace, which may have inhibited full expression of views. In addition, our findings may not apply to other forms of (non-HIV) care, other types of web-based assessments, or to clinics in resource-limited settings.

CONCLUSION

Providers value same-day, electronic patient-reported measures for use in clinical HIV care with the condition that PROs are 1) tailored to be the most clinically relevant to their population; 2) well-integrated into clinic flow; 3) easy to interpret, highlighting chief patient concerns and changes over time. Our findings suggest that electronic PROs are useful in outpatient HIV clinics in a setting that can obtain and support web-based touch screen technology. Additional research is needed in order to ascertain provider preferences in presentation of PRO feedback, as well as possible changes in providers’ relationships with PROs over time.