Interactive Decision Support for Risk Management: a Qualitative Evaluation in Cancer Genetic Counselling Sessions

Abstract

Genetic counselling for inherited susceptibility to cancer involves communication of a significant amount of information about possible consequences of different interventions. This study explores counsellors' attitudes to computer software designed to aid this process. Eight genetic counsellors used the software with actors playing patients. Clinicians' rating of expected patient satisfaction, content, accuracy, timeliness, format, overall value, ease of use, effect on the patient–provider relationship and effect on clinician's performance were evaluated via qualitative and quantitative analysis of interviews, training tasks and questionnaires. Most counsellors found the software effective. Concerns related to possible impact on consultation dynamics and content. Participants suggested countering these through appropriate new counselling skills and selective use of the computer. The REACT software could provide effective support for genetic risk management counselling.

Introduction

Although computerised decision support systems (CDSS) have the potential to positively affect clinicians' performance and patient outcome [1–3], few of these systems are eventually put into routine use, possibly in part because CDSS design rarely takes account of the interplay of contextual, organisational and personal influences that determine whether or not a system will be viewed as useful by clinicians [4, 5]. We report on work in progress evaluating the feasibility of introducing CDSS into the setting of breast cancer counselling to provide a shared focus for the patient–counsellor conversation.

The REACT (Risks, Events, Actions and Consequences over Time) care planning system [6] was designed to mitigate cognitive loads imposed by plan manipulation and decision making within the context of shared care planning. A graphical view of a care plan (Fig. 1) is shared by patient and counsellor, with events and actions shown against a time line. Using a mouse, these can be added, deleted and re-timed while simple graphs beneath the planning timeline respond immediately giving feedback of outcome measures such as risk. Textual arguments for and against proposed actions and conflicts and constraints within the plan are continually updated.

Fig. 1.

The REACT user interface. A plan is being developed for risk mitigation by manipulating actions in the planning chart (top). The outcome measure graph (middle) indicates the estimated risk of breast cancer, while arguments for and against prophylactic oophorectomy for this patient are reviewed in the information panel (bottom)

Materials and Methods

A qualitative study was designed to address two primary issues: to what extent is it practical to use the system in a clinical genetic counselling setting? And what are counsellors' attitudes to the introduction of such software? Eight cancer genetic counsellors from the clinical genetics unit of a major UK hospital (seven females and one male, all with at least 2 years experience in genetic counselling and 3 years experience using computers) were asked to counsel mock patients played by professional actresses. A 30-min training session was given to each participant approximately 1 week before testing. Each participant then led two simulated counselling sessions. In one, they were asked to follow their normal counselling practice. In the other, they were asked to use REACT whenever they felt it could be put to use. Two test scenarios were prepared each concerning a patient who had discovered in the previous year that they were carrying 185DelAG mutation in the gene BRCA1. In scenario 1, a post-menopausal woman of 54 years wanted to know what difference prophylactic mastectomy in addition to oophorectomy was likely to make to her risk of developing cancer. In scenario 2, a pre-menopausal woman of 30 years wanted to know what difference delaying oophorectomy would make to her risk. The order of scenarios and software use were both counterbalanced across participants.

Semi-structured interviews following testing were analysed using a general thematic approach [7–9]. Two researchers independently identified thematic units in each transcript which were then organised into broad theoretical categories. The coding schemes were iteratively refined through comparison of categories within and between transcripts and negotiation between the researchers, arbitrated by a third researcher, until it was felt that conceptual saturation had been reached. Interpretations were monitored by checking for deviant cases [10, 11]. Questionnaire results were reviewed to corroborate the overall interpretation [10, 11]. The themes emerging from interviews fell into three categories: benefits, problems, and suggestions for overcoming problems.

Positive Themes—Benefits of REACT

Perhaps the most positive comments were from four counsellors who said they wished REACT had been available in both counselling sessions. (Participant 1 [P1]: “And I felt that perhaps, having just used REACT for the first [simulated patient], I would have really liked to use REACT for the second one, because [...] I could feel that she would have found that useful, it would have helped clarify the things in her mind that were confusing…”)

Six counsellors commented that the provision of medical information was helpful. They perceived REACT as providing accurate, trustworthy facts and figures. Seven participants commented that the visual presentation style of REACT (particularly the dynamic, interactive display with its fast response time) was helpful. The visual impact of interventions on risk was felt to be stronger, clearer and more striking than that of verbal communication. (P6: “I sometimes felt […] the visual representation of what was there was better than what I could describe verbally.”)

Seven participants found the textual arguments for and against treatment options useful as a memory aid to make sure all issues were covered during the consultation or in preparation for clinic and in structuring consultations. (P6: “... some patients come, and their heads are full and they're all “muddly”, and you’ve got to [...] create some kind of structure for them, so that they're not so muddled. And something like REACT actually helps you do that, I think.”)

Most participants identified positive effects on their interaction with the patient. Two found the introduction of the computer screen improved interaction by providing a shared focus of attention. One mentioned the benefit of observing the patient’s reaction. (P6: “Because the patient is looking and taking in the graphs, you’ve got time to look at their expressions, and are they looking shocked, they’re looking surprised, they’re looking thoughtful […] so, you’ve got time, you can give them attention.”). Several participants felt that by easing the communication of facts and figures REACT made interaction with the patient smoother and that it increased their confidence and allowed them to be more patient-centred.

Negative Themes—Disadvantages and Problems

Concerns about REACT fell into two main groups. Firstly, a number of themes concerned intrusive effects on the patient–counsellor relationship. These primarily concern distraction caused by the software and a reduction in the counsellors’ communication and attention towards the patient. (P3: “[...] both our attentions were drawn to the screen [for] a lot of the consultation and there was less personal interaction.”). Seven participants found that using the software interfered to some degree with the usual flow of the consultation. Of these, five mentioned a reduction in their communication and rapport with the patient.

A second group of themes concerned problems with information display. Though the visual display of risk was considered by most participants to be one of the major advantages of REACT, three participants raised concerns regarding interpretation of graphical information and the emotional effect it might have on the patient. One felt that the graph might make risk seem artificially high. Another worried that it might over-emphasise the effects of interventions. Both thought the visual display of risk might produce a stronger emotional effect than verbal information. One participant commented that some patients would find the graph difficult to comprehend.

While most participants found the information panel useful, some reservations were raised. One believed this information could be distracting, and two, felt it might be redundant. Two were concerned that it could even undermine the counsellor's credibility with the patient by implying that they did not know their job. (P3: “[...] it sounds as if you don’t know what you’re talking about, if you're having to read it off a computer screen.”)

Constructive Themes—Suggestions for Overcoming Problems

The final group of four themes comprise proposals for overcoming problems, appropriate use of REACT and suggested improvements. Firstly, the question of how much information should be given to the patient emerged in most of the interviews. Some participants noted that they conveyed more information when using REACT than they would normally do, but were unsure how to decide which information to give. Three participants raised the possibility of information overload. (P1: “I think it's easy to get distracted by the bits at the bottom, so that you find yourself going “ooh, let's have a look at here, let's have a look at this”, you know, and did the patient really want that other bit of information? Did she ask for it?”)

Secondly, there was agreement among all participants that some patients would benefit from REACT more than others, in particular, those who were interested in facts and figures and in a thorough discussion of treatment alternatives, with specific queries or with several decisions to make. Patients' knowledge, educational level and attitude to computers were felt to be factors. Four participants commented that use should be also limited to certain parts of the consultation. (P5: “It was only towards the end of the consultation I realised that if I was actually doing a consultation, I wouldn’t want it to be there for the vast majority of it. Which, I guess, you learn with experience.”)

A third theme concerned new counselling skills that would be required to integrate REACT smoothly into consultations. These would include determining in what situations and with which patients REACT should be used, determining what information should be given to the patient, and managing time and patient focus in consultations. Generally, participants seemed willing to acquire these skills in order to benefit from the software. (P8: “I think that with prolonged use you would be able to [...] get more of the balance between [...] using it in a useful manner, and deciding when it’s not going to be useful.”)

A final theme concerned assurance of data quality. Seven of the eight participants expressed some form of concern here. Although they were keen to have access to accurate, detailed, up-to-date, scientific information during counselling they needed to know the source of the data and that it was being updated regularly. (P8: “I would want to know much more about the statements that were there […] and the evidence for the statements that were there [...] if I was actually going to use it in my clinical practice...”)

Discussion and Conclusions

Although the scale of this study is small, it confirms that the REACT software is usable in a simulated clinical setting. We expected that genetic counselling, an activity which is particularly focused on the clinician–patient relationship, would be a challenging area to introduce computer support. Four participants did indeed report that they were sceptical about using the software in a clinical setting before the study. However, after the study, seven of the eight participants expressed a willingness to use REACT clinically. Participants appreciated current, accurate, detailed and patient-specific information, and clinicians are more accepting of computer decision support when they see it as enhancing their capabilities by providing useful information [4].

Not surprisingly, some of the concerns that were raised relate to possible changes in the dynamics and content of the consultation—other studies [1, 4, 12] suggest that such concerns may constitute a major barrier to the implementation of CDSS. As in [12], participants reflected on the skill of using computers within a consultation. Their concern about deviation from a patient-led consultation has some empirical support. Computer use has often been found to increase clinician-centred speech and the number of medical topics raised at the expense of patient-centred activity [1]. On the other hand, although several studies find some increase in consultation length, in the long-term, it tended to decline to baseline levels [1], as our participants expected.

Participants themselves offered solutions to the problems they raised. The generally positive response to REACT, despite its shortcomings, might be surprising given the sometimes frosty reception clinical decision support software has received [e.g. 13]. We believe that a number of factors contribute. REACT was designed to provide support for the specific cognitive loads involved in planning, and it is effective at this [14]. However, the context of computer use is also clearly important [4]. REACT supports users in a specific, technical aspect of their task, which we expect that counsellors would not view as central to their clinical expertise (which lies in the psychological and emotional aspects of counselling). We believe that careful, user-centred design of software and learning to use software effectively within the counselling session, will both be significant in reducing the potential for negative effects on the patient–provider relationship and in increasing the benefit and satisfaction of patients and counsellors alike.