Computer support for recording and interpreting family histories of breast and ovarian cancer in primary care (RAGs): qualitative evaluation with simulated patients

Abstract

Objectives

To explore general practitioners’ attitudes towards and use of a computer program for assessing genetic risk of cancer in primary care.

Design

Qualitative analysis of semistructured interviews and video recordings of simulated consultations.

Participants

Purposive sample of 15 general practitioners covering a range of computer literacy, interest in genetics, age, and sex.

Interventions

Each doctor used the program in two consultations in which an actor played a woman concerned about her family history of cancer. Consultations were videotaped and followed by interviews with the video as a prompt to questioning.

Main outcome measures

Use of computer program in the consultation.

Results

The program was viewed as an appropriate application of information technology because of the complexity of cancer genetics and a sense of “guideline chaos” in primary care. Doctors found the program easy to use, but it often affected their control of the consultation. They needed to balance their desire to share the computer screen with the patient, driven by their concerns about the effect of the computer on doctor-patient communication, against the risk of premature disclosure of bad news.

Conclusions

This computer program could provide the necessary support to assist assessment of genetic risk of cancer in primary care. The potential impact of computer software on the consultation should not be underestimated. This study highlights the need for careful evaluation when developing medical information systems.

Key messages

• General practitioners are under increasing pressure to advise their patients about genetic predisposition to various diseases

• Computers could help doctors to give genetic advice by simplifying the construction and assessment of family trees and implementing referral guidelines

• This qualitative evaluation explored the context in which a computer program for assessing genetic risk of cancer would be used in general practice and issues surrounding its integration into a consultation

• Most of the doctors found the program easy to use, but it affected their control of the consultation—because of their desire to share the computer screen with the patient and their inability to anticipate the information that would be displayed

• The study identified important issues relating to the use of computers in consultations which may be of use in testing software for primary care in the future

Introduction

Primary care will inevitably play an increasing role in genetics because of rapid advances in genetic medicine and resultant pressures on specialist services.¹ The availability of tests for genetic predisposition to breast and colon cancer has resulted in increased referrals to genetic clinics, although many of these people are at low risk.² Most general practitioners lack the skills and knowledge required to provide a first line genetics service, particularly for multifactorial diseases.³ Computers could help doctors by simplifying the construction and assessment of pedigrees and implementing referral guidelines.⁴,⁵ Although programs exist for drawing pedigrees and assessing risk, they are too complex for primary care and do not provide advice about patient management.⁶,⁷

Computers increase the use of guidelines⁸,⁹ and can improve clinical performance.¹⁰ Despite this, decision support has not been adopted in routine general practice. This partly reflects limited clinical input in the development of medical expert systems.¹¹ The selective use of quantitative approaches to evaluate medical information systems has been criticised,¹² and multiple methods have been proposed that examine professional and organisational factors influencing use of information systems.¹³

Clinical scenarios used in simulated consultations

Case 1

Pam, aged 30, has come to see her general practitioner because her mother has recently been diagnosed with breast cancer again. Her mother first developed breast cancer when she was 56. At a follow up clinic six months later, a lump in the other breast was found, which has also turned out to be malignant. She is due to go in for surgery next week. Pam’s maternal aunt, Jane, developed ovarian cancer aged 50 and died nine months later. Pam’s maternal grandmother died of breast cancer aged 64. Pam doesn’t know much about this and doesn’t remember her well since she was quite young at the time. Pam’s mother is worried that her daughter will get cancer and has told Pam to see her doctor to see if anything can be done. Pam is not particularly concerned, and the issue has been discussed only recently when her mother picked up a leaflet at the hospital about cancer running in families. Pam is married and has no children. She resembles her aunt Jane physically and wonders whether this is important.

 Estimate of risk by Cyrillic program (version 2.1): heterozygote risk for BRCA gene=43%, lifetime risk of breast cancer=40%

Case 2

Sheila, aged 42, is concerned about her family history of breast cancer. She has read in a magazine about breast cancer genes and wants to be referred to a geneticist. Her mother, Shirley, had breast cancer diagnosed at age 62, three years ago. She had a lumpectomy and radiotherapy and is currently well. Her grandmother, May, had breast cancer diagnosed at age 73. She had a mastectomy but died two years later. One of May’s sisters also had breast cancer diagnosed, at age 50, and is still alive (age 84); she has had no problems since her mastectomy. May’s other sister is well, aged 88. Sheila’s family view breast cancer as a family condition and have often discussed it in these terms. Now that she has read the magazine article she wonders if gene testing holds the answer. The article also talked about prophylactic mastectomy for people who are found to be gene carriers. Sheila has had no breast symptoms but worries about developing breast cancer. She is married with a son aged 15 and daughter aged 18. She also worries whether her daughter will get breast cancer. Sheila has a brother, who is well.

 Estimate of risk by Cyrillic program (version 2.1): heterozygote risk for BRCA gene=7%, lifetime risk of breast cancer=13%

 Lifetime risk of breast cancer for general population of women=9%²⁰

We report a qualitative evaluation of RAGs (risk assessment in genetics), a computer program designed for primary care that draws pedigrees, assesses risk based on family history of breast and ovarian cancer, and suggests appropriate management. The software was developed through collaboration between JE and the ICRF Advanced Computation Laboratory using PROforma software.¹⁴,¹⁵ The aims of this study were to explore general practitioners’ opinions of using computers for genetic risk assessment and to study the use of RAGs in the consultation.

Participants and methods

We chose an “action design” method, in which practitioners use prototype software in a work setting to inform necessary changes to the program.¹⁶ A purposive sample of 15 Oxfordshire general practitioners was recruited between May and September 1998 covering a range of computer literacy, interest in genetics, age, and sex.¹⁷ The doctors were identified by colleagues who work in the General Practice Research Group and invited to participate by JE by telephone. Only one doctor declined to participate in the study.

Semistructured interviews

JE performed the interviews, presenting himself as a researcher from the General Practice Research Group. Each doctor was first interviewed about his or her attitudes towards computers and involvement in genetics. The doctor was familiarised with RAGs and then used the program in two consultations with an actor playing a woman concerned about her family history of cancer (see box). The pedigrees were based on patients referred to the Oxford Cancer Genetics Clinic. The consultations were videotaped and followed by a semistructured interview with both doctor and actor, the video being used as a prompt to questioning.¹⁸ Interviews principally explored the ease of use of RAGs in the consultation and the effect of the program on the doctor-patient interaction. Issues identified during the earliest interviews defined the subjects explored in more depth in subsequent interviews, thus establishing the field validity and transferability of emerging concepts.¹⁹ Interviews were audiotaped and transcribed verbatim.

Analysis

The interview transcripts were entered into the NUD*IST program²¹ and analysed using a grounded theory approach²² by JE and RW with advice from SZ. During the analysis, JE and RW met regularly to discuss the emerging themes and resolve any differences in interpretation. Each transcript was open coded, whereby each phrase was analysed to create key categories. A coding framework was developed and modified during the analysis of subsequent interviews according to the emerging concepts. Relationships between different categories were identified by constant comparison between and within transcripts and by comparison with existing literature. Videotapes of the consultations were reviewed, and observations were integrated into the developing conceptual framework.²³

All participants were sent a written summary of the analysis and invited to comment. We received responses from three doctors, who agreed with the main themes described. Additional minor comments helped refine the conceptual framework.

Results

The table gives background information about the study participants.

Several themes emerged from the data: controlling the consultation, accessing the report from RAGs, managing the “third party effect,” and concerns about time. These are presented in the context of two specific areas of inquiry: using computers for expert advice in genetics and using RAGs in the consultation. The box shows quotations selected from the interviews.

Quotations from interviews

Using computers for expert advice in genetics

Doctor 3 (female, age 39): “I think the use of genetics could be a useful tool for us. I think you need to be guided by the genetic experts, those that are knowledgeable about it. We need to be able to get the right information and have the right tools at our disposal so we know what to do with that information and we know how to advise people. It’s very easy to falsely reassure people without the information.”

 Doctor 8 (male, age 36): “Because it can get very easily very complex, and I don’t believe that we can really do that in our head, and assuming it gets updated regularly, [the computer advice] will be better than my knowledge I would assume, and it could be a source of help.”

 Doctor 9 (female, age 44): “I probably hadn’t put them [referral guidelines] in the right place, which is the fate of all bits of paper, so I tried, you know. I remembered one or two key points from it, but I couldn’t remember the details so I had to phone up and ask.”

RAGs in the consultation

Doctor 12 (female, age 29): “It was difficult because I was showing you the screen and it came up as I read, so I probably would have thought I’d quite liked to have read that and thought ‘How am I going to communicate this?’ rather than showing you.”

 Doctor 3: “Suddenly you get ‘High risk’ coming up on the screen .... Its much nicer to say it in words without seeing ‘High risk’ in black and white in front of you.”

 Doctor 9: “Once I read the reasons why [it was low risk] I began to see that it was slightly awkward to get that information myself and deal with someone who was getting a bit agitated about this apparent conflict of advice.”

 Doctor 3: “I was very aware she was sitting and I was saying nothing. I prefer to talk to people and look at them rather than spend a lot of time looking at the computer .... That made me feel uncomfortable.”

 Doctor 5 (male, age 31): “If I know it’s going to say ‘High risk’ it might be [that] one of the ways round sort of breaking the news [would be that] you can invite the patient back to break news rather than suddenly there it is on the screen.”

Using computers for expert advice in genetics

The participants viewed genetic information as specialist, complex knowledge pertaining to a new explanation of disease and allowing precise assessment of individuals’ risk. The general practitioners accepted their traditional gatekeeper role in the new subject of genetics and felt it appropriate for them to identify people at increased genetic risk and to refer them for specialist counselling. They admitted that they found assessing genetic risk difficult and felt uncomfortable when doing so. This was true even of the two doctors who had attended courses on cancer genetics. The doctors managed this discomfort in a variety of ways: some referred all patients concerned about their family history of cancer, some assessed risk using a heuristic approach reflecting an incomplete memory of referral guidelines, and others attempted to reassure patients in the face of uncertainty.

The local cancer genetics guidelines, mailed to all Oxfordshire practices in 1997, were unfamiliar to most of the participants. Two doctors did recall them: one sat on the local medical council, which appraises all Oxfordshire guidelines, and the other was involved in a project evaluating an electronic version of local guidelines. Participants mentioned the problem of “guideline chaos,” in which general practitioners were inundated with paper guidelines of varying quality, with resultant difficulty in storage and accessibility. Computers were seen as a potentially important method of implementing guidelines and restoring order to the perceived chaos. Furthermore, computers were seen to lend themselves to the complexity of genetic risk assessment for multifactorial diseases such as cancer.

Participants reflected on the skills required to use computers effectively in clinical practice and the limited training received. These included general keyboard and mouse skills, specific skills for particular programs, and, perhaps most importantly, the skill of using the computer within the consultation. Even the doctors who were more familiar with computers tended to enter little data during routine consultations because of pressures of time and concerns about the effect of the computer on doctor-patient communication (the “third party effect”).

Using RAGs in the consultation

Most of the participants found RAGs easy to use, but certain errors were commonly made because of the interface. An important concept that emerged related to control of the consultation. The doctors felt relatively in control during data entry and tended to stop considering the meaning of the information they were entering into RAGs. Thus, when the report was generated, with the risk score highlighted amid other text, doctors were surprised and felt a sudden loss of control. When the high risk message for case 1 appeared on the screen, doctors commonly felt panic and wanted to turn the screen away from the “patient” to allow them to regain control and break the bad news more gently. With the low risk message of case 2, doctors were often surprised and, indeed, may have already tried to pre-empt the computer and warn the “patient” that she was at increased risk. This conflict between information given by the doctor and that provided by the computer further impaired the doctor’s control of the consultation and made it difficult to reassure the low risk patient. The speed with which the computer generated the report added to this sense of losing control.

The doctors often felt unable to access additional information from the computer to support the management decision they were sharing with the patient. This related to poor highlighting and structure in the report. When they found the explanations provided by the program the doctors used this information to help regain control and reassure the patient.

Participants were concerned about the third party effect of the computer and missing important eye contact and subtle non-verbal cues. They were uncomfortable during silences when entering data and reading the report, which made them speak before they had a chance to assimilate the report fully. To counter the third party effect, doctors chose to share the computer screen with the patient. This worked well when generating the pedigree since there was a sense of joint effort and shared understanding. However, nearly all the doctors seemed uncomfortable sharing the screen when the report appeared, particularly for the high risk message.

Concerns about time were common, especially over recording a pedigree, as this was a period spent principally attending the “third party.” The consultations took 10-25 minutes, depending on the doctors’ computer skills and the actor’s responses. The doctors proposed various time management strategies. Some accepted the natural variation in length of consultation in primary care, some suggested double appointments or staging the consultation, and others opted to delegate data entry (such as to the practice nurse). Staging and delegation had the advantage that they would avoid generating the risk report with the patient present. Thus, the doctors could maintain control of the consultation and could prepare themselves to break the bad news without sharing the information on screen with the patient for the first time.

Discussion

This study explored the use of computer support for genetic advice in primary care, but several of its findings are relevant to the wider use of computers in general practice. Our sample of 15 doctors was purposively selected to represent a range of skills and opinions. Many of the themes that arose in this study about the role of primary care in genetics are common to those discovered in studies of general practitioners in Wessex and Nottingham (personal communication, S Kumar, E Watson). It is possible, therefore, that our findings are transferable to other general practitioners in the United Kingdom. However, qualitative research does not attempt to generalise but to provide a deeper understanding of phenomena and generate results of high validity.²⁴ Although our participants used RAGs in specific scenarios, the use of simulated patients has been shown to predict clinical performance,²⁵ while avoiding potential harm from using a prototype program with real patients. We used a combination of video-stimulated recall and rigorous data analysis using grounded theory. Such a method offers a simple, effective, and relatively cheap way of evaluating early prototypes and may be of value in the development of other general practice software.

Our participants thought RAGs was an appropriate application of information technology, given the complexity of genetic risk assessment for multifactorial disease and the detail of existing, paper based, referral guidelines. Using RAGs generated problems in the consultation because of the interface and the doctors’ varying ability to integrate the computer into the consultation. The potential impact of computers on general practice consultations has been described, both in relation to sharing sensitive information on the computer screen²⁶ and to doctor-patient communication.²⁷ Difficulties arose when RAGs generated the report and the doctor experienced a sudden loss of control. This occurred because of concerns about risk being disclosed prematurely, because of uncertainty about how to discuss this information, and because of the inaccessibility of supporting information. Participants proposed a variety of strategies to avoid this situation by assimilating the report in the patient’s absence.

RAGs is a generic tool for assessing genetic risk and will be extended to include other diseases such as colorectal cancer. This qualitative evaluation demonstrated the context in which RAGs will be used and the issues surrounding its integration into a general practice consultation. Many of the problems that arose were unpredictable and may not have been discovered using quantitative approaches. The development of medical information systems requires various stages of evaluation—investigating how the software functions and its impact on users, patients, and the health system.¹³ As a result of this study, we have identified important changes to the software to reduce the impact of the program on the consultation. We are now incorporating these changes and investigating the effect of RAGs on decision making.

Table.

Background information about the 15 general practitioners who participated in study. Values are medians (ranges) unless stated otherwise

Characteristic	Values
Age (years)	39 (29-59)
No of each sex	9 male, 6 female
Year of qualification	1982 (1964-1993)
No of doctors holding MRCGP	11
No of partners in practice	4 (1-6)
No of doctors in each type of practice	8 city, 7 rural