Interfaces for collecting data from patients: 10 golden rules

Abstract

Memorial Sloan Kettering Cancer Center has more than a decade's experience creating online interfaces for obtaining data from patients as part of routine clinical care. We have developed a set of “golden rules” for design of these interfaces. Many relate to the knowledge imbalance between professional staff (whether medical or informatics) and patients, who are often old and sick and have limited knowledge of technology. Others relate to the clinical nature of the encounter: data cannot be taken from patients as part of clinical care unless there is a plan to act on whatever information is prepared. We also note that the plethora of marketing questionnaires makes patients suspicious of surveys: patient trust is hard to gain and easy to lose. Addition of these golden rules to standard approaches to interface design will maximize our ability to obtain data from patients and thus improve communication between patients and clinicians.

INTERFACES FOR COLLECTING DATA FROM PATIENTS: 10 GOLDEN RULES

At Memorial Sloan Kettering Cancer Center, we have been building online questionnaires to obtain data from patients for close to a decade. These predominately collect patient-reported outcomes (PROs) such as pain, urinary function after prostate surgery, or cosmetic appearance after breast reconstruction. We have also built questionnaires that ask about medical history (eg, “do you have diabetes?”) or medical events (eg, “have you received treatment for a hernia since your cancer surgery?”). At the time of writing, we have created over 300 different questionnaires. A total of over 200 000 completed questionnaires have been submitted by more than 100 000 different patients and caregivers, the vast majority as part of routine clinical care.

Our extensive experience in this space has taught us several lessons about the best ways to create interfaces for collecting data from patients. Many of these are tried and true lessons of interfaces in general. For instance, Shneiderman's well-known “golden rules” of interface design includes “consistency” and “easy reversal of actions.”¹ We have similarly found that patients can be confused by differences in look, feel, or functionality and that they do change their minds about answers, hence making it essential that we allow patients to change those answers easily.

That said, some of the lessons we have learned are specific to patient data collection issues and have not been widely addressed in the wider literature. We share those lessons here as an extension of prior literature on user interfaces.

1. What seems obvious to an engineer (or informatics manager) may not be obvious to a patient. The famous hidden dog illusion (Figure 1) will look like just a jumble of dots to the uninitiated. However, once the dog is seen, it cannot be unseen, and it can be difficult to understand how the dog is not plain and obvious to any observer. In a similar way, once an engineer or informatician has worked on an interface for months, how that interface works just seems obvious; so much so, that it seems inexplicable that any user could not understand what to do. We have seen members of an informatics team literally roll their eyes when asked how the patient is able to add information to a web form (“Click on the pencil symbol!”). We also had an outside software engineer explain that the font on a questionnaire wasn't too small, because, after all, he found it easy enough to read. In response, we pointed out that he wasn't an 83-year-old with metastatic lung cancer. This isn't to say that engineers and informatics managers are unaware of or routinely ignore user design; indeed, we have found most to be thoughtful and sensitive. But the patient's perspective needs to be repeatedly stressed throughout development of a patient interface.

2. What seems quick and easy may strike the patient as burdensome. An experienced computer-user, who knows their way around an interface, can click through some links in a matter of seconds. This might well not be the case for a patient who is less computer savvy, may have neurological or other impairments that restrict vision or hand motion, and who has visited an interface only a small number of times if at all. No doubt a case can be made that, for instance, providing a link in an email to a patient portal, then having the patient click on “messages” then “New questionnaire available” and then the link in the message, takes no more than a second or 2. Indeed, we have seen this demonstrated at a meeting by a young informatics manager. Yet when we switched our own system from links to the portal to a direct link from the email to the survey, response rates increased dramatically. Similarly, we have seen an informatics manager demonstrate ease-of-use by quickly selecting a cancer from a drop-down list of over 100 cancer sites. But this is only because he knew that by typing the first few letters of the cancer, the interface would scroll down automatically and that the cancer was listed under “Sarcoma, bone” rather than “Bone sarcoma” or “Osteosarcoma”.

3. Questionnaires developed for research may not be appropriate for clinical practice. We have previously argued² that questionnaires developed for research are typically far too long, include unnecessarily complex language, include questions that make no sense to specific subgroups of patients, and often give inaccurate results for individual patients. In place of the common approach of using a standard, omnibus questionnaire for a large cross-section of patients (for instance, the EPIC-26 questionnaire asks about a wide variety of different symptoms experienced by prostate cancer patients), we first use different questionnaires for different groups of patients depending on their expected symptoms (for instance, we only ask radiotherapy patients about bowel symptoms and only those on hormonal therapies about hot flashes and breast tenderness). Second, we read each question extremely carefully to determine whether it is appropriate for all of the patients we would see in routine practice. It sounds like a simplistic approach, but we have been struck by the number of times that clinicians recommend questionnaires without a thorough knowledge of individual items. Third, we use skip and branch logic to ensure that we avoid asking irrelevant questions.

4. Many words used by doctors and researchers can be replaced by something simpler. Individuals who use medical language every day can forget how difficult it can be for nonmedics to understand. This goes beyond evidently complex or specialist medical terms, such as describing, say, “transient ischemic attacks” or “proband.” Many seemingly nontechnical terms such as “stool,", “bowel” or even “erection” are not universally understood note that whereas pretty much everyone in the US calls an elbow an elbow, terms used for defecation, urination, and sex vary enormously between social subgroups.³ Accordingly, we often add common expressions to items (eg, “how often have you had to urinate (pee) again less than 2 hours after you finished urinating?”). We also replace polysyllabic words with shorter words (eg, “Have you had pain?” rather than “Have you experienced pain?”) and avoid complex quantification (eg, “rarely” rather than “less than 1/5^th of the time”).

5. Mandatory fields and open text cause problems. There may be a good reason why a patient cannot or does not want to answer a specific question. It may seem logical to a group sitting around a conference table that, yes, we absolutely have to have an answer to that and, no, we can't think of any reason why a patient wouldn't answer but that's often not how it works in practice. We restrict mandatory fields to those 100% necessary to determine other critical questions. For instance, a question about which cancer a patient has may need to be completed in order to implement appropriate branching logic (eg, “prostate cancer” leads to questions about urinary function,“esophageal cancer” leads to questions about gastrointestinal function).With respect to open text, when asked by clinicians or investigators why an open text field is necessary, a typical response has been “just in case the patient wants to tell us anything else.” What a clinician is meant to do with that information, however, is often not completely thought through. Open text fields have a role, and we have used them in some questionnaires. But that is based on clear reasons to believe that they would be helpful, and we generally limit the number of characters that can be entered (for instance, not more than 150 characters, the length of a single short sentence).

6. Dont ask questions for clinical care unless you are prepared to act. Clinicians wanting to implement PROs in their clinic often default to using 1 or another standard validated questionnaire. The problem is that such questionnaires are developed for research purposes and often include items that, while of interest for research, are problematic in the clinical context. For instance, we have had surgeons recommend questionnaires that include items on spiritual concerns, financial problems, or interpersonal difficulties. But if you ask a patient about these sorts of problems, you have to be prepared to do something about it if the patient reports distress. Addressing problems such as a patient reporting that life has lost its meaning or that they have an abusive spouse is not typically the focus of a cancer surgeon's clinical practice.

7. Patients have to see that completing the questionnaire is in their best interests. The typical survey that any of us complete (“Tell us about your meal at Jo's Diner” or “Help research at St. Jo's”) is purely for the benefit of the survey provider, with the participant's role being purely altruistic. Patients need to understand throughout the process that clinicians are using their responses to provide them with better care. This point therefore has to be stressed repeatedly. The initial contact needs to be of the form “Dear Mr. Jones, this is Dr. Smith and I want to know how you are doing. Fill in the questionnaire and I'll look at it before we meet and discuss your responses at our appointment.” At the patient appointment, the clinician needs to say something like: “I saw how you responded online and I was concerned that you reported persistent pain. Let's talk about that.”

8. A subgroup of users can cause a great deal of additional work, but, unlike Amazon and Uber, you cant ignore those users. There are patients who don't understand simple medical terms, have trouble with email, or get confused by password resets. For a .com, the cost to adapt tools for difficult customers outweighs the value of those customers. A hospital, on the other hand, has to meet the needs of all patients. Be prepared to put in the extra work to make interfaces meet the needs of everyone.

9. Watch patients use your tool and ask about their experiences. We routinely visit clinics and approach patients in the waiting room to ask them to complete questionnaires in order to help us learn how to design them better. We explain to them that their exact answers do not matter particularly, and that their questionnaires will be discarded, but that we want to know how well they understand the questionnaire and if there is anything that confuses them. We then watch patients complete the questionnaires and ask questions such as “What did you understand by that item?” or “Explain why you gave that answer.” We almost always obtain insightful feedback that helps us with the layout or content of questionnaires, and often the feedback is unexpected. Just as examples, we have learned that patients didn't understand that they had to scroll down for some longer questionnaires, that button positioning was unclear, and that standard questions about gender and sexuality were distressing and confusing for many older patients.

10. Patient trust is hard to gain and easy to lose. It is not always easy to get patients to a place where they see PRO questionnaires as an important part of their care and not just another annoyance. But slip in a couple of satisfaction questionnaires (“Rate whether other staff at St. Jo's were polite and helpful”), ask a question that seems nonsensical to a patient (an item about breast tenderness to a man with prostate cancer who is yet to start treatment), or make it seem as though the left hand doesn't know what the right hand is doing (“Have you ever received chemotherapy?” to a patient returning to hospital for a chemotherapy-related toxicity), and that trust is lost.

Figure 1.

The spotted dog illusion. Just as the spotted dog is obvious to anyone who has seen it, and is often invisible to someone who hasn't, use of an interface can be obvious to the engineer who designed it and opaque to a new user.

Many of these “golden rules” might seem somewhat platitudinous. For instance, “what might seem obvious to an engineer might not be obvious to a patient” is not a complex or technical observation. In our experience, however, the “golden rules” need constant repetition so they are in the forefront of the design team's mind at all times. What otherwise tends to happen is that informatics teams take the easy option: using an established questionnaire, choosing an interface design that is the simplest to program, rather than doing the hard thinking required to do what is best for the patient. The addition of these golden rules to interface design will maximize our ability to obtain data from patients and thus improve communication between patients and clinicians.

FUNDING

This work was supported in part by funds from a National Institutes of Health/National Cancer Institute Cancer Center Support Grant (P30-CA008748) to Memorial Sloan Kettering Cancer Center.

AUTHOR CONTRIBUTIONS

This article is an overview of ongoing work by the 3 authors. All 3 contributed to the conception of the paper. AV wrote the original draft; LC and PS revised it critically for important intellectual content. All authors approved the final version and agree to be accountable.

Conflict of Interest statement

None to declare.