Skip to main content
Journal of the American Medical Informatics Association : JAMIA logoLink to Journal of the American Medical Informatics Association : JAMIA
. 1997 Jul-Aug;4(4):285–300. doi: 10.1136/jamia.1997.0040285

Consumer Informatics in Chronic Illness

Linda Tetzlaff 1
PMCID: PMC61246  PMID: 9223035

Abstract

Objective: To explore the informatic requirements in the home care of chronically ill patients.

Design: A number of strategies were deployed to help evoke a picture of home care informatics needs: A detailed questionnaire evaluating informational needs and assessing programmable technologies was distributed to a clinic population of parents of children with cancer. Open ended questionnaires were distributed to medical staff and parents soliciting a list of questions asked of medical staff. Parent procedure training was observed to evaluate the training dialog, and parents were observed interacting with a prototype information and education computer offering.

Results: Parents' concerns ranged from the details of managing day to day, to conceptual information about disease and treatment, to management of psychosocial problems. They sought information to solve problems and to provide emotional support, which may create conflicts of interest when the material is threatening. Whether they preferred to be informed by a doctor, nurse, or another parent depended on the nature of the information. Live interaction was preferred to video, which was preferred to text for all topics. Respondents used existing technologies in a straightforward way but were enthusiastic about the proposed use of computer technology to support home care. Multimedia solutions appear to complement user needs and preferences.

Conclusion: Consumers appear positively disposed toward on-line solutions. On-line systems can offer breadth, depth and timeliness currently unattainable. Patients should be involved in the formation and development process in much the same way that users are involved in usercentered computer interface design. A generic framework for patient content is presented that could be applied across multiple disorders.


Concurrent with a push from the medical industry to minimiz use of in-patient services is an emergence of grassroots and medical activism. The upshot has been the burgeoning of a home health industry and the advent of what is now being called the medical consumer.1 Computer technology and medical informatics are now converging with the prospect of providing health care information for home care using commercially available technology.

Following these trends, consumer medical informatics has begun to emerge as an independent topic within the larger rubric of medical informatics. Thanks to the explosion of medical information on the Internet and the ongoing, spontaneous materialization of disease-centered, peer support groups, patients and caretakers are developing into a meaningful informatics constituency.

By definition, what differentiates consumer informatics from its parent discipline is not so much its technical substrata as the users it serves. Medical informatics professionals and their technical colleagues are accustomed to focusing on the needs of providers. It will take a different discipline and perspective to service the consumer.

Typically one begins a development effort in computer systems with an evaluation of the users' environment and the tasks they currently perform to achieve their goals. In a work environment that may mean a cataloging of procedures, a detailed inspection of forms, and an analysis of the flow of work. However, the primary tasks of the chronically ill—such as getting well, becoming informed, and emotional coping—did not lend themselves to the usual analysis. Instead, a less formal but nonetheless informative picture of the environment was sketched, depicting the needs of a chronically ill patient/caretaker population.

Unlike most of the public health efforts which focus on wellness, prevention or first level triage, this work targets the spectrum of support that a computer could productively offer to patients with chronic illness or to their caregivers. Both caregivers and patients have responsibility for the home management of a complex, chronic illness in the context of potentially multidimensional informational, social, physical and psychological demands. This work is a vehicle for beginning to address these issues.

It has been a primary tenet of this work that the system developed should be driven by the needs of the patient, and while collaborating closely with medical staff, the primary focus has been on the patients and their information requirements. Patient education and reference material, increasingly available on-line, are an appropriate starting point for such support. This paper describes four patient-centered approaches employed to evaluate patient support needs.

Overview

A group of researchers at the TJ Watson IBM Research Center worked with the Pediatric/Hematology/Oncology (Pedi/HemOnc) Clinic at a major, tertiary care hospital to develop a pilot software system. The investigation was constrained to families of children, aged 2 to 12, with acute lymphocytic leukemia (ALL) because, while the treatment period is stressful and medically sensitive, the prognosis is good. ALL treatment protocols are stable and the clinical course is well understood. This environment provided a rich but tractable framework for exploring home support of chronic illness while obviating the need to address independently difficult problems, such as death and dying and the management of adolescent development, or compliance, which is generally quite high for young pediatric patients supervised by their parents.2

The plan was to prototype a multimedia workstation to respond to the needs of families during the 2-year outpatient treatment period. The focus was on the concerns of the pilot group, expecting the prototype to inform future efforts for a broader range of medical problems and populations.

Early interaction among researchers and the health care team identified major areas that such a system might profitably address, including: delivery of information about home care and ALL, symptom interpretation; instruction on home care procedures; communication between the patient families and the Clinic staff; peer communications among families; and emotional support. These topics paralleled work of others in patient informatics.3,4,5,6

Additionally, previous work was reviewed on the needs of families with a child with cancer,2,7,8,9,10,11 as well as materials produced locally by the Pedi/HemOnc department and other hospitals, materials available from various support organizations such as the National institutes of Health, the American Cancer Society, Candlelighters and the Leukemia Society of America, and independent authors. Following a week of intensive observation in the Clinic:

  • An open-ended questionnaire to elicit the primary concerns of parents was administered to the Clinic staff and to the parents themselves.

  • A structured questionnaire on informatics requirements was mailed to all families of a child with cancer in first remission.

  • Patients and Clinic staff were observed during procedural training to note spontaneously generated questions.

  • Parents were observed interacting with a computer prototype of an ALL information and support system.

The discussion concludes with recommendations linking findings of this needs assessment with existing demonstration projects to propose a template for the content of patient informatics applications for people with chronic illness.

Needs Assessment

The work was done with the full cooperation of the Pedi/HemOnc Clinic. The first three studies were done during approximately a 6-month period. Prototype evaluation was roughly a year later. Results of the studies served to direct ongoing design. Reflective analysis was at the end of the development activity.

Questions Asked by Parents

Method

The first questionnaire was administered to members of the Clinic staff since they ordinarily manage most of the questions posed by parents. Staff listed questions most frequently asked of them by ALL families, giving its source, an estimate of its frequency, and the circumstances under which it was asked. To obtain the parents' perspectives, the list of questions generated by the staff was appended to a version of the questionnaire soliciting any additional questions.

Results

A total of 368 responses were received from 14 staff members, including 2 physicians, 1 social worker, 3 secretaries, the medical technologist, and 7 nurses. Nurses reported substantially more questions (mean = 26.3) than other staff members (mean = 10.6). Ten parents (8 mothers, 2 fathers) returned questionnaires, yielding a total of 151 additional questions.

The results of both questionnaires were pooled to create a nonredundant list of 324 questions. Each question was then categorized by topic. Since some questions belonged to more than one topic (for example, “will a broviac leave any permanent scars on the heart” was categorized under both “broviac” and long term effects), the question total by category is larger than the total number of unique questions. The topics are summarized in Table 1.

Table 1.

ALL Question Topic Summary

Topic (Sample Question) Unique Questions Staff: Raw Totals Parent: Raw Totals
Patient care (How do I keep my child from pulling on the broviac?) 34 36 10
Caretaker support (How do I explain leukemia to my child?) 29 13 21
Activities/behavior (Can my child play with other kids when the counts are low?) 24 19 14
Treatment (Why is my chld sicker now than before treatment started?) 78 51 42
Symptoms (How do you know if the leukemia has come back?) 39 44 11
Disease (What causes leukemia?) 30 15 18
Prognosis (What is the latest information on cure rates?) 25 18 16
Other medical (Should my child get immunizations?) 23 16 16
Progress (How will we know if the chemo is working?) 22 25 4
Long-term effects (What are the effects of treatment on fertility?) 11 0 13
Clinic and hospitalization (Is there a support group for teens?) 32 31 5

The staff did a good job of representing the hospital and concerns about patient care, as evidenced by the relatively few additional questions generated by parents on these topics. Many questions were reported by both staff and parents about the medical concerns related to current treatment, doubtless because the complexity and importance of the topic precluded comprehensive coverage by either. The areas on which there was exclusive or significant additional focus by parents included child behavior, caretaker support, and long-term side effects.

The most frequent queries were about current blood counts, indicative of the progress of the disease and the vulnerability to infection; whether something would be painful or make the child sick; and what should be done about fever and vomiting. The largest number of unique questions was about treatment. At half that level were questions about symptoms and patient care. Other questions were distributed across both medical and psychosocial topics.

Underlying details of content and concerns specific to ALL were structural themes of requests for conceptual modeling of the disease, treatment, and care and requests for pragmatic day-to-day assistance. Usually the questions were concrete and directly relevant to the family's experience. Conceptual issues included common questions such as, “What is leukemia?” “What causes it?” Along with the abstract conceptual questions were more personal extrapolations: “Did I do anything to cause it?” “Is it contagious?” “What can I do to make it go away?” Other questions called for explantations of unpleasant procedures or those with no obvious rationale:

  • Why do I have to do mouth care even if there aren't mouth sores?

  • If the patient is in remission, why do you continue chemotherapy?

  • Why is a venipuncture necessary when the broviac is working?

  • Why can't you give spinal medication in the broviac?

  • Why is the patient sicker after treatment is begun?

The final major category of question had to do with the pragmatics of patient care and living with chronic disease. Many of the questions were of the “how do I...” or “what can I do...” variety. Some questions looked for practical tips; others seemed to elicit the need for role models to help chart this uncommon and frightening experience:

  • How do I explain about ALL to other people?

  • What do I do if the broviac pulls out a little?

  • What over-the-counter drugs can I use?

Some questions sought an active role in the healing process. For example: “Is there anything I can do that will help the counts to recover more quickly?”

Informatic Support Assessment

The identified questionnaire attempted to validate the needs during the initial investigation, and began to examine information acquisition preferences. The major categories investigated included managing at home, sources of information and other support, and experience with and expectations of interactive, programmable technologies. Figure 1 contains a summary of the questions. Most high level questions were followed up by more detailed and specific multiple-choice queries. For example, question 1 (what events do you track...?) was followed by “the time of your child's Clinic appointments,” “your child's blood and marrow counts,” etc. A few questions called for an open-ended, free-form response.

Figure 1.

Figure 1

Home survey questions (abbreviated text).

Method

Following a pilot with 14 Clinic parents, a clarified questionnaire was sent out to the 120 Clinic families with a child diagnosed with any cancer in first remission. Both parents or guardians in the household were asked to fill out a questionnaire. After 4 weeks a second mailing was sent out to those families who were not known to have responded to the first mailing. Questionnaires could be returned anonymously, but only nine people chose to do so.

Statistics were analyzed using SAS.12

Results

All results were pooled for reporting. All but one of those questionnaires administered in the clinic were returned; 25% of the first mailing and 25% of the second mailing were returned for a total of 101 responses from 62 families (56 mothers and 45 fathers). The mean age of the mothers was 36; the fathers, 39; and the patients, 9. Two thirds of the parents had at least some education beyond high school; 56% of the families had a child with ALL, but there was little discernible, difference in response based on diagnosis. Two thirds of the patients were 1 year or more past diagnosis.

How much do people know? It is possible that from the available books, pamphlets, and experts people come to know as much as they need to know, and that little beyond technical allure would be added by the computer. In fact, there is evidence that formal reference material is possibly secondary to interactive exchange with peers and experts in existing on-line patient support systems,4,13 though, of course, this could be related to the quality of the system content itself. The objective was to understand the effectiveness of the current information environment while exploring the potential value of an on-line system.

The providers and staff of the Pedi/Hem Onc Clinic are an exceptionally caring group of professionals dedicated to both the physical and psychological well-being of their patients and their families. In addition to their medical staff they also had two social workers, a parent consultant, and a nurse specialist who focused on patient education. Time, information, and emotional support flowed freely between families and staff, and there was some concern that there would be little a computerized system could offer above this baseline environment.

Indeed, parents did feel that they and their children were well informed. However, despite expressing satisfaction with the information they were receiving, parents still wanted to know more; 97% agreed that they “wanted to know everything they could about the disease.” Unsurprisingly, but nonetheless at odds with this assertion, was an ambivalence about encountering negative consequences: 31% didn't “want to hear about the bad things.”

While parents generally preferred to get information about their child's illness from the Clinic medical staff, there was at least some reluctance to ask questions, mostly for fear of imposing on the provider (Table 2). A few reported books were their primary source of information (Table 3). Quite a few reported more than one “primary” source, most commonly nurses and physicians. In general, people felt that they got a manageable amount of information from their primary source that was timely, relatively free of jargon, and easy to understand.

Table 2.

Reasons for Not Asking the Provider Questions

Reason (%) Selecting This Explanation
Not wanting to bother the provider 34
Feeling the question was too unimportant 34
Having trouble formulating the question 24
Being afraid of seeming stupid 24
Figuring they'd passed their question quota 21
Being afraid of the answer 19
Fearing the question was too difficult to answer 10
Discomfort with the provider 10
Table 3.

Primary source of Information

Source Sole Source One of Several
Clinic physicians 42 24
Clinic nursing staff 29 25
Books 5 10
Parents of children with cancer 0 7
Other medical sources 0 5
Friends 0 3
Number of parents (n = 103); some people selected more than 1 primary source.

In addition to their primary source, people utilized other resources, such as books, articles, pamphlets, and other families (Table 4). Traditional paper sources were the most common. However, the data suggest that, increasingly, traditional sources (articles, other families, books pamphlets and support groups) may be displaced by more the contemporary sources (newsletters, databases, video tapes, and hotlines).

Table 4.

Supplementary Sources of Support

Source of Support Mean Utility Current Users Projected Users Projected (%)
Articles 1.53 85 83 93
Other families 1.55 73 59 79
Books 1.57 86 82 93
Pamphlets 1.64 90 75 87
Support groups 1.72 36 34 59
Newsletters 1.86 44 68 82
Databases 1.91 11 47 80
Video tapes 1.92 24 55 74
Hotline 2.50 10 30 55
Utility was measured on a scale of 1 (very useful) to 4 (not at all useful.
Current users is the number who rates the resource.
Projected users is the number of people who said in the future they would be interested in using....
Projected % is projected users/(total respondents answering either yes or no).
The table is sorted by the score.

What do they want to know? An important issue is the scope and depth of information to be made available to patients and caretakers. On the one hand is an impulse to make as much information as possible available to as many as possible. On the other hand are concerns about its validity, volume, interpretability, and potentially threatening content. The hope was to discover uptapped opportunities to provide meaningful content.

In interviews, several people expressed a willingness to read medical materials at all levels of detail and complexity. However, most respondents read relatively medium-length, informative pamphlets directed to patients or caretakers, similar to those produced by the National institutes of Health or the American Cancer Society, or books primarily offering emotional support, such as Kushner's When Bad Things Happen to Good People.14 The most technical reference cited was Braken's Children with Cancer,15 which while clear and very informative was still not sufficient to address the spectrum of questions encountered:

  • What does it mean if the broviac can't draw blood?

  • Is blood in the cap OK?

  • Will the broviac leave permanent scars?

Corroborating the results of the first study, parents expressed interest in the disease and its treatment, its short- and long-term medical and emotional side effects, and how they and others could cope.

Issues of emotional support remained important. Anxiety about death and the fragility of their children continued well into remission and despite a good prognosis. While parents were generally comfortable talking about objective aspects of the disease, they were less comfortable talking about emotional aspects of their experience (Table 5). In the questionnaire, mothers expressed more reluctance than fathers to talk about feelings about their family. Stress in the marriage was a particularly sensitive topic for mothers. If indeed it is common that disease engenders distance from the very people from whom one would expect support, it may be that the more dispassionate support provided by on-line systems may have an unexpected contribution to make.

Table 5.

How Hard is it To Talk About... ? Percentage of parents reporting difficulty

Topic Mother (%) Father (%)
Anxiety about treatment outcome 53 59
Coping with the disruption to your life 40 41
Relationship to spouse since diagnosis 47 27
Feelings about child with cancer 44 32
Feelings about your other children 28 16
Cancer 21 22
Cancer treatment 16 14
Feelings about Clinic staff 2 5
N (mothers) = 43; N (fathers) = 37.

There was a possibility that caretakers' preferred source of information might depend on the nature of the question. Given the choices of Clinic physician, nurse, or other staff member, other expert, or parent (Table 6), the preferred source for information about cancer, treatment, and medical side effects was the physician; for home care and emotional side effects of treatment the nurses; and for coping with the inevitable stresses other parents. Aside from the doctors and nurses, Clinic staff was not regarded as a significant source of information, and “other experts” were rated surprisingly low across the board.

Table 6.

Preferred Sources of Information

Topic First Choice Second Choice
Cancer Physicians Nurses
Treatment of cancer Physicians Nurses
Medical side effects Physicians Nurses
Home care Nurses Physicians
Emotional side effects Nurses Physicians
How other people cope Other parents Nurses

Coping. It is possible that an important component of home care has to do with coping, with the secondary stress that living with a chronic disease imposed on the family. What was the impact, who was affected, and what kinds of services might be brought to bear on their mitigation?

Surprisingly few parents reported that their lives were disrupted by the exigencies of the disease, despite the rather considerable time their children spent in both the hospital and as outpatients.

The existing social support networks were explored. The most common form of interaction among families occurred in the hospital setting, with relatively little occurring outside. Write-ins indicated that there was solace in talking to others in similar circumstances, particularly when they could offer hope or share perspective on some yet unfamiliar aspect of the experience. The following is typical of a significant interaction: “During the first three days after diagnosis, my wife and I spoke with another family who was in consolidation. It gave us great comfort to speak with someone who had already been there.” The relief induced by shared experience is an important component of interpersonal self-help groups.16

The Clinic provided several groups aimed at supporting the emotional needs of its families. However, 71% of the respondents had never attended them, most commonly because of inconvenience. Fathers felt less in need of additional support and perhaps as a result less likely to interact with other families (Table 7).

Table 7.

Reasons for Not Attending a Parent Support Group

Reason Mothers (%) Father (%)
Inconvenience 70 50
Don't need additional support 29 49
Don't know much about them 35 32
Uncomfortable talking to others 15 27
Depressing 21 20
Not much in common with other parents 13 13
Time consuming 25 11
Don't like the way they're run 4 2
N (mothers) = 52; N (fathers) = 44.

About one third of the families had contacted such organizations as the American Cancer Society, the Leukemia Society of America or Candlelighters, and of those some two thirds judged that contact to be useful.

Programmable technology. Since the project goal was to develop a computer-based system, it was desirable to explore some of the feelings that people had about their interactions with media and programmable devices.

Might there be any systematic affinity of topics expression in particular media? Perhaps text might be preferred for exposition about disease and treatment, while video would be favored for exploration of psychosocial issues. Domain-specific learning preferences were explored.

Whenever possible, live interaction was preferred over an indirect medium, and video was preferable to text. Text was regarded as a particularly poor way of communicating about how other people cope.

It was possible that typing might be an uncomfortable way of interacting with a computer for the home user. However, the women, who were most put off by answering machines, were also the most skillful typists; 72% of the mothers could touch type, compared with 29% of the fathers. Only 5% of the mothers and 24% of the fathers did not type at all.

To what extent do people avail themselves of programmable technologies? Confirming the by now proverbial failure of the programmable VCR,17 respondents rarely recorded future programs, and were even less interested in future events on a microwave. On the other hand, three quarters reported using ATMs and in general found them easy and fun to use. While it is difficult to draw any significant conclusions from these observations, it would seem that critical differences between these devices may have to do with the nature of the interface (unstructured versus structured dialog) and the immediacy of the payoff (delayed versus immediate).

People only rarely recorded video or movies of their own. Almost two thirds of the families had answering machines either at home or at work. Those who didn't have them were uncomfortable with them (89%); they preferred to hang up and call back later or would reluctantly leave a message. This number dropped sharply to 49% of those who had a machine, so the discomfort evidently diminishes in time.

More direct inquiry into adult experience with computer-like interfaces revealed that 15% of the parents often play video games, and 95% had no objection to them in principle. They had generally positive feelings about computers: they felt comfortable using them (70%) and found them useful (93%), reasonably easy to learn (92%), and easy to use (82%). If they had a computer to support them with health care, they indicated that they would most likely keep it in the den, study, or living room (65%), with 24% favoring a bedroom.

Use of technology in home health care. The purpose of the system was to support home care. Were parents actually comfortable caring for their children at home, and what kind of activities did they engage in at home that a computer could suitably support.

Finally how would people say they felt about using a computer to support home health care? They were told: “The computer system we are designing will have information about cancer; it will help communicate with the Clinic staff and may help keep families whose children have cancer in touch with one another. We imagine that much of the system will use video like a TV, as well as having color pictures and text. You will be able to simply touch something on the screen to tell the system what you want to do.” Then they were asked, “Assuming that it were possible, how would you feel about using a computer to do the following?” The scale was 1 to 5, with 1 being the most positive response.

Nearly all parents (93%) reported being comfortable caring for their children at home. Many actively tracked critical aspects of care, with roughly 60% writing down blood counts, medication, treatments, and questions. About a third noted medical events such as fevers, vomiting, bruising, and the like; a third wrote down Clinic advice; a quarter kept some kind of diary of their feelings. Such event recording appeared unrelated to parents' levels of stress or their degree of education.

The overall response to the use of computers in home health care was positive. Mothers tended to be more enthusiastic, particularly regarding the use of the computer as a communication vehicle. (Table 8).

Table 8.

Rating a Computer To Support Home Health Care

Purpose Mean Rating
Learn about cancer 1.3
Remember what to watch for 1.3
Listen to the doctor 1.4
Assess an event (e.g., a fever) 1.5
Learn to perform a procedure 1.6
Decide about calling the hospital 1.6
Listen to other children 1.9
Listen to other parents 1.9
Create personalized videos 2.2
Communicate with another family 2.2
Screen phone calls 2.3
Perform relaxation exercises 2.5
Scale is 1 to 5, with 1 the most positive.

Individual differences. Users might want to tailor or organize information based on predictable differences in need. One possible difference was by gender: Common stereotypes lead to the expectation that men might be more information oriented while women were more concerned with interpersonal or emotional concerns.

Another kind of tailoring might be in terms of the user's education level. One might expect that more educated users would be interested in more complex or technical material or be more aggressive in their search for information.

Were there differences based on the stage of the disease? Did people simply want to know everything from the start, or were there different kinds of information that were appropriate as treatment progressed and families' priorities changed.

There were indeed differences between the responses of mothers and fathers, although the consequences for system design were unclear. There was no clear indication that a system might be used exclusively or even predominantly by one or the other, although one might reasonably expect differential patterns of use.

Mothers valued available resources more highly than did fathers (mother's mean = 1.57 versus father's mean = 1.82, p <.0001) for nearly all rated sources (18 out of 23), but both generally found what they needed to know. Mothers who reported the most difficulty in expressing their feelings were likely to be those who felt themselves out of control (Pearson correlation, r =.43, p <.005), although control and expression were uncorrelated for fathers. Curiously, fathers who found it hardest to express their feelings were also those who reported being most connected to other families (Pearson correlation, r = -0.42, p <.01); for the mothers, the expression of feeling was unrelated to their connectedness with other families.

While information was equally valued at all stages of the disease, the content requirements changed. It has been observed before that the information requirements in chronic illness change over time.18 The priority of the only family less than 6 weeks from diagnosis was for information about treatment. From 6 weeks to 6 months parents wanted to know how other people coped, but after a year the primary concern was about long-term medical side effects of treatment.

The second priority at all stages of treatment was the emotional effect of the disease/treatment process. Medication and prolonged hospitalization could have emotional consequences. It was difficult for parents to distinguish these changes from genuine aberrations in development requiring independent attention. Another emotional concern was the effect on the family as a whole.

Unexpectedly, the lowest priority at all stages was more information about home care. Perhaps because of the excellent training provided by the Clinic, and perhaps because of necessity, this is an area in which parents must quickly become expert.

Overall, there appeared to be no significant differences on the basis of education.

Information Foraging in Procedure Training

People receiving extended chemotherapy often have an intravenous catheter implanted into the chest wall to enable easy access to the circulatory system for administration of chemotherapy and blood testing. The catheter may have an external port, which must be kept clean and monitored for infection. Its dressing must be changed regularly, using a sterile procedure ordinarily taught in the hospital. The purpose of this study was to identify the information needs expressed by parents during procedure training.

Method

The typical training pattern included an initial lesson close to the surgical implantation of the catheter, possibly a follow-up lesson during the in-patient stay in the hospital, and then additional follow-ups during the first week at home until the nurse felt the parent had achieved sufficient competence to do it alone. The first lesson consisted of a demonstration of the procedure on a doll, followed by a closely supervised attempt by the parent. Six sessions were informally observed and video taped, noting instructional style, the problems parents experienced, and the questions that arose.

The instructor was a nurse practitioner who specialized in the care of cancer patients and in particular in the training of patients and caretakers new to intravenous catheters. The nurse was experienced both in home and clinical settings.

Results

The instructor was articulate, clear, and patient. She had a well-formed training routine for the procedures, interlaced with supplementary information and reassurance. The supplementary information covered such things as rationale (why you spiral out from the site when applying alcohol or betadine), common problems (forgetting to open the clamp), alternatives (large versus small tegaderms), things you don't have to worry about (which direction the gauze is facing), ways to remember what to do (to remember the swabbing order: a is for alcohol, b is for betadine), what is important (keeping sterile), personalization (since your child sweats, you'll want additional adhesive), and what to expect (it may be a little sore when you first get home).

Parents interrupted as they had questions, although most were requests to validate performance. Some of the other questions concerned procedural elaborations or exceptions:

  • What should I do if I can't adequately clean the work area?

  • The needle came off with the cap. What should I do?

Others seemed driven by dynamic development of a model of the task and associated concepts, such as these refinements of models of infection and sterility:

  • Do I need to wear a mask or gloves during heparin flushes?

  • Can I put the cap down once I've wiped it with alcohol?

During the parents' attempt at the task, the instructor provided encouragement, prompting, corrections, and answers. The instructional section moved along fairly quickly, causing a number of people to ask a little anxiously if written instructions would also be sent home. The directive style of the practice session seemed to enable a successful first experience for the parent. Sterility was the most difficult and important concept to fully integrate into caretaker practice. It appeared that sterile was not immediately differentiated from clean, and perhaps that the invisibility of bacteria made violations of procedure difficult to perceive.

Parents' Use of the Prototype Support System

Based on the review of the patient education, professional literature, and on the observations presented here, on-line content was developed for what was hoped to be broad coverage of the concerns germane to parents new to ALL and, in the area of broviac care at least, appropriately detailed as well. We included video interviews of parents talking about their experiences and feelings about living with ALL. Clinic publications were integrated with those from the public domain. Where possible relevant, recently released material from other sources was included, such as the New York Times and the Harvard Health Newsletter. The Clinic staff was interviewed, incorporating their perspectives and expertise. Video training materials were developed to respond to all the questions that had been previously raised about broviac care. While the broviac material was largely video, resource limitations constrained the remainder of the content to text. Part of the objective was to evaluate parent reaction to content and to the underlying system.

Training materials were developed for the broviac dressing change and heparin flush. The same nurse practitioner who did the live training was the instructor in the video. Training consisted of a text outline of the procedure, video demonstration and multimedia “footnotes” to amplify the main content. The video procedures completely covered the live training material, and the footnotes incorporated answers to all the questions discovered either in printed texts or through previous observations. The video training was developed using a minimalist philosophy19 that portrayed the main, error-free execution of the task without parenthetic commentary. Supplementary information was supplied in the footnotes that were made available by button press, whenever contextually relevant.

The prototype ran on an IBM PS/2, Model 95 with an attached laser disk and an additional 2 gigabytes of storage containing text, audio, and digitized full-motion video.

Method

Six parents were observed informally at the Pedi/Hem Onc Clinic exploring the system and working through the video procedure as a supplement to live training. As they worked they thought aloud about their activities20 and were debriefed at the end about their experiences.

Results

Parents were enthusiastic about the prospect of such a broad range of information available on-line and in the home. However, despite efforts to include information well beyond existing publications and routine training, experienced parents were already familiar with most of the content. They typically commented that this would have been a great facility to have had at the beginning and that they were sure if it became commercially available it would go into an appropriate level of detail.

While there were some parents who were a little tentative in approaching the system initially, their children were fearless. Parents worked with the system during the often lengthy waiting times at the Clinic, and occasionally their children would come back to visit. When they discovered the touch-sensitive screen, they would start to poke at it randomly to see what they could make it do. This eased the way for the parents, who became subsequently more comfortable explorers. The system had been designed for adult caretakers as users, but we were reminded of the possibly more general reality that it is often children who mediate adult introduction to computer systems.

Originally the content was organized in a strictly local hierarchy but encountered limits to its utility. Parents wanted ready access to emergency procedures. The authoring physician wanted to include the critical symptoms to watch for at home. System designers wanted content sufficiently interesting and salient to warrant continued exploration and to facilitate user training in use of the system. One patient reviewer commented that although (logically enough) interpretation of blood counts was included under clinic procedures, it was so important to patients that it deserved a much more prominent position.

Parents valued the ability to repeat steps in the procedure process but never spontaneously took advantage of the multimedia footnotes available by a button press.

Results Summary

While many issues surfaced during the course of the work, the main points can be summarized as follows:

  • Questions and concerns go beyond generally available patient education materials.

  • Information needs are concrete and task relevant.

  • Information itself can constitute emotional support by conferring conceptual control and by the same token can be emotionally threatening due to its inherent content or ambiguity.

  • Information and support needs may vary based on the stage of the disease as well as individual differences such as gender, but not evidently on the basis of educational background.

  • While materials should be organized for traditional hierarchical and topical browsing, special attention should be given to immediate and urgent needs.

  • Procedure training needs to respond to the needs of rapid acquisition, contextually situated modeling, and exception handling.

  • Computers are favorably regarded as a technology for addressing patient needs in the home, even with traditional keyboard data entry.

  • Consideration should be given to techniques for mediating the transition of computer novices to comfortable interaction with a consumer computer system. Here, parental use was facilitated by the fearless example of preschool children.

Discussion

Effective patient support at home implies attention to details and urgency of patient need. The most critical information enables understanding of what is happening, what to anticipate in treatment, symptoms and experience, how to plan for and manage the physical and emotional effects of the disorder, and the prognosis. The acquisition of this information can gradually confer a measure of control over an otherwise emotionally chaotic situation. These considerations suggest a taxonomy of medical information crucial to patient support:

  • Timely information relating to treatment and progress. Patients keep close tabs on the indicators of progress and want to understand exactly what they mean.

  • Information necessary to perform medical care. This may include essential information on what to look for, handling of emergencies, how to perform procedures, provider communications, etc. This should be succinct and readily accessed.

  • Emotional support. On the one hand, objective information itself provides a kind of emotional support. On the other hand, it is also necessary to validate patients' emotional response per se and help them understand any psychosocial implications of the disorder. Caretakers prefer to get information about emotional support from their peers, others who have “been there,” but there is also a place for metacommentary that surveys a broader range of response than an individual experience can provide.

  • Background information. Extended information on the disease and its treatment is needed. Some of this information needs to be made available immediately to help ground the patient and possibly to enable informed decision making. However, much of the explanatory material can be less conspicuously available.

With these considerations in mind, a template is proposed for patient information.

Template for Patient Information

The content design for the ALL prototype drew on materials currently in use at the Clinic,21,22 interviews with the Clinic staff, publicly distributed materials for childhood cancer, and technical literature about the psychosocial and informational needs of children with cancer and their families.

The intent was to service families' information and emotional support needs as fully as possible. This resulted in covering a broad range of categories and an attempt to cover information as deeply as resources would permit. While not all people want to know everything, there are many who do. Often there are more who do than the medical staff or information providers imagine.

The hierarchical organization was intended to make the material easy to browse. The organization was modified in response to feedback from user testing. It is hoped that this will provide a useful framework for subsequent health modules. Topics are described, and their generic subtopics expanded. The appendix outlines the topics in detail. What follows is the rationale underlying some of the choices.

Immediate Access Topics

When a patient is first diagnosed, there are likely to be a number of issues that are immediately pressing. It can take a while to become familiar with the content and organization of information resources, and initially the patient may have little patience or motivation for this kind of exploration. It may be desirable to generate a unit specifically to focus on the most immediate concerns of the new patient/caretaker, which can serve as an introduction to the materials while providing immediately useful information and support.

For ALL, video interviews with parents of a child with leukemia were included and information about managing during the initial extended stay in the hospital. Parents talked about how they felt at the beginning, reflecting on the experience. The messages were ones of empathy and optimism.

The Most Important Things to Keep in Mind

Almost by definition, patients in need of the system have a great deal to learn. While they need to do it quickly, it is often very difficult to remember everything because of the stress of the diagnosis. This section is the medical provider's response to questions about the most important things for the patient/family to remember when they get home. Each topic heading is itself an instruction to the patient. When the topic was selected, it expanded to provide more information on how to perform the instruction, the rationale for doing it, and frequently asked questions on the topic.

For example, for ALL the reminder list included:

  • Take a daily temperature

  • Report any temperature over 101°F.

  • Check for any signs of bleeding.

  • Watch for signs of unusual fatigue.

  • Be alert for exposure to chicken pox.

  • Do regular mouth care with a soft toothbrush.

What's New

A growingly common designation on WWW pages, “what's new” is an important feature of an on-line reference resource. People with a compelling interest in a topic may want to read everything available relevant to their particular circumstances. It is unlikely, however, that they will want to rely on scanning techniques to determine if something new has been added. While it is important to highlight new material, it should also be incorporated in place in the hierarchical exposition and structure. Extended lists of new material or of unedited questions and answers (a variant of “what's new”) become difficult to search and over time are decreasingly likely to address a user's interests in a coherent way.

Procedures

This includes instruction on any medical procedure that could be performed in the home. In the case of ALL, this ranged from taking the temperature (which could not be done rectally) to the care of the central venous catheters implanted in the chest wall and used for drawing blood and administration of chemotherapy.

For each procedure, provide the following:

  • An introduction to provide a rationale. This may highlight important considerations in performing it (for example, the importance of staying sterile) and, if the procedure is long or involved, a brief overview.

  • The procedure in its entirety, useful for overviews and reviews.

  • The procedure broken down into its component steps. Two levels were implemented:

    • Logical steps broke down the procedure into a relatively short sequence that constituted a high level overview of what had to be done. A text outline of the process was shown adjacent to the video instruction window. Changing the dressing for a broviac catheter decomposed into:
      • Setting up the work area.
      • Taking off the dressing and examining the site.
      • Setting up sterile materials.
      • Cleaning the site.
      • Putting on a clean dressing.
      • Checking the clamp.
    • Each logical step was then decomposed into the detailed component instructions. An instruction was a single physical act or a very small sequence of acts that could be remembered as a chunk, without repetition. “Putting on a clean dressing” was composed of:
      • Taking the small piece of gauze from the kit; touching it only on one side; placing the other side over the exit site.

    Taking the center square off the back of the tegaderm; removing the sticky side with the lettering.

    Centering the sticky side over the exit site and pressing down to get out all the air.

    Peeling off the outer rim of the backing while pressing down all around the edges to hold the dressing down.

  • Common mistakes and problems to provide useful related information not specifically part of the mainline procedure. This includes such things as rationale, alternative procedures, special purpose considerations, and responses to related questions.

Emotional support

While medical information itself is an important aspect of emotional support, material is often needed to address emotional support more directly. Inspirational and first-person stories of other people who shared the experience were included, as was metainformation about the experience to give perspective to the entire range of normal and pathological reactions, providing assistance in identifying, accepting, and working through feelings.

Glossary

The glossary contains medical terminology likely to be encountered by patients and caretakers, defined in straightforward, nontechnical terms. It should be comprehensive and elucidate medical jargon, acronyms, and other unfamiliar terms and usage. It should cover not only those terms intended to be shared with patients—the ones ordinarily included in patient education material—but also those that patients may encounter serendipitously in the literature or in the halls. Ideally, glossary terms should be pervasively accessible in context as well as in dictionary style forms.

Issues in Presentation of Consumer Medical Information

Other considerations have bearing on the creation of content. Although not all of them are stimulated by the empirical work in these studies, they have been included to provide a more comprehensive picture of the problem and to help situate the recommendations.

Readability and Detail

Based on the evidence from the use of existing applications such as CHESS13 and ONCOLINK23 the proliferation of medical newsletters such as those from Harvard, Berkeley, and Consumer Reports, the ever-increasing number of wellness and disease Web pages and Usenet groups, and the responses to the questionnaire, there is significant demand for health and disease information beyond the widely available lay literature.

Most of what patients read are relatively medium length, informative pamphlets or books primarily offering emotional support. However, a more elaborate need for information is evidenced in the emergence of on-line, disease-specific bulletin boards, mailing lists enabling peer networking among patients,4,13,24 and the development of on-line, question-and-answer services staffed by medical professionals.4,13 While there is a great deal of medical information on the World Wide Web, most of it is directed at professionals. Despite a number of efforts at consolidation,25 the Web is not very well organized; there are many points of entry and considerable redundancy. Only more recently has there been an eye to the support of patient populations. For examples, see the Cyberspace TeleMedical Office26 and Wellness Web—the Patient's Network.27

Patient education materials should be direct and concise. Without condescending, sentence structure should be simple. Familiar terminology should be used when possible and should be explained when it is not. Such materials are more readily absorbed by anyone, independent of educational achievement. Sick people and their caretakers may be under considerable stress and temporarily less adept at processing complex input.

Tolerance for additional detail is a function of individual disposition, the stage of the disease,28 specificity and urgency of need, and the degree of threat inherent in the material. Under conditions of personal stress people can be motivated to pursue salient technical material well beyond their customary reading level. The accessibility of such information can be improved with augmentations such as glossaries and provider annotations. The range of materials for patients could be extended at little cost by making available materials originally designed for general practitioners, nurses, or students. Multimedia materials developed for students and continuing education are often clear, simple, inviting, and informative. Although not specifically oriented around patient tasks or interests, they are an attractive and informative resource.

Effects of Situational Stress on Retention

Bourque and Perrault29 estimate the half-life of information conveyed to patients to be about 5 minutes. Chesler and Barbarin, while corroborating minimal retention, also report parents of children with cancer claiming detailed recall of the initial diagnostic conference.30 Possibly this reflects a different sampling by parents and professionals of what was recalled, paralleling work in cognitive psychology on recall under different conditions of stress.31

If in fact recall for the initial and perhaps longest—or even only—session with the physician is poor, it behooves the physician to make that information available in alternate and permanent forms. Ideally, and where appropriate, the information should be tailored to the individual patient,32 and critical information should be salient.

Conflicting Information

Clinical information can be confusing and may present contradictory views. Additionally, it is difficult to interpret technical material without adequate technical background. Problems are bound to occur with basic understanding, the ability to situate a particular paper, opinion, or finding within the realm of current thought or research, and to connect the information to a patient's individual case. Applying population statistics to an individual case is problematic, both because statistics are aggregate analysis by definition and medical cases can be highly variable, and because of biases inherent in human evaluation of statistical information.33

A number of strategies may be brought to bear on assisting the consumer in conflict resolution. The pilot included a video module of the Chief of the Pedi/Hem Onc Division acknowledging the tension inherent in the uncertainties of medical practice and suggesting strategies for conflict resolution. Explicit decision support materials, such as those developed by the Wennberg Foundation34 for prostate and breast cancer, currently available only in medical settings, could be made available in the home. Software that intelligently aggregates consumer preferences35 could be applied to medical applications and made widely available. Analytic programs could help consumers resolve statistical issues, such as the relative risk of heart disease versus breast cancer induced by estrogen therapy for menopause. Better encapsulation of the personal and interpersonal implications of treatment would complement the epidemiological work on treatment outcomes.

Emotional Character of Medical Information

Patients and caretakers may be anxious for the most up-to-date statistics on prognosis or current research, looking to discover fresh grounds for hope. However, there are impediments to effective presentation of this material. One difficulty lies in the different cultural and individual inclinations of providers to provide it, especially when the information is discouraging.6 Another is that caretakers and patients may primarily be looking for reassurance. Strategies for dealing with this conflict range from cultural injunctions against even confirming a diagnosis to unedited presentation of primary medical sources. Even consumer-oriented material may be quite frightening.

The design of on-line, patient-oriented systems should be sensitive to these simultaneous and potentially conflicting needs for hard information and emotional support. Information intended to reassure or confer mastery may be best presented as a video from an expert known to the listener. Threatening information might be less obtrusive as text from an unfamiliar expert or technical writer or maintained at lower levels in a topical hierarchy.

Frankly emotional issues are important, an area which may be less successfully addressed by existing sources. Among the most powerfully supportive experiences for patients and caretakers is the sharing with others who have already “been there.” Since community of interest does not necessarily coincide with physical proximity, there is good reason to foster the further development of on-line peer support for disorders of all kinds, particularly for those which are less common and as a result less likely to find a local, critical mass. One would anticipate that video presentation of peer support material would be particularly effective.

Information Source

People prefer information delivered in person from the most direct source. Unfortunately, current trends in the medical industry make it less and less likely that patients will have satisfactory personal access to their providers to address informational and psychosocial concerns. Even when patient questions are encouraged, people may be reluctant to speak up for fear of seeming foolish or of imposing on the professional.

For the well-meaning professional, it may be a burden to have to introduce the disease and treatment to each new patient and to repeatedly answer the same questions from patients as they gradually absorb the full meaning of their conditions. The computer's patience, availability, enormous memory, versatile means of expression, and low cost make it a valuable adjunct in patient education. In many situations it would provide a currently unavailable service. In other cases it would supplement existing efforts and improve the quality of patient involvement in their own care while freeing up time of the professionals.

The choice of presenter is important to patient content development and design. In multimedia presentation one can use actors, medical professionals, or one's own personal providers. While it is not clear at first blush which of these is the least expensive, actors will generally be more effective and efficient presenters. The unit time of medical experts may be more dear than that of either actors or personal providers. The use of personal providers would be of inconsistent quality and expensive in its redundancy. However, content personalization is important not only to the patient who wants individualized treatment but also to the provider who wants a differentiated practice.

Conclusion

The literature and ongoing experience with on-line systems point to the potential power of computer support to address a number of important needs of the chronically ill. Consumers appear positive disposed toward on-line solutions. On-line systems can readily offer a breadth and timeliness currently unattainable. Distributed and cooperative information development can capitalize nonredundantly on a range of expertise. Many users have the potential to access highly salient and ultimately personalized data.

The content of patient education material should be both broad and deep. It should cover the full range of issues relevant to the patient, including self-care; resources to interpret personal state, such as progress indicators and explanations of tests, procedures, and treatment; comprehensive guide to interaction with medical and ancillary services, such as the hospital, clinic, insurance, and suppliers, and emotional and medical support services; and explanatory information about all aspects of their condition and emotional support.

Patient education should be developed for all disorders (not merely the high volume disorders such as heart disease, cancer, and diabetes) with comprehensive detail. Since content production is expensive, there would be considerable overall savings in cooperative development and general dissemination of patient education material. On-line application support facilities such as glossaries, search mechanisms, language translation, and other important patient support mechanisms such as peer networking can be integrated into a shared client/server framework as they become available.

Information should be layered. Information needed early on, frequently, or in emergencies should be salient and easily retrieved. More discretionary information may be accessed using conventional categories and structures, such as a table of contents. Detailed or threatening information should be accessible but hidden, constrained, or annotated to prevent inadvertent access or intrusive exposure.

Materials should be clear and direct, targeted at a middle school comprehension level without being patronizing. There should be a comprehensive, hypertextually available glossary to elucidate medical jargon, acronyms, and other unfamiliar terms and usage. This should apply not only to those terms intended to be shared with patients—the ones ordinarily included in patient education material—but also those that patients may encounter serendipitously in their medical travels.

As with any other service or interface, patients should be involved in the development of materials intended for their use. They can be questioned, interviewed, consulted, and observed in natural situations. It should not be supposed that the providers' sense of what the patient should know entirely coincides with what the patient wants or needs to know. Much, perhaps nearly all, of the process that has been developed to improve the usability of software products,36 including early and continuous involvement of users, iterative development, and empirical evaluation of usability, applies equally well to content development if it is effectively to serve the needs of the users.

Acknowledgments

This work would not have been possible without the collaboration of the Pediatric/Hematology/Oncology Division of the medical center that participated in this work, nor without the gracious and generous cooperation of so many of its patients' families.

Team members Hamed Ellozy, Robert J. Schloss, Michelle Kim, and Wendy Kellogg all contributed both to the content and the impetus for this effort. Mary VanDeusen filmed the procedural observations. Dr. Kellogg helped collate the results of the first questionnaire and coauthored the second.

Dr. Patti Brennan provided extraordinarily patient and thoughtful commentary, suggestions, and support in bringing the manuscript to publication.

Appendix

Patient Information Template

  • Immediate access topics

    • The tasks and concerns most salient to the patient and immediately significant for patient care.
  • What's new

  • The most important things to keep in mind

  • Homecare

    • Communicating with the hospital
      • Emergency calls
      • Routine clinic calls
      • When to use system communications
      • Tracking medication
      • Tracking symptoms and questions
    • Physical health
      • Special disease-related concerns
      • Getting sick (from something else)
      • Interacting with the primary medical provider
    • Procedures
    • For each procedure provide:
      • An introduction
      • The procedure in its entirety
      • The procedure broken down into its component steps
        • 1. High-level overview
        • 2. Detailed steps
      • Common mistakes and problems
      • Useful related information that was not specifically part of the mainline procedure
  • Treatment

    • Overview/introduction
    • Phases of treatment
    • For each medication or treatment:
      • Effects—what should be expected and when
      • Side effects—focusing on the most common but also including the less common
      • Ways of coping with side effects
      • Dosages
      • Means of administration
      • Techniques for coping with administration of the drug or performance of the procedure
      • What to do in case of an overdose
    • For those treatments for which there may be discretionary choice on the part of the patient or caretaker, pros and cons of the alternatives are provided. When the considerations involve quality-of-life decisions, experiences of other patients/caretakers are included.
    • Procedures including those performed by a medical provider. Instruction is provided for procedures that can be performed at home (see Home care).
      • Details of what happens
      • Purpose
      • Description of the results; for example counts derived, or therapy achieved
      • Detailed discussion of medical protocols, including
        • How they work
        • Potential risks
        • Patient rights
        • Information about the protocols applicable to their condition
      • Interaction of treatment with vacation planning
  • Emotional support

    • Feelings
    • Coping strategies
      • Understanding and dealing with other people (family, friends, etc.)
      • Working with places of employment/schools
      • Community support
      • Self-help groups
    • Patient/caretaker empowerment
      • Speaking up
      • Making decisions
      • Extending personal caretaking responsibility
    • Relaxation techniques
    • Reservoir of personally meaningful material
  • Disease

    This is a discussion of the disease itself. It contains the following generic components:

    • Overview
    • Causes
    • Demographics
    • Prognosis
    • Progression of the disease
    • Key indicators of treatment progress
    • Detailed description
  • Hospital/clinic/provider handbook

    • Clinic philosophy
      • Doctor/family relationship
      • Calling during off-hours
      • Asking questions
        • Provider view
        • Strategies
      • Conflicting advice
    • Care team
      • Provider staff, describing both the roles in general and providing introductions to individuals on the staff
      • Working with the medical team (patient/caretaker role)
      • Role of medical providers outside the specialty team
      • Coordination of care among providers
    • Clinic operations
      • Schedule
      • Routine and emergency calls
      • Preparing for visits
      • Clinic procedures
      • Description of a typical clinic visit
      • Waiting
        • Why it can happen
        • Duration of common procedures
      • Day care
      • Billing and finances, including financial assistance
      • Clinic research
    • Hospital operations
      • Visiting rules
      • Overnight stays
      • Parking
      • Telephone calls
      • Communications among and with the staff
      • Theft
      • Teaching hospital considerations
      • Billing
    • Life in the hospital
      • Managing visitors
      • Making the hospital friendlier
      • Recreational opportunities
  • Services/resources

    A list of ancillary services and resources

    • Hospital/clinic services
      • Food (cafeterias, food machines)
      • Support services
      • Financial assistance
      • Medical and recreational libraries
    • Neighborhood resources
      • Nearby hotels
      • Places of worship
      • Restaurants
      • Florists
    • External support
      • External organizations.
        • For ALL this included such groups as the American Cancer Society and Candlelighters
      • Database and network (Internet) resources
      • References to books, pamphlets, videos; reference includes a summary and ordering information
      • Social security
      • Community resources
      • Self-help groups
      • Spiritual support
  • Glossary

References

  • 1.Jimison HB, Sher PP. Consumer health informatics: health information technology for consumers. Journal of the American Society for Information Science. 1995;46: 783-90. [Google Scholar]
  • 2.Varni JW, Katz ER. Psychological aspects of childhood cancer: a review of the research. Journal of Psychological Oncology. 1988;5: 93-119. [Google Scholar]
  • 3.Austin CJ. Information technology and the future of health services delivery. Hospital and Health Services Administration. 1988;34: 157-165. [PubMed] [Google Scholar]
  • 4.Brennan PF, Ripich S, Moore SM. The use of home-based computers to support persons living with AIDS/ARC. J Community Health Nurs. 1991;8: 3-14. [DOI] [PubMed] [Google Scholar]
  • 5.Jimison HB, Kieschnick T, Adler L. Informatics for patient education: beyond information exchange. Proc Annu Symp Comput Appl Med Care. New York: McGraw Hill, 1995.
  • 6.Nathanson MN. Meeting the educational and psycho-social needs produced by a diagnosis of pediatric/adolescent cancer. Health Educ Q. 1984;10: 67-75. [PubMed] [Google Scholar]
  • 7.Koltnow PC. 1972, The Family. Proceedings of the American Cancer Society's National Conference on Human Values and Cancer. American Cancer Society, June 22-24, 1972.
  • 8.Hymovich DP. Parents of sick children: their needs and tasks. Pediatric Nursing, September/October, 1976. [PubMed]
  • 9.Family Roles in Medical Decisions. The Candlelighters Foundation Quarterly Newsletter. 1982;6: 1-3. [Google Scholar]
  • 10.Barbarin O, Chesler M, Chesler J. Childhood cancer's impact on families: parents and the medical care organization. The Candlelighters Foundation Progress Reports. 1983;3: 9-12. [Google Scholar]
  • 11.Kramer RF, Perin G. Patient education and pediatric oncology. Nurs Clin North Am. 1985;20: 31-48. [PubMed] [Google Scholar]
  • 12.SAS Institute Inc. SAS User's Guide: Basics, Version 6 edition. Cary, NC: SAS Institute Inc., 1993.
  • 13.Gustafson DH, Hawkins RP, Boberg EW, Bricker E, Pingree S, Chan C. The use and impact of a computer-based support system for people living with AIDS and HIV infection. Proc Annu Symp Comput Appl Med Care. New York: McGraw-Hill, 604-608. [PMC free article] [PubMed]
  • 14.Kushner HS. When Bad Things Happen to Good People. New York: Avon Books, 1983.
  • 15.Bracken JM. Children with Cancer. New York, Oxford University Press, 1986.
  • 16.Lyman MJ. The parent network in pediatric oncology: supportive or not? Cancer Nurs. 1987;10: 207-16. [DOI] [PubMed] [Google Scholar]
  • 17.Norman DA. The Psychology of Everyday Things. New York: Basic Books, 1988.
  • 18.Kramer RF, Perin G. Patient Education and pediatric oncology. Nurs Clin North Am. 1985;20: 31-48. [PubMed] [Google Scholar]
  • 19.Carroll JM. The Nurnberg Funnel: Designing Minimalist Instruction for Practical Computer Skill. Cambridge: The MIT Press, 1990.
  • 20.Ericsson KA, Simon HA. Verbal reports as data. Psychol Rev. 1980;87: 215-51. [Google Scholar]
  • 21.Division of Pediatric Hematology/Oncology, Floating Hospital for Infants and Children, New England Medical Center, Parent/Child Handbook..
  • 22.New England Health Resources. Home Therapy Reference Manual, 1986.
  • 23.Buhle EL, Goldwein JW, Benjamin I. OncoLink: a multimedia oncology information resource on the Internet. Proc Annu Symp Comput Appl Med Care. New York: McGraw-Hill, 1995, 103-7. [PMC free article] [PubMed]
  • 24.Conroy C. Care takers. Compuserve Magazine. February 1994; 10-19.
  • 25.Medical Matrix Guide to Internet Clinical Medicine Resources. http://www.kumc.edu/matrix/
  • 26.Cyberspace TeleMedical Office. http://telemedical.com/drcarr/
  • 27.WellnessWeb. The Patient's Network. http://www.wellweb.com/wellness/
  • 28.Hamburg DA, Adams JE. A perspective on coping behavior: seeking and utilizing information in major transitions. Arch Gen Psychiatry. 1984;17: 277-84. [DOI] [PubMed] [Google Scholar]
  • 29.Bourque M, Perreault R. Developing content for a computer-based health education system. Proceedings of the First Annual Joint Conference of the AMIA. 1985.
  • 30.Chesler MA, Barbarin OA. Relating to the medical staff: how parents of children with cancer see the issues. Health Soc Work. 1984;9: 49-65. [DOI] [PubMed] [Google Scholar]
  • 31.Johnson C, Scott B. Eyewitness testimony and suspect identification as a function of arousal, sex of witness, and scheduling of interrogation. Presented at the meeting of the American Psychological Association, Washington, DC, September, 1976.
  • 32.Carenini G, Mittal VO, Moore JD. Generating patient-specific interactive natural language explanations. In: JG Ozbolt (ed). Proc Ann Symp Comput Appl Med Care. New York: McGraw-Hill, 1995. [PMC free article] [PubMed]
  • 33.Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science. 1974;185: 1124-31. [DOI] [PubMed] [Google Scholar]
  • 34.Kasper JF, Mulley AG, Wennberg JE. Developing shared decision-making programs to improve the quality of health care. QRB. June 1992; 183-90. [DOI] [PubMed]
  • 35.Shardanand U, Maes P. Social information filtering: algorithms for automating “word of mouth.” Human Factors in Computing Systems Proceedings. Annual Conference Series, New York: ACH SIGCHI, 1995; 210-17.
  • 36.Gould JD. How to design usable systems. In Helander M (ed): Handbook of Human-Computer Interactions. Amsterdam: North Holland, 1988; 757-89.

Articles from Journal of the American Medical Informatics Association are provided here courtesy of Oxford University Press

RESOURCES