Patients use an internet technology to report when things go wrong

Abstract

Background

As patients directly experience harm from adverse events, investigators have proposed patient‐report to complement professional reporting of adverse events.

Objective

To investigate how an automated health assessment system can be used to identify adverse events.

Design and setting

Internet survey responses from April 2003 to April 2005 involving communities and clinical practices across the USA.

Patients

44 860 adults aged 19–69 years.

Outcome

Patient perceptions of adverse events experienced during the previous year. Independent legal review was also used to estimate how many patient‐reports were serious enough to be potentially compensable.

Results

Although patient reports of possible adverse events was low (1.4%), the percentage of adverse events was eight times higher for patients with the greatest burden of illness than for those with the least (3.4% vs 0.4%). Two expert malpractice attorneys agreed that 9% of the adverse events seemed to be serious.

Conclusions

Patients will use internet technology to report their perceptions of health‐related adverse events. Some of the patient‐reported events reported will be serious.

A highly influential report from the Institute of Medicine, Washington DC, USA, “To err is human: building a safer health care system”, encouraged better documentation of errors and adverse events, so that causes for problems might be identified and remedied.¹ However, reporting of error and adverse events by health professionals depends on variables such as their awareness of the events, their judgment about events and their willingness to document the events. Incidence rates for adverse events vary 10‐fold when reported by health professionals; differences are attributed to professional attitude and judgment rather than the underlying rate of errors and adverse events.²,³ Since patients directly experience harm from adverse events, investigators have proposed patient‐report to complement professional reporting.⁴,⁵,⁶,⁷,⁸ Although the proposition that patient reporting of adverse events has merit, the feasibility of patient‐report is not known.

This study investigates how an automated outpatient health assessment system can be used to identify adverse events.

Method

The health assessment survey and the study population

More than 20 years ago, the Dartmouth Cooperative Practice‐Based Research Network identified important, wide gaps in the communication between patients and doctors.⁹ As an example, this research documented the need for and stimulated the development of patient‐report measures of functional status.¹⁰ The Cooperative Practice‐Based Research Network now widely disseminates a patient health assessment survey to improve patient–clinician communication.¹¹,¹²,¹³,¹⁴,¹⁵,¹⁶

Using computer‐generated branching logic, up to 120 items of the survey inquire about health status, symptoms, concerns, chronic disease management, preventive interventions, specific healthcare experiences and confidence with self‐care (http://www.howsyourhealth.org). Based on their own initiative or at the request of their doctor, people complete the health survey to obtain health and self‐management information customised to their needs.

As no personal identifiable information is stored by any means, patients decide on how to use their information. For example, they can deliver a clinical summary of the information (electronically or in person) to their doctor so that they and the doctor can review and manage the important problems identified by the survey in a better way.

Item completion rates are >95%. Whenever a patient reports that he/she has completed the survey in the previous 6 months, the recent information is not stored. The data are exempt from institutional review board review as no personal identifiable data is stored.

This report is based on 44 860 cross‐sectional responses by residents of the USA, aged 19–69 years during the 2‐year time frame from April 2003 to April 2005.

A measure of adverse events

Near the end of the survey one item asks:

Part A: “Describe here any medical errors (mistakes) that you or your family have experienced. Errors include such things as mixed up medications or poor treatment that result in harm or additional problems. If possible, be sure to tell us the cause of the error and how it might have been avoided. Your response will help us to improve future care delivery.”

Part B: “If you wrote in an error or harm, please help us by choosing ANY of the following categories for this error. (Please mark all that apply.) It caused harm, hurt or injury. It happened within the last year. It happened to me.”

What is an adverse event?

Although an individual clinician receiving a patient report of an adverse event might be able to corroborate it with the information in the medical record, for this large study population of patients only their verbatim comments were available to us. To contend with the possibility that the patients' reports might represent mere complaints about poor service, we used independent legal review to estimate how many, if any, patient‐reports were possibly serious—that is, the percentage of reports that might, after additional review, cross the threshold from nuisance to potentially compensable injury.

A lawyer (MH) with extensive experience in medical malpractice litigation reviewed all patient‐reported adverse events and selected those that seemed serious enough to represent compensable injury: when the costs of the legal remedy could be greater than the costs of litigation.

We derived a conservative estimate of potentially compensable injuries by subjecting the adverse events identified by the first legal review to another review. The cases that were agreed upon by two attorneys constitute our estimate of potentially serious adverse events. A verbatim list of these events is available at http://www.howsyourhealth.org/adverse.pdf.

Analysis

We have limited the analysis to descriptive statistics as the data are cross‐sectional and are based on a convenience sample of patients out of a much larger potential population of non‐respondents in clinical practices or communities.

Results

From the 2‐year convenience sample of 44 860 survey respondents, 2979 indicated a harm, hurt or injury to themselves or a family member, and 610 of these adverse events had happened to the respondents during the previous year.

Association of possible adverse events with burden of illness

Table 1 lists the characteristics of patients who reported health‐related adverse events and those who did not. All but four (age, hypertension, exercise and excessive alcohol consumption) of the many listed variables were statistically different between the two groups of patients. In the stepwise regression of the demographic and composite variables listed in table 1, burden of illness had by far the greatest association with a report of a possible adverse event.

Table 1 Health‐related adverse events and patient characteristics.

	Respondents reporting no health‐related adverse event in the previous year 98.6% (n = 44 250)		Respondents reporting a health‐related adverse event in the previous year 1.4% (n = 610)
Demographics
Female		65		79*
Age 19–49 years		68		63
Low financial status		15		27*
Bothersome dysfunction
Pain		13		44*
Emotional		10		24*
Social support		11		26*
Physical activity		7		23*
Daily activity		4		24*
Social activity		4		22*
Chronic disease
Hypertension		20		24
Respiratory		10		20*
Arthritis		14		26*
Diabetes		6		11*
Cardiovascular		4		8*
Symptoms
>1 of 14 bothersome symptoms		33		65*
Composite Burden of Illness Score† (dysfunction, disease, symptoms)		3.0		4.8*
Unique prescription medications†
⩾3		20		45*
Any hospital use in the previous year		8		33*
Lifestyle
Not exercising regularly		59		61*
BMI ⩾30		31		36
Poor injury prevention or poor eating habits		20		23
Smoking		14		22
Told to “cut down” drinking		7		8*
Composite Health Behaviour Score		3.0		3.4*

BMI, body mass index.

*Significance of difference is p<0.01.

†Burden of Illness ranges from 0 (none) to 11; Health Behaviour from 0 (no risk) to 15.

All values are in percentages except for Composite Burden of Illness Score and Composite Health Behaviour Score.

Figure 1 depicts the strong association between possible adverse events and burden of illness. The annual percentage of health‐related adverse events ranged eightfold, from 0.4% to 3.4% across quintiles of illness burden.

Figure 1 Annual rate per 100 of health‐related adverse events associated with the patients' burden of illness. Burden of illness is a composite score based on the number of chronic diseases, the number of bothersome symptoms and the number of bothersome functional problems. The 95% CI is shown for each quintile of burden from the lowest to the highest.

Severity of possible adverse events

In all, 574 of the 610 patients provided descriptive information about what went wrong. The majority of these events occurred in an outpatient setting; only 12% (66) of the possible adverse events transpired in a hospital.

Two expert malpractice attorneys agreed that 9% (52/574) of the adverse events seemed serious enough to represent a possible compensable injury. In absolute terms, twice as many serious events were experienced in the ambulatory setting (n = 35) than in a hospital (n = 17). However, in relative terms, 26% (17/66) of the hospital events were possibly compensable compared with 7% of those in an outpatient setting.

Comment

This study demonstrates that patients will use internet technology to report their perceptions of health‐related adverse events. Patients with a greater burden of illness are more likely to report adverse events than those with little burden. The legal review adds credence to the notion that some patient reports will be more serious than mere complaints about poor service.

These results suggest an inexpensive mechanism by which a clinician can learn about a patient's perceptions of previous poor care, acknowledge the problem, investigate its cause and initiate a remedy.¹⁷ By being patient‐centered and giving the perceptions of possible adverse events a human face, such systems could be useful for influencing the way the outpatient clinicians think about safety.¹⁸ Similar systems are also available for inpatients (http://www.howsyourcare.org).

Despite the promise of such automated systems for assessing health and healthcare, the findings have at least three significant limitations.

First, since the patient denominator is a non‐random sample from an unknown population, the adverse events reported in this study should not be interpreted as a benchmark rate based on a national probability sample of patients. A patient‐reported adverse event will be biased by the background of the person who chose to respond to an online “health check‐up” and their opportunities to interact with the healthcare system.

Second, in addition to the limitations intrinsic to our specific approach, both professional and patient‐derived reports of “when things go wrong” have biases that reflect the attitudes and experiences of the reporters, their incentives or disincentives to report, and the ease of use and fidelity of the reporting system.²,³,⁴,⁵,⁶,⁷,⁸,¹⁹

Finally, as yet there is no proof that patient‐reported perceptions of health‐related harms will improve the safety and quality of care.

In summary, we have shown how an internet‐based, patient assessment survey can produce information about health‐related adverse events. More research is needed to demonstrate the value of such automated systems in improving healthcare.

This research was supported by grants from The Commonwealth Fund and the Robert Wood Johnson Foundation.