Hospitalized patients’ participation and its impact on quality of care and patient safety

Abstract

Objective

To understand the extent to which hospitalized patients participate in their care, and the association of patient participation with quality of care and patient safety.

Design

Random sample telephone survey and medical record review.

Setting

US acute care hospitals in 2003.

Participants

A total of 2025 recently hospitalized adults.

Main Outcome Measures

Hospitalized patients reported participation in their own care, assessments of overall quality of care and the presence of adverse events (AEs) in telephone interviews. Physician reviewers rated the severity and preventability of AEs identified by interview and chart review among 788 surveyed patients who also consented to medical record review.

Results

Of the 2025 patients surveyed, 99.9% of patients reported positive responses to at least one of seven measures of participation. High participation (use of >4 activities) was strongly associated with patients’ favorable ratings of the hospital quality of care (adjusted OR: 5.46, 95% CI: 4.15–7.19). Among the 788 patients with both patient survey and chart review data, there was an inverse relationship between participation and adverse events. In multivariable logistic regression analyses, patients with high participation were half as likely to have at least one adverse event during the admission (adjusted OR = 0.49, 0.31–0.78).

Conclusions

Most hospitalized patients participated in some aspects of their care. Participation was strongly associated with favorable judgments about hospital quality and reduced the risk of experiencing an adverse event.

Introduction

Patients who participate in their care may reduce the risk of medical errors by providing clinicians with current information about their medical histories, medications and drug allergies [1]. They could notify clinicians of unexpected side effects or of difficulty completing recommended tests or treatments [2–8]. Some patients may recognize and inform clinicians of lapses in care in time to prevent an adverse event [9–13]. Patient participation could foster collaboration and empowerment [11], and in turn, enhance patients' perceptions of the quality of care [14–16].

Although there is limited empirical evidence that patient participation enhances patient safety, prominent organizations implicitly endorse this approach by producing consumer advisories that encourage patients to ‘speak up,’ ‘ask questions’ and follow recommendations for protecting themselves against medical errors [17–20]. Case reports [21, 22] and several small studies document patients' ability to identify errors and adverse events [6, 9, 12, 13, 23–25]. To our knowledge, no prior studies have measured the extent of patient participation in the hospital inpatient setting or its relationship to adverse events or quality.

To better understand the nature and extent of patient participation and its impact on care, we conducted a multi-faceted study of patient safety in US acute care hospitals [26–28]. In this study, we sought to address several questions: (i) Do patients participate in aspects of their care that may affect patient safety? (ii) Are some patients more likely to participate than others? (iii) Is patient participation associated with favorable perceptions of the quality of medical care? (iv) Do patients who participate actively experience fewer adverse events? We hypothesized that a majority of patients would participate in their own care, and that participation would be associated with higher quality ratings and fewer adverse events.

Methods

Sample

We created a two-stage probability sample of patients who were discharged from US hospitals in Massachusetts. We selected the five largest acute care hospitals in the state with certainty. The remaining 66 hospitals were assigned to three groups of 22, based on size, and 5 hospitals were then selected randomly from each group. We then sampled cases from within each group to obtain a self-weighting sample size of about 6000. Of 6003 patients identified by the study hospitals, 4163 met eligibility criteria: they were adults (age 18 or older) discharged from a medical or surgical service between 1 April and 1 October 2003. A detailed account of study methods is available [26, 27].

Survey instrument

The research team designed and tested a survey instrument to elicit information about patients' experiences during a recent hospital stay. The instrument queried patients about ‘negative effects’ that they attributed to their usual medications, to new medications given during the hospitalization, to surgery, or to tests performed during the admission, and asked patients about 10 specific complications or treatment-related injuries such as heart attack and falls. We also asked respondents about their sociodemographic characteristics, their personal health history and characteristics of the index hospitalization, including their assessment of the overall quality of care.

There are no extant validated instruments measuring patient participation in inpatient care. We therefore developed a series of seven questions, based on literature review and focus groups, intended to characterize hospitalized patients' participation in activities that might promote safe care [24, 25]. Participation comprises a continuum of passive and active behaviors that include the ability to interact with caregivers, seek or obtain information, be involved in decisions and ensure the patient's wishes are followed and care is delivered safely. The so-called ‘active’ patients may perform both passive and active participation behaviors during a hospitalization. Even debilitated patients may participate meaningfully in their care if providers communicate effectively and offer opportunities to ask questions and make decisions.

Accordingly, survey questions assessed patients' (i) knowledge about their medical problem, (ii) ability to talk with health providers, (iii) ease of finding a doctor or nurse to provide needed information, (iv) perception that a clinician had described the risks and benefits associated with treatment decisions, (v) ability to participate adequately in decision-making, (vi) experience with visitors who helped make sure patients' wishes were addressed and (vii) experience checking their own medications while in the hospital (see Table 2). Questions underwent cognitive testing with 10 former patients. Some activities rely on hospital characteristics and clinician behaviors to facilitate patient participation; however, they may also reflect patients' help-seeking activities. Informed by research on the relationship between self-efficacy and patients' use of safety-oriented behaviors [10], we asked each respondent to indicate ‘In general, when in the hospital, how much can you rely on your own knowledge and alertness to protect yourself from medical errors?’ To assess the respondent's perception of the quality of care, we asked ‘What overall rating would you give to the quality of medical treatment you received during that hospital stay?’

Table 2.

Patient-reported participation activities (n= 2025)

Survey question	n (%)
(1) During that hospital stay, how much did you know about the medical problem for which you were admitted?
A lota	1280 (63.2)
Somea	394 (19.5)
A little	220 (10.9)
Not at all	131 (6.5)
(2) During that hospital stay, how often did you feel well enough to be able to talk with your doctors and nurses?
Alwaysa	1345 (66.4)
Usuallya	397 (19.6)
Sometimes	260 (12.8)
Never	23 (1.1)
(3) When you wanted information about your care and treatment, how easy or difficult was it to find a doctor or nurse to tell you what you wanted to know?
Very easya	1024 (50.6)
Somewhat easya	732 (36.1)
Somewhat difficult	211 (10.4)
Very difficult	58 (2.9)
(4) During the hospital stay, when decisions had to be made, how often did your doctors and nurses describe the good and bad things about your treatment options?
Alwaysa	1230 (60.7)
Usuallya	396 (19.6)
Sometimes	295 (14.6)
Never	104 (5.1)
(5) Did you participate in the decisions your doctors made about your care … ?
Less than wanted	181 (8.9)
About the right amounta	1818 (89.8)
More than wanteda	26 (1.3)
(6) During that hospital stay, did you have a family member or a friend visit you? If yes, did that person help you make sure your health care wishes were being followed by the hospital staff?
Visitor always made surea	1237 (61.1)
Visitor usually made surea	364 (18.0)
Visitor sometimes made sure	182 (9.0)
Visitor never made sure	222 (11.0)
No visitors	20 (1.0)
(7) During that hospital stay, when you were given medicines, did you ever check to make sure that they were the correct ones? If yes, how often did you check the medicines given to you by the hospital staff?
Checked alwaysa	637 (31.5)
Checked usuallya	160 (7.9)
Checked sometimes	108 (5.3)
Did not check	1120 (55.3)
Number of participation activities
0	3 (0.1)
1	18 (0.9)
2	66 (3.3)
3	106 (5.2)
4	188 (9.3)
5	483 (23.9)
6	722 (35.7)
7	439 (21.7)
Mean ± SD [range]	5.45 ± 1.34 [0–7]
Participation
Low (0–4)	381 (18.8)
High (5–7)	1644 (81.2)

^aPositive participation activities.

The survey was developed and administered by telephone in collaboration with the University of Massachusetts Center for Survey Research. A total of 2582 patients completed the survey, a 62% response rate. Of the survey respondents, 2025 (78.4%) supplied information for all seven participation questions. Our analyses were restricted to these 2025 respondents.

Medical record review

The study team also collected and reviewed the medical records of a subset of subjects (n= 998) who completed the telephone survey and returned a written informed consent for chart review. Nurse reviewers, trained by investigators, completed chart abstractions using an explicit review form that was developed and tested in previous studies of adverse events affecting hospitalized patients [29]. Of the 2025 patients included in this study, 788 had their charts reviewed.

Identification of adverse events

Physician reviewer pairs identified independently potential adverse events from two sources: (i) survey respondents' reports of ‘negative effects’ related to their hospitalization and (ii) events identified by nurse reviewers during chart abstraction. The physicians then coded candidate adverse events from both medical record review and patient survey regarding the presence of an adverse event, defined as ‘unintended harm to the patient by an act of commission or omission rather than by the underlying disease or condition of the patient [30]’. Reviewers also rated severity (life-threatening, serious, significant or trivial or insignificant) and preventability (definitely, probably, probably not, definitely not). Reviewers excluded candidate adverse events if there was insufficient information to adequately understand or interpret the report, if internally inconsistent, or if the description was not clinically plausible. Interrater reliability of reviewers' pre-consensus judgments using the kappa statistic was satisfactory for the classification of adverse events (0.85), preventability (0.71) and severity (0.45) identified by survey; similar results were obtained for events identified on chart review (0.61–0.96). These figures are comparable to interrater agreement rates reported in previous studies [31, 32].

Data analyses

We analyzed the demographic and clinical characteristics of the sample, stratified by level of participation (high vs. low) using t-test, chi-square and Fisher's exact test, as appropriate. We dichotomized each participation activity by grouping positive responses (e.g. ‘always’ with ‘usually’) and negative responses (e.g. ‘somewhat difficult’ with ‘very difficult’). Because we had no a priori knowledge of the relative contribution of each item, we created a summary variable equal to the number of activities identified by each subject (possible range: 0–7). We coded low participation (0–4 activities) and high participation (5–7 activities) based on the response distribution (see Table 2).

We hypothesized that participation may be directly associated with patients' overall assessment of the quality of care during the hospitalization because high-quality hospitals may create opportunities for patients to participate in their care. Alternatively, patient participation may enhance care or create the perception of better care. In order to analyze this association, we first dichotomized patient-reported ratings of quality (excellent/very good vs. good/fair/poor). We then created a series of multivariable logistic regression models to analyze the relationship between patients' overall assessment of the quality of care during the hospitalization and each participation activity.

We also hypothesized that participation may be inversely associated with the presence of adverse events. If patient participation improves vigilance and communication with providers about potential hazards, this may prevent errors and injuries. Adverse events were coded as present if they were rated by the physician reviewer as significant, serious, life-threatening or fatal, and if the reviewer had strong or certain confidence in the rating. We also examined the subgroup of ‘serious’ adverse events, which included those rated as serious, life-threatening or fatal. ‘Preventable’ adverse events were limited to those adverse events with a physician rating of probably or definitely preventable. We created separate multivariable logistic regression models to analyze the relationship of types of participation activities and number of activities with the presence of adverse events, serious events and preventable events.

All models adjusted for demographic and clinical variables that were significant in the bivariate analyses. Models also adjusted for respondents' perceived patient safety self-efficacy [10]. We re-analyzed the data using generalized estimating equations to account for intra-hospital correlation. The results of these analyses were qualitatively the same, so we report only the results from the unclustered multivariable logistic regression models. Analyses used SAS version 9.1.3 (SAS Institute, Cary, NC).

The study protocol was approved by the institutional review boards of the investigators' institutions and of the participating hospitals.

Results

Characteristics of respondents and their hospitalization

The 2025 survey respondents had a mean age of 60.5, were 56.8% female, 3.6% Latino and 7.1% non-white. About one-quarter characterized their overall health as fair or poor. About half were admitted via the emergency department and 17.2% spent time in an intensive care unit. Additional characteristics are shown in Table 1.

Table 1.

Demographic and clinical characteristics by participation level (n= 2025)

	High participation (n= 1644), n (%)	Low participation (n= 381), n (%)	P-value
Age (mean ± SD)	60.8 ± 16.3	59.0 ± 16.6	0.046
Gender			0.01
Male	731 (44.5)	143 (37.5)
Female	913 (55.5)	238 (62.5)
Hispanic/Latino			0.19
Yes	54 (3.4)	18 (4.8)
No	1554 (96.6)	359 (95.2)
Race			0.59
Asian	11 (0.7)	3 (0.8)
Black/African American	78 (4.9)	13 (3.5)
White	1488 (92.8)	349 (93.3)
Native Hawaiian/Pacific Islander	2 (0.1)	1 (0.3)
American Indian/Alaskan	12 (0.7)	4 (1.1)
Other	12 (0.7)	4 (1.1)
Overall health			<0.001
Excellent	245 (15.0)	35 (9.2)
Very good	520 (31.7)	113 (29.7)
Good	500 (30.5)	99 (26.0)
Fair	280 (17.1)	93 (24.4)
Poor	93 (5.7)	41 (10.8)
Do you have one main doctor?			0.18
Yes	1499 (91.4)	340 (89.2)
No	141 (8.6)	41 (10.8)
For how many months or years have you been going to this doctor?			0.47
<6 months	98 (6.6)	29 (8.6)
7–12 months	113 (7.6)	27 (8.0)
1–2 years	129 (8.6)	32 (9.4)
>2 years	1153 (77.1)	250 (73.7)
Don't know	2 (0.1)	1 (0.3)
Number of hospitalizations (in 6 months of 2003)	1.6 ± 1.1	1.6 ± 1.2	0.51
Did you spend any time in an intensive care unit?			0.15
Yes	291 (17.2)	57 (15.0)
No	1301 (79.2)	306 (80.3)
Don't know	51 (3.1)	18 (4.7)
Did you spend any time in the emergency room?			<0.001
Yes	858 (52.3)	236 (61.9)
No	762 (46.4)	135 (35.4)
Don't know	22 (1.3)	10 (2.6)
When released, did you go home or somewhere else?			0.87
Home	1362 (82.9)	313 (82.6)
Somewhere else	280 (17.1)	66 (17.4)
Length of stay (days)			0.08
1–2	317 (19.3)	64 (16.8)
3–5	845 (51.4)	222 (58.3)
6–10	375 (22.8)	79 (20.7)
11–15	66 (4.0)	7 (1.8)
16+	41 (2.5)	9 (2.4)
Type of location			0.12
Urban	1407 (85.6)	314 (82.4)
Non-urban	237 (14.4)	67 (17.6)
Teaching status			0.044
Major	571 (35.6)	109 (29.7)
Minor	463 (28.9)	105 (28.6)
Non-teaching	568 (35.5)	153 (41.7)
Ownership			0.65
Non-profit	1351 (84.3)	306 (83.4)
Church operated	251 (15.7)	61 (16.6)
Bed size			0.01
1–100	178 (11.1)	41 (11.2)
101–200	382 (23.8)	82 (22.3)
201–300	342 (21.3)	100 (27.2)
301+	700 (43.7)	144 (39.2)
Charlson score (mean ± SD, n= 788)	1.34 ± 1.69	1.04 ± 1.47	0.048
How much can you rely on your own knowledge and alertness to protect yourself from medical errors?			<0.001
A lot/some	1416 (87.0)	271 (71.9)
A little/not at all	212 (13.0)	106 (28.1)

Patient participation in care

Nearly all (99.9%) patients reported at least one participatory activity during the hospitalization, as shown in Table 2. On average, patients reported 5.5 (±1.3) of the seven forms of participation during the hospitalization; 81.2% of respondents reported five or more (‘high participation’ group). However, some forms of participation were more prevalent than others. For example, 86.0% of respondents usually or always felt well enough to talk with their doctors or nurses and 79.1% had a visitor who usually or always made sure their wishes were followed. In contrast, only 39.4% of respondents usually or always checked to make sure they received the correct medications.

Characteristics of participatory patients

Table 1 presents the demographic and clinical characteristics of patients with high (5–7) vs. low (0–4) participation activities. The more participatory patients were slightly older than those who participated less, and included a higher proportion of men and a smaller percent of those who characterized their overall health as fair or poor. Patients reporting more participation were less likely to be admitted via the emergency department, were more likely to have been treated in major teaching hospitals and were more likely to express confidence in their ability to protect themselves from medical errors.

Participation and perceptions of hospital quality

Overall, 89.1% of patients in the high participation group rated the quality of care very good or excellent compared with 57.8% of the low participation group (P< 0.001), and there was a direct and consistent relationship between patients' specific participation activities and their rating of the quality of care (Table 3). In multivariable analyses, patients who endorsed each participation activity except checking one's medications were more likely to describe the overall quality of care as ‘very good’ or ‘excellent’. In fact, the odds of the patient endorsing very good or excellent quality care was more than 5-fold higher (adjusted OR: 5.46, 95% CI: 4.15–7.19) for patients with high compared with those with low participation.

Table 3.

Relationship between participation activities and patient-reported overall quality (n= 2025)

Participation activity	Very good or excellent quality, n (%)	Bivariate OR (95% CI)	Multivariable OR (95% CI)
Knew about the problem prompting admission (a lot/some)	1428/1673 (85.4)	2.14 (1.63–2.81)	2.05 (1.53–2.74)
Felt well enough to talk with doctor/nurse (always/usually)	1501/1740 (86.3)	3.43 (2.60–4.54)	2.99 (2.22–4.01)
Found a doctor/nurse to tell you what you wanted to know (very/somewhat easy)	1562/1756 (89.0)	9.57 (7.21–12.70)	8.28 (6.17–11.11)
Doctor/nurse described good/bad things about treatment options (always/usually)	1449/1626 (89.1)	5.64 (4.38–7.27)	6.03 (4.58–7.93)
Participated in decisions (more than wanted/about the right amount)	1601/1844 (86.8)	7.62 (5.51–10.53)	6.04 (4.30–8.48)
Visitor made sure wishes were followed (always/usually)	1350/1600 (84.4)	1.44 (1.10–1.89)	1.34 (1.01–1.79)
Checked your medicines (checked usually/always)	654/795 (82.3)	0.89 (0.70–1.13)	0.96 (0.74–1.24)
Participation (high)	1465/1644 (89.1)	5.98 (4.63–7.73)	5.46 (4.15–7.19)

Multivariable models were run individually, each controlling for age, gender, self-rated overall health, teaching status of the hospital and self-efficacy.

OR, odds ratio.

Participation and adverse events

Among the 788 surveyed patients who also agreed to medical record review, 163 patients experienced at least one significant, serious or life-threatening adverse event during or as a result of the hospitalization. Of these 163 patients, 108 were identified by patient survey only, 32 on chart review only and 23 by both methods. Fifty-six patients had at least one serious or life-threatening adverse event and 69 had at least one preventable adverse event.

In multivariable logistic regression analyses, the presence of at least one adverse event was less likely among patients who felt well enough to talk with their doctors or nurses, who said they could find a doctor or nurse when they wanted information, whose doctors and nurses described the pros and cons of their treatment options or who participated in decisions (Table 4). Patients in the high participation group were less likely to have at least one adverse event during the admission (0.49, 0.31–0.78). Many of these associations persisted in analyses examining the relationship between participation and the occurrence of preventable adverse events (i.e. medical errors).

Table 4.

Relationship between participation and adverse events reported by patient or identified on chart review (n= 788)

Participation activity	Patients with at least one adverse event (n= 163), multivariable OR (95% CI)	Patients with at least one serious adverse event (n= 56), multivariable OR (95% CI)	Patients with at least one preventable adverse event (n= 69), multivariable OR (95% CI)
Knew about the problem prompting admission (a lot/some)	1.15 (0.67–1.98)	1.18 (0.48–2.87)	1.16 (0.53–2.54)
Felt well enough to talk with doctor/nurse (always/usually)	0.33 (0.20–0.53)	0.57 (0.26–1.25)	0.38 (0.20–0.71)
Found a doctor/nurse to tell you what you wanted to know (very/somewhat easy)	0.50 (0.30–0.82)	0.73 (0.33–1.63)	0.44 (0.23–0.84)
Doctor/nurse described good/bad things about treatment options (always/usually)	0.60 (0.39–0.94)	1.01 (0.48–2.16)	0.74 (0.39–1.37)
Participated in decisions (more than wanted/About the right amount)	0.36 (0.20–0.65)	0.43 (0.18–1.02)	0.38 (0.18–0.82)
Visitor made sure wishes were followed (always/usually)	1.15 (0.74–1.79)	1.94 (0.85–4.44)	1.39 (0.71–2.69)
Checked your medicines (checked usually/always)	0.95 (0.65–1.40)	0.81 (0.44–1.49)	1.47 (0.85–2.52)
Participation (high)	0.49 (0.31–0.78)	0.96 (0.43–2.15)	0.63 (0.33–1.19)

Multivariable models were run individually, each controlling for age, gender, self-rated overall health, Charlson score, teaching status of the hospital and self-efficacy.

OR, odds ratio.

Discussion

In this study of patients treated in US acute care hospitals, virtually all respondents participated in their care during hospitalization in ways that might promote patient safety. Patients reported that they were aware of the reason for their admission, felt well enough to communicate with caregivers, found a clinician who could provide information, discussed the advantages and disadvantages of treatment options, and participated in decisions about their care. Fewer respondents reported the most ‘active’ participation activities, such as checking to make sure that their medications were correct. Those with greater participation were more likely to report high quality of care and, importantly, were less likely to experience adverse events.

The inverse association between participation and adverse events supports the theory that participatory patients observed, identified and communicated potential problems before they resulted in medical injuries. This association was demonstrated consistently for the more passive forms of participation, and persisted for analyses of serious and preventable adverse events. This suggests that safety is a product of the interaction between patients and families on the one hand, and providers and their hospitals on the other. Safety-oriented health-care organizations may create conditions that facilitate patients' access to their providers, ready provision of information and participatory decision-making. Patient participation and involvement is a nuanced concept, encompassing efforts and contributions of patients and their caregivers, their respective views and feelings, and their relationships with one another [33].

More active forms of participation, such as having a visitor advocate for the patient or a patient check his or her own medications, were not protective, perhaps because there were fewer opportunities to perform these functions or to perform them consistently enough to defend against occasional lapses of care [25]. Patients' willingness to perform certain tasks (e.g. asking providers if they washed their hands) depends on the perceived risk, level of confrontation required, patients' self-efficacy and perceived degree of control and effectiveness [10, 34–37]. Active participation measures may be mismatched with types of adverse events that hospitalized patients' experience. Drug checking would be unlikely to prevent procedure-related errors and post-operative infections. Conversely, a patient may more readily identify errors involving drugs they take routinely at home, compared with medications that are newly prescribed in the hospital.

The impact of patient participation on patient safety is complex and methodologically challenging. We believe the dominant effect links patient participation with heightened vigilance and effective communication with the care team, resulting in the identification of adverse events and interception of clinical lapses before adverse events occur. However, the experience of a serious adverse event could decrease a patient's capacity to participate as a consequence of the injury. On the other hand, observation of a medical error or an adverse event could increase patients' vigilance, increasing the probability of identifying adverse events, and potentially confounding the protective effect of participation. The final result will be a combination of these effects.

Our study suggests that patient-oriented patient safety initiatives are feasible among hospital inpatients, since patient participation activities were widespread even among this vulnerable population. A variety of models for promoting patients' self-protective behaviors have been proposed, including educational initiatives, patient-initiated incident reports and patient-activated rapid response teams [38–42]. However, given the beneficial effect of passive forms of participation, organizations should consider focusing their efforts on simple measures such as making it easier for patients to find a doctor or nurse, encouraging staff to address patients' questions and facilitating participatory decision-making.

This study has several limitations. First, patients who died during their hospitalization or before the surveys were administered were not included. If these patients were less able to participate in their care, our results overestimate the extent of participation among inpatients. Second, older and sicker patients may be less likely to report adverse events or to participate in their care. If these patients experience more adverse events, then their under-reporting would result in an underestimate of the observed association between participation and adverse events. Third, less than half of the cohort we surveyed authorized medical record review. It is possible that non-response bias affected ascertainment of adverse events or participation, as we found a small difference in the mean participation level of those who did and did not authorize medical record review (5.56 vs. 5.38, P= 0.002). Fourth, patients were surveyed 6–12 months after discharge, due to administrative delays, which may have contributed to possible recall bias. Social desirability bias may also be present, in which patients assumed that more participation was better, potentially overestimating their level of participation. Fifth, our study did not investigate hospital or provider attributes that may promote patient participation, potentially confounding the relationship between patient participation, quality and safety. Finally, ascertainment of adverse events was limited by the data available by patient survey and in the medical record. Physician reviewers used their best judgment about the plausibility of the evidence in making these judgments, but patient reports may be inaccurate and chart review methods likely underestimate the prevalence of adverse events. Nevertheless, we used standard methods for making these judgments.

In summary, our study provides evidence of the capacity of hospital inpatients to participate in their care in a variety of ways that are associated with enhanced perceptions of quality and reduced adverse events. Additional research is required to understand what interventions offer the most promise for encouraging participation and preventing adverse events. Organizations cannot shift the burden of responsibility for safe care to patients and families [41], but they can create an environment where patients and their families serve as an additional and often unaccounted source of strength and resilience.

Funding

This work was supported by a Cooperative Agreement from the US Agency for Healthcare Research and Quality to the Massachusetts Department of Public Health [U18 HS11928]; and the Center for Patient Safety, Dana-Farber Cancer Institute.