Patient‐Centered Insights: Using Health Care Complaints to Reveal Hot Spots and Blind Spots in Quality and Safety

Abstract

Policy Points:

• Health care complaints contain valuable data on quality and safety; however, there is no reliable method of analysis to unlock their potential.

• We demonstrate a method to analyze health care complaints that provides reliable insights on hot spots (where harm and near misses occur) and blind spots (before admissions, after discharge, systemic and low‐level problems, and errors of omission).

• Systematic analysis of health care complaints can improve quality and safety by providing patient‐centered insights that localize issues and shed light on difficult‐to‐monitor problems.

Context

The use of health care complaints to improve quality and safety has been limited by a lack of reliable analysis tools and uncertainty about the insights that can be obtained. The Healthcare Complaints Analysis Tool, which we developed, was used to analyze a benchmark national data set, conceptualize a systematic analysis, and identify the added value of complaint data.

Methods

We analyzed 1,110 health care complaints from across England. “Hot spots” were identified by mapping reported harm and near misses onto stages of care and underlying problems. “Blind spots” concerning difficult‐to‐monitor aspects of care were analyzed by examining access and discharge problems, systemic problems, and errors of omission.

Findings

The tool showed moderate to excellent reliability. There were 1.87 problems per complaint (32% clinical, 32% relationships, and 34% management). Twenty‐three percent of problems entailed major or catastrophic harm, with significant regional variation (17%‐31%). Hot spots of serious harm were safety problems during examination, quality problems on the ward, and institutional problems during admission and discharge. Near misses occurred at all stages of care, with patients and family members often being involved in error detection and recovery. Complaints shed light on 3 blind spots: (1) problems arising when entering and exiting the health care system; (2) systemic failures pertaining to multiple distributed and often low‐level problems; and (3) errors of omission, especially failure to acknowledge and listen to patients raising concerns.

Conclusions

The analysis of health care complaints reveals valuable and uniquely patient‐centered insights on quality and safety. Hot spots of harm and near misses provide an alternative data source on adverse events and critical incidents. Analysis of entry‐exit, systemic, and omission problems provides insight on blind spots that may otherwise be difficult to monitor. Benchmark data and analysis scripts are downloadable as supplementary files.

Formal health care complaints contain data that should be able to improve quality and safety.1, 2 Accordingly, there have been calls to integrate complaint analysis into service monitoring and learning.3 However, progress has been slow due to the lack of a reliable method for codification and analysis.4

Our study uses a reliable complaints analysis tool to provide a benchmark analysis of a national sample of complaints. We introduce an analytic process that unlocks the potential of complaints by revealing insights on unsafe (hot spot) and difficult‐to‐monitor (blind spot) areas of health care provision.

Health Care Complaints: A Resource for Improving Quality and Safety

A health care complaint is a formal communication reporting a failure in service provision that seeks an institutional response (eg, explanation, investigation, apology, change of procedure). Complainants include patients, their family and friends, and concerned professionals. Internationally and across services, there are between 1 and 9 health care complaints per 1,000 admissions.2, 5, 6, 7 Complaints occur when a threshold of dissatisfaction has been breached,8 with dominant motivations being to correct an ongoing problem or prevent recurrence.9, 10, 11 Complainants purport to have valid information that the institution does not know or has failed to take sufficiently seriously. People often refrain from complaining because it is effortful, time‐consuming, and sometimes perceived as futile.12, 13 Therefore, complaints may overrepresent the concerns of motivated individuals and events that complainants believe are being ignored.

Research on health care complaints has focused on complaint handling,14 complaint resolution,15 physician behavior,16 high‐risk clinicians,17 malpractice claims,18 and emotional impact on clinicians.19 Yet, research increasingly suggests that patient experience reflects the quality of care.20 Complaints, for example, are usually upheld,21 often contain adverse‐event information,22 have been identified as early warning signals in post‐hoc investigations,23 and are associated with surgical complications,2, 24 adverse events,25 and physician malpractice.26 The debate, we suggest, is no longer whether complaints contain useful information but how valid insights can be reliably extracted.27

Health care complaints can provide an independent check on quality and safety monitoring methodologies that rely on staff self‐report. Although reporting adverse events,28 recording near misses,29 measuring safety culture,30 and evaluating interventions31 offer useful data, inconsistent reporting means these data are often incomplete. Data omissions are attributable to incidents not being perceived or being perceived differently, reporting criteria not being understood, and staff being unwilling to report because of time constraints, poor systems, or concerns over professional consequences.31, 32, 33, 34, 35 Moreover, it is somewhat paradoxical to rely on staff embedded within a culture to self‐report on that culture because the culture itself shapes the practices of reporting.36, 37

Health care complaints contain data that are difficult to obtain from other sources. They report on the complete patient experience (from before admission until after discharge), covering both micro and macro issues, and focusing attention on issues proximal to patients. Specifically, patients have insight on issues such as continuity of care problems,38 low‐level systemic problems,39 and unfinished or omitted care.40 In support of this view, research has found that complaints provide unique data that are additional to patient notes32, 41 and incident‐reporting systems.22

In summary, health care complaints potentially provide valid, independent, and distinctive data for improving quality and safety. However, utilizing complaint data has been hampered by uncertainty in how to extract actionable insights.

Codifying and Aggregating Health Care Complaints

Central to extracting actionable insights from health care complaints is their reliable codification and aggregation. While health care organizations have rigorous procedures for responding to individual complaints, their broader potential remains untapped due to limitations in methodologies for codifying, aggregating, and distilling key learning points.3

A systematic review of 59 studies that analyzed health care complaints identified the following limitations4: no standardization of codes; minimal inter‐rater reliability; absence of theoretical frameworks to guide coding; conflation of problems (eg, poor hygiene, lost patient notes) and stages of care (eg, admission, diagnosis); little assessment of problem severity or consequences (eg, harm as used in other incident‐reporting systems); focus on frequency counting (problematic given barriers to making complaints12); and no theoretically informed analysis procedures to derive patient‐centered insights.

The Healthcare Complaint Analysis Tool (HCAT) was developed to overcome these limitations and reliably distinguish and codify problem type, problem severity (eg, small delay in receiving pain killers versus misdiagnosis of cancer), harm caused, stage of care where each problem occurred (eg, admission, examination), and the staff groups implicated.42 The HCAT framework has been successfully applied in Italy,43 Australia,44 the Netherlands,45 and the United States.46, 47

Despite these advances, there remains little guidance on the analysis of aggregated health care complaints data and how to obtain actionable insights. Furthermore, unlike adverse events, there are no standardized national benchmark data on the “typical” incidence of health care complaints in terms of reported problems, severity, stages of care involved, staff groups concerned, and consequences. Our research addresses these limitations by codifying, aggregating, and generating insights on a benchmark data set.

Generating Insights From Health Care Complaints

Drawing on human factors, patient safety, and social and organizational psychology research, we propose that health care complaints can be analyzed to reveal hot spots and blind spots in health care systems.

Using Health Care Complaints to Identify Hot Spots

Hot spots refer to the data‐driven mapping of risk onto places, systems, and processes. In health care, hot spots have been used to identify populations,48 communities,49 and individuals50 at risk. Focusing on the interaction between organizational structure and patient safety, we proposed 2 ways of mapping risks evident in health care complaints onto stages of care.

First, drawing on the adverse‐event literature, health care complaints can be analyzed to reveal “harm hot spots.” An adverse event is any unfavorable and unintended sign, symptom, or disease arising during a health care episode. Adverse events are typically captured through labor‐intensive retrospective case reviews, whereby patient case files are randomly selected and analyzed to detect unintended harm. Around 9.2% of cases report harm; analyses examine causes and where harm occurred.51 Unlike adverse events, health care complaints provide limited clinical data; however, they do provide insight on clinician behaviors (which retrospective case reviews do not) and the patient journey. Accordingly, we will use complaints to analyze hot spots of reported harm and where it occurs.

Second, drawing on the critical‐incident literature, health care complaints can be analyzed to reveal “near‐miss hot spots.” Critical‐incident databases are built through staff reporting on adverse events and near misses, with the goal of identifying opportunities for improvement.34 Near misses are problems that did not cause harm (because of error recovery52 or luck53), but had potential to. Identifying near misses enables organizations to preempt future problems34 and better understand error recovery.54 Interviews with patients after discharge from an emergency department demonstrated that patients can identify near misses.55 Accordingly, we will use complaints to analyze patient‐reported near‐miss hot spots along with associated problems and the stages of care in which they occurred.

In summary, our study examines whether health care complaints can reveal patient‐centered hot spots of harm and near misses. This will create an evidence base for using complaints to improve quality and safety by establishing an additional data source for identifying, quantifying, and localizing instances of patient harm; determining opportunities for improving health care by identifying developing risks; and focusing attention on improvements that will most directly enhance patient experiences. Such evidence may provide a counterpoint to the reliance on and limitations of case reviews, which are costly and provide incomplete behavioral data,56 and incident reporting, which relies on a good safety culture with enthusiastic, time‐rich staff.57

Using Complaints to Provide Insight on Institutional Blind Spots

A blind spot is a domain of individual or organizational functioning that is either unobservable or incorrectly observed. A characteristic of blind spots is that the individual or organization is unaware of the blind spot, and thus it is most visible from an external point of view. The concept has been used in social relations research based on the idea that people often have blind spots in relation to themselves.58 In health care organizations, blind spots arise from a lack of data or motivated dismissing of data.59 In terms of lack of data, complaint data can reveal issues that are not captured by adverse‐event reports, surveys, staff‐reported incidents, and quality inspections.22, 32, 41 In terms of motivated dismissing, behavioral commitment to a course of action or policy can make objective appraisal difficult,37, 59, 60 and thus, the external point of view provided by complaints can be valuable.

First, there is a blind spot for events that occur outside the institution, either before admission or after discharge.61, 62 For example, a case review study found that nearly 8% of adverse incidents related to discharge, none of which were picked up by incident reporting.33 Health care complaints may address this blind spot because patients can report on events occurring before and after staff are involved.

Second, there is a blind spot for problems that are systemic across the patient journey. Serious medical errors often involve a culmination of multiple problems (eg, misdiagnosis and communication failure) spread across stages (eg, examination and discharge); these issues are difficult to capture without the vantage point of multiple agents.62 A study on incident reporting in anesthesia found a tendency to capture isolated incidents (eg, medication errors) but not cascades of problems leading to incidents.63 Complaints may address this limitation because they often narrate the entire sequence of events as the patient moved through the health care system.

Third, there is a blind spot for errors of omission. An error of omission is an action that is not performed, whereas an error of commission is an action that is performed incorrectly.64 Errors of omission are widespread in health care,40, 65 and estimates of preventable harm would increase dramatically if errors of omission could be assessed reliably.64 Detecting errors of omission is difficult, because people rarely observe or take responsibility for what has not happened.66 Moreover, if the omission was deliberate, it is unlikely to be self‐reported. Health care complaints may provide data on omissions because patients usually experience their consequences.

In summary, we examine whether health care complaints can provide data on potential blind spots in health care provision. This will provide patient‐centered insights for managers and policymakers on aspects of quality and safety that are otherwise difficult to monitor.

Research Overview

We report a national sample of complaints received by England's National Health Service (NHS) hospitals and community health services (excluding general practices). Our study aims to provide benchmark data and descriptive statistics on the national sample; examine whether these complaints can reveal hot spots of reported harm and near misses; and examine whether these complaints can reveal blind spots in entry‐exit, systemic, and omission problems. Our overarching objective is to identify the potential added value a systematic analysis of health care complaints can provide for improving quality and safety.

Methods

Data Collection

Hospital and community health services in England are provided by 251 NHS health care conglomerates, or trusts. Between April 1, 2011, and March 31, 2012, these organizations received 107,259 complaints.67 We obtained 7% of the complaints received by 56 randomly sampled trusts (including university teaching hospitals, acute hospitals, children's hospitals, community hospitals, mental health trusts, and ambulance services) using the UK Freedom of Information Act. Local institutional ethical approval was obtained. Personally identifying information was redacted. To mitigate seasonal effects, each trust provided complaints from a randomized start date. This yielded 1,110 complaints, approximately 1% of all complaints received during the study period. The sample reflects the population of English NHS trust complaints with a 2.93% margin of error at a 95% confidence level. Secondary data were obtained from NHS Digital to calculate the number of complaints per 1,000 admissions and geographic region for each trust in the sample.

Coding Framework

Data were codified using HCAT.42 This tool is based on a systematic review4 that resulted in a taxonomy of problem types that has been successfully used in research.43, 44, 45, 46, 47 The taxonomy was iteratively refined using discriminant content validity and reliability testing to develop a comprehensive, theoretically informed, reliable, and free‐to‐use tool for coding health care complaints.68

HCAT records 3 complaint‐level variables: who made the complaint (patient, family member, or unspecified/other); patient's sex (female, male, or unspecified/other); and overall harm. Harm is assessed using the NHS's risk matrix, a 6‐point scale from no harm to catastrophic harm (Table 1).69 Harm is an absolute indicator of the overall negative consequences of 1 or more problems reported in a complaint.

Table 1.

Coding Harm

Level of Harm	Definition
None	No harm or no harm mentioned
Minimal	Minimal intervention or treatment required (eg, from a bruise or graze)
Minor	Minor intervention required to ameliorate harm (eg, from a sprain, anxiety)
Moderate	Significant intervention required to ameliorate harm (eg, from a grade 2‐3 pressure ulcer, health care–acquired infection)
Major	Patient experienced, or faces, long‐term incapacity (eg, from a dislocation, fracture, hemolytic transfusion, wrong medication side effect, post‐traumatic stress)
Catastrophic	Death or multiple/permanent injuries (eg, wrong‐site surgery, paralysis, permanent or chronic mental health problems)

HCAT hierarchically conceptualizes each problem reported in complaints using 7 problem types nested in 3 domains (Figure 1). Each problem has a definition, keywords, and severity indicators. Severity indicators were generated from the distribution of severity in each problem type;42 for example, lack of hydration can be for a few hours (low severity) or days (high severity). Severity is independent of outcome (harm). Therefore, a patient given the wrong test results is deemed high severity, even if the error was corrected and no harm resulted. Near misses were determined by identifying high‐severity problems associated with low harm. This is possible because harm relates to the overall outcome for the patient, and severity relates to the egregiousness of a given problem independent of harm.

Figure 1.

Hierarchical Coding Framework of Domains, Problems, and Severity Indicators

In addition to a severity score, each reported problem within a complaint is associated with 1 or more stages of care and staff groups. HCAT uses the stages of care coded in adverse‐event reports70: admission; examination and diagnosis; care on the ward; operation and procedures; discharge and transfers; and other. Staff groups are recorded using a high‐level distinction between administration, nursing (including midwives, physiotherapists, and support workers), medical (including all consultants and anesthetists), and other.

Coding Reliability

The 1,110 complaints were analyzed by 2 trained coders (master's‐level psychology graduates who completed a 5‐hour training course). Each coder worked with a separate batch of complaints, as anonymized PDF files, taking, on average, 9 minutes per complaint. Reliability was investigated using 274 complaints coded by both coders. Complaint order was randomized and the coders were blind to the complaints used for reliability. Intraclass correlation coefficients (ICCs) for category, domain, and overall scores were calculated. The guidelines used to assess coding reliability were as follows: .01‐.20 = poor/slight; .21‐.40 = fair; .41‐.60 = moderate; .61‐.80 = substantial; and .81‐1.00 = excellent.

The ICCs (single measures) for the HCAT problem categories and associated severity levels indicated substantial reliability for environment (ICC: .684, 95% confidence interval [CI]: .616‐.742), safety (ICC: .670, 95% CI: .599‐.730), institutional processes (ICC: .666, 95% CI: .595‐.727), and communication (ICC: .642, 95% CI: .567‐.707). Moderate reliability was found for respect and rights (ICC: .569, 95% CI: .483‐.644), quality (ICC: .457, 95% CI: .358‐.545), and listening (ICC: .418; 95% CI: .315‐.511). Harm showed excellent reliability (ICC: .863, 95% CI: .771‐.920).

Process of Analysis

First, we produced a descriptive analysis of complaints received by the NHS by calculating descriptive statistics for all HCAT data points.

Second, hot spots were analyzed by mapping harm and near misses onto stages of care. Near misses were operationalized as high‐severity problems associated with no more than minor harm. These hot spots are presented using mosaic plots that provide an area‐proportional representation of the interaction between 2 categorical variables.71 The size of each tile is determined by the observed frequencies (specified in text on the larger tiles), thus providing a visualization of a contingency table.

The interaction between the variables was tested using Pearson's chi‐square test of independence, a nonparametric test of association between categorical variables that makes no assumption about the distribution of the data, the residuals of which can pinpoint interacting categorizations that have more or fewer observations than expected. These residuals are represented in the mosaic plots: tile borders correspond to more (solid line) or fewer (dotted line) observations than expected. Tile shading corresponds to the absolute value of the Pearson chi‐square residual, comparing the observed distribution to the null hypothesis of no association. Cells with residuals less than 2 are white, cells with residuals between 2 and 4 have light shading, and cells with residuals above 4 have dark shading. Under the assumption of independence, residuals of 2 and 4 correspond approximately to 2‐tailed probabilities of .05 and .0001, respectively.72

Third, blind spots were analyzed by focusing on data subsets. To examine the entry‐exit blind spot, we identified the subset of problems pertaining to admission and discharge. The systemic problems blind spot was examined by identifying the subset of complaints reporting multiple problems. The omissions blind spot was examined by identifying the subset of problems pertaining to omissions (quality, communication, and listening problems). Mosaic plots and chi‐square tests were used to visualize and test relationships.

Qualitative analysis of the hot spots and blind spots was guided by the quantitative findings. The textual content of complaints concerning each hot spot and blind spot was extracted and reread. Thematic analysis was used to identify types of instances,73 and abductive analysis was used to make interpretative leaps where the data seemed surprising in light of expectations framed by relevant literature.74

Quantitative analyses and figures were produced using R software,75 with the vcd package for categorical data.71 Qualitative analyses were performed using NVivo 11. Coded data and R scripts are downloadable as supporting files (see Online Appendices).

Findings

Descriptive Benchmarks for Health Care Complaints in England

We obtained 1,110 complaints from 56 NHS trusts in England that received 4.69 (standard deviation [SD] = 2.35) complaints per 1,000 admissions. Our sample had 19.82 (SD = 12.58) complaints per trust and contained 1,132,245 words (mean: 1,020, SD = 1,105). The geographic distribution was as follows: midlands (including east, 40%, n = 444), south (including London, 32%, n = 357), north (28%, n = 309).

Complainants were categorized as follows: patients (50%, n = 554), family members (36%, n = 402), and others (13%, n = 149; mainly members of parliament, solicitors, professional services, and laypeople). Patient gender was distributed as follows: unidentifiable due to redaction (40%, n = 443), female (37%, n = 410), and male (23%, n = 257). The harm was coded as none (37%, n = 409), minimal (4%, n = 48), minor (22%, n = 248), moderate (14%, n = 152), major (15%, n = 163), and catastrophic (8%, n = 90). Most catastrophic harm was death (6%, n = 62), compared to physical injuries (2%, n = 27; including brain damage, blinding, amputation, and paralysis) and psychological impact (n = 1; patient becoming suicidal).

The 1,110 complaints reported 2,074 problems, with a mean of 1.87 problems per complaint (SD = 1.03). Problems were categorized as follows: institutional processes (25%, n = 523), safety (16%, n = 340), quality (16%, n = 328), respect and rights (13%, n = 274), communication (12%, n = 259), environment (9%, n = 192), listening (6%, n = 133), and other (1%, n = 25). There was a relatively even split when problems were aggregated into the clinical (32%, n = 668), relationship (32%, n = 666), and management (34%, n = 715) domains. Problem severity was distributed as follows: low (27%, n = 558), medium (50%, n = 1,030), and high (23%, n = 486).

The stages of care where problems occurred were admission (17%, n = 353), examination (21%, n = 443), care on the ward (22%, n = 465), procedures and operations (11%, n = 227), discharge and transfer (15%, n = 310), multiple stages (4%, n = 75), other (7%, n = 135), and unreported (3%, n = 66). Associated staff groups were medical (39%, n = 812), nursing (22%, n = 461), administration (11%, n = 232), multiple (6%, n = 130), other (16%, n = 341; midwives, physiotherapists, support workers, care assistants), and unreported (5%, n = 98).

Spearman's Rho tests found moderate associations between harm and clinical severity (r_s = .398, p < .001), management severity (r_s = .29, p < .001), and relationship severity (r_s = .259, p < .001). Despite these associations, 175 high‐severity problems resulted in only minor, minimal, or no harm, indicating potential near misses.

No regional differences were found for complaint frequency (F[2, 50] = 0.197, p = .822), patient gender (chi‐square [χ2] = 1.369, df = 2, p = .504), complainant (χ² = 4.657, df = 4, p = .324), or problem reported (χ² = 23.313, df = 14, p = .055). However, only 17% of complaints in the south reported major or catastrophic harm, compared to 22% in the midlands and 31% in the north (χ² = 32.527, df = 10, p < .001).

Hot Spots Reported in Health Care Complaints

Harm Hot Spots

Figure 2 presents the association between harm and stage of care across all 2,074 problems. Although all levels of harm occurred at all stages, there were statistically significant patterns (χ² = 173.47, df = 30, p < .001). The Pearson residuals from the chi‐square test are indicated in Figure 2 by shading (ie, revealing tiles that have more or fewer observations than expected by the null hypothesis of no association). The shading reveals a U relationship, with most harm in the intermediate stages. Admission was associated with fewer reports of moderate, major, and catastrophic harm and more reports of no harm (mainly delays in accessing services). Major and catastrophic harm was associated with examination (mainly misdiagnoses and omitted tests); procedures (mainly errors and poor communication about the procedure); and, especially, care on the ward (mainly patient monitoring and infection control), which accounted for 36% (n = 72) of problems reported in complaints with catastrophic harm.

Figure 2.

Association Between Harm and Stages of Care^a 

^aThe area of tiles is proportional to the number of problems within the cross categorization. Tile text reports the number of problems as a percent of all problems. Text did not fit in some small tiles. Tile border indicates more (solid) and fewer (dotted) observations than expected. Tile shading indicates chi‐square test residuals less than 2 (white, approximating p > .05), between 2 and 4 (light shading, approximating p < .05), and greater than 4 (dark shading, approximating p < .0001). [Color figure can be viewed at http://wileyonlinelibrary.com]

Figure 3 presents the subset of 489 problems associated with major and catastrophic harm, by problem type. Major and catastrophic harm was reported across problem types, with evident patterns (χ² = 216.04, df = 24, p < .001). Examination and care on the ward accounted for 60% of reported major and catastrophic harm. The most salient hot spots were safety problems during examination (16%; mainly misdiagnoses, but also tests not requested, unhygienic examination, and ignoring patient files) and quality problems on the ward (11%; inadequate patient monitoring, pain management, poor hygiene, dehydration, rough handling, and general neglect).

Figure 3.

Association Between Problem Type and Stages of Care for Problems With Major and Catastrophic Harm^a 

^aThe area of tiles is proportional to the number of problems within the cross categorization. Tile text reports the number of problems as a percent of all problems. Text did not fit in some small tiles. Tile border indicates more (solid) and fewer (dotted) observations than expected. Tile shading indicates chi‐square test residuals less than 2 (white, approximating p > .05), between 2 and 4 (light shading, approximating p < .05), and greater than 4 (dark shading, approximating p < .0001). [Color figure can be viewed at http://wileyonlinelibrary.com]

Listening problems were overrepresented during examination (4%; staff ignoring symptoms, concerns, requests for tests, and previous patient experiences). Communication problems were overrepresented during procedures (4%; inconsistent communication about operation, incorrect consent papers, patients uninformed about side effects or the rationale for procedures).

Institutional problems were associated with major and catastrophic harm during admission (7%; delayed ambulances, admissions, and procedures) and discharge (4%; incorrect discharge letters, lost patient files, care plans not passed on). These problems were rarely the proximal cause of harm, tending instead to compound other problems.

In summary, health care complaints can detect hot spots of reported harm. Although clinical problems dominated in reports of major and catastrophic harm, relationship and management problems were also important, often contributing to a harmful cascade of problems.

Near‐Miss Hot Spots

Near‐miss hot spots were examined using complaints that reported high‐severity problems, but minor, minimal, or no harm. Figure 4 presents the subset of 142 problems that met these 2 criteria and were not in other or multiple stages (no chi‐square test was conducted due to small expected frequencies).

Figure 4.

Association Between Problem Type and Stages of Care for Near Misses^a

^aThe area of tiles is proportional to the number of problems within the cross categorization. Tile text reports the number of problems as a percent of all problems. Text did not fit in some small tiles. Tile border indicates more (solid) and fewer (dotted) observations than expected. Tile shading indicates chi‐square test residuals larger than 2, approximating .05 significance. [Color figure can be viewed at http://wileyonlinelibrary.com]

The 14 high‐severity, low‐harm institutional process problems that occurred during admission included waiting more than 2 hours for an ambulance; delayed admission for a detached retina, vomiting blood, bleeding following surgery, stroke, and hernia; a patient admitted with the wrong file; and distressed stroke, Huntington's, and suicidal patients unable to access help.

The 13 high‐severity, low‐harm safety problems that occurred during examination were mainly missed diagnoses (lung cancer, lymphoma, burst appendix, aortic stenosis, infection, gallstones, meniscal tear) and misdiagnosis (hernia, poisoning, stage 3 ovarian cancer). In all cases, no more than minor harm resulted because the error was detected by a staff member or the patient (often through insistence or seeking a second opinion).

The 12 high‐severity, low‐harm environment problems that occurred on the ward included lack of cleanliness (blood stains, feces, and vomit); bathroom facilities or beds unavailable; lack of clean sheets or towels; and verbal aggression between patients. Some complainants reported family members cleaning and providing basic provisions.

Across these near misses there were instances of error recovery. Health care staff were reported correcting their own or colleagues’ diagnoses, resolving admission issues, and following up on issues. Patients, and sometimes their family and friends, also contributed to error recovery by bringing problems to the attention of staff, seeking second opinions, probing multiple routes to admission, and directly correcting hygiene issues.

In summary, health care complaints can detect hot spots of reported near misses. Analysis of the near‐miss hot spots revealed staff, patients, and family members engaging in error recovery.

Blind Spots Reported in Health Care Complaints

Entry‐Exit Problems Blind Spot

Nearly one‐third (32%, n = 663) of the 2,074 problems in our sample occurred on the boundaries of the health care institutions, either on entry (admission, 17%) or exit (discharge and transfer, 15%). Thus, although problems during admission were associated with less harm (see Figure 2), these problems were frequent.

Admission problems included denied and canceled care. For denied care, cases with the highest harm reported ambulances not dispatched and urgent home visits denied. However, the harm was generally low, with many problems being related to accessing nonemergency services (mental health, abortion, alcoholism, dementia, autism, cosmetic surgery). In terms of canceled care, the main issue was having appointments canceled (often repeatedly). Reported consequences were mainly short term (preparation for surgery, time off work, travel) but sometimes long term as well (deteriorating health, unemployment, isolation).

Complainants identified potential causes of entry problems, including unclear appointment processes, problematic telephone systems, and staff not returning calls or making agreed bookings. Some complainants reported being removed from waiting lists because they missed appointments impossible to attend (eg, notification arrived on the day of the appointment). One complainant wrote: “We are inundated with letters for appointments and cancellations. […] Someone is juggling with figures in an attempt to meet government targets and it is this that is creating more problems and distress to patients.”

Discharge problems rarely related to transfers (eg, no vehicle available) or being discharged (eg, disoriented and vulnerable patients sent home in taxis); most arose sometime after discharge. Clinical problems included medication (incorrect information, insufficient amount of medicine, and incorrect medication), side effects (medication‐induced personality changes, monitoring devices that prevented sleep, and surgery that made sexual intercourse impossible), and premature discharge (eg, missed diagnosis of broken rib, stitches not removed leading to infection). Some complainants reported readmission to the same unit, another unit, or a private facility.

Complaints also reported procedural failures at discharge, including patients discharged without risk assessment, care packages, follow‐up arrangements, medication information, or family being notified. A common pattern was that patients were promised a follow‐up visit, delivery of medications, or a referral that never occurred; then readmission was difficult because their situation was no longer sufficiently acute for emergency admission.

In summary, health care complaints contain valuable data on the entry‐exit blind spot. Problems during admission were frequent and 120 were associated with major or catastrophic harm. Indeed, research has found that admission delays are associated with longer hospital stays and poorer outcomes across many conditions,76 and discharged patients often have unmet needs.77 Entry‐exit problems are difficult to monitor, but understanding them better may facilitate cost‐efficient service improvements—for example, improving discharge processes may reduce readmissions.78

Systemic Problems Blind Spot

Most complaints (54%, n = 595) reported systemic problems (ie, multiple problems, stages, and/or staff). Systemic problems can be difficult to monitor because they are distributed across institutions as well as between institutions and care episodes. To test whether complaints reporting systemic problems were associated with harm, we used an ordered logistic regression model with harm as the ordinal response and number of problems, stages, and staff groups as predictors. Harm reported in complaints was predicted by the number of problems (B = .300, standard error [SE] = .08, odds ratio [OR] = 1.35, 95% CI: 1.154‐1.58, p < .001) but not by the number of staff (B = .127, SE = .126, OR = 1.135, 95% CI: .887–1.451, p = .313). The number of stages was borderline significant (B = .209, SE = .11, OR = 1.233, 95% CI: .994‐1.531, p = .057). For each additional problem in a complaint, the odds of an increase on the 6‐point harm scale were 1.35. Figure 5 visualizes this linear relationship: complaints reporting 1 problem had less harm than expected, whereas complaints reporting 3 or more problems had more harm than expected (χ² = 57.672, df = 10, p < .001).

Figure 5.

Association Between Harm and Number of Reported Problems^a

^aThe area of tiles is proportional to the number of problems within the cross categorization. Tile text reports the number of problems as a percent of all problems. Text did not fit in some small tiles. Tile border indicates more (solid) and fewer (dotted) observations than expected. Tile shading indicates chi‐square test residuals larger than 2, approximating .05 significance. [Color figure can be viewed at http://wileyonlinelibrary.com]

Complaints often reported a lack of continuity between stages of care and services (eg, delays, no follow‐up, test results not passed on, staff not reading patient file). This sometimes led to unclear lines of responsibility, with patients passed between uninformed staff members. One complainant wrote: “I would be delighted if SOMEONE would read my notes properly and get an idea of what's actually happened in the past and how things are today.” Additionally, sometimes multiple low‐level failures had diffuse but compounding effects. For example, one woman with ovarian cancer was admitted to an unclean ward; she acquired an infection that was not treated properly; she was regularly dehydrated; procedures were canceled; and questions were not answered. Her buzzer was ignored and her worsening symptoms were neglected. Before dying, she requested a formal complaint be lodged.

In summary, health care complaints can provide insight on systemic problems, especially continuity of care62 and low‐level caring neglect.39 Although systemic problems can be difficult to monitor, they are often directly reported within complaints that recount the patients’ trajectory of care. Analyzing systemic problems is important because, according to error theory,79 where such issues go unchecked, they cascade toward increasingly serious consequences.

Omission Problems Blind Spot

The distinction between errors of omission and errors of commission is implicit in the HCAT. Problem types associated with errors of omission were quality (neglect), communication (absent communication), and listening (ignoring). Problem types associated with errors of commission were safety (clinical errors) and respect and patient rights (disrespect and violations). Accordingly, 35% (n = 720) of problems related to omission, 30% (n = 614) related to commission, and 34% (n = 715) were indeterminate.

Examination of quality problems revealed many omissions, including neglecting patient hygiene, absence of hydration and/or nutrition, overlooking allergy bands, not reading patient notes, absent monitoring, not recording events, ignoring signs of decline, missed medication, not emptying catheters, and not responding to monitor alarms. There were a few errors of commission in quality problems, such as incorrect monitoring, mistakes in hand‐over notes, unhygienic dressing of wounds, and ineffective provision of pain relief.

Communication problems were predominantly errors of omission, including no discharge letter, no forewarning about side effects, not informing family about a patient's deteriorating health, no reasons given for transfer, no explanation of procedures, no information about diagnosis before receiving medication, and patients unaware of the treatment plan. Communication problems also included some errors of commission, such as poor and incorrect communication of diagnosis.

All listening problems were errors of omission. Complainants reported that their accounts of accidents, past experiences, and side effects or symptoms were ignored; requests for tests or pain relief were not acted on; their wishes in relation to treatment, medication, discharge, or care plans were overlooked; and they were unable to get urgent help with the call buzzer. One patient reported: “I knew something was wrong with me but no one would listen or investigate further. I was dismissed as neurotic and told that it would be better soon.”

In the context of widespread omissions, the initiative of patients and their advocates appears crucial. Family members reported feeding, hydrating, and washing patients or their bedsheets. They also reported reminding staff about care plans and medical details, struggling to obtain the correct medications, and insisting on clinical tests. One patient, whose multimonth “shambolic treatment” began with a routine operation, wrote: “I have had to constantly chase departments and I feel sure that if left to their own devices I would simply ‘fall between the cracks.’ […] I have had to contact the hospital every single step of the way to move to the next step. I have been pushed from pillar to post.”

A subset of communication and listening problems comprised omissions in acknowledging or responding to a reported problem. Figure 6 shows that communication and listening problems were more likely to be reported as secondary to an initial problem (χ² = 65.85, df = 12, p < .001).

Figure 6.

Association Between Problem Type and the Order That These Problems Are Reported Within Complaints^a 

^aThe area of tiles is proportional to the number of problems within the cross categorization. Tile text reports the number of problems as a percent of all problems. Text did not fit in some small tiles. Tile border indicates more (solid) and fewer (dotted) observations than expected. Tile shading indicates chi‐square test residuals larger than 2, approximating .05 significance. [Color figure can be viewed at http://wileyonlinelibrary.com]

Communication omissions secondary to an initial problem were mainly about the first problem not being acknowledged or communicated. For example, a patient reported removal of a pelvic cyst that resulted in distressing symptoms indicating her bowel had been perforated. The error was not communicated to her; she remained distressed by the side effects, and there was a delay treating the initial error.

Listening omissions secondary to an initial problem were predominantly patients and their family being ignored when trying to bring the first problem to the attention of staff. One complainant reported that her husband was prescribed metolazone despite her telling staff that this medication had previously caused serious side effects. Subsequently, the patient was readmitted to the hospital after bleeding from his bladder. Another patient with Crohn's disease wrote about his surgery to remove a buildup of pus and 12 centimeters of intestine, after which he was given a postoperative epidural. The patient wrote: “I was in unbelievable pain and no one was listening to me or seemed to care. The stomach spasms began a minute apart, jolting my body, interrupting my breathing. […] Once I recognized that the attacks wouldn't stop anytime soon, I began having panic attacks.” In the morning, it became apparent that the epidural had fallen out. In these cases, there was an initial error—not reading patient notes, incorrect epidural insertion—followed by a secondary error—patients and their family ignored when they provided information that could have detected the initial error.

In summary, health care complaints contain rich data on errors of omission. An important subset of communication and listening problems are secondary to an initial problem and relate to staff not acknowledging or dismissing evidence of a problem. Accordingly, health care complaints can contribute to the reported difficulties in obtaining good quality data on errors of omission65, 66 and processes of error detection and correction.36, 53, 54

Discussion

Our study reports the first analysis of a national sample of health care complaints using a reliable tool to produce theoretically informed insights. Eight percent of complaints reported catastrophic harm (equivalent to 9,000 complaints in English NHS trusts reporting catastrophic harm for the sampled year). This is broadly comparable to the percentage of adverse events that report death (7.4%) and permanent disability (7%).51 This benchmark revealed significant regional differences: complaints from the north of England reported nearly twice as much major and catastrophic harm compared to the south. Whether this is due to different thresholds for complaining or provides a new insight on differing north‐south mortality rates should be addressed in future research.80

The hot spot analysis detected hot spots of reported harm and near misses, revealing their prevalence and location in the health care system. Our findings support research on patients speaking up about safety issues81 and reporting near misses not included in medical records.55 A defining feature of a resilient organization is the ability to identify and recover from an error so that it becomes a near miss rather than an accident.34 Understanding near misses better can improve organizations’ capacity to preempt negative events.52

The analysis of the entry‐exit blind spot revealed that 32% of reported problems pertained to stages of admission and discharge. Previous research identified problems around admission and discharge as frequent, difficult to monitor, and something that patients can provide data about.61 Health care complaints can provide a benchmark indication of how often these problems occur, their nature, and their consequences.

The analysis of complaints reporting systemic problems found that the mean number of problems per complaint is 1.87, with considerable variation (SD = 1.03). As the number of reported problems increase, so does the reported harm. This association supports the validity of complaint data and is comparable to the positive association between the number of factors in an incident report and harm.82 Systemic problems can be challenging to monitor due to their distributed nature,39 yet they can indicate high‐risk hot spots and pathways, where relatively small errors compound toward more serious consequences.79

The analysis of errors of omission found them to be widespread. Complainants reported initiating error recovery by drawing the attention of staff to omissions (eg, pointing out that some aspect of care was missing) and directly correcting omissions (eg, cleaning patients or bedsheets). In either case, the institution is unlikely to be aware of the scale of these errors of omission because of compensating corrections.64, 66 Communication and listening omissions—for example, staff not acknowledging a mistake—revealed missed opportunities for error detection and prevention.

Implications for Monitoring and Learning

Health care complaints can be used to identify hot spots and blind spots for unsafe and poor‐quality care, providing new opportunities for patient‐centered monitoring and organizational learning. These analyses should be performed by managers and policymakers who wish to localize patient‐reported harm and near misses; monitor problems difficult to capture through existing methodologies; utilize patients’ spontaneous efforts to improve care quality; and develop interventions that directly address patients’ concerns.

We suggest a 2‐step mixed‐methods approach to move from individual complaints to actionable insights for organizational learning.83 The first step is quantitative monitoring of complaint frequency, hot spots (harm and near misses), and blind spots (entry‐exit, systemic, and omissions). Comparisons with benchmark data (historical or comparable institutions) identify trends that trigger the second step of analysis.7 The second step entails triangulation with other data sources (eg, incident reports, surveys, soft data),84 both to ensure representativeness and to identify discordant and potentially revealing discrepancies.85 Integrating these data sources provides the rationale for further in‐depth qualitative analysis of selected issues with high learning potential, ideally entailing a root‐cause analysis.

Health care complaints can be used for benchmarking. Benchmarking aims for continuous quality improvement by measuring and comparing institutional units and learning about the factors underpinning best practice.86 Broad participation in a benchmarking program would enable comparisons between organizations, regions, and countries, thus improving our understanding of variation between units, regions, and even entire health care systems. It could also provide a new way to monitor interventions and evidence good practice. To mitigate the unintended consequences of benchmarking, such as the reification of numbers31 and potential gaming,87 a collaborative approach should be cultivated, with analyses shared in the spirit of continuous quality improvement.86

Any use of complaints for monitoring, benchmarking, or learning needs to focus on the strengths of complaint data. Although patients have only partial access to the clinical details of their case, they can report what happened before admission, after discharge, and during each health care encounter along their trajectory. Indeed, the more stages and staff groups a patient trajectory entails, the more likely that the patient is the only person who was present for each encounter. Accordingly, health care complaints are particularly suited to providing insight into continuity of care issues.

Implications for Patient Involvement

Our findings reveal new potential for patient involvement in health care improvement. Patients are increasingly involved in identifying deficiencies in service delivery,88 discussing safety issues,81 and performing safety activities.89 However, all these interventions entail soliciting patient input; our analysis reveals that patients (and sometimes their family and friends) have already been trying to involve themselves in the delivery of safe care.

First, patients and family members are involving themselves when they submit a formal complaint. Rather than being random patients with an average (ie, unproblematic) experience, complainants are a subset of people who believe they have important observations. Arguably, there has been too much focus on data collection in health services and an insufficient focus on analyzing existing data.90 Complaints are a case in point: they provide a free, voluminous, continuous, and detailed data source that is widely recognized as valuable but insufficiently analyzed or theorized; turning complaint data into actionable insights for organizational learning is a form of patient involvement.

Second, patients and their family try to contribute to the quality and safety of their own care by raising concerns with staff. These concerns, verbalized before lodging a formal complaint, can be early indications of a problem. Early error detection, thereby enabling error recovery or mitigation, is crucial for a resilient organization.36, 54 Obtaining data on error recovery is difficult, as near misses are not usually conceptualized as errors.53 The near misses in the complaints revealed many instances of error recovery: staff were reported compensating for one another (correcting a diagnosis, following up a commitment); patients and family members were reported reminding staff about lapsed arrangements, pointing out overlooked care plans, insisting on tests, and even cleaning wards. However, patients and family members’ concerns were sometimes ignored or dismissed. For policymakers, this suggests that patients may be an underused and potentially obstructed resource for improving patient safety. Interventions should focus on both encouraging patients to speak up and training staff in the potential significance of patients’ speaking up for error recovery.

It has been difficult to demonstrate the role of patients in improving safety,91, 92 and the suggestion that patients should be integrated into hospital safety engineering has been challenged.27 This may be because most attempts have involved random samples of patients, most of whom experienced safe care. Patients with nontypical experiences who are reporting safety incidents and actively trying to volunteer information, may have more to contribute to improving safety. Ensuring that instances of speaking up are acted on and that complaint data are turned into actionable insights are low‐cost, and potentially high‐validity, forms of patient involvement.

Limitations and Future Research

Our data were limited to 1% of complaints received by 1 country in 1 year. A larger sample would enable comparisons across units; coding at subproblem levels for more granular hot spots; and tighter operationalization of harm hot spots (eg, only catastrophic harm or death). A longitudinal sample could identify areas that are improving (eg, in response to an intervention), worsening (eg, development of a new hot spot), or constant (eg, resistant to improvement). Cross‐unit comparisons could identify cases of best practice and widespread problems (eg, indicating national funding, training, or guideline issues). A cross‐national sample could identify different complaint profiles related to culture and health care configurations (eg, national, health insurance, direct payment, and charities).

Extending our approach to a cross‐national sample poses challenges. Although the HCAT problem categories were developed from international complaint data and are being used internationally,43, 44, 45, 46, 47 we recommend greater precision in coding harm (eg, distinguishing death from extreme physical and mental distress) and ensuring that the stages of care are cross‐nationally appropriate. The NHS, being a national health care system, enabled obtaining national data through Freedom of Information requests, but, equally we could not control for the quality of redaction. We recommend that health care institutions analyze their own complaints without redaction, but with inter‐rater reliability, using a common framework, and then share the numeric data.

Future research, whether national or international, needs to address 4 issues. First, what are the characteristics of people who complain,12 and why do many people not complain? Second, using a common conceptual framework, how do complaints vary according to the type of institution (eg, acute hospitals, residential care, mental health services)? Third, how do complaints compare to online comments and social media posts?93 We expect that complaints are more detailed and longer than online reporting, but are the nature and severity of the issues reported comparable? Fourth, how do complaints compare to patient records and incident reports? Studies comparing data sources report little overlap between complaints and patient records32, 41 and incident reporting.22 We suggest this stems from the patient‐centered perspective that complaints represent, most evident in entry‐exit, systemic, and omission problems. Future research should analyze data from multiple sources using a common framework to test whether complaint data are more likely to include data on these blind spots.

Our analysis was limited because it treats health care complaints as reported events, but it does not assess their veracity. We have, however, tested 2 relationships: the association between number of problems and harm within complaints and the association between English region and harm reported in complaints. A fuller test of concurrent validity should examine hospital‐level complaint profiles with hospital‐level outcomes, such as standardized mortality rates. Our focus has been to look within a national sample of complaints to consider in‐depth the nature and quality of the data, and future research should use the conceptualization of hot spots and blind spots to find associations with independent outcomes.

Finally, although we emphasized the limitations of monitoring based on staff reporting,31, 32, 33, 34, 35 health care complaints also have limitations. Health care complaints are not representative of patient experiences and some people are less likely to complain than others.12 Moreover, patients have their own blind spots, such as lacking medical expertise, not knowing resource constraints, and not participating in all clinical decision making. Health care staff can shed light on these issues. Accordingly, complaint data should be paired with data from other sources (eg, incident reports, staff surveys, patient experience surveys) as part of an integrated and comprehensive monitoring and learning suite.

Conclusion

Errors in health care are widespread. In the United States they are the third‐leading cause of death.94 Health care providers and policymakers have a responsibility to use all available data to better understand preventable harm. Despite containing valid, independent, and distinctive data, health care complaints have been insufficiently analyzed and benchmarked.3 To address this gap, we outlined and applied analytic steps that can shed light on patient‐centered hot spots of harm and near misses and reveal difficult‐to‐monitor blind spots.

Our analysis found health care complaints provide added value because they are patient‐centered, focus on problematic care episodes, and provide an end‐to‐end account of care that includes concrete details not captured in incident reports or case reviews. Insights obtained, such as the role of patients in detecting and recovering from errors, further demonstrate the value of systematically analyzing health care complaints.

Supporting information

Online Appendix 1. Coded Data

Online Appendix 2. R Scripts