Characterising patient complaints in out-of-hours general practice: a retrospective cohort study in Ireland

Abstract

Background

Patient complaints can provide valuable insights into the quality and safety of clinical care. Studies examining the epidemiology of complaints in out-of-hours general practice internationally are limited.

Aim

To characterise patient complaints in an out-of-hours general practice setting.

Design and setting

Retrospective cohort study of patient complaints to an out-of-hours service provider in Dublin, Ireland, over a 5-year period (2011–2016). This comprises nurse-led telephone triage and GP consultations for patients with urgent problems.

Method

A modified version of the UK Healthcare Complaints Analysis Tool (HCAT) was utilised to code complaints, which were reviewed independently in duplicate by two academic GPs.

Results

Of 445 598 telephone contacts, 303 085 resulted in face-to-face GP consultations. Of 234 patients who made 298 complaints, 185 (79%) related to GP care. The remainder related to nurse triage, other staff, and management issues. A total of 109 (46%) related to children aged ≤18 years, and 134 (58%) of complainants were female. There were 0.61 complaints per 1000 GP consultations. Most complaints (n = 126, 42%) were in relation to clinical care problems, largely diagnosis and prescribing. Common themes included unmet management expectations and clinical examination dissatisfaction. Inter-rater reliability was 90% (κ statistic 0.84, 95% confidence interval = 0.80 to 0.88). Following internal investigation, 158 (85%) of GP-related complaints were managed effectively by the out-of-hours service.

Conclusion

The majority of complaints related to clinical care problems and were successfully managed locally. Expectation management may be an important way to mitigate the risk of complaints.

INTRODUCTION

Patient complaints can provide valuable insights into the quality and safety of clinical care, acting as independent assessors of the healthcare service, and often reflect the expectations of society as a whole.^1–4 Their role is different from that of peer assessment, and fulfils another purpose in promoting patient safety. When a patient makes a formal complaint, a threshold of dissatisfaction has been reached, and such complaints have been shown to highlight deficiencies in the quality and safety of health care.⁵ To maximise learning, it is critical to have an evidence-based and robust system of reviewing and using such complaints, and to implement change where needed.²

GPs providing out-of-hours care are at risk of facing patient complaints. An Australian study that analysed 18 907 formal complaints against doctors (2000–2011) reported that 47% related to GPs.⁵ In a 1-year retrospective case review (n = 526 complaints against GPs) by the Medical Protection Society (MPS), a total of 86 (16%) related to care provided in the out-of-hours setting.⁶ A Dutch study of 250 randomly selected disciplinary complaints against family physicians (2008–2010) reported that 45 (18%) originated in out-of-hours care.⁷

In terms of complaint content, a systematic review which included 59 primary studies reported that approximately one-third of complaints relate to the safety and quality of clinical care, one-third to the management of the healthcare organisation, and one-third to healthcare staff and patient communication.¹ This review highlighted limitations in current approaches to analysing healthcare complaints, including the standardisation of the application of existing tools.¹

Interestingly, only four studies included in this systematic review were conducted either principally or partially in primary care, highlighting the need for further research in this setting. Owen et al retrospectively analysed 1000 UK complaints regarding GP principals (1982–1989) randomly selected from the MPS database.⁸ Failure to perform a home visit was the main reason for complaints in this study.⁸ A German study that examined 13 505 formal national healthcare commission complaints (2004–2007), reported that unjust policies (23.8%), refusal or restriction of drugs (23.8%), and refusal or restriction of non-drug treatments (23.9%) were the main categories of complaints.⁹ An Australian study that analysed 18 907 formal complaints against doctors (2000–2011), of which 47% related to GPs, reported that the main categories of complaint were in relation to clinical care (61%), communication (23%), and other issues (for example, fees, access to care, or confidentiality).⁵ Another Australian study that focused solely on complaints concerning informed consent reported that, of 218 formal complaints, 11 (5%) were made in relation to GPs.¹⁰ The aim of the present study was to characterise patient complaints in an out-of-hours general practice setting over a 5-year period.

How this fits in

Research focusing on patient complaints in out-of-hours general practice is limited. This study found that the overall prevalence of patient complaints was 0.61 per 1000 consultations. The most common reason for complaints related to clinical care (diagnosis and prescribing) problems, in particular unmet patient expectations regarding management, and dissatisfaction with clinical examination. The majority (85%) of GP complaints were successfully resolved locally.

METHOD

The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines were used in the conduct and reporting of this study.¹¹

Study design, population, and setting

This retrospective cohort study examined all patient complaints recorded by Northdoc in Dublin, Ireland, over a 5-year period (1 January 2011 to 31 December 2016). Northdoc provides GPs for D-Doc, the out-of-hours GP service for all patients registered with a participating GP in North Dublin, Ireland, presenting with urgent medical problems. The Health Services Executive (HSE), the body responsible for public healthcare provision, provides the infrastructure for the service, including employment of triage nurses and administrative staff. The service operates from 18.00 to 08.00 Monday to Friday, and additionally provides a 24-hour service on Saturday, Sunday, and all public holidays. The service consists of nurse-led telephone triage and GP consultations in five treatment centres across North Dublin, and via home visits where necessary. A robust complaints process is in place, with complaints managed centrally and investigated internally by a designated complaints manager who is a GP and holds responsibility for maintaining records.

Complaint definition and coding of complaints

For the purposes of this study, a patient complaint was defined as ‘a verbal or written complaint made to and addressed by the Northdoc out-of-hours GP service complaints manager’. Anonymised designated complaints files containing written records of each complaint and any additional relevant correspondence were reviewed independently in duplicate by two experienced academic GPs, independent of Northdoc, with disagreements resolved by a third reviewer. In addition, anonymised patient consultation records relevant to the patient complaint were reviewed. A standardised template was used to extract relevant data.

A recent systematic review that included 59 studies reporting healthcare complaint coding tools revealed significant limitations with the way healthcare complaints are analysed.¹ These issues include absence of an established taxonomy for categorising healthcare complaints, minimal standardisation of the procedures involved (for example, coding guidelines and training), absence of testing for reliability, and no assessment of complaint severity.

To address these limitations, the Healthcare Complaints Analysis Tool (HCAT) was developed in 2015 by aggregating the coding taxonomies from studies included in the systematic review, revealing 729 uniquely worded codes that were then refined and conceptualised into seven categories and three broad domains.^1,12 The three domains are clinical, management, and relationship problems, and subcategories are outlined in Appendix 1. The HCAT has been tested for both reliability and accuracy in assessing healthcare complaints relating to UK inpatient care.¹³ A detailed handbook for applying the tool has been developed, in addition to online training materials.¹²

Applying the HCAT

According to the HCAT, there are four phases to coding a healthcare complaint. Phase A involves identifying the presence of a problem category (and subcategory if required) using the coding taxonomy provided, and assessing the severity using examples provided in the HCAT to guide the process (Appendix 2). Phase B involves specifying the stage of care in which the problem arose. Phase C involves rating the level of harm experienced by the patient as a result of the reported problem. Phase D involves providing descriptive information about the complaint.¹² Patient level of harm is rated according to the UK National Reporting and Learning System and was coded from 1–5, as outlined in Appendix 3.¹⁴ Assessment is focused on the harm caused by the problems raised in the detail of the complaint and is rated independently of the severity of the complaint. For example, a high-severity complaint, such as a serious prescribing error, may not have resulted in any patient harm.

Piloting of the HCAT and modifications

As the HCAT was developed for use in inpatient settings, it required some modifications to the problem subcategories and severity indicators (Phase A) and stage of care (Phase B) for application to GP complaints. The HCAT was piloted on a sample of 20 general practice out-of-hours patient complaints by two academic GPs, who first completed online training regarding the application of the tool.

Following piloting, an additional sub-categorisation of ‘fees’ was included in the ‘management problems’ section. In contrast to the UK, where the HCAT was developed, Ireland has a mixed private–public healthcare system. Approximately 47% of the population receive free medical care through the General Medical Services (GMS) scheme and Doctor Visit Only (DVO) schemes.¹⁵ These schemes are means tested, but in 2015 free GP care (including out-of-hours) was introduced for all children aged ≤6 years. Second, further examples of severity indicators relevant for GP complaints were included, summarised in Appendix 2. Third, further options were included for the stage of care where the problem arose, as follows:

• clinical examination;

• diagnosis;

• prescribing;

• referral; and

• other.

Illustrative examples of applying the HCAT to letters of complaint are available from the authors on request.

In addition to the HCAT coding, the outcome of the complaint following investigation was recorded as follows:

• successfully defended to the mutual agreement of both parties;

• defended but closed without agreement between parties;

• complaint upheld and formal apology; or

• complaint not GP related so referred to an alternative agency.

Statistical analysis

Descriptive statistics for the study participants are presented, in addition to the prevalence and incidence of patient complaints.

RESULTS

Descriptive statistics

There were a total of 445 598 telephone contacts to the out-of-hours nurse-led triage service from 2012–2016, of which 303 085 resulted in face-to-face GP consultations. A total of 234 patients made 298 complaints; 185 (79%) related to GP care. The remaining complaints related to nurse triage, other administrative staff, and other management issues (Table 1). Of the 234 patients who made complaints, 109 (46%) related to children aged ≤18 years, with the majority of these relating to children aged ≤5 years (Table 1). A total of 134 (58%) complaints related to female patients, and 98 (42%) were in receipt of free medical care (GMS card).

Table 1.

Descriptive statistics of study population 2012–2016 (n = 234)

	n (%)
Age, yearsa
≤5	85 (36)
6–17	23 (10)
≥18–64	103 (44)
≥65	22 (9)
Sexb
Male	98 (42)
Female	134 (58)
General Medical Services (GMS) cardc
Yes	98 (42)
No	127 (55)
Complainant
Parent/guardian	109 (46)
Patient	88 (38)
Son/daughter	11 (5)
Spouse/partner	10 (4)
Other family member	9 (4)
Healthcare professional	5 (2)
Other	2 (1)
Staff member complaint relates to
GP	185 (79)
Triage nurse	8 (3)
Administrative staff	6 (3)
Multiple healthcare professionals	3 (1)
Other	32 (14)

^aAge was missing for one person.

^bSex was missing for two people.

^cGMS card details missing for nine people.

The overall prevalence of patient complaints against GPs over the 5-year period was 0.61 per 1000 GP consultations. The annual rate of patient complaints per 1000 GP consultations remained relatively stable over time, with 0.81 in 2012, 0.56 in 2013, 0.69 in 2014, 0.56 in 2015, and 0.45 in 2016. Following the completion of an internal investigation by the complaints manager, 30 complaints (16%) against GPs were upheld and resulted in a formal apology to the complainant. This represents an annual rate of 0.18 per 1000 GP consultations. The remaining GP-related complaints were successfully defended to the satisfaction of both parties (n = 128, 69%), or were closed without agreement (n = 27, 15%).

Of the 30 upheld complaints, 13 related to a clinical problem, five to a management problem, and eight to a relationship problem. Four complaints were coded across multiple categories. Of the clinical problems, eight related to prescribing and four to diagnosis.

Categorisation of patient complaints

A total of 298 complaints relating to care received by 234 patients were made (Figure 2). Inter-rater reliability was 90%, with a κ statistic of 0.84 (95% confidence interval [CI] = 0.80 to 0.88). The majority of complaints (n = 126, 42%) related to clinical care. These complaints were further subcategorised according to clinical safety complaints (n = 76) and quality of care complaints (n = 50), and largely related to issues with diagnosis, prescribing, and referral (Table 2). Common themes included dissatisfaction with clinical examination and unmet expectations regarding management (for example, parental dissatisfaction that oral antibiotics were not prescribed to a child diagnosed with a viral infection).

Figure 2.

Complaints according to complaint category, subcategory, and severity (n = 298 complaints relating to n = 234 patients).

Table 2.

Clinical problem category complaints: stage of care and level of patient harm (n = 126)

	n (%)
Stage of care
Clinical examination	20 (16)
Diagnosis	45 (36)
Prescription	46 (37)
Referral	9 (7)
Other	6 (4)
Level of patient harm
No/minimal	102 (81)
Minor	19 (15)
Moderate	4 (4)
Major	1 (1)
Catastrophic	0 (0)

Of 45 complaints regarding diagnosis, five objective misdiagnoses were identified, which resulted in either moderate (n = 4) or major (n = 1) patient harm (Table 2). No misdiagnosis resulted in catastrophic harm. The majority of patient complaints regarding diagnosis related to the development of a recognised complication of their index visit condition, or a deterioration in a correctly diagnosed condition, subsequently misconstrued as a misdiagnosis. Similarly, of 46 complaints related to prescriptions, seven objective prescription errors were identified (for example, wrong dose of medication, prescription of a medication to a patient with known medication allergy, or incorrect treatment protocol used), which resulted in negligible or minor patient harm. The majority of prescription-related complaints resulted from unmet patient expectations regarding management (for example, oral antibiotics not prescribed when not clinically indicated), the type of medication prescribed (for example, oral formulation prescribed rather than intramuscular administration), the dose prescribed, or the duration of the prescription. In some cases, these complaints arose as a result of comments made to the patient by other healthcare professionals, such as other doctors or pharmacists (n = 14).

A total of 86 complaints (29%) related to relationship problems between the healthcare provider and patient. These were further subcategorised as issues regarding respect and patient rights (n = 47), communication issues (n = 36), and listening problems (n = 3). Common themes included perceived rudeness, abrupt manner, inadequate explanation of diagnosis or management plan, or dissatisfaction with the approach of the GP to the consultation. Finally, 86 complaints (29%) related to management problems subcategorised as issues regarding fees (n = 51), institutional processes (n = 30), and the environment (n = 5). Common fee complaint themes included dissatisfaction with payment for review consultations, refund requests if the patient subsequently attended their own GP or self-presented to the emergency department in the course of the same illness, and refund requests if patient expectations of the consultation were not met (for example, prescription of medication or referral). Other themes related to waiting time to see the GP, suitability of infrastructure (for example, building or consultation rooms), and triage processes (for example, triaged to treatment centre rather than a home visit, or triaged as routine rather than urgent home visit).

DISCUSSION

Summary

This study found that approximately one out of every 2000 GP consultations in this out-of-hours setting resulted in a patient complaint. This rate is similar to a previous Irish study examining out-of-hours GP complaints (2010–2013) in a different geographical location, where the reported complaint rate ranged from 0.49 to 0.77 per 1000 GP consultations.¹⁶ In the current study, complainants were most frequently parents on behalf of their child.

Approximately 40% of complaints related to clinical problems, with the remainder concerned with management issues and relationship problems between the patient and healthcare provider. Clinical problems were related to diagnosis, prescribing, and management. For diagnosis, the majority of complaints were due to the development of a known complication or subsequent deterioration in a correctly diagnosed index condition. There were five objective misdiagnoses that resulted in moderate (n = 4) or major (n = 1) patient harm. No misdiagnosis resulted in catastrophic harm. The majority of prescription complaints related to unmet management expectations. There were seven objective prescription errors identified, which included prescription of a medication to a patient with a known allergy, incorrect medication dose prescribed, and incorrect treatment protocol utilised. No prescription error resulted in moderate or severe patient harm.

The remainder of complaints related to management problems (29%) and relationship problems (29%). Management problems concerned issues with fees, institutional processes, and the environment. Fee complaints largely related to requests for refunds and dissatisfaction with paying for multiple out-of-hours consultations. Relationship problems were largely about breakdown in communication where the complainant perceived the healthcare provider to be rude, abrupt, or lacking empathy. Other relationship issues related to inadequate explanation of the management plan.

The majority (85%) of patient complaints against GPs were managed effectively by the out-of-hours service This highlights the value of local complaints resolution structures in general practice settings.

This study offers valuable insights into the epidemiology and content of patient complaints in out-of-hours general practice. The majority of complaints related to clinical care problems and were successfully managed locally. Expectation management may be one way of mitigating the risk of complaints.

Strengths and limitations

This study includes 5 years of data from a large out-of-hours care provider with a robust complaints system in place. All complaints were independently reviewed in duplicate by two experienced academic GPs using a validated coding system, amended for use in general practice. The number of participants with missing data was very low (Table 1). A limitation of the study was that the authors did not examine healthcare provider factors (for example, age, sex, years of clinical experience, or history of previous complaints) associated with complaints. Additionally, the case mix in the out-of-hours setting is different from routine general practice, as this service is designed to assess and manage urgent rather than routine problems. Therefore, the types of complaints encountered in out-of-hours care may not reflect routine general practice complaints. In addition, this study was conducted in an out-of-hours GP provider with an established complaints service, which may have influenced resolution rates.

Comparison with existing literature

International literature focusing on out-of-hours general practice complaints is very limited.^17,18 A recent Dutch study examined the concept of patient safety culture across 16 out-of-hours GP cooperatives and two call centres in the Netherlands.¹⁹ Of 784 responders (of which 470 were GPs, and 189 were triage nurses), healthcare providers were most positive about teamwork, climate, and job satisfaction, and less about communication, openness, and safety climate. In the UK, the National Statistics Office produces an annual overview of written complaints against primary care healthcare professionals (including GPs, pharmacists, clinical therapists, and dentists). During 2016–2017, 65 637 complaints against GPs were resolved: 24 243 (37%) were upheld, 8809 (13%) were partially upheld, and 32 315 (50%) were not upheld.²⁰ In the current study, 16% of all complaints were upheld, representing a smaller proportion of total complaints compared with routine UK general practice. However, out-of-hours Irish general practice is a different setting, with a different case mix from routine UK general practice. There is currently very limited literature on which to base comparisons. For GP practices, most complaints related to issues regarding communication (16.9%), clinical treatment (13.7%), and staff attitude, behaviour, and values (12.6%).²⁰ Most complaints in the current study related to clinical problems (42%), with the remainder relating to management and relationship problems.

It is important to consider the reliability of current complaints coding systems and, more broadly speaking, how useful patient complaints are in identifying patient safety incidents. A systematic review published in 2014 identified several different taxonomies developed for the purpose of coding patient complaints, but methodologies varied considerably.¹ A retrospective Dutch study of 1145 medical records concerning patient contacts with four out-of-hours GP cooperatives identified 27 patient safety incidents, an incident rate of 2.4% (95% CI = 1.5% to 3.2%).²¹ The most frequent incident type was treatment related (56%), with the majority not resulting in patient harm (70%).²¹ Another Dutch inpatient study (n = 5375, 14 hospitals) examined how reliable patient complaints, malpractice claims, and healthcare professional incident reports were in identifying patient safety incidents.²² Of a total of 498 adverse events detected, only 18 (3.6%) were identified by patient complaints and claims and/or healthcare professional incident reports.²² This highlights that, although patient complaints are an important part of examining overall patient safety and patient experience, they are not a substitute for having robust systems in place to regularly audit care. Therefore it is important to have multiple approaches in place to identify patient safety incidents, as all modalities have their limitations.

Implications for research and practice

In the current study, unmet patient expectations were a driver for many complaints, including parental expectation of antibiotic treatment for their child that was not deemed clinically indicated by the GP. There are many contributing factors to parental expectation of antibiotic treatment, including prior experience with receiving oral antibiotics and knowledge regarding the appropriate use of antibiotics.^23,24 A large-scale Irish survey (n = 7487 participants aged ≥15 years) published in 2017 reported that 49% of responders believed that antibiotics were effective against viruses.²⁵ It is important for GPs to actively address patient expectations during the consultation, while accepting that providing evidence-based care is not without risk of complaints. A recent Cochrane systematic review examining clinician interventions to influence antibiotic prescribing behaviour for acute respiratory infections in primary care reported that point-of-care testing (for example, for C-reactive protein) and shared decision making were effective in reducing antimicrobial prescribing.²⁶ Another systematic review that focused on interventions to reduce antibiotic prescriptions for childhood upper respiratory tract infections reported that educational interventions targeting both parents and clinicians were most effective.²⁷

A systematic review examining the epidemiology of malpractice claims in primary care internationally reported that the commonest medical misadventure resulting in claims was failure to or delay in diagnosis, which represented 26–63% of all claims across included studies.²⁸ However, the majority of claims were successfully defended.²⁸ In the current study, the number of objective misdiagnoses and prescription errors were very small. Interestingly, the development of a recognised complication of a correctly diagnosed condition, or a deterioration in a condition, were often misconstrued by complainants as diagnostic errors. It is important for GPs to communicate potential complications and prognosis, where feasible. However, as it is impossible to predict all clinical outcomes, safety netting is essential.²⁹

In the UK, an external review of out-of-hours GP providers reported that some providers did not inform patients how they could make complaints about the service.³⁰ From a clinical governance perspective, having robust complaints procedures in place is essential.

Appendix 1.

Healthcare Complaints Analysis Tool (modified): categorisation of complaints

Complaint category and subcategories	Examples of complaint type
Clinical problems
Quality of care (clinical standards of health care and behaviour)	Inadequate clinical examination, illegible prescription, inadequate detail provided in referral letter
Safety (errors and clinician competency)	Misdiagnosis, prescribing error
Management problems
Environment (facilities, clinical equipment, staffing levels)	Problems with facilities provided (for example, waiting room), staffing levels
Institutional process (waiting times, accessing care)	Waiting time, nurse telephone triage, opening times
Fees	Issue with fees charged, refund requests
Relationship problems
Listening	Questions not answered, patient felt dismissed or ignored
Communication	Management plan not communicated, follow-up plan not communicated
Respect and patient rights	Patient felt healthcare provider was rude, healthcare provider lost temper, confidentiality breach

Appendix 2. The Healthcare Complaints Analysis Tool (HCAT). Reproduced with permission of the authors.¹³

Appendix 3. Coding of patient harm: UK National Reporting and Learning System

Code	Description
1) Negligible/minimal	No or minimal intervention required to ameliorate harm
2) Minor	Minor intervention required to ameliorate harm — for example, needed to see GP
3) Moderate	Significant intervention required to ameliorate harm — for example, needed to be admitted to hospital
4) Major	Results in long-term incapacity
5) Catastrophic	Death, permanent injury

Funding

This research is funded by the Health Research Board of Ireland under grant reference number HRC-2014-1.

Ethical approval

Ethical approval was granted by the Irish College of General Practitioners (ICGP) Research Ethics Committee.

Provenance

Freely submitted; externally peer reviewed.

Competing interests

The authors have declared no competing interests.

Discuss this article

Contribute and read comments about this article: bjgp.org/letters