Quantification of Complaint and Compensation Cases by Introducing a Danish Translated and Cross-Cultural Adapted Edition of the Healthcare Complaints Analysis Tool

Abstract

Purpose

The study aimed to translate and cross-culturally adapt the English version of the HCAT to produce a Danish HCAT version and to test the Danish version’s reliability.

Methods

We used best-practice guidelines for linguistic translations and cultural adaptations. For cross-cultural adaptation, we conducted forward and back translation followed by expert committee review. Subsequently, two researchers assessed 140 complaint cases to test intra- and inter-rater reliability of the Danish HCAT version. We used descriptive statistics for distributions and tested for differences between English and Danish editions Intra- and inter-rater reliability used Gwet’s AC1 statistics, applying quadratic weights to assign more weight to large discrepancies.

Results

The back translations showed both semantic and conceptual differences, and the expert committee thus discussed the meaning of the wording in the HCAT guide and coding form to ensure that the Danish version would be conceptually similar to the English version but also culturally appropriate for Danish settings. There was discussion about how to use the coding form to graduate problem severity, and this led to some altered wording. Pilot testing revealed the need for two new categories of “hospital-acquired infection” and “involvement of patients’ relatives”. The problem categories of the HCAT-DK showed “substantial” intra- and inter-rater reliability (0.79, and 0.79 to 0.85). In addition, there was a “substantial” agreement (0.70 to 0.73) between the original HCAT and the HCAT-DK version.

Conclusion

The study translated and cross-culturally adapted the English HCAT version to produce a Danish HCAT version. Cultural and conceptual differences led to adjustments and to addition of two extra items in the HCAT-DK. The Danish version showed “substantial” intra- and inter-rater reliability and is considered suitable for coding complaint and compensation cases in Danish health care.

Introduction

Patients are willing to provide information that can enhance patient safety if they can be assured that their voices will be heard and their input used.¹ One way that patients and their relatives can provide input and contribute to health care improvement is by making complaints about the health care received. Despite this potential source for supplementary information about the need for health care improvements, patient complaints are currently underused in quality improvement research.² Complaints can be considered uncensored patient opinions on the quality of care,³ and most complainants state that they complain to avoid other patients being exposed to the same errors that they themselves were exposed to.^4,5

Most countries have formalized systems to register adverse episodes, complaints, and compensation cases that could be used to promote patient safety and to avoid repetition of healthcare errors. Challenges arise, however, when these data are to be quantified and analyzed.⁶ The lack of systematic and reliable methods to extract data from qualitative and heterogeneous complaint letters hinders their use for quality improvement.⁷ It has previously been stated that if healthcare settings were better supported to standardize, quantify, and analyze the information from complaint and compensation cases, this information could have a larger impact on the quality of care.⁸

The Healthcare Complaints Analysis Tool (HCAT)⁷ was developed in the UK as a standardized tool for coding and analyzing complaints in a systematic manner. It is based on a classification of health care complaints identified in a systematic literature review and contains three overall problem domains (clinical, management, and relationship) and seven problem categories (quality, safety, environment, institutional processes, listening, communication, and respect and patient rights). The seven problem categories each have four levels of severity (0–3), and the stage of care to which the complaint refers is coded into six categories. The HCAT was found to have good inter-rater reliability in a UK sample of healthcare complaints.⁷ In a Danish study, the English version of the HCAT was used to code a sample of Danish patient compensation claims and was found to be reliable for categorizing problem types in claims for injury compensation cases.⁹

Further determination of whether the HCAT could be a useful tool for quality improvement by extracting and coding information from compensation cases in the Danish health care setting requires a Danish language version. It is commonly recommended that instruments used across cultures should be translated and undergo some degree of cross-cultural adaptation to ensure content validity at a conceptual level.¹⁰

The aim of the current study was thus to translate and cross-culturally adapt the English version of the HCAT to produce a Danish HCAT version. Additionally, the intra- and inter-rater reliability of the Danish HCAT was tested on a sample of actual Danish complaint cases.

Methods and Materials

The linguistic translation and cultural adaptation process comprised seven stages (Table 1) and was based on recommended international guidelines^10,11 and previous translations conducted at our research organization.^9,12

Table 1.

The Phases of Linguistic Translation and Cross-Cultural Adaption

Stages of the Translation Process		Tasks Conducted at Each Stage
1	Initiation	Contact to developers
		Forming an expert group (Steering group)
2	Translation	Translating the HCAT coding manual from English to Danish
3	Synthesis	Comparing the two translations
		Reaching consensus
		Produce initial Danish version
4	Back Translation	Re-translating the Danish translation back into English
		Creating two back translations
5	Expert committee review	Review of all translated versions
		Reaching consensus on a pilot version
6	Pilot testing	Testing among 3–4 raters (research assistants)
		Rating 10 cases – 5 as a group
		Consensus on any conceptual questions
		Final version
7	Clinical testing	Testing of the final coding taxonomy
		Determining the reliability of the HCAT-DK

The English HCAT consists of a coding manual and a coding form.⁶ The coding manual (or “Step-by-Step” guide) includes four sections. Section A describes the three problem domains, giving examples for each of the problem categories within each domain and indicating how to assess the level of severity for each category. Section B specifies the stages of care in which a problem can occur. Section C explains how to assess the level of harm, and Section D specifies the descriptive details that are collected on each complaint. The HCAT coding form is a one-page report form for recording the data extracted from each complaint. Besides using the translation of the HCAT manual to develop a Danish version, the translation will be used in the development of online teaching material to assure ongoing quality in education of health care staff.

Stage 1: Initiation

To support the ongoing collaboration between our research institutions, the authors contacted the English developers to inform them of the translation plans and to avoid copyright infringements. The developers offered input to the process when needed and allowed us to use the original template for the Danish version so as to standardize the layout of the HCAT across countries.

We formed a research panel consisting of five members from different branches of health care (physiotherapy, clinical medicine, and midwifery) who all are researchers in their own field. Four are native Danish speakers, and one is a native English speaker. The panel members' remit was to ensure a high-quality translation process, and they also acted as an expert committee at stage 5.

Stage 2: Forward Translation

The HCAT Step-by-Step guide and coding form were translated from English into Danish by two members of the research panel who had prior knowledge of the Danish complaint system and the HCAT concept. The translators worked independently of each other, and each provided a written version of the HCAT documents (T1 and T2).

Stage 3: Synthesis

The two forward translators met to synthesize the translations into one common Danish version on the basis of the original HCAT manual (T1-2). They discussed and agreed on the coding form first and then used phrases from this to ensure uniform phrasing in the Step-by-Step guide. Any discrepancies in wording between the two translations were debated until they were resolved through consensus.

Stage 4: Back Translation

The common Danish version (T1-2) of the HCAT was translated back into English by two translators working independently. One translator was a native English speaker and a health care academic with in-depth knowledge of questionnaire translation. The other was a professional translator who was a native Danish speaker but had lived for 20 years in the US. This translator had no particular knowledge of the health care complaint system or of Danish health care. The two back translators only received the T1-2 Danish version and did not have prior knowledge of the original English HCAT version. This process produced two back translations (BT1 and BT2).

Stage 5: Expert Committee Review

All members of the research panel received for review the original HCAT manual and all the translated versions (of the Step-by-Step guide and the coding form). It proved not possible to involve the English developers in this part of the process. The research panel met to discuss the translations, and any discrepancies were discussed until the panel had reached consensus. This produced a pre-final Danish version.

Stage 6: Pilot Testing

The pre-final Danish version was sent to three research assistants with academic background employed in our department who were asked to use it to code the complaint cases illustrated in the online HCAT educational module developed in the UK. The English teaching program gives a short introduction to the HCAT and gives some examples of complaint cases. The three raters then used the pre-final version of the Danish HCAT to code five fictive complaint cases. They were also asked to note any ambiguities they noticed in the Danish manual, especially in relation to its use in a Danish health care setting. Afterwards, we met with the raters to hear their feedback and to discuss how the manual performed. The results of this pilot test were used to assess experiential, semantic, idiomatic, and conceptual equivalence¹⁰ of the manual to produce a final Danish version of the HCAT, the HCAT-DK (See Appendix 1).

As an extra check of the face validity of the severity grading examples provided in the HCAT, we asked the head of the Department of Patient Dialogue in the Region of North Jutland to review the Danish version. This department works professionally with patient complaints and compensation claims and assesses over >150 complaint cases annually.

Stage 7: Clinical Testing

The HCAT-DK was used by two independent raters (two researchers from the research panel) to code 140 actual Danish complaint cases. One rater coded all the cases again after four weeks in another random order. The same cases were used in an earlier Danish reliability study⁹ in which four raters used the English version of HCAT to code the 140 cases (again one rater coded the cases twice), and the intra- and inter-rater reliability was tested. By using the same procedure with the Danish version on the same cases, we could assess intra- and inter-rater reliability in a comparable manner and were able to compare the ratings from the English HCAT and the HCAT-DK.

This clinical testing was conducted at the local office of the Patient Compensation Association in Odense. Both raters were members of the research panel and had participated in the translation process as translators. Prior to using the HCAT-DK to assess the 140 cases, they completed the English online HCAT educational module. They then completed a calibration exercise where they coded and discussed ten complaint cases: three cases were coded together by the two raters, the next three cases were coded individually section by section and then discussed, and the final four cases were coded individually as a whole and then compared and discussed until consensus.

Data Analysis

Much of the analyses were descriptive to provide transparency to the translation process and to enable evaluation of the various stages. For the intra- and inter-rater reliability, we used Gwet’s AC1 agreement coefficient,¹³ applying quadratic weights to assign more weight to large discrepancies. We used the same guidelines for interpretation of reliability¹⁴ as used in the previous Danish reliability study on HCAT,⁹ ie, 0.01–0.20 denotes poor/slight agreement; 0.21–0.40 fair agreement; 0.41–0.60 moderate agreement; 0.61–0.80 substantial agreement; and 0.81–1.00 excellent agreement. Statistical analyses were performed in Stata, version 15 (StataCorp LP, College Station, Texas).

Ethics

The Danish Data Protection Agency and the PCA approved the data handling (project approval 17/18411). Danish law does not require approval from the ethics committee for studies of the present type, according to the Act on Research Ethics Review of Health Research Projects (2020). According to this law, informed consent was not required from either the patients or the individuals testing the HCAT tool.

Results

Stages 2 and 3: Forward Translation and Synthesis

Initially, the two forward translators focused on a literal translation of the HCAT manual by keeping the Danish translation as close as possible to the original English. This resulted in some discrepancies related to different perceptions of the nuances in the English wording and the inadequacy of a direct translation into Danish. To meet this challenge, the translators reviewed the versions again focusing on semantic translation to ensure that the original meaning of the English wording was retained in the Danish wording. The two translators discussed all observed discrepancies until reaching consensus and a common Danish version emerged (T1-2).

Stages 4: Back Translation

The two English back translations of the Danish T1-2 version showed some discrepancies compared to the original English version. Some of these were semantic differences where single words had similar meanings (eg, leadership vs management; responsiveness vs receptiveness; conduct vs behavior), while others were conceptual differences where the meaning in the back-translated version differed to that in the original HCAT (eg, the concept of severity was discussed due to conceptual differences). These differences were noted and brought to the attention of the research panel in the next step of the translation process.

Stage 5: Expert Committee Review

The research panel consisted of the two forward and two back translators and the two researchers skilled in using the HCAT coding form. A line-by-line review of the translation documents revealed disagreements, especially where the coding form is used to graduate severity of a certain problem (eg, use of severity vs seriousness; interpretation of 0 = “not evident”) and in the specific examples of different problem areas. The panel spent most time debating the conceptual equivalence of the wording used in the Step-by-Step guide and the coding form in an attempt to produce a Danish version that would be conceptually similar to the English version but also culturally appropriate for a Danish setting.

Despite the attempts made by the panel to simplify the wording in the coding manual, it was discussed whether the Danish wording was still too sophisticated. As the HCAT coding manual will be used by health care and administrative staff with specific knowledge and training in dealing with complaints and compensation cases, however, the panel found the conceptual level of the translated version acceptable under each problem category, the HCAT Step-by-Step guide provides lists of keywords that complainants typically mention in their complaint letters. These were all translated into Danish using idiomatic words and phrases that patients would use when filing a complaint. The committee also agreed in principle that further keywords could be added if necessary, as new knowledge emerges from future complaint cases. Finally, the research panel agreed that the coding form should be as user-friendly as possible but must still reflect and be in accordance with the Step-by-Step guide. Overall, it took the research panel two meetings (approx. 4 hours in total) to reach consensus on a pre-final version of the HCAT-DK.

Stage 6: Pilot Testing

The pilot testing of the HCAT-DK by our three academic colleagues led to considerations and adjustments concerning experiential equivalence. Firstly, the raters considered that the examples provided for rating the severity of complaints did not cover all the areas of importance in Danish health care. For instance, there were no examples related to “hospital-acquired infection”. There was some discussion of where this example should be added, however, as it could be relevant for several of the HCAT problem categories. ‘Hospital acquired infections’ could be relevant to allocate under two different domains (clinical problems or management problems). If placed under clinical problems, it might be referred to as a quality issue or a safety issue. If placed under management problems, it could fit best under “environment”. The two suggested allocations with severity categories are displayed in Appendix 2. It was decided to await the clinical testing to see if “hospital-acquired infection” would emerge as a common source of Danish patient complaints.

Secondly, the massive focus on involvement of patients’ relatives in Danish health care was considered not to be reflected in the HCAT-DK. The pilot testers suggested that examples of insufficient family involvement were added to the severity graduation (Table 2).

Table 2.

Severity Graduation Under the Problem Category of “Communication” in the Domain of “Relationship Problems”

1. Low Severityc	2. Medium Severityc	3. High Severityc
Insufficient information given to relatives despite prior agreement on this [Da; Manglende information til pårørende trods aftale om dette].	Re-scheduling of meeting leaving relatives without important information despite prior agreement on this [Da; Møde rykket så pårørende ikke modtog vigtig information trods aftale om dette].	Relatives not present when information on critical illness was given to patient despite prior agreement on this [Da; Manglende tilværelse af pårørende ved information om kritisk sygdom trods aftalt om dette].

Note: ^cSeverity graduation form 0–3 (0= severity not identified, 3= high severity).

Overall, despite minor comments on difficulties of distinguishing between the categories of “quality” and “safety” and how exactly to define levels of harm, the three raters found the HCAT manual and coding form to be easy to use. Like the expert group, they felt that the manual would benefit from continuous addition of examples and keywords suitable for the Danish context as knowledge from future complaint cases emerge.

In the second part of pilot testing, the head of the Department of Patient Dialogue in the Region of North Jutland gave feedback on the graduated examples in the manual. In her opinion, the examples were recognizable in a Danish setting and reflected the issues presented in Danish health care complaint cases. She also considered the severity grading of the complaints to be relevant guides for Danish complaint cases. However, she emphasized the importance of a qualitative approach in handling complaint cases to ensure that all underlying circumstances in a filed complaint are identified.

Stage 7: Clinical Testing

The reliability testing showed that the seven problem categories of the HCAT-DK reached substantial inter-rater reliability (0.79 [95% confidence interval (CI) 0.76–0.83] for Rater 1 and 0.85 [0.82–0.88] for Rater 2) and intra-rater reliability (0.79 [0.75–0.83] for Rater 1). Agreement between the original HCAT and the translated HCAT-DK version was lower but still substantial, ranging from 0.70 [0.65–0.75] to 0.73 [0.69–0.77] (Table 3).

Table 3.

Reliability of the HCAT-DK, and Comparison of HCAT and HCAT-DK

Intra- and Inter-Reliability of the HCAT-DKa	Agreement Coefficient [CI95%]
Rater 1, first rating – Rater 2	0.79 [0.76–0.83]
Rater 1, second rating – Rater 2	0.85 [0.82–0.88]
Rater 1, first rating – Rater 1, second rating	0.79 [0.75–0.83]
Inter-Reliability Across the HCAT and HCAT-DKa
Rater (HCAT) - Rater 1(HCAT-DK), first rating	0.72 [0.68–0.76]
Rater (HCAT) - Rater 1(HCAT-DK), second rating	0.70 [0.65–0.75]
Rater (HCAT) - Rater 2(HCAT-DK)	0.73 [0.69–0.77]

Note: ^aGwet’s AC1 agreement coefficient was used, applying quadratic weights to assign more weight to large discrepancies.

Regarding “hospital-acquired infection”, this was found to be a useful addition to the HCAT-DK and viewing the cases during the clinical testing, we found that the item was best included under the problem category of “environment” in the domain of “Issues with management and service”. The wording shown in the lower half of Appendix 2 was used. The new category was included in the final HCAT-DK version (Table 4).

Table 4.

New Category in the HCAT-DK Classification

A2 Management Problemsa
Location Could Be “Environment”b
1. Low Severityc	2. Medium Severityc	3. High Severityc
The patient experienced poor ward hygiene during admission, potentially exposing the patient to a hospital-acquired infection [Da; Patienten oplevede dårlig hygiejne på afdelingen, med risiko for eksponering for hospitalserhvervet infektion]	The patient experienced a hospital-acquired infection that required treatment, but it did not prolong recovery [Da; Patient udsat for behandlingskrævende hospitalserhvervet infektion]	The patient experienced a hospital-acquired infection that required treatment and considerably prolonged the recovery [Da; Patient udsat for behandlingskrævende hospitalserhvervet infektion med længerevarende sygdomsforløb til følge]

Notes: ^aHCAT domain, ^bproblem category, ^cseverity graduation from 0-3 (0=severity not identified, 3=high severity).

Discussion

This study aimed to translate and cross-culturally adapt the English version of the HCAT to produce a Danish HCAT version, the HCAT-DK. The translation process followed internationally accepted guidelines, and the resulting version was shown to have good intra- and inter-rater reliability and to be relevant for extracting and coding information from compensation cases in the Danish health care setting.

During the translation process, we identified cultural and conceptual differences between the English and Danish versions. One explanation for this may be the differences in how health care standards are perceived in the two countries,¹⁵ but it may also be due to different points of focus in the two national health care organizations. In Denmark, for example, the health care authorities place a strong emphasis on the involvement of the patient’s relatives in treatment and illness management and especially in the communication between patients and health care providers. This consideration led to an additional “involvement” item in the Danish version (under “Communication”) that does not appear in the English version.

Similarly, we found that a number of Danish complaint cases can refer to the risk or occurrence of “hospital-acquired infection”, and this item was thus also added to the HCAT-DK. It could be argued that this item should be placed under the domain of “clinical issues” if the hospital-acquired infection relates to the behavior of the clinical staff (for example, poor hand-washing practices). Under this domain, the problem category could either be “quality” if it is considered a side-effect of hospitalization or “safety” if someone missed the signs of a hospital-acquired infection. Alternatively, hospital-acquired infection could be placed under the domain of “issues with management and service” using the problem category of “environment” if an infection is considered to be a result of poor standards of service caused by low staffing levels. In the pre-final version of the HCAT-DK, it was decided to include both graduations under each domain, as the scores did not precisely indicate where to place it. In the final edition (see the HCAT-DK in the Supplementary Materials), “hospital-acquired infection” was placed under the problem category of “environment” in the domain of “issues with management and service” because we considered the issue to be more of an organizational problem than relating to an individual patient. As the HCAT-DK coding form (see Appendix 2) becomes more widely used in the future, we will re-visit this issue to confirm the placement of “hospital-acquired infection” under “environment”.

Strengths and Limitations

In producing the HCAT-DK, we followed a rigorous translation process and supplemented this with pilot testing, clinical expert opinion from another region in Denmark, and clinical testing with “real-life” complaint cases. The work follows suggested guidelines for translations of questionnaires and measurement tools,^10,11,14 and have earlier been used in translations of other questionnaires.^12,16

A limitation of the study is that we did not include the whole HCAT manual in the formal translation process. Much of the original manual is background information containing, for example, reasons for coding complaint cases in a systematic way. We did not find it relevant to include these paragraphs in the Danish version, because they contained also much UK-specific information that could be considered confusing or less relevant in a Danish context. Nonetheless, we later realized that two pages of the introduction to the Step-by-Step guide were thus not included in the formal translation process. We do not believe this has influenced the functioning of the manual itself, however, because it did not contain any methodological introduction to the coding.

Another potential limitation is the lack of patient involvement in the translation process. It could be considered relevant to include the lay-person perspective in a research project focusing on health care users’ assessment of health care expressed in complaint letters. In our study, the lay perspective was only represented during back translation, where a bilingual translator with no health care expertise contributed with input from a non-professional viewpoint. Including patient representatives in the process might have added further patient-relevant severity categories under each main problem category.

Implications

All Danish complaint and compensation cases are currently treated on an individual basis by both the central administration and the hospitals. Although the HCAT-DK offers a new way to classify and analyze these cases, it is not the intention to replace the individual management of complaint cases. The merit of the HCAT tool is to help identify trends and areas within health care that could benefit from analysis across settings and departments. This would be a major contribution to the detection of both blind spots and hot spots in the complaints field, thus helping to improve the quality of health care.¹⁷

The HCAT approach has implications for other countries and health care settings as well. To be able to classify and analyze complaint cases, there needs to be centralized reporting, registration and identification of these cases. This might be challenging in some health care settings, eg, private institutions or in countries without central registers. To our knowledge, this is the first translation and adaption of the HCAT tool to non-English language countries. Our experiences suggest that translated and slightly modified HCAT versions may serve as templates for future analyses in multiple countries, thus allowing comparative studies and knowledge exchange.

Conclusion

In this study, we translated and cross-culturally adapted the English version of the HCAT to produce a Danish HCAT version, the HCAT-DK. Cultural and conceptual differences between the English and Danish health care settings led to the addition of two extra items of “involvement of the patient’s relatives” and “hospital-acquired infection” in the HCAT-DK. The Danish version was shown to have good intra- and inter-rater reliability and is recommended for extracting and coding information from complaint and compensation cases in Danish health care settings.

Disclosure

The authors report no conflicts of interest in this work.