Validation of the French Version of the Auditory Hallucination Rating Scale in a Sample of Hallucinating Patients with Schizophrenia: Validation de la version française de l’échelle d’évaluation des hallucinations auditives dans un échantillon de patients souffrant de schizophrénie et ayant des hallucinations

Abstract

Objective:

The aim of this study was to validate the French version of the 7-item Auditory Hallucination Rating Scale (AHRS) so as to facilitate fine-grained assessment of auditory hallucinations (AH) in native French-speaking patients with schizophrenia (SZ) in clinical settings and studies.

Method:

Patients (N = 66) were diagnosed with SZ according to the Diagnostic and Statistical Manual of Mental Disorders. The French version of the AHRS was developed using a forward–backward translation procedure. Psychometric properties of the French version of the AHRS were tested including (i) construct validity with a confirmatory one-factor analysis, (ii) internal validity with Pearson correlations and Cronbach α coefficients, and (iii) external validity by correlations with the Scale for Assessment of Positive Symptoms (SAPS-H1), the Positive and Negative Syndrome Scale (PANSS-P3; concurrent), the PANSS-Negative subscale and age of subjects (divergent), and inter-rater intraclass correlation coefficients (ICCs).

Results:

(i) The confirmatory one-factor analysis found a root mean square error of approximation (RMSEA) = 0.00, 90% confidence interval = [0.000 to 0.011], and a comparative fit index = 0.994. (ii) Correlations between AHRS total score and individual items were mostly ≥0.4. Cronbach α coefficient was 0.61. (iii) Correlations with PANSS-P3 and SAPS-H1 were 0.42 and 0.53, respectively. In a subset of participants (N = 16), ICC values were extremely high and significant for AHRS total and individual item scores (ICCs range 0.899 to 0.996)

Conclusion:

The French version of the AHRS is a psychometrically acceptable instrument for the evaluation of AH severity in French-speaking patients with SZ.

Introduction

Schizophrenia (SZ) is a major psychiatric disorder that affects 1% of the population worldwide.¹ Despite markedly divergent clinical courses across patients, three core features are classically described: positive symptoms (e.g., hallucinations, delusions), negative symptoms (e.g., avolition, flattened affect, reduction in spontaneous speech, social withdrawal), and impairments in cognitive functions (e.g., working memory, social kills).² Among positive symptoms, auditory verbal hallucinations (AH) are experienced by 60% to 80% of all patients diagnosed with SZ.³ AH are often associated with high levels of distress, behavioral dyscontrol and altered quality of life. Moreover, AH resist to traditional antipsychotic medications in 25% to 30% cases,⁴ which places this symptom among the most burdensome for these people. Therefore, it is critical to validate reliable tools as pertains to fine-grained assessment and monitoring of AH in clinical settings and clinical studies.

To date, AH and other positive symptoms can be evaluated with more than 10 standardized psychometric scales (for advantages and disadvantages of each scale, see references^5–7). However, evidence of change in clinical trials is available for only half of them (rev. in⁵). Another limitation is the lack of structured clinical rating scales as the vast majority of hallucinations scales assess symptoms in a self-report format.⁵ Self-report measure may be influenced by cognitive dysfunction, through limiting the ability to concentrate, synthesize mental content related to the AH, and make abstract judgments as part of the rating process.⁸ Finally, the impact of AH on the individual, including both influence on behavior and induced distress, is not sufficiently assessed by measures that are currently available. Given that these dimensions are of considerable clinical importance, the validation of a measure investigating these features associated with AH would provide an important variable for outcome assessment.

The 7-item Auditory Hallucination Rating Scale (AHRS) is an English-language structured tool developed by Hoffman and colleagues to measure the effect of repetitive transcranial magnetic stimulation in hallucinating patients with SZ.⁹ Its format and fine-grained measures of AH dimensions may address some of the abovementioned issues about AH scales. The AHRS presents as a brief, structured rating scale that generates detailed description of AH for the last 24-hr time period. The measured features are (i) frequency, (ii) vividness, (iii) loudness, (iv) length (single words, sentences, phrases, or extended discourse), (v) attentional salience (the degree to which hallucinations capture attention and alter ongoing thought and behavior), (vi) degree of distress associated with the AH, and (vii) number of distinct speaking voices. AHRS reliability was investigated across 50 patients with SZ involved in a repetitive transcranial magnetic stimulation trial.⁹ Authors demonstrate acceptable internal consistency (Cronbach’s α = 0.60), inter-rater reliability (intraclass correlation coefficient [ICC] ranging from 0.80 to 0.98), and test–retest reliability (individual items ICCs > 0.70 except for vividness 0.51 and length 0.57). As compared to broadly used instruments such as the Positive and Negative Syndrome Scale (PANSS) and the Scale for Assessment of Positive Symptoms (SAPS), the AHRS present the advantage of focusing only on the auditory modality. The AHRS distinguishes itself from other measures for AH, as besides the severity and qualitative characteristics of AH, it also assesses the number and attentional salience of voices (in contrast to the Psychotic Symptom Rating Scales [PSYRATS]¹⁰) as well as the form of voices (in contrast to the BAVQ-R¹¹). Among AH scales, AHRS has the highest inter-rater reliability and is one of the shortest assessment available. In addition, evidence of change in noninvasive brain stimulation trials has been repeatedly demonstrated for the original English AHRS version by several teams around the world^9,12–15 (in contrast to the Auditory Vocal Hallucination Rating Scale Questionnaire⁷ and Psycho-Sensory hAllucinations Scale [PSAS]⁶).

To date, several studies conducted in French-speaking clinical settings have been using a French-translated AHRS version as outcome and showed promising interest of its use for AH monitoring in patients with SZ.^16–23 However, this French version of the AHRS has not yet been psychometrically validated. The aim of this study was to validate the French version of the AHRS so as to facilitate fine-grained assessment of AH in native French-speaking patients with SZ in clinical settings and studies.

Methods

Participants

From April 2009 to January 2019, both in- and outpatients diagnosed with schizophrenia or schizoaffective disorder according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV or DSM-5) criteria and experiencing daily AH were recruited at a tertiary care facility (Centre Hospitalier Le Vinatier University Adult Psychiatry unit, Bron, France). These subjects were part of two clinical trials investigating the effect of transcranial electrical stimulation in treatment-resistant schizophrenia (NCT00870909 and NCT02744989). All measurements used in our study were performed at baseline by three different trained clinicians, in accordance with relevant guidelines and regulations. All raters were trained using video-based assessments followed by a detailed correction of their ratings.

All patients were interviewed with the French version of the Mini-International Neuropsychiatric Interview (MINI²⁴) to assess the presence of AH symptoms and diagnosis of SZ. The MINI is a reliable tool to diagnose SZ with strong validity according to DSM criteria. Additionally, all patients completed a battery of structured clinical interviews in French versions. The SAPS²⁵ assessed the presence of positive symptoms through four subscales (7 items, “Hallucinations”; 13 items, “Delusions”; 5 items, “Bizarre Behavior”; and 9 items, “Positive Formal Thought Disorder”), with items rated from 0 (none) to 5 (severe). The PANSS²⁶ assessed the presence of schizophrenia clinical dimensions through three subscales (7 items, “Positive Symptoms”; 16 items, “General Psychopathology”; and 7 items “Negative Symptoms”) with items rated from 1 (absence) to 7 (extreme). The SAPS has one specific item for AH (SAPS-H1 “Auditory Hallucinations”), while the PANSS has one specific item for hallucinations in general (PANSS-P3 “Hallucinatory Behavior”).

French Version of the AHRS

To conduct this study, we used the original English AHRS version freely given by the author on request and published by Hoffman and colleagues in the Archives of General Psychiatry in 2003.¹² The French version of the AHRS was developed using a forward–backward translation procedure. The forward step involved two independent translators who translated the English into French. The backward step involved a third translator who retrotranslated the French version back into English that was validated by Hoffman. The final French version was elaborated after resolution of the discrepancies between the original and the retrotranslated versions. A team of clinicians and scientific methodologists consensually rated and approved the final French version. This version is presented in the Supplementary Material S1 along with its rating manual.

The original French version of the AHRS includes 7 itemized questions on a 5- to 9-point Likert-type scale, ranking from lowest (0) to highest intensity (5, 6, 7, or 9 depending on individual item) of the AH feature. The same scoring method was used as in the original paper presenting the AHRS.¹² The mean of the item responses was calculated for each of the seven AH features: AH1—frequency (0 to 9), AH2—vividness (0 to 5), AH3—loudness (0 to 5), AH4—length (0 to 5), AH5—attention disturbance (0 to 7), AH6—distress (0 to 5), and AH7—number of voices (0 to 6). The mean time of AHRS assessment was 10 to 15 minutes.

Analyses

Descriptive statistics of the obtained data included means and standard deviations of continuous variables. For the scale validation process, we analyzed the psychometric properties of the AHRS French version including three major criteria (i) construct validity, (ii) internal validity, and (iii) external validity. Significance threshold was set at P < 0.05. Analyses were performed using R software version 3.3.3.²⁷

Construct validity

We performed a confirmatory factor analysis (CFA) to assess the construct validity and the one-factor structure of the AHRS. As AHRS measures several features of a unique symptom (AH), we posit that all items will fit in a single factor. Structural equation modeling (Lavaan package) was used with indicators of acceptability of comparative fit index (CFI) > 0.90²⁸ and root mean square error of approximation (RMSEA) < 0.08.²⁹

Internal validity

1. Internal structural validity was investigated by the internal item consistency (IIC) of the AHRS. IIC is reflected by correlations between the scale items and the factor that they are hypothesized to represent. Correlation indices ≥0.4 are recommended for supporting IIC.³⁰ Associations between each AHRS item score and the total (i.e., one-factor) score were conducted using Spearman nonparametric testing (r _s).

2. Internal consistency reliability was assessed by Cronbach α coefficient (umx package). To reach acceptable internal consistency, a coefficient ≥0.70 was expected.³¹ In addition, we recalculated the coefficient after deletion of each item. It is recommended that new coefficients should not be increased by items deletion.³² As response option differs across AHRS items, we used standardized Cronbach’s α. Finally, we examined the distribution of the clinical severity by reporting floor and ceiling effects for all items (i.e., proportion of individuals who obtained the lowest and the highest scores at each item, respectively).

External validity

1. External validity. To explore the concurrent external validity, the correlations between AHRS items and total scores and hallucination items of the SAPS (SAPS-H1) and PANSS (PANSS-P3) were measured using Spearman r _s coefficients, with the ranges of <0.3, 0.3 to 0.5, and >0.5 corresponding to small, moderate, and large correlation, respectively. To explore the divergent external validity, the correlations between AHRS items and total scores and PANSS-Negative subscale score and age were measured.

2. Inter-rater reliability. The inter-rater reliability was explored in a subset of participants who were assessed with AHRS the same day by two different clinicians during the same interview, without therapeutic intervention between measures. ICC (absolute agreement) was computed (ICC Package).

Results

Sample Characteristics

A total of 66 native French-speaking patients with a current diagnosis of SZ were included. None reported any difficulties in understanding the individual items of the AHRS.

The mean age was 36.08 ± 9.36 years old (range 18 to 58). Thirty-five (53.03%) were male. The mean level of education was 11.80 ± 2.73 years. Means for duration of illness and chlorpromazine equivalents were 11.80 ± 7.89 years and 938.79 ± 763.08 mg/day (https://psychopharmacopeia.com/antipsychotic_conversion.php, accessed 2019 Dec 7), respectively. Intensity of positive symptoms was assessed with PANSS-Positive subscale (20.08 ± 4.83) and SAPS scale (grand total: 48.95 ± 22.10, Hallucinations subscale total: 12.45 ± 4.94). Other symptoms were measured with the PANSS-Negative (20.21 ± 6.02) and PANSS-General Psychopathology subscales (35.17 ± 8.32). Mean scores for AH were 5.47 ± 1.18 (PANSS-P3) and 4.26 ± 1.04 (SAPS-H1). Demographic and clinical data are detailed in Table 1.

Table 1.

Demographic and Clinical Data.

Variables	N (% of Missing Values)	Mean (SD)	Median (Min to Max)
Demographics
Age (years)	66 (0.0)	36.08 (9.36)	36.50 (18 to 58)
Sex	66 (0.0)
Male		35 (53.03%)a
Female		31 (46.97%)a
Education (years)	66 (0.0)	11.80 (2.74)	12 (14 to 19)
Chlorpromazine equivalents (mg/j)	29 (56.1)	938.79 (763.08)	800 (200 to 3,600)
Clinical features
Illness duration (years)	61 (7.6)	11.80 (7.89)	11 (1 to 30)
PANSS (total)	66 (0.0)	75.45 (15.54)	73.5 (45 to 113)
Positive	66 (0.0)	20.08 (4.83)	19.5 (12 to 33)
Negative	66 (0.0)	20.21 (6.02)	20.5 (9 to 33)
General psychopathology	66 (0.0)	35.17 (8.32)	34.5 (18 to 58)
SAPS (total)	56 (15.2)	48.95 (22.10)	46 (14 to 108)
SAPS-Hallucinations	56 (15.2)	12.45 (4.94)	12 (0 to 25)
Auditory hallucinations
PANSS-P3	66 (0.0)	5.47 (1.18)	6 (2 to 7)
SAPS-H1	56 (15.2)	4.26 (1.04)	5 (1 to 5)

Note. PANSS = Positive and Negative Syndrome Scale; SAPS = Scale for Assessment of Positive Symptoms.

^a Dichotomous variables are expressed as number and percentages.

Psychometric Validity

Construct validity

Statistical indicators generated by the CFA showed a good fit for the one-factor model: RMSEA < 0.001 (90% confidence interval = [< 0.001 to 0.113]); CFI = 0.994. Standardized estimates of each AHRS item associated with the CFA are reported in Supplementary Material S2.

Internal validity

1. Internal structural validity. IIC was satisfactory for the one-factor, each of the 7 item achieving the 0.40 standard threshold (r _s = 0.41 to 0.67), except for AH3—loudness (r _s = 0.32; Table 2).

2. Internal consistency reliability. Cronbach’s standardized α coefficient was 0.61 for the overall scale and was not significantly higher after individual items were removed. Floor and ceiling effects ranged from 3.03% to 25.8% and from 1.52% to 48.5%, respectively (Table 3).

Table 2.

Relationships between AHRS Individual Items Scores, Total Score, the PANSS-P3, the SAPS-H1, the PANSS-Negative Subscale and Age (Spearman Correlation Analysis).

AHRS items	AH1	AH2	AH3	AH4	AH5	AH6	AH7	PANSS-P3	SAPS-H1	PANSS-Negative	Age
AH1—frequency	—							.40**	.30*	−.22	.02
AH2—vividness	.17	—						.15	.27*	−.24*	.14
AH3—loudness	−.15	.34*	—					.15	.24	−.11	.18
AH4—length	−.16	.18	.13	—				.21	.13	.09	.19
AH5—attention disturbance	.13	.22	.25*	.24	—			.11	.15	−.07	−.14
AH6—distress	.21	.25*	.28*	.07	.37**	—		.42*	.40**	−.21	−.04
AH7—number of voices	.14	.09	.09	.13	.11	.12	—	.31*	.17	−.18	−.02
AHRS (total)	.67**	.48**	.32*	.41**	.49**	.53**	.53**	.53**	.42**	−.28*	.12

Note. AHRS = Auditory Hallucination Rating Scale; PANSS = Positive and Negative Syndrome Scale; SAPS = Scale for Assessment of Positive Symptoms; AH = auditory hallucinations.

*P < 0.5. **P < 0.005.

Table 3.

French AHRS Characteristics and Intraclass Correlation Coefficient (ICC) for Interjudge Test–Retest Reliability.

AHRS Item	Mean (SD)	Median (Min to Max)	Floor (%)	Ceiling (%)	ICC [95% IC]
AH1—frequency	4.89 (2.86)	5 (1 to 9)	18.2	19.4	0.962 [0.891 to 0.987]
AH2—reality	4.03 (1.30)	4 (0 to 5)	3.03	48.5	0.899 [0.710 to 0.965]
AH3—loudness	2.79 (1.20)	3 (1 to 5)	19.7	12.1	0.955 [0.872 to 0.984]
AH4—length	3.17 (1.18)	3 (0 to 5)	3.03	12.1	0.971 [0.918 to 0.990]
AH5—attention disturbance	4.29 (1.29)	4 (1 to 7)	3.03	1.52	0.930 [0.800 to 0.976]
AH6—distress	3.52 (1.39)	4 (1 to 6)	10.6	1.52	0.936 [0.816 to 0.978]
AH7—number of voices	3.33 (1.89)	3 (1 to 6)	25.8	21.2	0.996 [0.989 to 0.999]
AHRS (total)	26.02 (6.08)	27 (12 to 37)	4.54	3.12	0.985 [0.958 to 0.995]

Note. N = 66 for French AHRS characteristics. N = 16 for Interjudge test–retest reliability. AHRS = Auditory Hallucination Rating Scale; AH = auditory hallucinations.

External validity

1. Concurrent external validity. The correlations between the AHRS total and item scores and SAPS-H1 and PANSS-P3 scores were heterogeneous. Only AHRS total, AH1—frequency, AH6—distress and AH7—number of voices items were significantly correlated with both SAPS-H1 and PANSS-P3 (see Table 2). Largest correlations were observed between AHRS total—PANSS-P3 (r _s = 0.42) and AHRS total—SAPS-H1 (r _s = 0.53). Divergent external validity: The correlations were only significant between AH2—vividness, AHRS total, and PANSS-Negative scores with small effect sizes. No significant correlations were observed for age (Table 2).

2. Inter-rater reliability. A subset of subjects (N = 16) participated in AHRS assessments by two clinicians (i.e., two measurements). ICC values were extremely high and significant for AHRS total score (ICC = 0.985, IC95% [0.958 to 0.995]) and individual item scores (ICCs range 0.899 to 0.996; Table 3). Moreover, the mean % variability between AHRS total score measures was low (0.91 ± 7.64%).

Discussion

The main objective of this study was to validate the French version of the AHRS for assessing AH in French-speaking patients with SZ in the context of clinical practice. The psychometric evaluation was performed in a sample of 66 patients with SZ. Overall, the results reached several acceptability thresholds for construct, internal and external validity of the scale. To the best of our knowledge, this is the first study that has provided exhaustive psychometric validity evidence for the use of the French AHRS in French-speaking settings.

Summary of Major Findings

The mean scores of the AHRS individual items were above 3, which relate to a significant clinical intensity of AH symptoms.¹⁹ As our sample was refractory to antipsychotic treatment, it is likely that the ceiling effect might have been inflated. The CFA for construct validity demonstrated that all the items fit in a one-factor model as hypothesized. That is, conversely to the majority of available scales, which measures AH among other positive symptoms, the AHRS allows assessment of a unique symptom within the positive dimension. Therefore, the AHRS might be useful for clinicians to obtain fine-grained and specific information on AH outcomes, especially in trials that primarily target this symptom in SZ. Among hallucination scales, only the factor structure of the PSYRATS has been investigated for AH items, but number and content of factors remain heterogeneous across studies.³³

Internal structural validity analyses demonstrated satisfactory correlation levels between AHRS individual items and total, with the exception of AH3—loudness. This level of structural validity was not demonstrated in other French-validated tools that directly assess AH (i.e., PSYRATS and PSAS). Nevertheless, replication and further analyses with larger samples are well warranted to control for item overlap (i.e., estimating the correlations between 1 item and the sum of all other items in one domain).

The calculated Cronbach’s α of 0.61 corresponds to questionable level, so the internal consistency should be taken with caution.³⁴ However, when the number of items is small (below 10), Cronbach’s α value can be diminished in comparison to longer scales with similar internal consistency.³⁴ This value of internal consistency was comparable to that of the validation study of the original AHRS version (α ∼ 0.60),⁹ which suggests similar specific assessment of AH by both versions. Moreover, the AHRS presents the highest level of internal consistency in comparison to other French-validated scales.

We also provided evidence that the AHRS total score is significantly correlated with other commonly used French versions of validated scales that measure hallucinations with moderate-to-large effect sizes (i.e., PANSS-P3²⁶ and SAPS-H1²⁵), even though these scales assess symptoms within different time frames (24 hr for AHRS vs. 1 week for PANSS and SAPS). The correlation between AHRS total and age of participants was not significant, and the correlation between AHRS total and PANSS-Negative, although significant, was small. Previous correlation measures between PANSS-P3 and French PSYRATS³³ and SAPS⁶ total scores yielded similar effect sizes (r _s ∼ 0.42), while the divergent validity has never been measured before. Taken together, these results revealed a fair level of external validity of the AHRS.

Finally, similarly to the original scale validation study,⁹ inter-rater reliability of the scale as measured by the ICCs was extremely high for both total and individual item scores (ICCs range 0.899 to 0.996), and comparable to the French PSYRATS³³ (ICC = 0.90 to 1.00) and PSAS⁶ (ICC = 0.78 to 1.00) versions. This result demonstrates accuracy and consistency in the assessment of AH by the French AHRS version.

Relevance of AHRS for Clinical Use

Only a minority of AHRS individual items were significantly correlated with other items measuring AH (PANSS-P3 and SAPS-H1). This might also be explained by previous evidence showing that factors determining AH severity (i.e., measured by the 7 individual items) vary considerably across patients.^9,13 Therefore, trials using AHRS should rather use its total score instead of individual item scores for a suitable outcome measure. Alternatively, Hoffman and colleagues used the AHRS-related “Hallucination Change Scale” (HCS) as primary outcome to assess changes in hallucinations severity in trials rather than the AHRS total score.¹² To evaluate the HCS score at baseline, participants are required to describe their AH for the last 24-hr time period, using the narrative of the AHRS, which is assigned a score of 10. The HCS is scored on subsequent evaluations by requesting the participant to generate a new narrative description of the AH. Thus, follow-up severity scores range from 0, corresponding to no hallucinations, to a maximum score of 20, corresponding to hallucinations twice as severe as baseline. Another explanation for nonsignificant correlations between AHRS and PANSS-P3/SAPS-H1 items is their low specificity for hallucinations measurement. Indeed, PANSS-P3 quantifies hallucinations regardless of the sensory modality,²⁶ while SAPS-H1 measures AH that can be voices or sounds,²⁵ conversely to AHRS items that only assess voices. Nevertheless, AHRS individual items remain useful to investigate precise phenomenological features of AH and their respective neural correlates.³⁵ Further studies aiming at testing concurrent validity of individual items should use more detailed instruments such as the PSYRATS.¹⁰

A major caveat of the AHRS scale is the relationship between individual items and total scores. For instance, a specific phenomenological feature might be increased and another decreased after treatment, which would yield a same total AHRS scores but distinct experiences of symptoms. The clinician should be aware of which features induce more distress in a specific patient. Although the 7-item AHRS provides a fine-grained assessment of AH, the number of feature items remains low, whereas others allow measurement of further features such as voices source location, controllability, associated beliefs (PSYRATS¹⁰), and behavioral expression of AH (PSAS⁶). Furthermore, the weight of AH features varies across items (5 to 9 points), which limit the possibility of homogeneous comparisons between individual ratings of items. Finally, it is worth noting that the 24-hr time period of the AHRS assessment might induce bias in the evaluation of nondaily hallucinated patients. Hence, the AHRS may not capture patients who alternate from multiple and severe daily episodes of AH to no AH during several days. Nevertheless, since SZ patients display consistent deficits in episodic memory,⁸ a longer time period would increase the risk of mistakes in recalling information about experienced AH.

Limitations

Several limitations should be acknowledged when interpreting the results of this validation study. First, our sample was recruited from a single center in a tertiary care hospital and involved patients with more severe symptoms and impairment. Hence, the validation of the French AHRS cannot be generalized to other groups of individuals who present with AH such as first-episode psychosis. Future validation studies should be conducted in more diverse health-care settings. Second, the overall sample size was relatively small, which prevailed us from testing for invariance of measurements across relevant variables (e.g., age, gender, illness duration, and treatment). In addition, the small subgroup of participants involved in the inter-rater reliability analyses (n = 16) limits the robustness of the intraclass correlation indices and warrants further studies in larger populations to investigate external validity of the scale.³⁶ Third, correlations between AHRS and functional impairments could be examined to further explore the impact of AH on “real-life” outcomes.

Conclusion

In sum, the French version of the AHRS is a psychometrically acceptable instrument for the evaluation of AH severity in patients with SZ. This scale, which allows a short fine-grained assessment of several AH phenomenological features and impact, can be incorporated into clinical evaluation of French-speaking SZ patients.

Future work is needed to validate the scale in broader and larger SZ samples and to assess sensitivity to change after pharmaceutical or psychological interventions that specifically target AH in these people. Finally, availability of the AHRS in additional languages is warranted to facilitate large-scale studies investigating AH over multiple countries.