Validity, Internal Consistency, and Test-retest Reliability of the Montenegrin 10-item Voice Handicap Index

Abstract

The purpose of the this cross-sectional study was to assess clinical validity, internal consistency, and test-retest reliability of the adapted Montenegrin translation of the Voice Handicap Index 10 (VHI-10). It included 50 patients with voice disorders, divided into three subgroups according disease aetiology: structural, neurological, and functional and a control group of 50 vocally healthy participants. Mean patient VHI-10 score of 21.1±7.6 was significantly higher than the 2.3±2.5 score of controls (p<0.001). Each of the three patient subgroups also scored significantly higher than control (p<0.001). Spearman’s rank correlation coefficient of 0.90 (p<0.001) indicated a very strong correlation between the Montenegrin VHI-10 score and self-reported perception of the severity of voice disorder. Excellent internal consistency was found in the patient group, with a Cronbach’s alpha of 0.94. Test-retest reliability was also excellent, with intra class correlation coefficient of 0.98. The translated Montenegrin version of VHI-10 is a valid, reliable, and clinically useful tool for self-assessment of the severity of voice disorders in individuals with voice problems in daily practice and in research projects.

About 10 % of the general population presents with voice disorders, and this percentage reaches 50 % among voice professionals (1). Voice disorders can have a significant impact on the quality of life (QoL), but clinical impact evaluation can be quite challenging, as it should involve instrumental, aerodynamic, and sensory measurements, as well as patient’s self-perceived severity of the voice problem. Generic, non-voice-specific health questionnaires such as the Short Form Health Survey (SF-36) cannot measure specific characteristics of patients with dysphonia (2, 3). Various voice disorder-specific instruments have been developed to measure QoL in dysphonic patients, including the Voice Outcome Survey (VOS) (4), Voice-Related Quality of Life (V-RQOL) (5), Voice Activity and Participation Profile (VAPP) (6), Voice Symptom Scale (VoiSS) (3), and Voice Handicap Index 30 (VHI-30), developed by Jacobsen et al. (7). VHI-30 is a self-reported questionnaire consisting of 30 items to cover three aspects (functional, physical, and emotional) of perceived handicap in daily life owed to voice problems (7). It has been validated and translated into multiple languages, including German, European Portuguese, Polish, Hebrew, Italian, Greek, Arabic, Brazilian Portuguese, Croatian, and Serbian (8, 9, 10, 11, 12, 13, 14, 15, 16, 17). Later, the VHI-30 version was shortened by Rosen et al. (18) to 10 most clinically relevant questions (VHI-10) to facilitate application for “busy clinicians”, which has been translated and/or adapted into languages such as Arabic, Brazilian Portuguese, Chinese, Danish, Hebrew, Italian, Serbian, Spanish, and Urdu, and validated for internal consistency and test-retest reliability (19, 20, 21, 22, 23, 24, 25, 26, 27). The Montenegrin translation of the original English version of VHI-10, involved a bilingual translator and a health professional who is a native Montenegrin speaker, which was then evaluated by a bilingual expert panel of three otorhinolaryngologists. A revised version was back-translated into English and compared with the original English VHI-10 by a qualified independent translator, whose native language is English and who had no prior knowledge of the questionnaire. The proposed final version was then pilot-tested with 10 patients with dysphonia, all native Montenegrin speakers, and amended according to their suggestions. The final result was a culturally modified Montenegrin VHI-10 (Table 1).

Table 1.

The Montenegrin version of the Voice Handicap Index-10 (VHI-10)

Indeks glasovnog oštećenja (VHI-10) Instrukcije: Naredne izjave se često koriste u opisivanju uticaja izmjene glasa na kvalitet života. Molimo Vas da ih pažljivo pročitate i potom zaokružite odgovor koji pakazuje njihovu učestalost u Vašem životu. 0-nikad; 1-skoro nikad; 2-ponekad; 3-skoro uvijek; 4-uvijek
F1	Ljudi me slabo čuju zbog mog glasa.	0	1	2	3	4
F2	Ljudi me teško razumiju u bučnoj sredini.	0	1	2	3	4
F8	Problemi sa glasom ograničavaju moje lične i društvene aktivnosti.	0	1	2	3	4
F9	Osjećam da sam isključen/ -a iz razgovora zbog problema sa glasom.	0	1	2	3	4
F10	Manje zarađujem zbog problema sa glasom.	0	1	2	3	4
P3	Ljudi me pitaju: "Šta nije u redu sa tvojim glasom?"	0	1	2	3	4
P5	Osjećam napor kada hoću da govorim.	0	1	2	3	4
P6	Ne mogu da predvidim jasnoću mog glasa.	0	1	2	3	4
E4	Problem sa mojim glasom me uznemirava.	0	1	2	3	4
E6	Osjećam se hendikepirano zbog svog glasa.	0	1	2	3	4
Ukupno:_____

The aim of this study was to evaluate its validity and consistency for application in vocally healthy and dysphonic Montenegrin population.

Participants and methods

The study was approved by the Ethics Committee of the Clinical Centre of Montenegro, Podgorica, Montenegro (approval number: 03/01-5067) and carried out in compliance with the Declaration of Helsinki. All study participants signed an informed consent before participation, and data were collected in an anonymised database.

To assess the validity and reliability of the Montenegrin VHI-10, we enrolled two groups of participants in 2021 and 2022. The first group consisted of 50 patients (10 men and 40 women) aged 24–63 years (mean age 48.26±9.13 years) who came to our clinic for initial ear, nose, and throat examination over voice-related complaints. Their voice disorders were then diagnosed by an otorhinolaryngologist and the patients divided into three subgroups according to the aetiology of their disorder: neurological (e.g. vocal fold paralysis; n=5), functional (e.g. hypokinetic dysphonia and muscle tension dysphonia; n=13), or structural (e.g. intracordal cyst, polyp, nodules, papillomas, vocal fold atrophy, Reinke’ s oedema, phono trauma, leukoplakia, and early laryngeal cancer; n=32). We enrolled only those patients whose voice disorders were expected to continue for the following two to four weeks.

The second group consisted of 50 matching vocally healthy controls to eliminate the influence of demographics (10 men and 40 women aged 25–61 years, mean age 47.12±9.03 years) recruited from hospital staff, students, and patients at the otorhinolaryngology department who had no voice complaints or disorders, gastric reflux, sleep apnoea, swallowing disorder, allergies, asthma, or chronic cough. We excluded candidates with a history of voice disorders requiring treatment or of self-reported voice complaints, save for the history of mild flu or cold.

All participants self-completed the Montenegrin VHI-10 questionnaire in no longer than 3 min.

Instrument validation

The validation of the questionnaire’s psychometric properties was carried out in accordance with the guidelines proposed by the Scientific Advisory Committee of the US-based Medical Outcomes Trust (28). It involved comparison with another instrument, in this case a single-item questionnaire in which all participants self-assessed their dysphonia by scoring it on a 7-point Likert scale ranging from none (1 point) to severe (7 points). Scores were then grouped into three levels of voice disorders: none or mild (1−3 points), moderate (4−5 points), and severe (6−7 points). The VHI-10 questionnaire was validated by correlating its total score with the single-item self-assessment of dysphonia by means of Spearman’s correlation analysis. Correlation was considered strong with coefficients of 0.60−0.80 and very strong with coefficients >0.80.

The ability of VHI-10 to distinguish between the three aetiological subgroups of patients with voice disorders and controls was assessed by the Kruskal-Wallis test.

Internal consistency

The internal consistency (correlation between items measuring the same thing in a questionnaire) of the Montenegrin VHI-10 was estimated with Cronbach’s alpha. Coefficients greater than 0.7 were considered as “satisfactory”, greater than 0.8 “good’’, and greater than 0.9 “excellent” (29).

Test-retest reliability

Test-retest reliability of the Montenegrin VHI-10 questionnaire was assessed by repeating the questionnaire in 10 randomly selected patients and 10 controls two weeks later. It was measured using the intraclass correlation coefficient (ICC) for the total score (30). The ICC of ≤0.50 points to poor, of 0.51−0.74 to moderate, 0.76−0.9 to good, and >0.9 to excellent test-retest reliability.

Statistical analysis

The level of significance was set to 0.001. For all analyses we used the Statistical Package for the Social Sciences, version 18 (SPSS Inc., Chicago, IL, USA).

Results and discussion

Table 2 shows answers to the single-item assessment of vocal disorder and VHI-10 scores. Spearman’s rank correlation coefficient of 0.90 (p<0.001) confirmed very strong correlation between the Montenegrin VHI-10 score and dysphonia assessed with the single-item questionnaire.

Table 2.

Scores of the Montenegrin Voice Handicap Index 10 (VHI-10) in relation to patient self-assessed severity of voice complaints

	Control group (n=50)		Patient group (n=50)
Disorder severity	None/Mild	None/Mild	Moderate	Severe	Overall
Self-assessed single-item score (N of participants/total)	50/50	15/50	24/50	11/50	50/50
VHI-10 score (mean±SD)	2.3±2.5	12.5±3.3	21.6±3.4	31.5±2.9	21.1±7.6
VHI-10 score range	0–10	8–18	17–28	27–37	8–37

SD – standard deviation

Mean VHI-10 score in the patient group and each subgroup was significantly higher than control (p<0.001) but did not differ significantly between the subgroups (Kruskal-Wallis test, p=0.31). Mean VHI score for patients with neurological disorders (n=5) was 25.4±3.6, for those with functional disorders (n=13) 20.5±7.9, and for those with structural disorders (n=32) 20.6±7.8. There were no statistically significant correlations between VHI-10 score and age (r=-0.04, p=0.79) or gender (r=0.23, p=0.11).

The internal consistency for the VHI-10 questionnaire was good to excellent (Table 3), and the ICC for total score was 0.98 in the patient group and 0.96 in the control group, which confirmed excellent test-retest reliability.

Table 3.

Internal consistency of the Montenegrin Voice Handicap Index-10 (VHI-10) in patient and control groups by item (Cronbach’s alpha)

	VHI-10	Q-1	Q-2	Q-3	Q-4	Q-5	Q-6	Q-7	Q-8	Q-9	Q-10
Patient group	0.94	0.94	0.93	0.93	0.92	0.94	0.92	0.93	0.92	0.92	0.93
Control group	0.85	0.81	0.81	0.83	0.83	0.83	0.82	0.87	0.83	0.85	0.85
All	0.97	0.96	0.96	0.96	0.96	0.97	0.96	0.96	0.96	0.94	0.97

Our findings confirm the validity of the Montenegrin VHI-10 version in line with normative values laid down by Arffa et al. (31) and is, in fact, well above its cut-off level (>11). Our results are also in agreement with findings reported for the original English version (18) and subsequent translations into, Arabic, Brazilian Portuguese, Chinese, Hebrew, Serbian, and Spanish (19, 20, 21, 22, 23, 26) inasmuch as they are good instruments to distinguish between normal and abnormal voice conditions. Furthermore, the highest mean VHI-10 score in our study was found in patients with neurological voice disorders, which confirms earlier reports (20, 21, 27). The strong internal consistency and excellent test-retest reliability of the Montenegrin VHI-10 also put it side by side with the Danish, Hebrew, Italian, and Serbian versions (21, 24, 26, 27).

The main limitation of our study is within-group heterogeneity. The sample size for test-retest reliability was low, considerably lower than the sample size used in the original study. Responsiveness effect was not evaluated.

In conclusion, the Montenegrin VHI-10 version has met all the requirements to serve as a quick and reliable instrument for screening patients with voice disorders.