Validity and Reliability Aspects of a Newly Developed Questionnaire for Auditory Localization

Dear Editor,

Recently, I read a published article written by Neelamegarajan et al. ^[1] entitled “Development of Questionnaire for Auditory Localization” with great interest. The authors should be commended for their effort in developing a questionnaire to specifically measure the essential aspects of auditory localization (rather than the general binaural hearing). Having a clinical questionnaire for documenting auditory localization deficits can be advantageous for screening and diagnostic purposes. Having said that, I would like to highlight some issues that might be worthy of consideration.

When developing a new questionnaire, the validity and reliability aspects must be properly addressed. The authors did perform a content validity task by considering the comments from the experts but this can be enhanced with the use of content validity index (CVI). By employing CVI, the content validity results can be presented in a more objective manner ^[2–4]. As such, a particular questionnaire is said to have good content validity if its item-level CVI is at least 0.78, which is rated by at least 3 experts ^[3]. Having good CVI values for this newly developed questionnaire would provide evidence for the robustness of its psychometric property.

Furthermore, I failed to see any reliability task conducted by the authors. Common internal consistency assessments such as Cronbach’s alpha and item-total correlation can be conveniently performed to provide the reliability information of this 5-point Likert scale questionnaire. In relation to this, I noticed that in comparison with other items, item A3 had the highest mean score (2.15). From this viewpoint, it is essential to conduct a reliability check to determine whether the questionnaire has achieved good internal consistency or item deletion/amendment is required prior to its administration.

To compare the total scores between the subsections, the authors performed the Friedman test followed by Wilcoxon signed rank test, which was statistically correct. As such, for group I, significant differences in the total scores were found for most of the subsection pairs. However, the use of the total score in this within-group analysis can be problematic as the number of items in each subsection is different. For example, a significant difference in the total score can easily be predicted when comparing TA (with 5 items) and TC (with only 2 items). From this viewpoint, using the average score, instead of total score, would produce more realistic study outcomes when performing the within-group comparisons.

To unveil the sensitivity and specificity of the questionnaire, the authors conducted receiver operating characteristic (ROC) analysis, which was a good step. In fact, the data obtained from ROC can be useful to provide evidence on the discriminant validity of the questionnaire. The area under curve (AUC) analysis found the questionnaire to be excellent (0.987) in separating normal and hearing-impaired groups and hence, excellent discriminant validity. However, further information is required, particularly on the rationale of choosing 42.5 as the cut-off point. Was it based on the Youden index? Once the cut-off point has been chosen, the sensitivity and specificity values should also be provided to equip readers with “full package” information. It is worth noting that if the data were heavily skewed, the conventional ROC approach is not recommended and some modifications are required, such as the use of partial AUC and false discovery rate (FDR) ^[5].

Finally, prior to its intended applications, conducting more validation studies can be useful to further unveil the psychometric property of this questionnaire. Nevertheless, the authors are to be congratulated for their effort in developing a specific questionnaire for auditory localization that is currently lacking in clinical settings.