The Application of Pupillometry in Hearing Science to Assess Listening Effort

Abstract

In recent years, the fields of Audiology and Cognitive Sciences have seen a burgeoning of research focusing on the assessment of the effort required during listening. Among approaches to this question, the pupil dilation response has shown to be an informative nonvolitional indicator of cognitive processing during listening. Currently, pupillometry is applied in laboratories throughout the world to assess how listening effort is influenced by various relevant factors, such as hearing loss, signal processing algorithms, cochlear implant rehabilitation, cognitive abilities, language competency, and daily-life hearing disability. The aim of this special issue is to provide an overview of the state of the art in research applying pupillometry, guidance for those considering embarking on pupillometry studies, and to illustrate the diverse ways in which it can be used to answer—and raise—pertinent research questions.

It has always been known that measures of raw performance during a speech-in-noise task, be they percent-correct at a given signal-to-noise ratio (SNR) or SNR for a criterion percent-correct, are imperfect indicators of hearing (dis)ability or hearing intervention benefit. Over time, our understanding of daily-life verbal communication and hearing disability has improved, and their multifaceted natures have become increasingly apparent. In parallel, hearing device technology has also improved, but as the “easy wins” of better controlled audibility are gradually accomplished, any further increments in user benefit become harder to demonstrate via raw speech-in-noise performance in idealized laboratory tests. Thus, it is increasingly untenable to rely exclusively on test methodologies whose only outcome is a performance measure.

One supplementary outcome domain which has attracted increasing attention in recent years is typically labeled “listening effort” (McGarrigle et al., 2014; Pichora-Fuller et al., 2016). It turns out that listening effort itself is a multifaceted concept, in which observations in the domains of self-report, behavior, and physiology all illuminate different but related aspects (McGarrigle et al., 2014). While the term “effort” is generally associated with conscious processes (Pichora-Fuller et al., 2016), it remains to be resolved whether there are unconscious processes which also deserve the term (Strauss & Francis, 2017). For the purposes of this Special Issue, “effort” is to be understood as the allocation of mental resources, whether consciously or unconsciously done, and regardless of whether a test participant would self-report an expenditure of effort.

In addition, the assessment of listening effort provides more insight into the interaction between bottom-up (sensory) processes and top-down (cognitive) processes, and thereby also complements performance-based measures. As such, besides the relatively applied research to the benefit obtained from hearing rehabilitation techniques, it gives a means to contribute to more fundamental research questions addressing auditory processing. Focusing on listening effort furthermore acknowledges that listeners with hearing impairment often experience difficulties that relate to increased listening effort and fatigue (Hornsby, 2013; McGarrigle et al., 2014; Nachtegaal et al., 2009; Wang et al., 2018).

Observation of the pupil diameter provides a relatively convenient window into internal processes relating to the allocation of mental resources (Beatty & Lucero-Wagoner, 2000). Furthermore, pupil size is not under direct volitional control, removing one source of potentially misleading biases. That is not to say that pupillometry is free of contextual confounds; on the contrary, the pupil response is sensitive to a diverse array of effects. Some of these might be thought of as relevant to our field of enquiry (e.g., sound level, motivation, properties of the tasks and stimuli, age), thus increasing the potential value of pupillometry. Others (e.g., luminance level, medication use, eye disease) are obstacles to measurement and interpretation. The panoply of variables to which the pupil is sensitive reflects a complex underlying network of activity in underlying systems (including the autonomic nervous system in general and especially the hypothalamic–pituitary–adrenal axis). While the pupil response can provide evidence of mental activity at a low experimental cost, it does not provide direct observation of that activity. Hence, there is a need to carry out validation studies in which pupillometry is compared against other imaging techniques. Also, more knowledge is needed regarding the pros and cons of the different statistical methods and procedures used to analyze pupillometric data.

The Ear & Hearing section of the Amsterdam UMC location VUmc has one of the longest track records of work involving pupillometry and listening effort. In 2015, and again in 2017, this group organized workshops to connect researchers experienced in such work to others who aspire to apply the techniques.

The 2017 workshop (for program, see http://www.ac-vumc.nl/onderzoek/workshop2017/index.htm) was attended by 33 researchers, of whom some were experienced in applying pupillometry to listening tasks, many were setting out in the field, and others were interested in either pursuing similar research questions by slightly different means or applying pupillometry to different research questions in audition. The aim of the workshop was to share expertise and to provide an overview of recent pupillometry research. Discussion focused on different (statistical) analysis techniques, the pupil response parameters extracted from the signal, and the interpretation of these parameters. Also, practical issues regarding the measurement of the pupil size were discussed. Established experts in the field were invited to give keynote talks on primary technical and conceptual themes, and newcomers submitted talks on their intentions and early experiences.

The workshop concluded with a session to collect views on topics for which summary overviews would be of benefit to aspiring researchers and to determine outstanding questions in need of further work. It was apparent that the community of aspirants and experts is now large enough that there is both a need for some guidance and sufficient collective experience to provide it. In addition, the specific topic of pupillometry and listening effort has now been explored sufficiently to reveal critical conceptual and methodological issues which require attention. For these reasons, it was decided to assemble contributions to make a special issue on the topic. The coauthors of this editorial issued a general invitation for papers related to the subject of the workshop, so that the resulting collection of work would not be limited to those authors who had attended the event itself. We also invited specific cross-institutional groupings of experts to combine their expertise to produce consensus or survey papers. The results are to be found in this issue, and we believe they represent a particularly useful contribution to the field.

The collection of papers in this special issue provides an overview of the state-of-the-art in research applying pupillometry and addresses specific issues of experimental apparatus and design, statistical analysis techniques, and interpretation of parameters extracted from the pupil signal in the context of relevant theoretical models. In addition, a few of the papers consider alternative approaches to understanding the nature of listening effort itself, thus ensuring that this is not taken for granted. The overall ambition is that by bringing together a significant number of articles on closely related topics, including work by leading researchers in this field, this special issue will rapidly become a compact and popular reference source of knowledge for those taking new or further steps in this burgeoning research area.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Coauthor GN was funded by UK Medical Research Council grant MR/S003576/1 and by a grant from the Chief Scientist Office of the Scottish Health Directorate. Coauthor TK was supported by Oticon Fonden (Foundation) grant 16-0463. This work also received funding from the European Union’s FP7 Research and Innovation funding program under grant agreement No. 607373 (LISTEN).