Operationalization of the Brief ICF Core Set for Hearing Loss: An ICF-Based e-Intake Tool in Clinical Otology and Audiology Practice

Supplemental Digital Content is available in the text.

Abstract

Objectives:

According to the International Classification of Functioning, Disability and Health (ICF), functioning reflects the interplay between an individual’s body structures and functions, activities, participation, environmental, and personal factors. To be useful in clinical practice, these concepts need to be operationalized into a practical and integral instrument. The Brief ICF Core Set for Hearing Loss (CSHL) provides a minimum standard for the assessment of functioning in adults with hearing loss. The objective of the present study was to operationalize the Brief CSHL into a digital intake tool that could be used in the otology–audiology practice for adults with ear and hearing problems as part of their intake assessment.

Design:

A three-step approach was followed: (1) Selecting and formulating questionnaire items and response formats, using the 27 categories of the Brief CSHL as a basis. Additional categories were selected based on relevant literature and clinical expertise. Items were selected from existing, commonly used disease-specific questionnaires, generic questionnaires, or the WHO’s official descriptions of ICF categories. The response format was based on the existing item’s response categories or on the ICF qualifiers. (2) Carrying out an expert survey and a pilot study (using the three-step test interview. Relevant stakeholders and patients were asked to comment on the relevance, comprehensiveness, and comprehensibility of the items. Results were discussed in the project group, and items were modified based on consensus. (3) Integration of the intake tool into a computer-based system for use in clinical routine.

Results:

The Brief CSHL was operationalized into 62 items, clustered into six domains: (1) general information, including reason for visit, sociodemographic, and medical background; (2) general body functions; (3) ear and hearing structures and functions; (4) activities and participation (A&P); (5) environmental factors (EF); and (6) personal factors (mastery and coping). Based on stakeholders’ responses, the instructions of the items on A&P and EF were adapted. The three-step test interview showed that the tool had sufficient content validity but that some items on EF were redundant. Overall, the stakeholders and patients indicated that the intake tool was relevant and had a logical and clear structure. The tool was integrated in an online portal.

Conclusions:

In the current study, an ICF-based e-intake tool was developed that aims to screen self-reported functioning problems in adults with an ear/hearing problem. The relevance, comprehensiveness, and comprehensibility of the originally proposed item list was supported, although the stakeholder and patient feedback resulted into some changes of the tool on item-level. Ultimately, the functioning information obtained with the tool could be used to promote patient-centered ear and hearing care taking a biopsychosocial perspective into account.

INTRODUCTION

The consequences of ear and hearing problems are multifaceted and often go beyond the level of ear and auditory impairments in structures and functions: various aspects of functioning in daily life and general health can be negatively influenced. Examples are restrictions in social relationships, inability to perform work, and depressed mood (Bennett & Haggard 1999; Bennett et al. 2001; Pronk et al. 2013; Barker et al. 2017; Vas et al. 2017). Promoting, maintaining, and improving overall functioning from a holistic perspective, instead of applying a mere focus on impaired body structures and functions, are increasingly recognized as the primary target and point of departure in audiology, both in clinical practice and research (Boothroyd 2007; Hickson & Scarinci 2007; Danermark et al. 2013; Grenness et al. 2016; Meyer et al. 2016; Vas et al. 2017). To successfully assess the level of functioning of an individual with hearing problems, it is necessary to capture the whole spectrum of a person’s impairments, activity limitations, participation restrictions, and relevant contextual factors (Danermark et al. 2010). According to Hopfe et al. (2018), such a bio-psychosocial perspective would form a good basis for identifying all relevant aspects that should be addressed in the care pathway (WHO 2013; Hopfe et al. 2018). When applying such a bio-psychosocial approach to the general otology or audiology clinical practice, a challenging issue is the lack of a universal definition and an instrument describing functioning in a standardized way (Bentler & Kramer 2000; Danermark et al. 2010, 2013; Granberg 2015; van Leeuwen et al. 2017; Alfakir et al. 2019).

The International Classification of Functioning, Disability and Health (ICF) of the World Health Organization provides a comprehensive framework to describe functioning. It is based on a bio-psychosocial model of health (WHO 2001). According to the ICF, an individual’s level of functioning is the outcome of complex interactions between a health condition, body function and structures (emotional, cognitive, and physical functions and anatomy), activities (tasks and demands of life), participation (engagement in life situations), and contextual factors. Contextual factors are divided into environmental factors (EF) (e.g., physical, social and attitudinal elements that can act as a barrier to or facilitator of an individual’s functioning) and personal factors (potentially influencing how a disability is experienced such as gender, age, habits, lifestyle, coping styles) (WHO 2001). To make the ICF hearing specific, two ICF Core Sets for Hearing Loss (CSHL) were developed, a brief one and a comprehensive one (Danermark et al. 2013; Granberg 2015). CSHL are shortlists of ICF categories (covering body functions, body structures, activities, participation and EF) that are considered most relevant for describing the functioning of an adult with hearing loss. While the Brief ICF CSHL provides a minimum standard, the Comprehensive ICF CSHL is meant for multiprofessional comprehensive assessment (Danermark et al. 2013; Granberg 2015). The Core Sets were developed through a WHO-defined process including three phases: preparatory phase, phase I, and phase II. The preparatory phase consisted of four scientific studies. These were conducted to identify ICF categories that were considered relevant by three different stakeholder groups: (1) Researchers: A systematic literature review was performed on outcome measures used in research, including adults with hearing loss, and these outcome measures were linked to the ICF categories (Granberg et al. 2014a, b), (2) Experts: An internet-based international expert survey among hearing health professionals was performed (Granberg et al. 2014c), and (3) Patients: Qualitative focus group interviews with Dutch and South-African adult patients were organized (Granberg et al. 2014d). The information collected during the preparatory phase was presented at a consensus meeting (phase I), at which consensus was reached on the final set of ICF categories to be included in the CSHL (Danermark et al. 2013). Phase II is currently ongoing, aiming to validate and test the Core Sets in practice (Selb et al. 2015). As mentioned, the Core Sets provide a minimum standard to describe the typical spectrum of problems in functioning. This standard may be extended for any purpose stated, such as according to the needs of the specific setting (Bickenbach et al. 2012). In two previous studies, we examined the “overlap” between the content of the ICF CSHL and the intake documents used in the oto-audiology practices in the Netherlands and the United States (i.e., the percentage of CSHL categories included in the intake documentation). Both studies showed substantial overlap (50 to 100%), supporting the CSHL’s content validity (van Leeuwen et al. 2017; Alfakir et al. 2019). However, there was also partial “non-overlap”, especially in psychosocial topics, indicating that current intake procedures may not cover all aspects relevant to patients with ear and/or hearing problems (as indicated by the CSHL). In addition, the ICF’s category sleep function and various personal factors (currently not included in the CSHL) emerged from the intake documents as potentially relevant for functioning. This finding suggests that the CSHL may need to be expanded.

While the CSHL cover lists of aspects that would need to be considered to describe functioning, it is not known how this should be done. In other words, operationalization of the CSHL can take different forms. The aim of the current study was to operationalize the Brief CSHL into a tool to be used as an intake (admission) instrument for patients visiting the oto-audiology department. Given that an individual’s functioning is best assessed from the patient’s perspective (FDA 2009), we chose to operationalize the Brief CSHL into a self-reported diagnostic screening tool. This tool is further referred to as “ICF-based e-intake tool”. The goal of our tool is to use it to screen adults with ear and/or hearing problems (for simplicity, these are further indicated as: “ear and hearing problems”) to be able to identify the problems and environmental and personal factors that are relevant to their functioning. This screening will be done prior to their treatment, and is meant to support the intake procedure and subsequent treatment or intervention. Ultimately, by using the ICF-based e-intake tool in oto-audiology practice, we aim to support and enhance patient-centered care and shared decision-making by: (1) providing an overview of the patient’s responses (i.e., his/her “functioning profile”) both to the clinician and the patient before the intake appointment; (2) discussing the profile during the intake appointment, and (3) providing tailored follow-up actions or treatment opportunities within the tool. Figure 1 illustrates how we envisage incorporation of the intake tool may support patient-centered care planning for individuals with ear and hearing problems.

Fig. 1.

Logic model of the intake tool’s feedback mechanisms, in which an integral assessment of the patient is obtained. *Prefereably the patient completes the intake tool at home, at a time and moment of their own choosing. When this is not possible, the intake tool can be completed in the waiting room. ICF, International Classification of Functioning, Disability and Health. Modified from Greenhalgh et al. (2017).

The objective of this article is to describe the process of developing the self-reported intake tool. The development of a self-reported instrument usually comprises the following six steps: (1) definition and elaboration of the construct intended to be measured, (2) choice of measurement method, (3) selecting and formulating items, (4) choice of response formats, (5) content evaluation, 6) field-testing (De Vet et al. 2011). Steps 1–2 have been described above. This study focuses on steps 3–5. A mixed method design was used and included: the selection of appropriate items from a pool of existing, commonly used patient-reported outcome measures (PROMs), a formal decision-making process, and qualitative content assessments. In addition, the integration of the ICF-based e-intake tool in a computer-based system is described.

MATERIALS AND METHODS

Selecting and Formulating Items and Choice of Response Formats

A). Selection of Categories to Be Represented in the ICF-Based e-Intake Tool •

Additional categories to the Brief CSHL were selected based on our previous study (see van Leeuwen et al. 2017) and based on expertise of clinicians (i.e., experienced audiologist, ENT surgeon, and psychologist). The ICF categories of the Brief CSHL are provided in Appendix 1 in Supplemental Digital Content 1, http://links.lww.com/EANDH/A637.

B). Formulating Items for the Selected ICF Categories •

The method used to formulate items for the Core Set categories involved a formal decision-making and consensus process in the multidisciplinary project team consisting of an ENT surgeon, audiologist, psychologist, and researchers with relevant experience in oto-audiology research.

First, a pool of items was created by linking the items from existing questionnaires to the ICF categories of the Brief CSHL and the selected additional categories. Three sources were used to create an item pool: (1) existing ear and hearing questionnaires relevant for the field as shown by the review study by Granberg et al. (2014a). These were questionnaires available in the Dutch language; (2) additional questionnaires routinely used in Dutch clinical oto-audiology practices; and (3) general functioning questionnaires based on the concepts of the ICF (e.g., WHO Disability Assessment Schedule 2.0 [WHODAS 2.0]; Üstün et al. 2010), World Health Survey [WHS]; WHO 2012). This item pool was used to select specific items that were considered appropriate to screen the ICF categories. Each member of the project team evaluated and indicated the relevance (yes, no) of each item and provided additional comments to justify their choice (Phase A).

Second, the results of Phase A were discussed in various meetings until consensus was reached about operationalization of each ICF category. New items were created in cases where existing items could not be linked to the particular category, or where they were considered unsuitable. For the formulations of particular constructs of these items, we used the official descriptions of the ICF categories as formulated by the WHO (e.g., e3 support and relationships: “people or animals that provide practical physical or emotional support, nurturing, protection, assistance and relationships to other persons, in their home, place of work, school or at play or in other aspects of their daily activities”; ICF 2017). For all items, rules were drawn up to secure uniform formulations (e.g., regarding the recall period and the experienced degree of difficulty).

C). Determining Response Formats •

For existing items that were adopted verbatim, the response format was based on the original response categories. For the items formulated by the project group, the ICF qualifiers were used to describe the extent of a problem in a particular domain (i.e., no problem (0); mild problem (1); moderate problem (2); severe problem (3); complete problem (4); WHO 2013).

Phases A–C resulted in a preliminary item list agreed upon within the project team.

Content Evaluation

The aim of this part was to test whether the item list was judged relevant (all items should be relevant for the construct of interest within a specific population and context of use), comprehensive (no key aspects of the construct should be missing), and comprehensible (the items should be understood by patients as intended) (Terwee et al. 2018). The preliminary item list was therefore administered to a panel of relevant stakeholder representatives. After that, it was piloted in a group of patients.

D). Expert Survey •

An expert survey was conducted among Dutch representatives of all relevant stakeholder groups, that is, patients, audiologists, ENT surgeons, a general practitioner, and a clinimetrician/methodologist. The selection of experts was based on a convenience sampling method (Knudsen et al. 2012) and recruitment took place through the contacts of the project team members via email. When an expert indicated to be willing to participate, L. v. L. explained the study in more detail via email or telephone and sent the expert survey via email. Consent was implicit by agreeing to participate in the expert survey via email, after which the survey was sent. The representatives were asked to score each item on its relevance and comprehensibility. In addition, the item list was rated on comprehensiveness and the order in which the domains and associated items were queried. At the end of the survey, respondents were able to provide additional comments. See Appendix 2 in Supplemental Digital Content 2, http://links.lww.com/EANDH/A656 for the survey questions. In addition to the expert survey, the main developer of the ICF CSHL (Dr. Granberg) was consulted for feedback on the item list. This was done by using survey questions via email. Specific attention was asked for the operationalization of the hearing-related categories. This was done because the description of ICF categories relating to hearing, listening, and communication are unclear and overlapping (as previously pointed out by the developers; Granberg et al. 2014b).

E). Patient Pilot Study •

The modified item list was tested in a small sample of patients who were randomly selected from the VUmc patient pool. These were new patients who had their first appointment scheduled. Patients were recruited at Amsterdam UMC, location VUmc in Amsterdam, The Netherlands.

Patients were included who visited the outpatient clinic of the VUmc for an ear and/or hearing problem for the first time, were 18 years or older, and who spoke Dutch. A maximum variation strategy (Knudsen et al. 2012) was applied to select participants, with regard to patients’ ear/hearing problem(s), gender, and age. This was done to create a heterogeneous group of patients, covering the full spectrum of oto-audiology diseases/complaints, with an equal gender distribution and a wide age range. Recruitment of patients took place via the secretary of the department, who sent an information letter 2 weeks prior to the scheduled intake visit by email. When a patient indicated to be willing to participate, L. v. L. explained the study in more detail and scheduled the study interview. Recruitment of new patients ceased when variation was achieved. Patients were interviewed directly prior to their appointment with the audiologist or ENT surgeon. They were therefore asked to arrive half an hour earlier.

All patients were interviewed at the outpatient clinic of VUmc. Prior to the interview, written informed consent was obtained. The intake tool was administered in a digital format. Interviews were held in Dutch. The aim of the pilot study was to study the relevance, comprehensibility, and comprehensiveness of the intake tool. This was done based on the “three-step test” interview (TSTI) (Hak et al. 2008). The TSTI combines observational and interviewing techniques to identify how items are interpreted and whether problems occur during completion of the item list. The TSTI comprises three consecutive steps: concurrent thinking aloud, retrospective interview, and a structured interview using an interview guide.

• – During the first step, the interviewer observed the patients as they were completing the item list. Patients were asked and encouraged to verbalize their thoughts while doing so. The interviewer used prompts to encourage the patient to verbalize his/her thoughts. The patient’s comments and interviewer’s observations were written down by the interviewer. The time needed to complete the item list was also noted by the interviewer.

• – During the second step, patients were interviewed regarding their response behavior and comments made during the first step.

• – During the third step, a brief structured interview about the comprehensibility and comprehensiveness of the item list was conducted. The format of the intake tool and how the patient preferred to view the results of the completed item list was also discussed. In addition, patients were invited to share any additional comments about the intake tool.

The interviewer prompts and the interview guide are shown in Appendix 3 in Supplemental Digital Content 3, http://links.lww.com/EANDH/A657.

The digital item list was pre-tested by colleagues who needed around 15 minutes to complete the list. Hence, it was decided to reserve a time slot of 30 minutes for completion of the item list (step 1; 15 minutes) and the interview (steps 2 and 3; 15 minutes) to minimize patient burden. In one case, the intake consult was postponed somewhat (with the consent of the patient and the clinician) so that sufficient time would be available for the interview. No repeat interviews were carried out.

Data Analysis

For the data collected in the expert survey, results and comments were summarized by L. v. L. and discussed within the project group. Items were modified based on consensus in the project group.

All patients were interviewed by a researcher who was trained and experienced in qualitative research methods (L. v. L.) (see Appendix 4 in Supplemental Digital Content 4, http://links.lww.com/EANDH/A658 for the researcher’s characteristics, which have been reported according to the COREQ criteria; Tong et al. 2007). All patient interviews were audio-recorded and transcribed verbatim. Qualitative content analysis was used (Knudsen et al. 2012) to analyze the data. Coding was on item level (except for comments made in step 3 which concerned the item list as a whole and lay out of the intake tool), across the 3 steps of the interview. Comments and problems were labeled based on content and subsequently grouped into categories. Transcription and coding were performed by L. v. L, under supervision of M. P. and S. K. Transcripts were not returned to participants for comment or correction. Results were discussed and items were modified based on consensus in the project group.

This study was approved by the Medical Ethics Committee of the VU University Medical Centre, Amsterdam, The Netherlands (reference number 2013-067).

Digital Format

We explored various options to integrate the intake tool in a digital format as this was the preferred mode of administration. Digital administration enables a rapid provision of the patient’s “functioning profile” to the patient and clinician during the intake procedure.

RESULTS

Selecting and Formulating Items and Choice of Response Formats

A). Identification of Categories to Be Represented in the ICF-Based e-Intake Tool •

A total of 39 categories were chosen to be covered in the intake tool, including the 27 categories from the original Brief CSHL and 12 additional categories.

Additional categories were added based on our previous research. These categories were as follows:

• – Sleep functions (i.e., b134) and Personal Factors. Our previous study showed that sleep functions and personal factors are important for patients with ear and hearing problems, but that these categories are not part of the Core Set (L.M. van Leeuwen et al. 2017). Literature substantiates the relevance of these categories for this patient group (Hallam 1996; Asplund 2003; Cox et al. 2005; Hume 2011; Test et al. 2011; Granberg et al. 2014d), and therefore the project team decided to include them in the intake tool.

Additional categories added based on clinical expertise within the team were as follows:

• – The subcategories of the ICF categories (i.e., third-level) b230 “hearing function” and b240 “sensations associated with hearing and vestibular functions” (i.e., b2301–b2304 and b2400–b2405). The project team decided to include these categories as the Brief CSHL includes only second-level categories (Danermark et al. 2013). Hearing impairment and ear complaints are at the core of ear and hearing care, and therefore more detailed information on hearing functions and ear functions was regarded relevant; and

• – The ICF categories b250 “taste function” and b255 “smell function”. These were included because in the field of otology these are considered important indicators for nerve damage to the auditory organ.

Please note that Personal Factors are not yet classified within the ICF. However, a list of examples is available from the ICF and these include demographics, other health conditions (HCs), coping styles, social background, education and profession, past life events, overall behavior patterns, and other factors playing a role in disability (WHO 2001). In addition to demographics, other HCs, social background, education and profession, other operationalized personal factors were mastery and coping behaviors in communication situations. These constructs were selected, because with our intake tool we aimed for (1) a global view of personal factors indicating how people deal with setbacks such as diseases (including hearing impairment/ear problems) (i.e., mastery), and (2) a specific view of personal factors indicating how the patient deals with his/her ear and hearing problems at the moment (i.e., coping behaviors in communication). Mastery is the extent to which a person perceives one’s life as being under one’s own control in contrast to being fatalistically ruled (Pearlin & Pearlin 1978). It is considered a relevant psychosocial resource when coping with stressful life events. For example, a higher sense of mastery is associated with better psychosocial adjustment to the hearing impairment in older adults (Kramer et al. 2002). Regarding coping behaviors, evidence shows that applying maladaptive (as compared to adaptive) coping behaviors can lead to higher levels of hearing disability, and subsequent psychosocial problems in people with hearing impairment (e.g., Tesch-Römer & Nowak 1995). B-C).

Operationalization and Response Format •

The ICF categories were divided into the following domains: (1) general information, including reason for visit, sociodemographic and medical background-related items; (2) general body functions; (3) ear and hearing structures and functions; (4) activities and participation (A&P); (5) EF; and (6) mastery and coping. The sections below describe how the ICF categories of each domain were operationalized.

General Information (Personal Factors)

In a previous qualitative study, patients indicated that they would like to start the intake tool with reporting the reason for their visit to the outpatient clinic. This way, the focus of the visit would be clear to the professional (van Leeuwen et al. 2018). Therefore, the category “reason for visit” was included as the first item.

For the operationalization of demographics, other HCs, social background, education and profession-related factors, items were based on similar items used in large national cohort studies (i.e., LASA, see Hoogendijk et al. 2016; and NL-SH, see Stam et al. 2016).

General Body Functions

For the operationalization of body functions, items were based on the content and wording of the Speech Spatial and Qualities Questionnaire items (Gatehouse & Noble 2004), items used in a large national cohort study (LASA, Hoogendijk et al. 2016), WHODAS 2.0, WHS and WHO’s official descriptions of ICF categories. Items were formulated as “How much difficulty do you have … [with sleeping]”. The response format was based on the ICF qualifier to specify the degree of difficulty.

For the operationalization of body functions category “temperament and personality functions”, the construct self-esteem was selected. This was done on the one hand because it is known that a poor hearing status can negatively affect self-esteem (e.g., Chen 1994; Kramer et al. 2002). And on the other hand, the level of confidence/self-esteem can influence the management of hearing loss, for instance through applying certain coping strategies (Gatehouse 1991; Erler & Garstecki 2002). Moreover, it is known that involvement from the social environment can positively address incurred hearing losses and lead to important benefits including higher self-esteem (Southall et al. 2010). Lastly, hearing loss management through taking up hearing aids could negatively influence one’s confidence levels (stigma), while it could also improve self-esteem (because communication is improved). “Emotional functions” was operationalized through the constructs feelings of loneliness, depressive complaints, and anxiety complaints. These constructs are known to be commonly affected by ear and hearing problems (e.g., Heine & Browning 2002; Kramer et al. 2002; Lee et al. 2010; Pronk et al. 2011).

Ear and Hearing Structures and Functions

For the operationalization of the ICF categories on ear structures, a figure was made in which the patient could indicate where he/she thinks his/her ear and hearing problem is located. Also the response option “I don’t know” was added. It was decided that it would be relevant to know how well the patient would be able to indicate the location of the hearing or ear problem, to discuss this during the intake and to be able to correct perceptions.

For the operationalization of the hearing, listening, and communication ICF categories (i.e., b230, d115, d310, d350 and d360), the project group agreed to use the validated, 28-item version of the Amsterdam Inventory for Auditory Disability and Handicap (AIADH; Kramer et al. 1995). The AIADH is being used widely in the Dutch hearing aid dispensing practice. The AIADH assesses self-reported disabilities and handicap in everyday hearing. The AIADH includes five hearing domains (subscales): auditory localization, intelligibility in noise, intelligibility in quiet, detection of sounds, and distinction of sounds. For each of the five subscales, we selected the most discriminating item based on Item Response Theory (see Boeschen Hospers et al. 2016). For instance, for the subscale “auditory localization”, the item “Can you hear from what corner of a lecture room someone is asking a question during a meeting?” was chosen, because this item had the highest discriminative ability to indicate auditory disability. In addition to selecting the items with the highest discriminatory power, the items on “conversations over the telephone” and “conversations in quiet” were selected to ensure coverage of all ICF categories in the intake tool. The original four-point response scale was used, “never, sometimes, often, always”.

For the operationalization of ear problems, wording was based on clinical expertise, and the operationalization ran parallel to, and was influenced by, the development of the Otology Questionnaire Amsterdam (see Bruinewoud et al. 2018). The ICF qualifier system, by which the severity of the complaint can be graded, was used as response scale.

Activities and Participation and Environmental Factors

For the operationalization of ICF categories in the A&P and EF domains, formulation was based on the wording of WHODAS 2.0 and WHS items and WHO’s official descriptions of ICF categories. Items in the A&P domain were formulated as “How much difficulty do you have in … [participating in community activities]”. Items in the EF domain were formulated as “To what extent do you feel supported/hindered in your daily functioning by … [your healthcare providers]”. The ICF qualifier system, to specify the degree of difficulty (for the A&P domain) and degree of perceived support and degree of impediment (for the EF domain), was used as response scales.

Mastery and Coping Behavior (Personal Factors)

The construct of mastery was operationalized using an abbreviated five-item version of the Pearlin Mastery Scale (Pearlin & Schooler 1978). The scale measures the extent to which an individual regards his/her life chances as being under their personal control rather than being fatalistically ruled. The original five-point Likert scale, ranging from “strongly disagree” to “strongly agree”, was used as response scale.

For the operationalization of coping behavior relating to hearing impairment, items of the subscales “communication strategies” and “personal adjustment” (including embarrassment and acceptance of the ear and hearing problem) of the Communication Profile for the Hearing Impaired (CPHI) were chosen. CPHI items with the highest discriminating power were included (as reported in Mokkink et al. 2010). The original five-point Likert scale, ranging from “strongly disagree” to “strongly agree”, was used as response scale. In addition, the response option “not applicable” was included to allow patients to indicate whether an item applied to them or not.

Content Evaluation

D). Expert Survey •

All invited experts responded positively to the invitation and expert survey. In total, the preliminary item list was assessed by 10 stakeholders: four patient representatives from Dutch patient organizations, two audiologists (one from a secondary center and one from an academic center), two (resident) ENT surgeons (one from a secondary hospital and one from an academic hospital), a general practitioner, and a clinimetrician/methodologist.

With regard to the relevance of the items, most experts rated all items as relevant, but clinicians indicated that items in the A&P domain in general should be reworded. These questions would be more relevant when explicitly asking how the patient’s ear and/or hearing problems influence functioning in daily life.

With regard to comprehensibility, items were generally well understood, but some suggestions for a better formulation of items or response categories were made.

With regard to the comprehensiveness of the total item list, no important domains were considered to be missing. One of the patient representatives indicated the need for the opportunity to further explain his/her given pre-defined answers (open space). The order of the item list was found adequate.

E). Patient Pilot Study •

Forty-seven patients were invited, and 11 patients participated in the TSTI (response rate 23%). Table 1 shows their characteristics. The categorization according to the International Classification of Diseases version 2010 (ICD-10) – chapter VIII, “Diseases of the ear and mastoid process”: diseases of the external ear, diseases of the middle ear, diseases of the inner ear, and other diseases – shows that the broad range of ear and hearing problems that can generally be encountered in the oto-audiology practice was represented in this group of participants.

TABLE 1.

Characteristics of participants involved in pilot testing (N = 11)

Variable	Total	Otology Patients	Audiology Patients
Number of participants	11	6	5
Gender, male/female	5/6	2/4	3/2
Age in years, mean (range)	59.8 (44–75)	60 (45–75)	59.5 (44–68)
Diagnosis, N
Diseases of external ear (H60–H62)	1	1
Diseases of middle ear and mastoid (H65–H75)	1	1
Diseases of inner ear (H80–H83)	1	1
Other diseases of the ear (H90–H95)
Hearing loss	6	2	4
Tinnitus	2		2
Cochlear implant	1	1
Education level, N
High	5	2	3
Moderate	4	4
Low	2		2

The mean time to complete the item list was 16 min (range: 9-24 min).

Steps 1 and 2: Thinking aloud and retrospective interview

The data collected in steps 1 and 2 showed that every patient encountered problems with at least one of the items of the intake tool. All patients filled in every item. Three categories of comments/problems were identified: (1) problems with response options; (2) difficulty with formulations; (3) response to the item would depend on the specific situation. These categories are discussed below.

Problems With Response Options

One respondent mentioned she found it difficult to choose between the response categories indicating the degree of difficulty experienced.

“Then I think ‘maybe it is not so bad [the ear problem]’, for example compared to others. I find it very difficult to say such a thing about yourself”.

Two respondents indicated to have problems with the item about localization of the ear/hearing problem. They did not know how to answer this question.

Difficult Formulations

Almost every patient encountered problems with answering the EF items. Problems related to the fact that each category was questioned twice, that is, first to what degree the category acted as a barrier to the person’s functioning and then to what degree the category acted as a facilitator of the person’s functioning. Patients suggested that only one item per category should be asked, and this could be either in the formulation as a barrier or a facilitating factor. In addition, the item about the accessibility to care was not well understood. One respondent reported to have problems with the item about which chronic diseases are experienced “at this moment”. The respondent indicated to have had problems, but he “did not suffer from it at this moment”, and therefore did not know how to answer this item. Another respondent thought the item on feelings of loneliness was difficult to understand.

Response Would Be Dependent on Specific Situation

Some patients indicated that the answer on items “depended on the situation”, but could always answer the question after some consideration. For example, regarding the item about difficulties when attending education, one respondent reported that the answer on this question would depend on whether the education material was provided orally or in a written fashion. Another example was the items on coping behavior (personal factors). It was reported that whether or not being able to cope well would depend on the specific (social) situation. One respondent suggested to include the option to provide comments in the items, to be able to better explain the chosen response category.

Instructions Were not Read

It was observed that patients consistently did not read the instructions at the beginning of each domain or subset of items.

Step 3: Structured Interview

The data collected in step 3 showed that all patients thought that the intake tool was relevant. Regarding the content of the item list, patients stated that the items were relevant to them and comprehensible (except for the items on EF). Regarding the comprehensiveness of the item list, some patients indicated that more detail on some specific complaints would be desirable but they did not miss any key concepts. They also agreed on the general nature of the intake tool and mentioned that further specification may not be feasible. Regarding the layout of the item list, it was mentioned twice that the font size should be somewhat bigger. Patients found it difficult to comment on presentation of the (future) functioning profile because they found it hard to envisage how this would look like. The option to save or print the filled-out form was regarded as mostly convenient to them. Regarding the layout of the intake tool, a simple format and a low quantity of questions per screen was preferred.

Amendments to the Intake Tool

Based on the responses of the experts, changes were made in the instructions of the items covering A&P and EF so that these would specifically address factors in relation to the patient’s ear and hearing problems. The description was adjusted into “The following questions are about the influence of your ear and/or hearing problem on your daily activities” (A&P) and “The following questions address the influence of different EF on your daily functioning. With regard to your ear and/or hearing problem, indicate to what extent these provide support for your daily functioning” (EF). In addition, some items were modified to improve the wording.

Based on the problems patients encountered while answering the EF items, these items and response categories were adapted. From the literature it is known that positive items are generally preferred. Therefore, only items about the facilitating effect of the item were retained. In addition, the item about the accessibility to care was simplified. Items adopted from existing questionnaires were retained despite the (few) identified problems. According to patient’s suggestions, the instructions were written in a bold font style and were copied on every new page (in case of a page break).

The table with the final item list is available in Supplemental Digital Content 1, http://links.lww.com/EANDH/A637.

Digital Format

The online portal “KLIK” was chosen to implement the intake tool. KLIK provides an online environment to administer PROMs digitally. The use of KLIK is as follows. Prior to the intake visit, patients are asked to register to the online portal (www.hetklikt.nu). After completion of the questionnaire, the patient’s outcomes are digitally presented and converted into a “functioning profile”. A three-color traffic light system was chosen to be used to indicate in which area(s) further detailed examination(s), action(s) and/or intervention(s) are needed. Figure 2 provides an example of such a functioning profile. Because the cutoff points can only be determined after sufficient data collection, the traffic light system could not be utilized for the first version of the tool. The functioning profile can be saved as PDF and/or printed. This way, it could be used by patients in preparing for and during the intake appointment. Moreover, the PDF format allows it to be added to the patient’s medical file such that it is visible to clinicians.

Fig. 2.

Example of electronic functioning profile, domain activities, and participation, using traffic lights. Note: This figure is purely illustrative and not based on cutoffs.

DISCUSSION

This study aimed to operationalize the recently developed ICF Brief CSHL (Danermark et al. 2013) into an self-reported diagnostic screening tool for patients with ear and hearing problems visiting the audiology or ENT outpatient clinic. This study is part of Phase II of the WHO’s Core Set’s development process (Selb et al. 2015). The ICF-based e-intake tool assesses the functioning of an individual with ear and hearing problems and includes the assessment of potentially influencing environmental and personal factors. The current version of the intake tool covers 39 ICF categories. It comprises 62 items, and it takes approximately 16 min to complete it.

Content validity is the most important measurement property of a self-reported instrument (Terwee et al. 2018). The results of the current study present preliminary evidence to support the content validity of the tool as an instrument to screen for ear and hearing problems relating to functioning and the environmental and personal factors that may interact with these problems. Furthermore, overall, the intake tool was perceived to be relevant and to have a logical and clear structure, as indicated by the stakeholder representatives and the patients who participated in the pilot study.

The tool was integrated into a digital, web-based patient system called KLIK. The integration of the intake tool into such a system will facilitate its use by clinicians (van Leeuwen et al. 2019). It offers options to create routing pathways by presenting additional items based on a patient’s response on a previous item. Also, a summary of the patient’s answers in a graphical functioning profile can be generated. KLIK has been adopted and implemented for self-reported questionnaires in different settings and in different hospitals across the Netherlands, in both child and adult care (KLIK 2018). The feasibility and user-friendliness of our intake tool in oto-audiology patients will need to be further evaluated to optimize its intended use in the otology and audiology practice.

Clinical Implications

Patient-Centered Care •

The intake tool is developed with the ultimate aim to improve patient-centered care in oto-audiology practice. It is important to recognize that the intake tool in itself will not directly cause patient-centered care (Epstein & Street 2011). Rather, the functioning profile resulting from the intake tool may act as a facilitator of patient-centered care. It is considered a starting point of the intake process, enhancing communication between the clinician and the patient about the experienced challenges in functioning, clarifying priorities for care, and fostering equal partnership in determining treatment (e.g., Snyder et al. 2012). It is important to emphasize that the goal of the intake tool is not to replace the intake appointment but to serve as an aid to facilitate the intake conversation. Several studies have addressed the impact of self-reported instruments on the (intake) appointment with the clinician. Reviews provide evidence of improved patient-clinician communication, better identification of psychosocial problems, and better guidance in clinical decisions made in response to patient-reported symptoms (Valderas et al. 2008; Greenhalgh 2009; Basch et al. 2011; Snyder et al. 2011; Johansen et al. 2012; Locklear et al. 2014). However, whether the intake tool will indeed facilitate patient-centered care will partly depend on its successful implementation. That will imply changes in practice for both patients and clinicians in order to accommodate the collection and the feedback of the patient-reported information. Changing practices is known to be challenging (Greenhalgh et al. 2005; Velikova et al. 2008; Noonan et al. 2017). In parallel studies, we identified the perceived barriers to and enablers of using the intake tool (van Leeuwen et al. 2018) and used this information for the development of an implementation intervention (van Leeuwen et al. 2019).

A Tool for Clinical Oto-Audiology Practice •

With our intake tool, we opted for an integrated and uniform approach to collect functioning information in the initial contact, independent of the specific oto/audiology discipline the patient encounters first. Information about a person’s functioning documented during the intake should facilitate a proficient and interconnected collaboration between the team members during the care process, that is, by using the standardized intake tool in both disciplines.

Operationalization of Other ICF Core Sets

Over the past few years, operationalization of ICF Core Sets for use in clinical practice occurred in other domains. Examples are the Brief Core Set Questionnaire of Breast Cancer for Screening in cancer care (Yang et al. 2014), the Work rehabilitation Questionnaire for vocational rehabilitation (Finger et al. 2014), a health index for patients with ankylosing spondylitis (Kiltz et al. 2015), the Neuromuscular disease impact profile for neuromuscular diseases (Bos et al. 2015), and the ICF CS-based questionnaire for non-traumatic spinal cord injury (Coelho et al. 2017). Contrary to our diagnostic screening tool, these instruments were developed to measure the effect of treatments or interventions on functioning. None of these cover contextual factors. We chose to create a tool that can provide a quick, standardized screen of ear and hearing–related functioning. It highlights aspects that need further examination and/or actions. It is known that having only one to two items to measure a construct generally yields insufficient reliability for evaluative purposes (Frost et al. 2007). Including more items per construct was discussed within the project team, but this would yield a too lengthy questionnaire and therefore would result in an unacceptable patient burden. The current version of the tool is not suitable for the measurement of the effectiveness of treatments. For that purpose, it would have to be expanded to provide a more detailed assessment of sub-constructs of functioning. It could be combined with validated symptom-specific questionnaires. For example, to measure improvement in self-perceived disability and handicap in everyday hearing, the full version of the AIADH could be incorporated. Similarly, to measure the effect of treatment or interventions on patient’s coping behavior, the full CPHI could be added. Also other PROMs not part of the intake tool may be used. Examples are the Dizziness Handicap Inventory to measure dizziness. Such multi-item scales would be suitable for follow- up measurements as they have better sensitivity and responsiveness than one or two-item scales. Development of an ICF-based instrument that can be used to evaluate treatment effectiveness was beyond the scope of the current project.

International Perspective

This project is part of Phase II of the WHO-defined process to develop ICF Core Sets. Other than the described purpose of the intake tool, it can be used to serve additional objectives. It can be used to (1) promote and guide further development of Core Sets for use in clinical practice, research and education in the field of Audiology, (2) develop strategies for the implementation of the ICF Core Sets for HL in clinical practice, (3) encourage international collaboration and alignment in these processes. Similar activities to operationalize the Brief Core Set into a self-assessment instrument are ongoing in the United States (Alfakir et al. 2015a, b, 2019) and in Sweden (“ICF-core sets for hearing loss; validation and operationalization of Brief ICF-Core set for hearing loss into a self-assessment instrument”). The experience gained in our study, in combination with the other initiatives, are of importance to achieving the WHO’s goals with the Core Sets.

METHODOLOGICAL CONSIDERATIONS

Operationalization

We chose to operationalize the ICF-category “emotional functions” into feelings of loneliness (item 14, Supplemental Digital Content 1, http://links.lww.com/EANDH/A637), sorrow, sadness, depressive complaints (item 15, Supplemental Digital Content 1, http://links.lww.com/EANDH/A637), and feelings of worry and anxiety (item 16, Supplemental Digital Content 1, http://links.lww.com/EANDH/A637). With regard to psychological personality traits in the component personal factors, “mastery” and “coping behavior” were selected. This was done based on the literature showing that these provide a representative picture of a patient’s personality/intrinsic factors potentially influencing someone’s living with ear and hearing problems. Nevertheless, the choice for including only these two categories may seem arbitrary and other additional categories could have been considered. An example is frustration, which is a well-known consequence of hearing impairment (e.g., Heffernan et al. 2016; Vas et al. 2017). Another consideration concerns existing difficulties with regard to the conceptualization and categorization of personal factors (Geyh et al. 2011; Müller & Geyh 2015). For example, the psychological assets in the personal factors component (e.g., emotional reactions) seem to overlap with the categories of mental functions of the BF component. This was also the case in the current study. We tried to adhere to the descriptions of the ICF categories, but the choice for the operationalization of embarrassment as a personal factor rather than an emotional reaction (see items 53 and 55, SCD 1) may therefore be regarded as somewhat arbitrary.

Another possible shortcoming of the operationalization process may be the consensus being based on expertise from a small group of experts from one hospital setting. Consequently, choices were made based on preferences within this setting and thus may not apply in other (hospital) settings. However, we validated our choices as much as possible by testing the draft item list in a broader expert group and in a heterogeneous sample of patients.

Different response formats were selected for the different domains in our intake tool. Previous research showed that mixed response scales may be confusing for respondents (e.g., Heffernan et al. 2018). Moreover, it is known from the literature that the patient’s self-reported data should be easy to interpret by the clinician in order to facilitate its implementation (Locklear et al. 2014). Mixed response scales may hamper that. However, both experts and patients included in the content assessment did not report important problems with the response scales (except for the domain of EF, which was adapted accordingly). With regard to clinician burden and ease of using the intake tool, our other study in which we identified the barriers and enabler to use the intake tool indicated that clinicians indeed preferred a simple overview of easy-to-interpret results (van Leeuwen et al. 2018). At this point in the development process, such an overview has not been developed and considered for review by the clinicians yet. This will be addressed during next steps of the development and testing of the tool (see further under “Future directions”).

Content Assessment

With regard to the data of the patient pilot-study, bias could have occurred because the interviewer was also part of the project team. However, the aim of the pilot study was to ensure that the questionnaire content would match the target group, so the interviewer was motivated to know all the critical points in order to be able to improve the content of the item list. Therefore, we do not expect this was a negative factor. A limiting factor was the use of closed-ended questions in the interview guide, which may have limited the respondents’ answers and more detailed explanations of their experiences with the item list.

Generalizability

Another possible limitation is that the tool is developed in Dutch, and decisions were made based on the Dutch health care system. Instruments must fit into the health care system where they should be applied (ISOQOL 2011). The current version of the intake tool is intended for use in the Dutch otology and audiology system, which – for now – limits its use to Dutch speaking patients. Its application and generalizability to other countries and care systems would need to be addressed in future work.

It may be argued that this study was limited in the sense that the consensus meeting on the selection and initial formulation of the items did not include patient representatives. As already mentioned in the Introduction, the development of the ICF CSHL did include patients’ participation in various stages of the Core Sets’ development and consensus process. The patient perspective on functioning with hearing loss was carefully mapped in a qualitative focus group study (Granberg et al. 2014d). The current study did include the patients’ voice in the pilot study, and a wide range of ear/hearing problems was included. Nonetheless, this concerned only a limited absolute number of highly motivated patients who thus may not be representative of the average patient.

FUTURE RESEARCH DIRECTIONS

The suitability and use of the intake tool for all patient groups will need further evaluation in a large-scale field-test study. In addition, to make the clinician’s and patient’s use of the intake tool as efficient as possible, the ease of reviewing and interpreting the patient’s responses will need to be addressed. For clinicians, a system that has been shown to be easy to use is the traffic light system. It is also easy to read (provides a graphical summary format) and can deliver concrete actions to take. Such a traffic light system was successfully applied in pediatric cancer care (Schepers et al. 2017). However, applying it requires relevant cutoffs for the each item and/or underlying domains. Moreover, a follow-up decision tree is needed to guide clinicians on their actions (e.g., treatment options, referral to another health care professional) (See also van Leeuwen et al. 2019). A field-test study and the input of and consensus among clinicians will be needed to determine meaningful cutoffs. This is essential for clinicians’ motivation to use the tool (e.g., Hildon et al. 2012).

CONCLUSION

The current study describes the development of an ICF-based e-intake tool to be used by patients and clinicians to assess functioning in individual adults with ear and hearing problems. Based on stakeholders’ responses, item instructions for A&P and EF were adapted and explicitly related to patients’ ear and hearing problems. Patients’ responses resulted in changes to the items of EF. Overall, the intake tool was perceived to be relevant and to have a logical and clear structure. In addition, the tool showed sufficient content validity. The findings of the current study cover important developmental steps taken toward creating an intake facilitating individualized clinical otology and audiology services using a biopsychosocial perspective.

ACKNOWLEDGMENTS

We gratefully acknowledge stakeholders for participating in the content evaluation studies: Chantal Emaus, Willem Dekker, Marcel Maré, Henk van Rees, Jiska van Stralen, Caroline Terwee, Karen van den Toren, Marein van der Torn, Niek Versfeld, Susanne van Wijk. We are thankful for Dr. Granberg’s input on the development of hearing-related items. We thank the secretary and clinician assistants of the Department of Otolaryngology-Head and Neck Surgery of Amsterdam UMC, location VUmc, who assisted in the recruitment of patients.