A Store-and-Forward Tele-Audiology Solution to Promote Efficient Screenings for Ototoxicity during Cisplatin Cancer Treatment

Abstract

Background

Tele-audiology improves access, controls cost, and improves efficiency of many aspects within health care. We have developed and validated a device, the ototoxicity identification device (OtoID), which enables remote hearing monitoring by a patient during chemotherapy treatment. Aspects of the design such as patient self-testing and texting of results to the audiology clinic are important features of this device.

Purpose

The purpose of this article is to present the efficacy and effectiveness of the OtoID hearing screener.

Research Design

A repeated measures design was used in this study.

Study Sample

Twenty-one veterans undergoing cisplatin chemotherapy were recruited in this study.

Data Collection and Analysis

Participants were tested using the OtoID at each cisplatin treatment by an audiologist using the manual mode of test and the participant using the automated mode of test. Test sensitivity and specificity were developed from the detection (yes/no) of an American Speech-Language-Hearing Association (ASHA) change in hearing.

Results

The OtoID had a test sensitivity of 80.6% and specificity of 85.3%. A logistic regression model analysis of the probability of an ASHA shift identified by the automated OtoID was conducted. Separate models were fit to establish effects of age, average baseline thresholds in the sensitive range for ototoxicity (SRO), and dose of cisplatin on the probability of a positive hearing change result. Interactions were also included to evaluate these effects on the sensitivity and false-positive rates of the automated test. Results indicated no statistically significant effects of age, of baseline hearing in the SRO frequencies, or of cisplatin dose.

Conclusions

The OtoID automated test can be recommended for use. The automated test provides significant personnel efficiencies. The modem with simple text messaging function recently added to the device improves on these efficiencies.

BACKGROUND

Technological advances have allowed telehealth—the delivery of health services at a distance—to improve the access, cost control, and efficiency of many aspects within health care. Telehealth not only increases connectedness of facilities and clinicians, but also enables health-care delivery outside of centralized settings allowing health-care professionals to monitor indicators of health from a patient's home or in geographic areas where easy access to professional services is limited or nonexistent. An issue brief by the American Speech-Language-Hearing Association (ASHA) in support of the Telemedicine for Medicare Act of 2013 states that telepractice can “alleviate provider shortage by extending clinical services to ... underserved population” (ASHA 2015). They define telepractice as synchronous (real-time interaction with the client) and asynchronous (store and forward). In this brief, eligible telepractices for audiology that are reimbursable included adult hearing screening and auditory brainstem testing, among others. Reflecting its importance to the delivery of speech and hearing services, ASHA now has a special interest group (ASHA SIG 18) devoted entirely to the promotion of telepractice to its members. A recent survey by ASHA indicated that over half of audiologists and speech-language pathologists provide services through telepractice including assessments (50%), treatment (89.3%), and follow-up/monitoring (63.9%) (ASHA 2014).

At this time, most telepractice done in audiology tends to depend on the synchronous model because of our reliance on equipment and devices. Though commonly used now, Givens et al (2003) developed the method for remote, real-time hearing assessment using a remote audiometer with a computer-driven controller and a micro-Webserver that connected the patient to the audiologist who conducted the hearing test. They found that Web-based hearing thresholds were equivalent to the thresholds obtained in traditional “brick and mortar” settings. This preliminary system with later refinements (Yao et al, 2010) demonstrated the ease and feasibility of remote auditory testing and promoted this type of practice for populations lacking easy access to audiological professionals. However, an automated hearing test would remove the requirement of real-time professional presence and the need for audiology assistants in attendance with the patient at the time of test. This could save significant professional time and further reduce costs. Moreover, it could provide a more patient-centered approach in that the testing could be done in the comfort of the patient's home at a time when he/she feels up to the task. Self-testing could be accomplished if the equipment had self-testing software that was automated, proven reliable, and easy to use. This asynchronous model of telepractice is commonly found in other medical practices (e.g., radiology).

A 2005 technical report from ASHA stressed the necessity to develop and validate telepractice clinical protocols. The development of a hearing monitoring system that provides reliable and sensitive early detection indicators of threshold change and incorporates telepractice into the reporting of this change while minimizing cost should garner widespread support. This is especially true for routine procedures such as hearing screening or monitoring for hearing change caused by ototoxic agents and excessive noise exposure.

Injuries to the ear are one of the most common, single injury types among soldiers (Gondusky and Reiter, 2005). Auditory impairment and tinnitus represent the most prevalent service-connected disabilities, resulting in 1 billion dollars in associated health-care costs. As a result, prevention of these disabling conditions is a top research priority for the Veteran Affairs (VA) and Department of Defense. For example, blast trauma incurred as a result of military deployment often requires treatment with broad-spectrum antibiotics, which may be ototoxic, to reduce the risk of infection during and after transport to a medical facility. As our nation's veteran population ages, cancer incidence increases with many late stage cancers requiring treatment using ototoxic chemotherapeutic agents. Cisplatin is an effective, though toxic, chemotherapy that requires close monitoring for nephrotoxicity, myelosuppression, neurotoxicity, and ototoxicity. Treatment with cisplatin can result in irreversible cochlear damage in up to 50% of patients and new tinnitus in 40% of patients (Reavis et al, 2008; Dille, Konrad-Martin, et al, 2010; Dille, McMillan, et al, 2010). Cisplatin preferentially damages the outer hair cells of the organ of Corti in a base to apex manner and the stria vascularis (Rybak 2007; Hellberg et al, 2009). In addition, as damage progresses, the inner hair cells and supporting cells are affected (Berglin et al, 2011). Concomitant damage to the spiral ganglion cells can also occur (Lee et al, 2003). The damage to the auditory system caused by exposure to ototoxic medications may be minimized with early detection since serial monitoring, using either hearing testing or otoacoustic emission testing, or preferably both, provides an opportunity to change the exposure before debilitating hearing loss occurs. Furthermore, when the exposure is unavoidable, aural rehabilitation can occur in a timely manner. Yet despite substantial evidence and clear implications, early identification and monitoring practices have not been implemented as the standard of care for patients receiving ototoxic drugs in most medical care settings, including those within the VA and Department of Defense. This may be attributed to the time-consuming procedures associated with repetitive serial diagnostic tests, some of which may be unnecessary. There exist substantial barriers to implementing evidence-based hearing monitoring practices.

Research investigators at the VA Rehabilitation Research and Development Service, National Center for Rehabilitative Auditory Research (NCRAR) pioneered ototoxicity monitoring with the development of early detection protocols (Fausti et al, 1999; Wilmington et al, 2011). Ototoxic damage has a relatively predictable course of action within the cochlea. Initially, hearing change occurs near the high-frequency limit of hearing and progresses apically through the cochlea. Therefore, the most sensitive screening procedure should employ testing at or near the highest frequencies heard in each ear by each individual. When sampled using air conduction audiometry in 1/6-octave increments, an ear-specific, 1-octave sensitive range for ototoxicity (SRO) was found to detect 90% of all initial changes in hearing (Fausti et al, 1999, 2003, 2007). Compared to a traditional comprehensive (air- and bone-conduction, speech, and immittance) test, SRO hearing screening represents an impressive time-saving test with no reduction in sensitivity. A further advantage of SRO testing is that its sensitive range is typically above the important frequencies for speech. Currently, there are no commercially available portable audiometers capable of supporting the precision testing (up to 20 kHz in 1/6th octave steps with output of 105 dB SPL) required for the SRO protocol. To address this clinical need, the NCRAR investigators developed this instrumentation, the portable ototoxicity identification device (OtoID) (Jacobs et al, 2012), to optimize efficiency and cost-effectiveness of ototoxicity early identification practices.

To improve its efficiency, the OtoID supports self-testing using a simple, validated protocol (Dille et al, 2013). At the baseline test, the patient is trained by the audiologist in the use of the device. After the patient is seated in a quiet area, the device “walks” the patient through pretest checks such as earphone placement and room noise monitoring and then begins the self-testing protocol. The patient is alerted to each listening interval (“listen now”) and then instructed to push the “yes” button on the screen if a tone was heard or “no” if a tone was not heard. The device then guides the patient through a seven-frequency hearing screen in each ear using the modified Hughson-Westlake procedure. After testing is completed, the results of the test are then automatically transmitted to the audiologist specified in the software of the device using simple messaging system (SMS) texting. When the patient returns for chemotherapy, the nurse gives the OtoID for self-testing. The simplicity of using the device means that patients can test their hearing in just about any setting, even, potentially, in their own homes when daily hearing monitoring might be recommended such as for aminoglycosides treatments of serious or chronic infections.

PURPOSE

In practice, veterans from rural and highly rural areas and their families must travelgreat distances to central VAs to obtain cancer care. On treatment day, laboratory results must be obtained, oncology must assess for medication side effects, treatment decisions must be made, and radiation oncology must be seen. The chief barrier to current hearing monitoring practices is finding time in the veteran's schedule for time consuming, comprehensive hearing assessments. This has the potential to create unevenly applied treatment practices favoring those veterans who live nearby and can be tested on days other than the treatment day. The OtoID was developed to address this potential disparity in health care delivery and to optimize the ability for all veterans to receive efficient and timely ototoxicity monitoring. In this article, our purpose is to evaluate the accuracy of the OtoID self-test, to report on our findings of ototoxicity monitoring research and to glimpse the future for tele-audiology applications incorporated in ototoxicity monitoring using evidence-based practices.

RESEARCH DESIGN

Study Sample

Treatment schedule and length varies with the type and stage of cancer. Hearing testing is done up to 7 days before treatment and within 24 h after treatment since the hypothesized effects of cisplatin are not immediate. Hearing does not tend to recover once it is shown to be significantly shifted, and further treatments have potential to continue hearing damage by increasing the spread of involved frequencies and the amount of hearing loss. Predicting which patients will experience ototoxic hearing loss is not possible without a prior pretreatment hearing evaluation. Although the risk for developing hearing loss from ototoxic drugs is generally related to the dose, duration, frequency, and method of medication administration, there is marked individual variability in these relationships (Vermorken et al, 1983; Rademaker-Lakhai et al, 2006).

Participants were recruited from veterans preparing for cisplatin chemotherapy in the chemotherapy treatment unit (CTU) at the VA Portland Health Care System. These veterans tended to be male and in their sixth decade of life. All veterans prescribed cisplatin were asked to join the study unless oncology personnel felt that study participation was not appropriate. Criteria for excluding potential participants from this study were (a) cognitively, physically, or psychologically unable to participate; (b) unable to provide reliable behavioral threshold responses; and (c) participant or medical record report of Meniere disease, retrocochlear disorder, or active or recent history of middle ear disorder. All participants were consented to participate in the study following the guidelines of the VA Portland Health Care System Institutional Review Board and were compensated for their time.

The OtoID, shown in Figure 1, was used for all assessments. It is comprised of an advanced reduced instruction set computer machine-based processor with a touch screen monitor running Windows CE (Microsoft; Redmond, WA) and OtoID firmware (locally-designed at VA NCRAR), a custom audiometer circuit board with extended high-frequency audiometer (HFA) capability, and Sennheiser HDA200 circumaural headphones (Sennheiser; Old Lyme, CT). The OtoID is ergonomically designed to be sturdy and comfortable to use by both professional and nonprofessional personnel. In the development of the OtoID, demanding acoustic performance was required such that each test frequency (500–20000 Hz, in 1/6th octave step size) has a dynamic range of 115 dB (−10 to 105 dB SPL). A specification summary of the OtoID type 4 and HFA is shown in Table 1. A fully detailed description of the technologies employed in the OtoID device and performance of the device using healthy volunteers, with and without hearing loss, who were tested in a hearing booth and in the CTU, is available elsewhere (Jacobs et al, 2012; Dille et al, 2013). All calibration procedures used the Bruel & Kjaer (B&K; Naerum, Denmark) 2250 sound level meter and a B&K model flat-plate coupler equipped with a B&K 4192 microphone. Before calibration of the OtoID, the B&K 2250 sound level meter and 4192 microphone were adjusted for level accuracy using 4231 calibrator. Sennheiser headphones were centered on the flat-plate coupler, and frequency accuracy, harmonic distortion, linearity, and tone switch compliance (rise–fall and duration parameters) were evaluated from 500 to 20000 Hz in 1/6th octave intervals annually. Linearity extended from 105 to −10 dB. The OtoID complied with American National Standards Institute (ANSI, 2010) S3.6-2010 except for rise times, which were slightly prolonged (rise time = 65 msec). The Maico easyTymp (Maico International; Berlin, Germany), used in the study, was calibrated annually.

Figure 1.

A veteran using the OtoID audiometer in the automated test mode. The OtoID prompts the veteran to “Listen now.” In this listening interval, a tone may or may not be presented to one ear. After the interval, the veteran is asked to report if a tone was heard “yes” or “no.” The modified Hughson-Westlake procedure is used to establish threshold. Catch trials, intervals in which a tone is not played, are inserted into the test protocol to increase reliability of the test. Catch trials can be programmed to vary from 0% to 100%.

Table 1.

Specifications of the OtoID

Parameter	Value
Frequency range (Hz)	500–20000
Frequency resolution (Hz)	5
Frequency accuracy (%)	±1
Output level (dB SPL)	–15 to 105
Attenuator range (dB)	0.0 to –119.9 dB
Attenuator resolution (dB)	0.1
Attenuator accuracy (%)	±0.4%
Power amplifier resolution (dB)	0.1
Ambient noise frequency (Hz)	500–14000
Harmonic distortion + noise (%)	<0.1
Data storage format	ASCII text files

Note: ANSI S3.6-2010 Class Type 4 and HFA specifications for reference equivalent SPL, frequency accuracy and purity, attenuator accuracy and linearity, tone switch characteristics, and absence of unwanted acoustic signals.

DATA COLLECTION AND ANALYSIS

Procedures for all participants included (a) a brief hearing history questionnaire, (b) otoscopy, (c) tympanometry, (d) pure-tone air conduction thresholds bilaterally from 500 to 8000 Hz in 1/2-octave steps, and (e) thresholds in the SRO frequencies with 1/6th octave precision done by a licensed audiologist. The audiologist first obtained thresholds using the traditional manual mode of operation. The participant was then instructed in the OtoID self-testing mode of operation and proceeded to test his or her pure-tone air conduction thresholds using the automated self-testing mode. Both test procedures (manual and automated) used the modified Hughson-Westlake testing protocol (Carhart and Jerger, 1959). The SRO was defined as the uppermost frequency, R, with a threshold of ≤100 dB SPL followed by the next adjacent six lower frequencies in 1/6th octave steps, R-1 through R-6, all <100 dB SPL. Table 2 illustrates an example of hearing thresholds obtained from the manual and automated tests (SRO only).

Table 2.

Example Veteran Audiogram

Frequency (Hz)	Baseline Hearing Threshold (dB SPL)	Automated SRO Thresholds (dB SPL)
500	25
1000	25
2000	30
3000	45
4000	50
6000	60
6350	70	70	R-6
7130	75	70	R-5
8000	80	80	R-4
9000	80	80	R-3
10000	85	85	R-2
11200	90	90	R-1
12500	100	100	R
14000	105
16000	NR
18000	NR
20000	NR

Note: The shaded area identifies the individualized (by ear) SRO. Example values represent behavioral thresholds in SPL. R is the highest frequency with a threshold of ≤100 dB followed by the next six frequencies with thresholds <100 dB in 1/6th octave steps. NR = no response (threshold >105 dB).

At each test interval, otoscopy and 0.226 kHz tympanometry were done before testing to verify bilateral normal middle ear function. Tympanometric measurements were considered to be normal when compliance ranged within 0.2–1.8 cm³ and peak pressure ranged within +100 to −150 decapascals. Tympanometry was measured using the Maico easyTymp portable immittance screening device. Pure-tone threshold results of each participant were stored in the OtoID, which has the capability to recall previous test results via a patient identifier.

The research audiologist used a quiet patient care area in (or nearby) the CTU to test each participant in both ears. To verify patient reliability, baseline measures were immediately repeated (i.e., “baseline recheck”), and retest reliability of ±5 dB was required. This baseline evaluation provided the reference from which all subsequent monitoring tests were compared. At each treatment interval (monitor sessions), participants were tested by the audiologist and then performed the self-test using the automated test mode. All testing was done within 24 h of each cisplatin treatment. Finally, at 1 and 3 mo after chemotherapy treatment ceased (post-treatment sessions), the same tests were performed.

Before each self-test, the audiologist provided the participant with a brief explanation and orientation to the OtoID device including a demonstration of the response required. The audiologist remained with the participant during all testing to insure that the study protocols were correctly implemented, the test room noise was sufficiently quiet, and to pause testing for chemotherapy-related tasks (e.g., radiation treatment, blood testing). This occurrence provided an additional rationale for administering hearing testing near the CTU since patients remained readily available for oncologic care.

Before beginning the automated testing sequence, the OtoID steps through two important patient and testing set-up sequences. First, the OtoID instructs the participant how to place the earphones. Earphone placement instructions on the screen read: “Place the blue earphone on the left ear and the red earphone on the right ear.” After this instruction, a tone was played in one ear after which the screen prompts the participant to select the ear in which the tone was heard, “left” or “right.” If the tone was played in the left ear and the participant reported that it was heard in the right ear, the instructions on earphone placement were shown again. Once the OtoID verified that earphone placement was correct, ambient room noise was measured using microphones mounted on the earphone cup. If the noise (broadband) is found to be <70 dB SPL, testing was initiated. If not, the participant was instructed to either quiet the room or to find a new location for testing. After three unsuccessful attempts to quiet the room or find a test location, the OtoID instructed the participant to seek the help of the nurse or audiologist. In this study, the audiologist was present during all testing and was able to control the room noise.

Room noise monitoring was implemented on the OtoID very early in the development of the device. The ultimate application of room noise monitoring was to provide a reliable hearing test when the patient tests him- or herself alone (no audiologist present to control the room for excessive noise). Before the start of the project, the noise monitoring microphones were calibrated in a sound suite using Knowles Electronic Manikin for Acoustic Research (KEMAR; Knowles Electronics, Itasca, IL). The Sennheiser headphones were mounted on KEMAR that was placed 1 m from a speaker. Broadband noise was introduced into the environment through the speaker. Room noise was evaluated using a B&K 2250 sound level meter with the microphone hung just above the center of KEMAR's head. Using the OtoID software, the room noise monitor microphones were calibrated in 5-dB increments from 60 to 85 dB for broadband noise and in 1/6th octave fine frequency steps from 500 to 20000 Hz. In the software, any of the calibrated level steps can be selected for monitoring during testing. The initial room noise measurement is done using the broadband measurement. Once testing begins, the OtoID monitors room noise surrounding the test frequency (±1/6 octave). The frequency range of the SRO in veterans typically ranges from 4000 to 12500 Hz, well beyond most energetic masking effects.

Following these test preparations, the participant entered the automated SRO threshold test. Participants were first alerted to an upcoming listening interval in which a tone may or may not be presented with a “Listen Now” prompt on the screen. After each listening interval, the participant was required to indicate whether a tone was heard (“yes” or “no”). If the participant reported hearing a tone when the tone was presented, the level of the tone was decreased by 10 dB. If the participant reported that no tone was heard when a tone was presented, the level was increased by 5 dB. This continued until a threshold was obtained (two out of three ascending behavioral responses at the same dB level) for each of the SRO frequencies. Ten percent of the presentations were randomly presented as “catch” trials, intervals in which no tone was presented, to detect false-positive behavioral responses. If the participant reported hearing the tone during a catch trial, the screen message read “listen carefully for the tone.” At this point, a tone may or may not (presentation of another catch trial) be presented. In addition, the OtoID continued to sample room noise before each tone presentation. If room noise levels exceeded 70 dB SPL, testing was halted and the participant received (again) the on-screen message to quiet the room or find a quieter room.

ASHA (1994) guidelines for audiological management of individuals receiving cochleotoxic drug therapy were used to identify ototoxic change, which include (a) ≥20 dB change at any one test frequency, (b) ≥10 dB change at any two consecutive test frequencies, or (c) loss of response at three consecutive test frequencies where responses were previously obtained. If hearing was found to be significantly shifted in the manual test mode, the oncology nurse was verbally alerted to the hearing shift. The nurse was apprised of all test findings at each treatment interval regardless of whether hearing was or was not changed using the computerized patient record system, the VA's electronic medical records.

The goal of the analysis was to (a) estimate the accuracy of the automated OtoID for identifying cisplatin-induced hearing shifts; (b) evaluate the effects of patient characteristics (e.g., age and hearing level) and treatment (e.g., cumulative dose of cisplatin at the time of testing) on the accuracy of the automated OtoID; and (c) propose an evidence-based ototoxicity monitoring protocol that minimizes the use of audiology personnel resources for ototoxicity resources. In addition, this article demonstrates the potential benefit of incorporating tele-audiology practices when considering adoption of ototoxicity monitoring in clinical practices.

Accuracy of a diagnostic test was operationally defined in terms of sensitivity and specificity measures. Sensitivity was the rate at which the automated OtoID identified actual hearing shifts using the gold standard pure-tone SRO frequency thresholds obtained by the audiologist. The false-positive rate (1 minus the specificity), is the probability with which the automated OtoID incorrectly identified an individual ear hearing shift for which no change was detected by the gold standard. In our analysis, the gold standard was the testing conducted by an audiologist using the OtoID and done in the chemotherapy unit. Hearing shift was operationally defined as an ASHA significant shift in the SRO frequencies in an ear, as defined previously.

The unit of analysis in this study was the test result in an ear at a particular monitoring appointment. In general, each participant gave two test results (left and right ear) at each appointment, which varied in time depending on the chemotherapy regimen and the participant's health. A crude estimate of the sensitivity of the automated OtoID can be given as the number of ASHA shifts identified by both the automated OtoID and the audiologist divided by the total number of ASHA shifts identified by the audiologist. A crude estimate of the false-positive rate of the automated OtoID can be given as the number of ASHA shifts identified by the automated OtoID but not identified by the audiologist, divided by the total number of tests during which the audiologist found no ASHA shift. These estimates are crude in the sense that they ignore the correlation between ears on a participant and, over time, within ears. Accordingly, estimates of the precision of the estimated sensitivity and false-positive rate using conventional methods (Collett, 1991) will be incorrect. Instead, to avoid this problem, a model-based approach as described by Pepe (2003) was followed. The probability that the automated OtoID gave a positive test result as a function of the manual result provided by the audiologist was modeled using logistic regression. Regression coefficients were estimated using generalized estimating equations, to account for correlation among repeated measures. This approach also allowed investigation of the effects that age, baseline hearing levels, and progress through treatment had on the OtoID test accuracy using interactions between each effect and the audiologist test result. Details of this approach, along with several examples, are given in the work of Pepe (2003).

RESULTS

Twenty-one participants were recruited for ototoxicity monitoring. Each participant provided data for left and right ears (total = 42 ears) over 54 monitoring appointments, or n = 108 ear-level monitoring appointments. Participants provided between one and five test appointments (mean = 2.6 tests). Mean cisplatin treatment period between the first and the last treatment was 46 days (minimum = 7 and maximum = 98). Two participants were too ill during one session to provide automated test results, so a total of four automated tests (two left ear results and two right ear results) were eliminated from the analysis. The final sample size for the accuracy analysis was n = 104. As an aside, participants who become too ill for testing are an important finding. These two participants, if untested, are at higher risk than their counterparts who receive monitoring since they may sustain hearing loss that is undetected before administration of an additional dose of cisplatin. These participants provide incentive for inclusion of evidence-based objective tests, such as distortion product otoacoustic emissions (DPOAEs), to be done as part of any ototoxicity monitoring program.

Table 3 summarizes the participant and treatment characteristics of the sample. Mean age was 66.1 yr (range = 54–80 yr). All participants were men. Most participants (n = 12) were being treated for lung cancer, with the remainder nearly evenly split between bladder (n = 5) and head and neck cancer (n = 4). Starting dose of cisplatin ranged from 50 to 100 mg/m², with a mean of 72.1 mg/m². Dosing regimens varied by cancer diagnosis (staging and type). Of the 22 participants, 14 (67%) displayed evidence of an ASHA-criterion hearing shift in the SRO frequencies of either ear.

Table 3.

Characteristics of the Participant Sample

N	21
Age
Mean	66.1
Min	54
Max	80
Cancer type
Bladder (N)	5
Head and neck (N)	4
Lung (N)	12
Starting cisplatin dose (mg/m2)
Mean	72.1
Min	50
Max	100
Hearing change during treatment
N	14
%	67

Note: Mean age is generally consistent with the average age of the veteran population at large. Most participants had lung cancer. Fourteen participants (67%) experienced hearing change, which was somewhat higher although relatively consistent with other reports of hearing change during the treatment for cancer with cisplatin chemotherapy.

Figure 2 shows pretreatment hearing test results from 500 to 8000 Hz of all 21 participants in each ear. The dark line represents the mean of all thresholds. Generally, veterans enter treatment with significant hearing loss. Our past research has demonstrated over multiple studies that cisplatin treatment leads to further hearing loss in approximately half of all patients (Reavis et al, 2011; Dille et al, 2012) and new tinnitus in nearly 40% of patients (Dille, Konrad-Martin, et al, 2010).

Figure 2.

Left and right ear pretreatment pure-tone thresholds (PTT in dB SPL) from 21 cisplatin patients recruited for the study. Dark lines represent the sample mean at each frequency (0.5–8 kHz) for each ear. Thin lines are patient-specific baseline audiograms by ear from 0 to 105 dB SPL, the test limit of the OtoID.

Table 4 shows the confusion matrix comparing the collection of tests done on each ear of each veteran to detect the presence or absence of ASHA-criterion hearing shift (change versus no change in hearing) cross-tabulated by the audiologist (columns) and the automated (rows) testing. The bottom right cell is the number of ear-test sessions that generated an ASHA shift by both methods. The percentage in that cell is the cell count (29) divided by the row total (36), or an automated test sensitivity (80.6%). The bottom left cell is the number of tests in which the automated OtoID documented an ASHA shift, but the audiologist did not find a shift. That cell count (10) divided by the column total (68) is an estimate of the false-positive rate (14.7%). Put another way, the automated OtoID will correctly identify an ASHA shift in slightly more than 80% of monitoring sessions, stable hearing in over 85% of tests, but will incorrectly identify an ASHA shift in about 15% of sessions.

Table 4.

Confusion Matrix of Possible Change When Two Tests are Performed and Compared

	Audiologist (Gold Standard)
	No Change		Change
Automated Self-Test	n	%	n	%
No change	58	85.3	7	19.4
Change	10	14.7	29	80.6

Note: N = 104 tests. The gold standard is designated as the Audiologist test; the comparator test (OtoID in the automated mode) is evaluated in terms of sensitivity, specificity, false positive, and false negative (miss) rates. The OtoID was found to have a sensitivity (bottom right value) >80% and a specificity >85% (top left value). Tolerable false-positive (>14.7%; n = 10 tests) and false-negative (19.4%; n = 7 tests) rates were found.

A logistic regression analysis of the probability of an ASHA-shift identified by the automated OtoID was conducted using generalized estimating equations methodology and is shown in Table 5. Separate models were fit to establish effects of age, average baseline thresholds in the SRO, and dose of cisplatin on the probability of a positive hearing change result. Interactions were also included to evaluate these effects on the sensitivity and alse-positive rates of the automated test. Results indicated no statistically significant effects of age, of baseline hearing in the SRO frequencies, or of cisplatin dose. Therefore, there was no evidence that the accuracy of the automated OtoID was affected by these characteristics.

Table 5.

Logistic Regression Model Results for the Interaction Effects between the Audiologist-Conducted Hearing Test and Patient Age, Patient Baseline Test in the SRO, or Cisplatin Dose (mg/m²)

Parameter	Estimate	Empirical Standard Error Estimates	p Value
Audiologist test × age	–0.0120	0.0781	0.8778
Audiologist test × base SRO	0.1006	0.1048	0.3370
Audiologist × dose	0.0064	0.0125	0.6119

Note: The interaction coefficient (estimate) and associated standard error of the estimate are shown. Significance of each model is shown in the final column. No significant effects were found with any of these factors.

DISCUSSION

We found that self-testing with the OtoID correctly identified 80.6% of ASHA-significant shifts in the SRO frequencies and 85.3% of monitoring occasions where no actual shift occurred. No significant effects of age, baseline hearing, or treatment progression on the OtoID self-test accuracy were found. These results demonstrate that the OtoID is a reasonably accurate clinical testing modality, though confirmation testing by an audiologist following an apparent ASHA shift obtained in the automated test mode is necessary to reduce the false-positive rate.

One purpose of this article was to investigate the efficacy of the OtoID hearing screening device when employed within an evidence-based and efficient ototoxicity monitoring program. This device, developed specifically for this task, enables veteran capability to test his/her hearing just before each chemotherapeutic treatment. While the device's effectiveness is well documented when compared to other HFAs (Jacobs et al, 2012), and on participants not undergoing chemotherapy but who vary in age and hearing ability (Dille et al, 2013), this is our first report of findings with cancer patients. The aim of this study was to establish the sensitivity and specificity of an abbreviated air-conduction hearing test done by the OtoID in a seven-frequency range, known to be most sensitive to early ototoxic hearing change, as a means to monitor hearing for change during cancer treatment. A secondary purpose of the article was to report on tele-audiology efficiencies with a look toward our future of comprehensive and efficient patient care at lower costs. Cost-effectiveness is of particular importance when the addition of any new hearing program is being considered. The OtoID includes a newly incorporated modem with supporting software capable of sending test results using SMS text format. When a veteran chemotherapy patient arrives at the oncology clinic, he or she is given the OtoID by nursing staff to test his or her hearing. On completion of the test, results of the hearing screen are assembled into a report format by the OtoID and sent to a specified e-mail inbox in the audiology clinic, shown in Figure 3. Test results are then inspected for reliability and compared to the baseline test results. Embedded in the text message report are test findings that include the number of intensity level reversals to threshold, the number of false-positive responses (by frequency) to the catch trials, and total test time. These statistics can be used to assess if a patient was capable of providing reliable air conduction screening results. If after comparison with the original testing and review of the testing statistics, the audiologist finds that the hearing results indicate no change, no further action is necessary except to enter a medical chart note of no change in hearing. Screen failures warrant further testing with results communicated to the patient and oncology providers. This simple though highly effective and secure patient-to-provider communication improves professional efficiency.

Figure 3.

An example of a text message that might be received by the audiologist after the automated test is completed. Thresholds are reported by ear and frequency (in kHz). Included in the text are metrics that can be used to determine if the test was reliable: reversals in intensity to obtain threshold and overall test time in seconds.

Fifty-eight tests indicated that no hearing change had occurred. Had this been a clinical setting, no testing would have been required by an audiologist. When compared with “usual care” protocols in which the patient is tested in an audiology clinic by an audiologist at each treatment interval using a comprehensive testing approach (air and bone conduction, speech testing, and otoacoustic emission testing), it is evident that the OtoID self-test screening protocol with telehealth efficiencies represents significant potential for time savings for the clinician. Only those tests with a hearing change need to be verified by an audiologist to insure that results are reliable.

The more difficult problem of balancing false-negative (missed) and false-positive screening audiograms is more difficult to address. False-positive errors represent unnecessary diversion of professional time away from other patients while false-negative errors have negative repercussions for a patient in whom hearing change is missed. All clinical hearing tests have some error. One must assess the balance that exists between the cost of professional time needed to eliminate any and all error in a test against the consequences of an error and the clear benefit of testing. To somewhat address this issue, we performed an additional analysis on our data to determine if there were any “systematic” bias, between the two tests, that could suggest that hearing change measured by the automated test could be accounted for a systematic “error” in the test itself. Figure 4A and B (baseline and monitoring tests, respectively) shows the result of this analysis. Except for the increased variability with hearing severity, we did not find any systematic bias in the data as evidenced by the flat, dotted trend line in both figures. Retest variability (y axis) is equivalent as a function of severity of hearing loss suggesting no systematic bias in either test: baseline or monitoring. The decision for the clinician is the consequence of an error. It could be argued that the negative consequence of error for the veteran in treatment is great since changed hearing is missed, and increases the potential for further change with the next dose. However, the SRO methodology is an “early” detection protocol such that earliest loss typically occurs first in frequencies outside of the most critical communication frequencies (>4000 Hz).

Figure 4.

Shown are the differences between the averaged SRO thresholds obtained by the clinician and the automated SRO thresholds obtained by the veteran, shown as a function of the averaged thresholds in the SRO obtained by either method: (A) the baseline SRO obtained before treatment and (B) the data from all monitoring visits. The dotted lines represent any systematic difference or bias between the two methods of testing. No trend is shown as a function of increasing hearing loss nor between tests (monitor versus baseline). Though there is a slight increase in the variability of thresholds with increasingly severe hearing loss, most change is within 5-dB retest variability.

Automated test sensitivity was greater than 80%, suggesting a very sensitive proxy for a traditional audiologist-based hearing screen though we continue to consider ways to improve test sensitivity. These data were collected under the watchful eye of an experienced audiologist and might, therefore, be less vulnerable to intermittent and unpredictable room noise effects than if the test were done without an audiologist present. This could create bias between the tests though we do not believe that this bias was present in the data reported here since both tests were with an audiologist present. Nevertheless, our noise monitoring system has been upgraded to be compliant with ANSI (1999) maximum permissible ambient noise levels for audiometric test rooms, a potentially more stringent standard for acceptable room noise by the device. We will continue to search for ways to improve the sensitivity of this test; however, all diagnostic tests have some associated error.

Expansions of our current ototoxicity monitoring program are planned that rely on the telehealth capabilities of the OtoID. Our current program targets cisplatin ototoxicity monitoring done in a hospital setting, however, aspects of our program can be used for monitoring the ototoxic side effects of other agents (e.g., other antineoplastic drugs, aminoglycosides) and be implemented in patients’ own homes for added convenience. In our current research, we are nearing completion of our planned migration of the OtoID software and firmware to a tablet-based system. This migration also includes the addition of DPOAE testing as part of ototoxicity monitoring. We have found that DPOAEs are sensitive to early changes in cochlear function and may even provide detection of potential (not only actual) hearing change. More research is needed in this area since DPOAEs are subject to retest variability that increases with increasing time between testing intervals. Defining normal variability of DPOAEs is an area of intensive work within our research group. In addition, we envision that the OtoID will have nearly unlimited options beyond ototoxicity monitoring. The OtoID, designed as an intelligent automated system that measures background noise and ensures test validity, is an ideal platform to support various telehealth initiatives in audiology, and for hearing surveillance approaches such as hearing conservation programs by military and industry, and bedside newborn hearing screening.

CONCLUSIONS

This VA Rehabilitation Research and Development–supported research project is a clinical trial in which we are comparing the ototoxicity monitoring standard of care practice (less frequent serial monitoring using a full diagnostic audiometric test battery) with our comprehensive ototoxicity monitoring program that is administered chairside in the chemotherapy treatment unit using the OtoID, and relies on a set of evidence-based screening measures. As part of this research, we are using the OtoID modem to send text messages from the chemotherapy treatment unit to the audiology clinic e-mail inbox. This provides significant personnel efficiencies in this patient series, which we feel will be important to the success of a program that relies on limited professional resources. Although an estimated 50% of veterans experience hearing change at some point during treatment, the majority of tests result in no hearing change. It is those sessions only with changed hearing that an audiologist's expertise and time are required. We found that the automated OtoID test will correctly identify an ASHA shift in slightly more than 80% of monitoring sessions and stable hearing in over 85% of tests. The OtoID automated test can therefore be recommended for clinical use.

Abbreviations

ASHA

American Speech-Language-Hearing Association

B&K

Bruel & Kjaer

CTU

chemotherapy treatment unit

DPOAE

distortion product otoacoustic emission

HFA

high-frequency audiometer

KEMAR

Knowles Electronic Manikin for Acoustic Research

OtoID

ototoxicity identification device

SRO

sensitive range for ototoxicity

VA

Veteran Affairs