A Comparative Study of the Sensitivity and Specificity of the Ishihara Test With Various Displays

Abstract

Objectives

Visual colour differentiation in clinical research requires colour-competent (CC) participants. The Ishihara colour charts (ICC) have established themselves as the standard for CC screening of colour vision deficiencies (CVD). However, the extent to which the results can be compared with a presentation of the colour charts on a smartphone display (SD) is currently unknown. The aim of this in vitro study was to determine the sensitivity and specificity of the Ishihara colour deficiency test depending on the presentation mode.

Methods

Dental students (female n = 28; male n = 10; mean age, 23.5 ± 2.65 years; median age, 23.0 ± 13.0 years) evaluated 25 Ishihara test plates on their SD (n = 38) and/or a calibrated monitor (HP monitor, 22-inch; n = 18). The median size of the SD was 6.0 inches. Datasets with more than 2 failed charts were scored.

Results

When the Ishihara test charts were presented on a PC screen, the sensitivity was 94.4% and the specificity was 82.4% (0 mistakes: n = 14, <3 failures: n = 3, 14 false answers: n = 1). On the SD, a sensitivity of 96.0% and a specificity of 94.7% were calculated (89.5% were correct; 4 participants [10.5%] made <3 errors; and 1 participant made 21 errors). No significant difference between display modi (PC vs SD) was evaluated (P > .05).

Conclusions

The presentation of ICC on an SD is useful and can be used for the investigation of a possible CVD of large groups. Comparable results to data projection can be achieved with a high degree of certainty. For CVD screening of larger groups (eg, students in preclinical training as part of CC training), the presentation of ICC on the SD can be used. This research was able to demonstrate that the sensitivity and specificity of the usual presentation method (Ishihara's booklet or data projection) is comparable.

Introduction

As 8% of the male population and 0.5% of the female population have congenital colour vision deficiencies (CVD),¹ it is crucial to consider illumination, test conditions, and examiner experience as essential requirements in colour discrimination studies.2, 3, 4 For the detection of possible colour blindness, Professor Shinobu Ishihara developed 38 colour charts in 1917 which can reveal potential red-green visual defects.⁵ The Ishihara test set contains a total of 38 test charts with pictures of dots in different sizes and colour shades. The dots and circles in the images are arranged in specific patterns to form numbers or figures that people with normal colour vision are able see. People with CVD are only able to recognize the numbers on the Ishihara charts to a limited extent. So some figures and numbers on the Ishihara test charts are only visible to people with CVD and invisible to other people with normal colour vision. The term colour vision deficiency refers to all hereditary or acquired disorders of colour perception, as a result of which the function of the colour-sensitive cones on the retina of the eye is impaired. The consequences are weaknesses in the perception of individual colours (dichromasia) or all colours (achromasia). Red-green deficiency and red-green blindness are the most common forms of CVD. In this study, CVD was defined as impairment in the ability to recognise colours and contrasts. The symptoms of a colour vision disorder can range from mild to severe or vary in severity. Achromatopsia (colour blindness) refers to the condition of those who cannot distinguish colours and shades of colour. In summary, protanopia (red-blindness), deuteranopia (green-blindness), tritanopia (blue-blindness), and achromatopsia (colour blindness) can be distinguished. Nowadays, the most commonly used test for CVD besides the Ishihara test (10.88%) is the Farnsworth-Munsell (7.04%), followed by the Farnsworth-Munsell 100 Hue (6.4%) and Hardy-Rand-Rittler (3.2%) tests. Whilst web-based online tests and the anomaloscope have been introduced as the gold standard (1.28%), they tend to be complex to use or require specialised equipment.⁶

The Ishihara test results give an indication of whether a colour vision defect is present or not; however, a quantitative statement can only be made if the results are cross-checked with one of the abovementioned options. The Ishihara test was originally developed for the detection of CVD and is considered the most effective test for this purpose, albeit when testing larger groups. Here in particular, it must be pointed out as a limitation that the test is very time-consuming, especially when examining large cohorts. van Staden et al⁷ also point out that the Ishihara test is not able to recognise the severity of visual impairment. In this case, the authors recommend a more thorough, individualised examination by trained professionals if participants are found to have visual impairment. Other authors also confirm that this single test is considered to be very time-consuming.8, 9, 10 In this regard, there is a need to improve and standardise the Ishihara test. To accomplish this, the authors modified the Ishihara test by performing a modification of the presentation format and modifying the Ishihara test for mass screening purposes. Ganley and Lian¹¹ and Gündogan et al¹² have used projected slides in addition to the classical Ishihara test in mass screening. In the comparison, both methods achieved a specificity and sensitivity of 100%.^{11, 12}

The research group led by Campbell et al¹³ investigated the effectiveness of printed and digital iPad versions (Apple) of the Ishihara charts in screening CVD charts. They found that the use of electronic presentation forms provided comparable results.¹³ The high specificity of the electronic form of presentation was also noted by Ing et al,¹⁴ who, in addition to computer emulation of the Ishihara test, used the optical Hardy-Rand-Rittler (HRR) test. In a study by Khizer et al¹⁵ on the comparability of different mobile smartphone screens and the Ishihara booklet, no significant difference between the 2 smartphone groups (iPhone or Android) and the group with the Ishihara booklet was shown. The authors concluded that smartphones could be a viable alternative to the standard Ishihara booklet test. However, different smartphone screens present a challenge in standardising colour vision, which is not the case when using the Ishihara booklet.¹⁵ Sorkin et al¹⁶ stated that users of CVD apps (Eye2Phone and Color Vision Test application [CVT app]) should be aware that, compared to the Ishihara test booklet, some apps may have different sensitivity for the detection of CVD, limiting their clinical usefulness.¹⁶

With the advancement of smartphones, explicitly the display technology, whether they can also be used for the evaluation of CVD must be explored. The currently available displays are divided into displays with the designation LCD or TFT. This technology is based on liquid crystals which are made to glow. The IPS displays (in-plane switching technology) with better contrast and viewing angle stability differ from the OLED technology ("AMOLED," "Super AMOLED" display) to other display types in that no backlight is used. Due to the self-luminous organic diodes, high black levels can be expected thanks to the self-illuminating organic diodes, as well as strong colour saturation and good viewing angle stability. Another parameter for the display quality is the resolution and how many pixels are accommodated on the display. The currently available display sizes vary from 4 to 6.5 inches.

Unfortunately, the literature is lacking information regarding the sensitivity and specificity of the electronic presentation on smartphone displays (SDs) and computer screens. The aim of this study was to close this scientific gap and eliminate the lack of knowledge. The hypothesis of this study was that no significant differences between the display methods (calibrated computer screen vs smartphone) in Ishihara's CVD test can be found.

Methods

Dental students (female n = 28, male n = 10) from the University Medicine Leipzig, the University Dresden, the Free University Berlin (Charité), and the University Greifswald voluntarily participated in the study (Figure 1). The mean age (±SD) was 23.5 ± 2.65 years, with a median of 23.0 ± 13.0 years (Figure 2). The study was reviewed and approved by the ethics committee of the University Greifswald (BB 175/22). Student cohorts evaluated 25 colour plates of the Ishihara CVD test in different presentation formats. All participants used their personal smartphone to complete the initial screening test. Eighteen participants also carried out the Ishihara test on a computer monitor (HP monitor, 22 inch). The monitor screen was calibrated using the free calibration monitor software Monitor Calibration Wizard 1.0 (hex2bit.com), which is a colour profile creator for the Windows system. The application helps you to remove unsightly colours in the display and restore the display to its factory default state. Calibrating an SD is necessary if the preconfigured values are not optimal or the display has changed over time. To improve the display, the brightness and contrast are primarily improved; a change in the colour scale is possible with additional applications. As the age of the smartphones was <3 years, standard calibration was not used and the individual display settings (brightness and contrast) were used. In contrast, the monitors, whose lighting illuminates a larger area as evenly as possible and whose surfaces are at risk of ageing due to burnt-in images, were calibrated. For every run, 25 Ishihara charts were displayed for 15 seconds each. The total duration of the test was 7 minutes. The participant's individual viewing distance to the monitor was about an arm length, and on the smartphone about 20 to 35 cm. Participants recorded the results of the monitor test according to a multiple-choice procedure on prepared protocol sheets. The evaluation includes the percentage of incorrect decisions or the number of errors depending on the 2 display methods (SD vs PC screen). Datasets of all participants were scored. Result's significance was evaluated using the Student t test. The significance level was determined to be α = 0.05.

Fig. 1.

Gender distribution in the proband's cohort (%).

Fig. 2.

Students’ age distribution within the cohort with a mean age of 23.5 ± 2.65 years.

Results

The results obtained by the participating students were comparable and are therefore summarised. The smartphone's mean (±SD) age was 2.83 (±1.2) years and the median was 3.0 years. The display's mean (±SD) size was 5.88 (±0.6) inches and the median was 6.0 inches. When displaying the Ishihara test charts on a PC screen, 14 participants made no mistakes. Three participants made fewer than 3 errors: 2 participants made 1 error and 1 study participant made 2 errors, respectively; 1 participant made 14 (5.6%) errors (Figure 3). Of the 18 participants, one individual (5.5%) showed red-green weakness in the Ishihara test on closer evaluation and was demonstrably colour vision–deficient. In the present study, the sensitivity of the Ishihara test per monitor presentation was 17/18 = 94.4%, and the specificity was 1/17 = 82.4% (Table).

Fig. 3.

Percentage error distribution depending on the presentation mode.

Table.

Sensitivity and specificity depending on the presentation mode.

		Sensitivity	Specificity
	PC display (n = 18)
Normal vision	17	94.4	-
Colour blind	1	-	82.4
	Smartphone (n = 38)
Normal vision	36	96.0	-
Colour blind	2	-	94.7

No significant differences between the presentation modi were evaluated (P > .05).

When the Ishihara test was presented to the participants on the smartphone, 34 participants (89.5%) gave correct answers (true positive). Four participants (10.5%) made 1 or more errors: 2 participants made 1 error and 1 participant made 3 errors. Of the 38 test participants, one showed CVD (with 21 false detections) when presented with the Ishihara test chart on the SD. During the results’ verification and interview, the participant stated that he had red-green CVD, which he was aware of in advance. The false-negative result of 3 or more errors of 2 participants leads to a sensitivity of 24/25 = 96.0% and a specificity of 94.7% (Table). In the gender comparison, the participant with the comparatively high number of failures was male. No significant difference between the display methods (SD vs PC screen) was seen (P = .503)

Discussion

In the present study, the results of 2 presentation modes were compared to perform the Ishihara test for detecting colour perception deficit, especially red-green CVD. Previous research has shown that the Ishihara test exhibited a sensitivity of 97% and specificity of 100%.¹⁶ However, in the current study, the sensitivity and specificity for the monitor projection were 94.4% and 82.4%, respectively. For the smartphone projection, the specificity was 94.7% and the sensitivity was 96.0%. The criterion used for evaluation was an error rate of 3 or more mistakes. The results indicated that the different screens (PC monitor vs smartphone screen) did not significantly influence colour differentiation, and the null hypothesis regarding sensitivity was not rejected but the hypothesis related to specificity was rejected.

Bratner et al^17,18 evaluated the sensitivity of a presentation using a data projector and achieved a sensitivity of 99.6% and specificity of 100% along with a sensitivity of 21.4% for the Farnsworth 15 test. Cole et al¹⁹ reported a sensitivity of 100% and specificity of 97.5% for 2 errors and 98% for 3 errors, respectively, for the HRR test. Using a group of 111 participants, Ganley and Lian¹¹ compared the extent to which printed Ishihara colour charts could be replaced by HRR test charts using slide projection. They concluded that the sensitivity of both projection methods was 100%; for the Ishihara panels, they reported a specificity of 98.1% but only 20.8% for the HRR test panels. Comparable results were found in a pilot study by Gündogan et al.¹² In their study with a young cohort of 104 individuals aged 18 to 25 years, the Ishihara test panels were presented conventionally and as a slide projection, and the results were subsequently compared. The authors were able to demonstrate a sensitivity and specificity of 100% each and recommended the imaging method (slide projection) in large cohorts.¹² The sensitivity and specificity of colour vision smartphone applications, as reported by Sorkin et al,¹⁶ were 100% and 95.2% (Eye2Phone application) and 100% and 54.8% (CVT app). The authors found no significant difference between the conventional Ishihara test chat presentation method and smartphone applications (P < .001). When reviewing our results, it should be kept in mind that standardising different smartphone screens could be a challenge. For standardisation, a reference to the background colour of the SDs might be necessary. Some smartphones have power-saving modes that reduce the backlight's intensity. With regard to this study's results, we must also discuss whether the same conditions can be assumed when carrying out the test. When using smartphones, a reflection on the screen surface and the ambient light brightness could have an influence on the results. Unfortunately, no information is available as to whether the screen of the smartphones was protected by an antiglare film or a protective film. These films generally reduce the brightness of the screen but do not affect colour reproduction. In a study conducted by this group of authors, the influence of grey-tinted glasses on colour selection was negated.²⁰ The influence of yellow-tinted glasses on colour differentiation was also not proven in a further study.²¹ Another aspect that needs to be discussed in connection with those presented here is the colour reproduction on SDs. The smartphones used in the study had an average age of around 3 years. Thanks to advances in display technology, the colour reproduction corresponds to the full range of the spectrum (sRGB colour standard or 16 million colours), meaning that small colour nuances can also be reproduced. Further studies dealing with the suitability of smartphones should provide recommendations for use or mention key points for implementation.

For the assessment of a possible CVD or further studies in large groups, the web-based display of the Ishihara test charts on the smartphone screen can be used. For the screening of CVD in larger groups, smartphone screens can be used in addition to the classic display of Ishihara color maps, whereby comparable results can be achieved with both methods.

Conclusions

The present study was able to demonstrate the effectiveness of screen presentation of the Ishihara test charts on both the SD and PC screen with comparable results. For further studies with a large group of participants or the evaluation of possible CVD of large cohorts, the display on the smartphone can be used and comparable results can be obtained.

Funding

This work was supported by VITA Zahnfabrik, H. Rauter GmbH and Co, KG, Germany.

CRediT authorship contribution statement

Thomas Klinke: Conceptualization, Formal analysis, Investigation, Writing – original draft, Writing – review & editing. Wolfgang Hannak: Conceptualization, Investigation, Writing – review & editing. Klaus Böning: Conceptualization, Investigation, Writing – review & editing. Holger Jakstat: Conceptualization, Methodology, Formal analysis, Resources, Writing – review & editing.

Conflict of interest

None disclosed.