Accessibility analysis using WCAG 2.1: evidence from Indian e-government websites

Abstract

E-government is a global phenomenon. Many governments throughout the world are using e-government websites to deliver government services to their stakeholders. Consequently, it is now quite crucial for the governments to make sure that e-government websites must be accessible to all stakeholders regardless of their visual, cognitive, and hearing ability. However, many prior studies have shown that most of the e-government websites in different countries do not meet the accessibility guidelines prescribed in the Web Content Accessibility Guidelines (WCAG) of the World Wide Web Consortium (W3C). In this article, we present the evaluation of the accessibility of Indian e-government websites using a sample of 65 websites of various ministries based on the WCAG 2.1 standard. We found that the majority of e-government websites do not meet Level A conformance with WCAG 2.1. Our findings suggest that designers and developers of e-government websites should pay due attention to the accessibility features during the design and development of these websites to achieve universal accessibility.

Introduction

In this information age, the internet plays a pivotal role in accessing various information and services. Today people frequently use information systems to communicate and access information ubiquitously. Due to this reason, many governments have introduced e-government websites to cater government information and services to their various stakeholders [16–22]. According to the US general accounting office, e-government services are categorized into four models, namely, (i) government-to-citizen (G2C), (ii) government-to-business (G2B), (iii) government-to-government (G2G), and (iv) government-to-employee (G2E) [30]. The key objective of e-government websites is to provide equal opportunity in accessing information and services. Furthermore, governments should make sure that e-government websites should be accessible to every citizen to achieve citizens’ trust and adoption of these websites. Therefore, accessibility is one of the important factors that determine the quality of e-government websites [1]; it measures how differently abled people access and interact with e-government sites. Accessibility is determined by certain parameters, such as color choice, page layout, readability, etc. Hence, it plays a key role in building trust and ensuring successful e-government execution and service delivery [29].

Though accessibility is an important aspect that determines the quality of an e-government website, earlier studies have reported accessibility issues present in e-government websites [16–22]. Most of the e-government websites of various countries need to improve the accessibility dimension [16, 17, 20–22]. Also, many prior studies used the Web Content Accessibility Guidelines 1.0 (WCAG 1.0) or WCAG 2.0 to assess the accessibility of e-government websites, while very few studies used both. In 2018, the World Wide Web Consortium (W3C) introduced WCAG 2.1 [29]. WCAG 2.1 consists of a wide range of accessibility guidelines and has backward compatibility with WCAG 2.0. However, there is barely any study that can be found in the literature which utilized the latest accessibility guidelines provided by WCAG 2.1 to evaluate e-government websites.

Therefore, this study attempts to address the prevailing literature gap by examining the accessibility of e-government sites based on the latest website accessibility standard WCAG 2.1, in the context of India, a developing nation. Moreover, this study presents implications for practitioners to improve the accessibility of Indian e-government websites by addressing accessibility issues.

The remainder of the paper is organized as follows. In the second section, we present related works. The third section deals with the methodology used to evaluate accessibility of e-government websites. In the fourth section, the results are described. In the fifth section, we discuss implications for practitioners and researchers. Finally, the sixth section concludes the paper.

Related works

Overview of e-government in India

E-government has impacted significantly both developed and developing nations [35]. It uses similar tools and techniques as e-commerce to offer government services to their stakeholders [27]. Many countries have implemented e-government services to their citizens [13–22, 32–34]. The key benefits of e-government include minimizing corruption, increasing efficiency, transparency, convenience, cost reductions, etc. [36]. However, there are three major issues, namely universal accessibility, citizen focus in government management, and privacy & confidentiality that governments should consider for the successful implementation of e-government services [28].

Like other nations, India has also taken major steps toward the development and wide implementation of e-government services. In 2015, the digital India campaign was launched by the Government of India (GoI) to ensure efficient and effective delivery of government services to citizens [26]. Ever since e-government has played a critical role in service delivery to various stakeholders.

However, despite all efforts to improve the adoption of e-government websites, the United Nations E-Government Survey 2020 published by the United Nations Department of Economic and Social Affairs (UN DESA) ranked India 100th in e-government development index (EGDI) [11]. After analyzing the EGDI ranking of India based on all published United Nations E-Government Survey reports from 2003 to 2020, we conclude that the ranking has gradually declined (from 87th in 2003 to 100th in 2020) as shown in Table 1 [2–11]. This shows that e-government initiatives still have a scope to improve in.

Table 1.

EGDI and world ranking of India order to meet international standards

Year	2003	2004	2005	2008	2010	2012	2014	2016	2018	2020
EGDI	0.3730	0.3879	0.4001	0.3814	0.3567	0.3829	0.3834	0.4637	0.5669	0.5964
World Rank	87	86	87	113	119	125	118	107	96	100

Website accessibility

Accessibility refers to making websites accessible to all persons irrespective of their cognitive, visual, and hearing abilities. Websites should provide equal opportunity to everyone to access information [28]. The W3C accessibility standards provide widely recognized guidelines for the accessibility of any website. The WCAG 1.0 was the first standard introduced by the W3C in 1999 to test the accessibility of a website [25]. In 2008, WCAG 2.0 came into existence with backward compatibility having a more comprehensive set of accessibility guidelines [23]. Furthermore, in 2018, WCAG 2.1 was introduced by W3C, which extends WCAG 2.0 with a wide range of recommendations for making web content more accessible to a wider range of persons with disabilities [29]. The main objective of WCAG 2.1 was to improve accessibility for three major groups, i.e., users with low vision, cognitive disabilities, and disabilities on mobile devices.

Accessibility evaluation of e-government websites

A number of earlier studies examined the accessibility of e-government websites in various developed and developing nations [13–22, 32–34]. These studies have highlighted the presence of accessibility issues in e-government sites across various countries based on accessibility guidelines of WCAG 1.0 and WCAG 2.0 standards. Most of these websites failed to meet conformance levels specified by WCAG standards. The accessibility studies on e-government websites in different countries are shown in Table 2.

Table 2.

Accessibility studies

Author (s) (year)	Country of study	Accessibility Guidelines	Sample Size	Tools used	Findings
Abanumy et al. (2005)	Saudi and Oman	WCAG 1.0	27	Multiweb, LYNX, W3C validator service, Bobby	None of the e-government sites of Saudi and Oman conform to all priority 1 checkpoints of WCAG 1.0
Shi (2007)	China	WCAG 1.0	324	Bobby	None of the Chinese e-government sites passed priority 1 accessibility checkpoints of WCAG 1.0
Kuzma et al. (2009)	European Union (EU), Asia, and Africa	WCAG 1.0	72	TAW	Most of the e-government websites of developed and underdeveloped countries did not meet checkpoints of WCAG 1.0
Isa et al. (2011)	Malaysia	WCAG 1.0	155	EvalAccess 2.0	Malaysian e-government sites did not meet accessibility standard
Al-Faries et al. (2013)	Saudi	WCAG 2.0	20	Accessibility Evaluators	None of the Saudi e-government websites comply with WCAG 2.0 guidelines
Al Mourad & Kamoun (2013)	Dubai	WCAG 1.0	21	TAW	Most of the e-government websites did not meet the lowest conformance level of WCAG 1.0
Kamoun & Almourad (2014)	Dubai	WCAG 2.0	21	WaaT	Not a single e-government website met the Level A conformance of WCAG 2.0
Lujan-Mora et al. (2014)	South America and Spain	Section 508, WCAG 1.0, WCAG 2.0	6	TAW, Total Validator, AChecker, eXaminator, and WAVE	E-government websites did not meet the accessibility guidelines
Adepoju et al. (2016)	Nigeria	WCAG 2.0	36	TAW and site analyzer	None of the websites met WCAG 2.0 standard
Akgul & Vatansever (2016)	Turkey	WCAG 1.0 and WCAG 2.0	25	eXaminator, AChecker, TAW 1.0, TAW 2.0, Total Validator, EvalAccess 2.0, HERA	The prevalent priority-1 accessibility problems related to text equivalents for non-text elements were identified
Ismailova (2017)	Kyrgyz	WCAG 1.0	55	EvalAccess 2.0	The results showed an accessibility error rate of 69.38%
Mtebe & Kondoro (2017)	Tanzania	WCAG 2.0	22	SortSite	Most of websites had more than 100 accessibility issues
Paul & Das (2020)	India	WCAG 1.0 and WCAG 2.0	65	EvalAccess 2.0 and AChecker	The outcomes highlighted accessibility error rate for priority 1 and priority 2 of 39.70% and 83.88%, respectively
Csontos & Heckl (2020)	Hungary	WCAG 2.0	25	WAVE	None of the websites adhere to WCAG 2.0

Moreover, most of the earlier studies have used either WCAG 1.0 or WCAG 2.0 to analyze the accessibility of e-government websites. However, few studies have also used both accessibility standards [21, 38, 40]. In 2018, W3C came up with a new accessibility standard WCAG 2.1, but there is barely any study in the literature, which has utilized accessibility guidelines specified by WCAG 2.1 to analyze the accessibility of e-government websites.

Hence, the current study perform accessibility evaluation of Indian e-government sites using WCAG 2.1 standard.

Methodology

Accessibility evaluation

Previous works have used many automated accessibility testing tools, such as TAW, WAVE, AChecker, eXaminator, EvalAccess 2.0, TAW 1.0, TAW 2.0, SortSite, etc., for analyzing the accessibility of e-government websites [15–21, 37]. Most of these tools evaluate the accessibility of a website against WCAG 1.0, WCAG 2.0, and Sect. 508. However, WCAG 2.1 is the current accessibility standard published by the W3C in 2018. WCAG 2.1 provides three diverse conformance levels (Level "A", Level "AA", Level "AAA") for web content accessibility. Table 3 shows the different conformance levels of WCAG 2.1.

Table 3.

WCAG 2.1 conformance levels

Conformance levels	Description	Symbol
A	The web page satisfies Level A success criteria. This is the minimum level of conformance
AA	The web page satisfies Level A and Level AA success criteria
AAA	The web page satisfies Level A, Level AA, and Level AAA success criteria

In this study, an automated website accessibility testing tool, SortSite, was used [37]. This tool has been developed to evaluate whether a website adheres to WCAG 2.1 and Sect. 508 guidelines or not. The tool provides accessibility testing of first ten web pages of a site in the online version.

Data

In the present study, a total of 65 e-government websites were selected across various ministries, such as information and broadcasting, law and justice, finance, electronics and IT, external affairs, tourism, science and technology, communications, road transport and highways, and corporate affairs of GoI. Out of these 65 websites, 27 sites belong to G2C, four websites are of G2B, and 34 are G2G sites. All websites belong to central and state government agencies. The dataset consists of a representative sample of Indian e-government websites. Information about these websites is available at [12, 24]. These websites are available in English.

Results

We analyzed the accessibility of 65 e-government websites using the SortSite tool. We identified that the average accessibility checkpoints errors across all 65 websites for Level A and Level AA are 8.34 and 1.74, respectively, as shown in Table 4. Our findings indicated the evidence of accessibility errors based on both Level A and Level AA of WCAG 2.1 in the Indian e-government websites. However, none of the websites had accessibility errors based on Level AAA. We also found that out of 65 e-government websites, seven sites satisfy conformance Level AAA. Most of the accessibility errors are Level A, which is the minimum conformance level that every e-government website must satisfy for essential website accessibility. Otherwise, some groups of users find it impossible to access information on the websites.

Table 4.

Level-wise total number of errors and average errors present in e-government websites

E-gov. website category	Accessibility errors
	Level A	Level AA	Level AAA
G2C	329	56	0
G2B	37	8	0
G2G	176	49	0
Total	542	113	0
Mean	8.34	1.74	0

Moreover, on average, 1.74 Level AA errors were found in the Indian e-government websites. The presence of such errors makes websites challenging to access for some groups. We found 655 accessibility errors across all checkpoints of Level A, AA, and AAA. The percentage of total accessibility issues at various accessibility levels of WCAG 2.1 is shown in Fig. 1.

Fig. 1.

Percentage of total accessibility errors in different levels

We found a number of violated checkpoints, including WCAG 2.1 A F54, WCAG 2.1 A 3.1.1, WCAG 2.1 A 4.1.1, WCAG 2.1 A F68, and WCAG 2.1 A F16 in Level A; WCAG 2.1 AA 1.4.3, WCAG 2.1 AA F24, WCAG 2.1 AA 2.4.5, and WCAG 2.1 AA 2.4.6 in Level AA. The three most ignored checkpoints of WCAG 2.1 were WCAG 2.1 AA 1.4.3 (69.23%), WCAG 2.1 A F54 (64.62%), and WCAG 2.1 AA F24 (60%) (See Table 5). Thus, the accessibility error rate for Level A was 89.23%, Level AA was 78.46%, and Level AAA was 0.00% in the e-government websites.

Table 5.

Summary of violated WCAG 2.1 checkpoints by Indian e-government websites

Violated Checkpoints	No. of e-government websites	Description
WCAG 2.1 A F54	42	Every onmouseout handler should have an equivalent onblur handler. Also, onclick handler should have an equivalent onkeyup or onkeydown handler
WCAG 2.1 A 3.1.1	24	Lang attribute should be used to identify the language of the page
WCAG 2.1 A 4.1.1	29	The same id is used on more than one element
WCAG 2.1 A F68	31	Hypertext Markup Language (HTML) form control has no label
WCAG 2.1 A F16	21	Marquee text is tough to read for low vision users
WCAG 2.1 AA 1.4.3	45	Set the text and background colors with enough contrast
WCAG 2.1 AA F24	39	Set the colors on the body or elements consistently
WCAG 2.1 AA 2.4.5	13	Provide information about the general layout of a site using a “Sitemap” or table of contents
WCAG 2.1 AA 2.4.6	12	Form field labels should be unique on a page

Discussion

E-government websites play a crucial role in delivering government services to different stakeholders. Moreover, the importance of these websites has grown significantly in India during the present COVID-19 situation since most government services are being provided through online mode. Therefore, the accessibility features of these websites play a critical role in delivering services to a wide range of users.

The present study contributed to the web accessibility literature by analyzing the accessibility of the Indian e-government websites using a sample of 65 websites of different ministries. The results reveal accessibility checkpoints errors based on Level A and Level AA of WCAG 2.1. Only a few Indian e-government websites adhere to accessibility guidelines of the WCAG 2.1. Our findings are similar to the existing accessibility studies, which have used WCAG 1.0 and WCAG 2.0 standards [21, 38, 40]. The outcomes highlight that many Indian e-government websites do not have conformance to Level A and Level AA of WCAG 2.1. Since these conformance levels are vital to improving universal accessibility, e-government sites should be designed and developed based on the accessibility guidelines prescribed by WCAG 2.1. The most violated checkpoint was WCAG 2.1 AA 1.4.3 (45 websites), followed by WCAG 2.1 A F54 checkpoint (41 websites) and WCAG 2.1 AA F24 (39 websites). Outcomes indicate that there is scope to improve the accessibility of Indian e-government websites. The following improvement measures such as onmouseout handlers should have an equivalent onblur handler, onclick handlers should have an equivalent onkeyup or onkeydown handler, unique id should be used for all elements, setting lang attribute to identify the language of the page, adding span element instead of scrolling marquee text, text and background colors have enough contrast, text color on the body and background color must be set correctly due to browser defaults, etc. should be considered during the design and development of e-government sites. The National e-Governance Plan (NeGP) of GoI aims to make all e-government services available to their stakeholders [26]. Hence, every e-government website should be designed and developed according to accessibility standards to improve the quality of these websites and for better citizen participation.

Implications for practitioners

The present study has a number of implications for e-government policymakers, website designers, and developers. The accessibility analysis results highlight specific critical accessibility issues based on WCAG 2.1, due to which some groups may find it impossible or even difficult to access information on the Indian e-government websites. Thus, the results reveal that accessibility features have not been given due attention during the design and development of e-government sites. Therefore, designers and developers should pay due attention to resolve the accessibility issues present in e-government websites: they should consider necessary accessibility improvement measures, such as adding an equivalent keyboard event handler to help users who are unable to use a mouse; unique id should be used for every webpage element; the lang attribute should be set to identify the language of the page; the marquee element should be replaced by a static element like span to allow low vision users sufficient time to read the text; text and background colors should have adequate contrast, as some users find it hard to read text on a white background, dark gray text on a black background and white text on a red background; text and background color must be set correctly according to browser defaults, etc.

Implications for research

The current study extends the accessibility literature of e-government websites in two ways. First, to the best of our knowledge, the present study is the first of its kind that evaluates the accessibility of Indian e-government websites using the WCAG 2.1 standard. This study can also be used to evaluate the accessibility of e-government websites in other countries. Second, this study highlights the accessibility problems and the necessary measures to improve the accessibility of e-government websites.

Conclusion

Accessibility is one of the critical dimensions to determine the quality of e-government websites. India is the second largest country in the world in terms of population, and as per the Census 2011, about 2.68 crores of people are disabled. Therefore, this study attempts to analyze the accessibility of Indian e-government websites using online evaluation tools. The outcomes clearly illustrate accessibility issues based on Level A and Level AA checkpoints of WCAG 2.1. Thus, we can conclude that the accessibility aspect has not been given due attention during the design and development of e-government websites. Poor accessibility may lead to ineffective service delivery and poor adoption among differently abled users. Therefore, there is a necessity to improve the accessibility of e-government websites, and this may help to improve the EGDI ranking of India.