Read and accepted? Scoping the cognitive accessibility of privacy policies of health apps and websites in three European countries

Abstract

Objective

Trust and accessibility are vital to adoption of health and wellness apps. This research scoped three elements of cognitive accessibility of health app privacy policies: availability, ease of navigation, and readability.

Methods

For this cross-sectional study, quantitative data collected in the Netherlands, Sweden, and the United Kingdom included: whether privacy information was in a country's official language (availability); number of distracting visual elements (ease of navigation); word count and Common European Framework of Reference (CEFR) reading level (readability). Health app privacy policies were compared to policies from a purposively selected sample of websites, and to benchmarks, including CEFR reading level B1.

Results

Health app privacy policies were less often available in countries’ official languages compared to sampled websites (Chi-Square [1, 180]  =  57.470, p < 0.001) but contained fewer distracting visual elements. More UK privacy policies were in the country's official language, whereas Swedish privacy policies contained fewest words and fewest potentially distracting design elements. Only one privacy policy met the CEFR reading level benchmark.

Conclusions

Lack of privacy information in non-Anglophone app-users’ native languages and high reading levels may be major barriers to cognitive accessibility. Web and app developers should consider recommendations arising from this study, to stimulate trust in and adoption of health and wellness apps.

Introduction

Online data privacy is an important legal and ethical issue, about which many people are concerned.¹ Legal instruments such as the European General Data Protection Regulation (GDPR) protect people’s rights to access information about how their data is processed, so that they can make informed choices.² With respect to health, it is a fundamental principle of healthcare ethics that we respect autonomy, which includes respecting an individual's right to make informed choices.³ From a practical perspective, perceived transparency of privacy information has been positively associated with trust in and adoption of eHealth systems.^4–7 Transparency around privacy online therefore impacts the extent to which people are willing and able to access online environments and benefit from technological advances promoting health. Transparency and informed decision-making are particularly important, as data-collection from app and web users occurs on a large scale, sometimes in ways which have the potential to compromise their health privacy.⁸ These issues loom large as at least 350,000 health and wellness apps are now available via the leading app stores worldwide.⁹

Unfortunately, legally mandated disclosure of information about data collection and processing does not always seem to reassure technology users. Under the provisions of GDPR, for example, all organizations must inform service users about the fundamental aspects of how their personal data is collected and processed, which is frequently presented in the form of a privacy notice or privacy policy.^10,11 However, many people still seem to be uncertain about how online activities may impact their health privacy and overly-complex and inaccessible information may in fact contribute to ‘privacy fatigue’, where people feel a loss of control about their online personal data.^12,13 Indeed, previous research has demonstrated that it could take the average adult internet-user 244 hours per year to read all the privacy policies relevant to their online activities.¹⁴ During the authors’ own research into technology use by people with cognitive impairment, privacy concerns also seemed to influence use of apps on a tablet by this group. During the FindMyApps study, experience from a process evaluation indicates that participants frequently struggle to find answers to three questions with respect to an app: What information is collected?; Who has access to the information; and For what purpose is the information used?¹⁵ The accessibility, and particularly the cognitive accessibility, of privacy policies merits further investigation.

Cognitive accessibility is an emerging conceptual extension of web accessibility. Web accessibility, as operationalized in the Web Content Accessibility Guidance (WCAG) is a long-established concept in the design of digital technologies, which aims to provide a standard based on which all people can “perceive, understand, navigate, and interact with the Web.”¹⁶ Cognitive accessibility has been understood to mean that “digital services are simple, consistent, clear, multimodal, error-tolerant, and attention-focusing to use, taking into account all users,” including those with cognitive impairments.¹⁷ Accessing and interacting with websites places demands on multiple cognitive domains including complex attention, language, and learning and memory.¹⁸ For people with impairments in one or more cognitive domains, online environments may be less easily accessible than for those without cognitive impairment. Cognitive accessibility is therefore an important factor in ensuring equitable access and universal design.¹⁹ At least one internationally recognized standard to operationalize requirements around ‘cognitive accessibility’ is currently in development.^20,21 To our knowledge, the cognitive accessibility of online privacy policies is yet to be evaluated. We focused particularly on privacy policies associated with health and wellness apps, and also with the websites of charitable, governmental, and health and care organizations (organizations providing medical, nursing, or social care services). We anticipated that such organizations should be the most likely to provide cognitively accessible privacy policies, because users of their website are more likely to experience cognitive impairments (in the case of relevant charitable and health and care organizations), or because they are expected to be accessible for all citizens (in the case of governmental organizations) or because of a strong ethical imperative to support autonomy (in the case of health and care organizations). Accordingly, the primary goal of this research was to scope the potential cognitive accessibility of privacy information associated with health and wellness apps and websites. We sought to identify the nature and scale of challenges, and to identify examples of good practice, in order to make recommendations to app and website developers, policy-makers, and individual app and website users.

While a definitive operational definition of cognitive accessibility is yet to be established, a previous literature review of studies addressing cognitive accessibility identified—alongside privacy as an issue in itself—“content and language” and “design of the digital service” as two important themes.¹⁷ In line with these themes, our research questions were: to what extent is privacy information available for health and wellness apps, compared to selected websites? How easy is it to navigate to the privacy information regarding an app or website, given the user-interface? What is the readability of the text of the privacy policy, considered in terms of structure of the text and linguistic complexity? Our hypothesis was that the availability, ease of navigation and readability of privacy policies associated with the selected websites would be better than those associated with the apps. The results of this research will inform ongoing development of an internationally recognized operational definition and standard for cognitive accessibility.

Materials and methods

To answer our research questions, a cross-sectional, multicenter study was undertaken. GDPR is a European regulation, which may be implemented in different ways in different countries. Cultural and linguistic differences might also be expected to influence cognitive accessibility of privacy information between countries. We collected data in three countries which have implemented GDPR: the Netherlands, Sweden and the United Kingdom. For each research question, variables were defined, and quantitative and qualitative data were collected to analyze the observable properties of the privacy policies.

Availability of privacy information

Researchers recorded observations with respect to the following binary variables: was a link to a privacy policy identified? Was a summary of the privacy policy available? Was information available in an official language of the country in question? Were contact details for further information or questions available.

Ease of navigation to information and potentially distracting design elements

We used a limited selection of low-level state-of-system heuristics to structure data collection.^22,23 With respect to the design of the privacy policy, we defined potentially distracting design elements based on the potential of certain design features to make reviewing the privacy policy less “simple, consistent, clear, multimodal, error-tolerant, and attention-focusing.”¹⁷ Specifically, we considered that any of the following design features could be distracting: pop-ups, GIFs, and videos (if the content was not related to the privacy policy); and hover/rollover effects (features which change in appearance when placing the cursor over them).

Assuming a scenario in which a visitor arriving on the app store page of an app or the homepage of a website, seeks reassurance by navigating to and reviewing the privacy policy before installing the app or further interacting with the website, we observed: how many clicks were necessary to navigate from the homepage or relevant app store page to the privacy policy, by what the researcher found to be the shortest possible route; and the number of potentially distracting design elements within or on the same page as the privacy policy. The researchers also kept qualitative notes on anything which they found unclear, which complicated the process of navigating to the privacy policy, or which may have constituted a distracting design element but did not meet our criteria.

Readability (text structure and content)

To assess readability, a communicative purpose—the result that the communicator is hoping to achieve—should be defined.²⁴ We summarized the hypothetical communicative purpose of the privacy policies in our research context as follows: “a person with cognitive impairment should be able to understand how their privacy may be affected by using the app/website, so that they can make an informed decision about whether to use the app/website, resulting in more efficient adoption of technology.” We operationalized the structure and content aspects of readability with this purpose in mind.

With respect to structure, we observed: the length of the text (number of words and reading time); the grouping, ordering and granularity of headings used to divide the information into sections, specifically noting whether or not there were headings pertaining to the three most frequently asked questions in the FindMyApps study (“What information is collected?”; “Who has access to the information”; and “For what purpose is information used?”), and if so, which number subheading these were.¹⁵

With respect to content, different scales for linguistic complexity exist for each of the languages investigated in this study: the Lix scale for Swedish, the Flesch Reading Ease scale in English, and a modified Flesch Reading Ease scale for Dutch, for example.^25–27 For this study we recorded the observed level of each text using the Common European Framework of Reference (CEFR).²⁸ An advantage of the CEFR for this study is that it places all European languages on a common scale, with six levels of increasing complexity: A1, A2, B1, B2, C1, and C2. While there has been criticism of CEFR's use to assess readability and linguistic complexity, it remains widely used and the Dutch government agency responsible for government communications uses CEFR level B1 as a cut-off to define its policy on readability of texts.^29,30 An English-language description of the B1 level for a language-user is: “I can understand texts that consist mainly of high frequency every day or job-related language. I can understand the description of events, feelings and wishes in personal letters”.³¹ Previous research of readability of texts on Dutch local government websites has found that the B1 target is rarely met.³²

Sample of apps and websites

We aimed to identify 30 health apps and 30 websites per country (total n  =  180 privacy policies). Within the Google Play Store and Apple App Store, as accessed from the researchers’ smartphones in the Netherlands, Sweden and the UK, in October 2021, we identified the top 15 apps listed under the category of ‘Health and Fitness’ in each store. We selected this category instead of ‘Medical’ because many of the ‘Medical’ apps are targeted at healthcare professionals, whereas we were particularly interested in apps designed for non-professional target users. In the Google Play Store this was achieved by selecting ‘Categories’, then ‘Health and Fitness’ and expanding the list of apps. In the Apple App Store this was achieved by expanding the list of ‘Top Categories’, selecting ‘Health and Fitness’ and then expanding the list of ‘Top Free’. The list of ‘Top’ apps in each store is likely influenced by characteristics of the user, and changes over time, as both Apple and Google use complex algorithms to curate these lists. To avoid duplication, each app was sampled only once. If the organization which owned the app was based in one of the three countries, it was sampled in that country. If the organization was not based in any of the three countries compared in this research, but in an Anglophone country, it was sampled in the UK, and otherwise it was sampled in whichever country it appeared higher up the list of “top apps.” To retain approximately equal numbers of apps in each country, replacement apps were sampled after deduplication, taking the next app on the list of top apps in the category, and again checking for duplication, until a sample of approximately the same number of unique apps from each country was reached. Governmental organizations, health and care providers and relevant charitable organizations in each country were selected by the researchers by purposive sampling. The aim was to include a variety of national and regional organizations within each sector. The organizations all had publicly accessible websites.

Data collection

Apps were initially sampled in 2021 in the Netherlands from 31^st October (from Google Play Store) and 2^nd December (Apple Store), 5^th December in Sweden and 1^st December in the UK. Between August and December 2021, we visited the pages in the app stores of each of the apps identified, and followed the provided link to access the privacy policy. We visited the websites of the selected governmental, healthcare and charitable organizations and navigated from the homepage to the privacy policy. Website data were initially sampled in 2021 from 9^th September in the Netherlands, 7^th September in Sweden and 30^th September in the UK. All privacy policies were copied to Microsoft Word. Each researcher used the tool constructed for this research to record data regarding background information, availability of the privacy policy and ease of navigation to the privacy policy while retrieving the information. Once retrieved, the researcher recorded observations regarding structure and linguistic complexity of the text in the same tool. The tool used was validated and discussed by the research team following collection of data regarding the first six privacy policies, to ensure consistent use by all researchers.

We used the built-in word count tool in Microsoft Word to generate word counts for the texts. These word counts were converted to estimated reading times, based on recent data regarding average reading speeds of adult native-speakers under the age of 60, without reading problems or cognitive impairments, in each of the three languages.³³ Reported reading rates were: English 236 words per minute (wpm), Dutch 228 wpm and Swedish 218 wpm. Given that reading speeds have been reported to decline with age, and that these estimates are drawn specifically from a population without any cognitive impairment, these estimates may be viewed as a highly conservative estimate of the time taken to read the privacy policies by our target group.

To estimate the CEFR level, different algorithms must be applied in each language. For texts in Dutch, we used the “Leesniveau tool” provided by Stichting Accessibility.³⁴ The Leesniveau tool has previously been shown to better correlate with other metrics of language complexity than alternative Dutch language tools for estimating CEFR levels.²⁹ For texts in English we used the tool “Text Inspector,” provided by Weblingua Ltd and The Centre for Research in English Language Learning and Assessment (CRELLA) at the University of Bedfordshire.³⁵ The most recent large-scale validation study of this model classified 44% of texts at exactly the same CEFR level as human raters, and 91% of texts were classified to either the same level or an adjacent level.³⁶ Texts in Swedish were analyzed using the tool “Lärka-verktyg,” provided by the University of Gothenburg.³⁷ This tool has been validated on a large dataset and achieved an accuracy of 81.3% (F-score = 0.8) of texts correctly classified to the same CEFR level as a human rater.³⁸ Lärka-verktyg does not distinguish between level C1 and C2, and we therefore interpreted texts estimated at C1 level to be “at least C1.”

Data analysis

Following initial data collection, researchers’ qualitative notes were used as part of a triangulation process: the data collected by each researcher was independently reviewed by the other researchers, and any discrepancies and accompanying notes discussed, to decide whether to expand the initially defined list of potentially distracting design elements.

We used SPSS to generate descriptive statistics with respect to quantitative data (mean and standard deviation for continuous variables, counts and percentages for dichotomous present/absent variables), broken down by country and by sector for privacy policies associated with websites and apps. Differences between the sector to which the owner of the privacy policy belonged, between the three countries and between the three languages were tested for using one-way ANOVA, Kruskal–Wallis tests or Chi-squared tests, depending on level of measurement and distribution. The sample size was chosen based on a priori sample size calculations performed for one-way ANOVA with G*Power v3.1, assuming alpha = 0.05 and power = 0.8 and at least a medium effect size. We considered small effect sizes unlikely to be of practical importance for this scoping research. We expected that at least medium effect sizes may be found, given that different sectors have different target users, and different motivations in communicating about privacy. For example, charities representing people with cognitive impairments might reasonably be expected to put more effort into making their privacy information more readable for that target group, compared to private sector organizations producing general health and wellness apps. In the private sector, the collection, processing and sale of user-data can be an important part of an organization's business model, which they may wish to obfuscate, whereas public and charitable organizations may have more incentive to build trust through transparency.⁸

Results

In total, 180 privacy policies were analyzed, from 90 “health and fitness” apps (Table 1) and 90 websites (Table 2). After an initial round of sampling, a number of apps were identified in multiple countries. As shown in Table 1, the deduplication process resulted in slightly different numbers of apps sampled in the Netherlands and Sweden.

Table 1.

Number of health and fitness apps selected from the Google and Apple app stores, broken down by country and by sector to which the organization owning the app belongs.

Apps n  =  90	Charitable organization	Health or social care provider	Governmental organization	Other private sector	Total
Netherlands	1	0	0	28	29
Sweden	0	0	2	29	31
United Kingdom	0	0	0	30	30
Total	1	0	2	87	90

Table 2.

Number of websites identified by purposive sampling broken down by country and by sector to which the organization owning the website belongs.

Websites n  =  90	Charitable organization	Health or social care provider	Governmental organization	Other private sector	Total
Netherlands	8	9	11	2	30
Sweden	12	7	11	0	30
United Kingdom	10	10	10	0	30
Total	30	26	32	2	90

The full list of apps sampled and data collected are available in the Supplementary Materials (Document S1). Table 3 shows the results of comparisons between countries with respect to measures of availability, ease of navigation, structure and linguistic complexity.

Table 3.

Availability of, ease of navigation to, and structure and linguistic complexity of privacy policies associated with websites and health and fitness apps in The Netherlands, Sweden and United Kingdom.

	NL	SV	UK	Test statistic	p
Availability of privacy policy (researchers able to locate privacy information)
PP available, n (%)	57 (97)	59 (97)	59 (98)	χ2  =  0.413	0.814
PP available in country's official language, n (%)	35 (59)	38 (62)	59 (98)	χ2  =  28.890	0.000*
Summary of PP available, n (%)	11 (19)	1 (2)	2 (3)	χ2  =  14.569	0.001*
PP contains contact details for enquiries, n (%)	56 (95)	55 (90)	57 (95)	χ2  =  1.993	0.369
Ease of navigation (researchers able to navigate to privacy information)
More than one click to arrive at PP, n (%)	19 (32)	15 (25)	8 (13)	χ2  =  6.350	0.042*
Potentially distracting visual elements present, n (%)	16 (27)	1 (2)	12 (20)	χ2  =  15.187	0.001*
Readability—structure (length and organization of text)
Word count, official language PP, mean (SD) [range]	2084 (1932) [196–10979]	1213 (955) [220–145]	3507 (2911) [37–15553]	K–W  =  22.683	0.000*
Minutes reading time, official language PP, mean (SD) [range]	9.1 (8.5) [0.9–48.2]	5.6 (4.4) [1.0–19.0]	14.9 (12.3) [0.2–65.9]	K–W  =  20.975	0.000*
Number of subheadings used, mean (SD) [range]	11.2 (5.34) [0–26]	10.8 (6.84) [0–34]	11.7 (5.68) [0–29]	F  =  0.301	0.740
Section heading ‘which data are collected’, n (%)	40 (68)	29 (48)	48 (80)	χ2  =  13.950	0.001*
Section heading ‘purpose of data collection’, n (%)	41 (69)	36 (59)	52 (87)	χ2  =  10.214	0.006*
Section heading ‘with whom data are shared’, n (%)	38 (64)	19 (31)	41 (68)	χ2  =  21.115	0.000*
All three section headings present, n (%)	28 (47)	13 (21)	35 (58)	χ2  =  18.845	0.000*
Heading number ‘which data are collected’, mean (SD) [range]	1.8 (1.03) [1–5]	2.0 (1.21) [1–6]	2.3 (1.81) [1–9]	K–W  =  1.810	0.405
Heading number ‘purpose of data collection’, mean (SD) [range]	3.2 (2.39) [1–12]	3.2 (1.53) [1–7]	3.4 (2.26) [1–13]	K–W  =  1.516	0.469
Heading number ‘with whom data are shared’, mean (SD) [range]	4.7 (2.07) [1–10]	6.3 (4.41) [1–22]	5.9 (3.32) [2–16]	K–W  =  2.772	0.250
Readability—linguistic complexity (complexity of language used)
CEFR reading level B1 or below in official language, n (%)	1 (2)	0 (0)	0 (0)	n.a.	n.a.

*  =  statistically significant difference between groups. Underlined values are the results closest to normatively desirable values, where a statistically significant difference between groups was identified. Abbreviations: NL  =  Netherlands, SV  =  Sweden, UK  =  United Kingdom, PP  =  privacy policy, SD  =  standard deviation, K–W  =  Kruskal–Wallis, CEFR  =  Common European Framework of Reference.

In all countries, all websites had links to privacy policies but several health and fitness apps did not have privacy policies linked from the Google or Apple app stores. The data with respect to word-count, reading time and mean heading numbers of key information were non-normally distributed (Shapiro–Wilk test, p < 0.05) and Levene's test of homogeneity of variance returned a significant result (p < 0.05). Kruskal–Wallis tests were therefore performed to test for between-group differences with respect to these variables. Although the Shapiro–Wilk test was also significant with respect to the mean number of subheadings used, visual inspection of a histogram suggested that the data were not grossly non-normally distributed. Levene's test of homogeneity of variance was non-significant and ANOVA was therefore performed. No statistical test was performed to compare the linguistic complexity between groups as only one privacy policy met the threshold of B1 reading level. With respect to reading times, pairwise comparisons (Bonferroni adjusted) demonstrated no significant difference between the Netherlands and UK, but significant differences between Sweden and the UK and between Sweden and the Netherlands.

Table 4 shows the results of comparisons between the purposively selected websites from all three countries, and the top-ranking health and fitness apps from the Google and Apple app stores in all three countries. Where assumptions of the Chi-squared test were not met, Fisher's exact test was performed.

Table 4.

Availability of, ease of navigation to, and structure and linguistic complexity of privacy policies associated with websites, and health and fitness apps from The Netherlands, Sweden and United Kingdom.

	Websites	Apps	Test statistic	p
Availability of privacy policy (researchers able to locate privacy information)
PP available, n (%)	90 (100)	85 (94)	Fisher's exact	0.121
PP available in country's official language, n (%)	90 (100)	43 (48)	χ2  =  57.470	0.000*
Summary of PP available, n (%)	8 (9)	6 (7)	χ2  =  0.310	0.578
PP contains contact details for enquiries, n (%)	85 (94)	82 (91)	Fisher's exact	0.745
Ease of navigation (researchers able to navigate to privacy information)
More than one click to arrive at PP, n (%)	23 (26)	19 (21)	χ2  =  0.337	0.561
Potentially distracting visual elements present, n (%)	26 (29)	3 (3)	χ2  =  20.899	0.000*
Readability—structure (length and organization of text)
Word count, official language PP, mean (SD) [range]	2289 (2485) [132–15553]	2872 (2345) [37–9677]	U  =  2283	0.094
Minutes reading time, official language PP, mean (SD) [range]	9.9 (10.6) [0.6–65.9]	12.3 (9.9) [0.2–41.0]	U  =  2262	0.116
Number of subheadings used, mean (SD) [range]	10.9 (6.13) [0–34]	11.5 (5.82) [0–30]	t  =  0.650	0.516
Section heading ‘which data are collected’, n (%)	53 (59)	64 (71)	χ2  =  5.904	0.015*
Section heading ‘purpose of data collection’, n (%)	58 (64)	71 (79)	χ2  =  9.136	0.003*
Section heading ‘with whom data are shared’, n (%)	51 (57)	47 (52)	χ2  =  0.009	0.924
All three section headings present, n (%)	31 (34)	45 (50)	χ2  =  5.731	0.017*
Heading number ‘which data are collected’, mean (SD) [range]	2.2 (1.57) [1–9]	2.0 (1.34) [1–7]	U  =  1528.500	0.403
Heading number ‘purpose of data collection’, mean (SD) [range]	3.2 (2.13) [1–13]	3.3 (2.12) [1–12]	U  =  2075.500	0.821
Heading number ‘with whom data are shared’, mean (SD) [range]	6.1 (3.86) [1–22]	4.8 (2.12) [2–10]	U  =  946.500	0.070
Readability—linguistic complexity (complexity of language used)
CEFR reading level B1 or below in official language, n (%)	1 (1)	0 (0)	n.a.	n.a.

*  =  statistically significant difference between groups. Underlined values are the results closest to normatively desirable values, where a statistically significant difference between groups was identified. Abbreviations: PP  =  privacy policy, SD  =  standard deviation, K–W  =  Kruskal–Wallis, CEFR  =  Common European Framework of Reference.

Discussion

The primary goal of this research was to scope the cognitive accessibility of online privacy information associated with health and fitness apps, and websites. We analyzed the extent to which privacy information in three European countries which have implemented the EU GDPR is “simple, consistent, clear, multimodal, error-tolerant, and attention-focusing to use, taking into account all users.”^17,20,21 We investigated the availability, ease of navigation to, and readability (structure and linguistic complexity) of a sample of privacy policies from health and fitness apps and a range of public-facing websites. We did not have a priori hypotheses about the nature or direction of differences between countries. We anticipated better availability, ease of navigation and readability of privacy policies from the purposively selected sample of health and care providers, public sector organizations and healthcare charities, compared to the privacy policies of commercial health app providers. In general, we found more statistically significant differences in the features of privacy policies between countries, than between the sample of apps when compared to the sample of websites, with respect to each of the investigated aspects of cognitive accessibility.

Availability

In line with our hypothesis, the availability of the website privacy policies was better than the health apps. As noted, under the EU GDPR all providers of digital technologies have a responsibility to inform potential users of how data is collected from them and processed.² It could be that privacy policies for some apps were temporarily unavailable for some reason, perhaps due to technical errors, or due to the information being updated. However, there were no notices to this effect. Alternatively, if the app does not collect any data and a privacy policy was not legally required, the developers may have felt that this was unnecessary. However, we found an example in our research of a succinct 37-word privacy notice which clearly stated that the app in question did not collect any data. Given the limited effort required to write such a brief notice and the importance of trust in adoption of apps, this could be a better approach, even if an app collects no data.^4–7

Only 5 of 29 (17%) health and fitness apps in the Netherlands and 8 of 31 (26%) in Sweden, had links to privacy policies in the official language of those countries. In the other cases we investigated, where information was available, it was in English. It has been estimated that in the Netherlands, approximately 70% of the population speaks English at least well enough to hold a conversation, and in Sweden this is approximately 38%.³⁹ However, this level of English may not be sufficient to understand privacy policies at B2 reading level or above and all other potential app users would need to have the information translated. The process of getting information translated either by computer or a human translator would require additional, cognitively demanding steps, which people with cognitive impairments may struggle to take.

Ease of navigation

In most cases, privacy information was accessible from the app stores or website home pages with a single click. This was particularly true in the UK, in part because a number of apps in the Netherlands and Sweden were linked by default to privacy information in English, but included the option with one or more additional clicks to access the information in Dutch or Swedish, respectively. The notably less frequent occurrence of potentially distracting visual elements in the Swedish sample may in part be explained by the Swedish government's law on accessibility to public services (“Lag (2018:1937) om tillgänglighet till digital offentlig service”), which is based on earlier guidance such as the WCAG 2.1, and EU Directive 2016/2102/EU.^16,40 Since September 2020, all public sector websites and mobile applications in Sweden must be perceivable, operable, understandable and robust.⁴¹ The Swedish Agency for Digital Government (DIGG) implements the law, providing official guidelines for developers, designers and digital managers to develop accessible websites and applications. Website users can report accessibility issues to DIGG, who will initially contact the relevant public authority to address the complaints and a penalty may be imposed if the accessibility remains below legally mandated standards.⁴² Given that the majority of Swedish websites included in our analyses were from the public sector, these websites were likely influenced by Lag (2018:1937).

Contrary to our hypothesis, in all three countries, the privacy policies associated with health and fitness apps were less likely to contain potentially distracting visual elements than the websites included. This is partly because several dynamic visual features with the potential to distract users, such as hover/rollover effects, are not a feature of the standard user interface on a touchscreen device as there is by default no cursor.

Readability (structure)

The privacy policies from the Netherlands were more likely to include a summary of the information. Significant differences between countries were found in mean word-count and reading time, being lowest in the Swedish sample, and highest in the UK. Privacy policies in the UK were more likely, and in Sweden less likely, to contain headings clearly sign-posting the information identified as particularly important to people taking part in research around eHealth in dementia. This suggests that in each country, a different strategy for simplifying the structure of privacy policies may dominate: summarizing in the Netherlands, shorter texts in Sweden, and sign-posting key information in the UK. The reason for these different approaches may be down to chance, or to cultural differences. It is not clear why only one of these approaches in each country should dominate, since the combination of multiple strategies should result in the simplest structure and greatest cognitive accessibility, though it may simply be that this is seen as too resource-intensive or unnecessary. It should also be noted that linguistic differences might explain differences in word counts. For example, there is no word expressing the definite article (a translation of “the”) in Swedish, which could result in equivalent texts which in English heavily feature definite articles being shorter in Swedish.

Contrary to our hypothesis, privacy policies of health and fitness apps were more likely to contain all three headings sign-posting key information than those of the websites identified. This may be due to greater heterogeneity in the layout and content of the website privacy information, whereas the researchers noted relatively little variation in the layout and format of privacy policies for health and fitness apps. This could be because developers of new apps are more likely to simply copy the privacy policy format of existing apps.

Readability (linguistic complexity)

We found only one privacy policy, in the Netherlands, which met that country's guidance to write at the CEFR level B1, with the rest of the sample estimated to be of greater complexity by tools which have been previously validated in each of the languages concerned.^30,34–38 This is in line with previous findings regarding the reading level of website texts in general in the Netherlands.³² It may be that the results in our study were affected by the ways in which the tools we used estimated the CEFR level of texts, and there are many other measures of linguistic complexity, which were not considered in this study.^26–28 On the other hand, there are reasons why linguistic complexity of the privacy policies analyzed may be high. The texts concern technologies and legal matters, and contain specific terms or jargon relating to both of these fields, which do not occur with high frequency in each language in general. As the texts concern processing of data under particular circumstances or conditions, there is also significant potential for use of conditional clauses, which increase sentence length and complexity as measured by the three tools used in this study. Developers might argue that such constraints make it impossible to write a privacy policy which meets all of the legal requirements and remains cognitively accessible. Our initial work developing a cognitively accessible privacy for the FindMyApps intervention suggests that it is possible to write a cognitively accessible and legally valid privacy policy, though the cognitive accessibility of the FindMyApps privacy policy is yet to be researched at scale. Even if it were impossible to write a single policy to meet all demands, it is always possible to write a plain language summary, and our research shows that this is also rarely undertaken. High linguistic complexity of privacy information places unnecessarily high burdens on language and short-term memory, runs counter to the spirit of the EU GDPR, contributes to privacy fatigue.^2,13,18

Strengths and limitations

An internationally agreed operational definition of cognitive accessibility is still emerging.^20,21 This scoping research was anchored within the aspects of cognitive accessibility so far defined. By including a large number of privacy policies across multiple sectors, countries and languages, the results of this study help to highlight broadly relevant issues which may inform the further specification of an internationally recognized standard for cognitive accessibility, particularly with respect to privacy information. More precise evaluations of cognitive accessibility of privacy policies should follow once a standard is available. In the absence of a detailed operational definition of cognitive accessibility, we focused on collecting observable, quantifiable data. We developed operational definitions of relevant concepts such as “potentially distracting visual elements” within the research team and engaged in a process of triangulation, wherein privacy policies were reviewed by a second investigator and any divergent results discussed to arrive at consensus. Using the CEFR scale allowed direct comparisons of linguistic complexity between languages, and allowed the use of validated software to estimate linguistic complexity. While critics of scales which focus solely on properties of a text point out that reader-specific factors such as background knowledge or interest in the subject are not taken into account, software-based assessment of texts allowed a large number of privacy policies to be included in this research, which was useful for the purposes of broadly scoping linguistic complexity of the corpus of texts sampled.^43,44 The tools used to estimate CEFR levels do not correlate perfectly with the gold standard of human expert evaluation, so our results are, at the level of each individual privacy policy, an estimate of the CEFR reading level, rather than a definitive measurement.^30,36,38 More detailed, qualitative evaluation of privacy policies and how app and web-users interact with individual privacy policies should follow.

Recommendations

Based on our results, initial recommendations have been developed for policy-makers, health app and website developers, and those making use of health apps.

Policymakers should:

• Require that privacy information be available in an official language of their country

• Implement and enforce regulatory frameworks such as that in Sweden

• Adopt, once available, ISO standard for cognitive accessibility into specifications for public apps and websites and preferred provider criteria

Web and app-developers should:

• Comply with the spirit of GDPR and provide information on privacy even if no data is collected

• Make privacy policies available in an official language of all countries in which they are active

• Review and re-write privacy policies using accessible language, with a benchmark such as CEFR level B1

• Consider providing plain language summaries of privacy information

• Adopt a ‘Swedish-style’ design approach to make user interfaces more “attention-focusing

Web and app-users should:

• Make the best use possible of available privacy information, if necessary with support of someone they trust

• Make use of developer contact details to request further information, translations or support

• Provide feedback to developers regarding their experience attempting to access and read privacy information

• Contact app store providers if experiencing problems with privacy policies of multiple apps, or if developers do not respond

• Include people with cognitive decline when developing and user-testing new apps/websites

Collaboration with charities and other organizations dedicated to making (healthcare) information more accessible, such as Pharos in the Netherlands, the Patient Information Forum in the UK, or the Swedish Agency for Digital Government (DIGG) in Sweden, may help developers to put these recommendations into practice.

Conclusions

This scoping research identifies a number of concerns regarding the cognitive accessibility of privacy information associated with a large number of health apps and websites in three countries which implemented the EU GDPR. There are a number of specific examples of good practice from which website and app developers and policy-makers could learn, in order to improve cognitive accessibility of privacy information. Future research should seek to understand the experience of individuals interacting with specific privacy policies, in order to develop more detailed recommendations for developers. Such user-centered research should form part of all developers’ collaborative design process, and should take account of emerging operational definitions of cognitive accessibility. The results and recommendations from the present study should allow organizations committed to equitable accessibility of online environments to get a head-start in improving the cognitive accessibility of online privacy information to support autonomy for people with cognitive impairments. By building public trust in organizations to protect their privacy online, this should remove an important barrier to the adoption of technology, in healthcare and beyond.