A cross-language analysis of urolithiasis patient online materials: Assessment across 24 European languages

Abstract

Introduction

Urolithiasis is a prevalent condition with several etiological factors, affecting up to 20% of the population and exhibiting high recurrence rates. Its strain on healthcare systems, exacerbated by high incidence and recurrence, often results in insufficient time for thorough diagnostics and counselling. Consequently, many patients seek easily accessible online sources of information. This study aimed to assess the readability and availability of online urolithiasis materials across 24 official European languages to compare readability across different source types.

Material and methods

The phrase “kidney stones” was translated into all official European languages, and the first 50 search results for each language were retrieved. Non-functional websites, those requiring accounts or payments, and duplicates were excluded. Relevance was assessed using Google Translate to filter out results lacking medical information. Only patient-oriented materials were included for analysis. Obtained results were then classified by source category, and their readability was assessed using LIX formula.

Results

A total of 723 articles were analysed. The English term yielded the highest number of results, followed by Spanish and Portuguese. Overall, the English articles performed best, being the only language with a mean LIX score below 40, which marks the threshold between “somewhat hard” and “hard” to read. Finnish, Lithuanian, and Hungarian materials had LIX scores significantly exceeding the threshold of 50, classifying them as “very hard to read” and among the most difficult to comprehend.

A subgroup analysis revealed no statistically significant differences across the source classification.

Conclusions

Online materials on kidney stones are generally too complex for patients, limiting their understanding and treatment adherence. Simplification of patient-oriented materials along with artificial intelligence utilisation could enhance comprehension. Improved awareness may promote adherence to preventive measures and help reduce the incidence and economic burden of urolithiasis.

INTRODUCTION

Urolithiasis is a multifactorial condition with a complex aetiology influenced by genetic, metabolic, environmental, and dietary factors, as well as comorbidities and their corresponding treatments [1–4]. Consequently, the same diagnosis encompasses patients with widely varying prognoses, ranging from individuals with incidental, asymptomatic stones to those progressing to end-stage kidney disease (ESKD). Moreover, this condition is relatively frequent. Depending on the demographic, urolithiasis affects up to 20% of the population, with its incidence steadily increasing over the past 30 years [5, 6]. Furthermore, the risk of kidney stone recurrence reaches as high as 50% within five years of the initial episode/finished treatment [7].

In the aforementioned context, overloaded and often inefficient healthcare systems, already facing long waiting lists for endoscopic procedures, frequently lack the time needed for comprehensive diagnostics and counselling on the causes of urolithiasis [8]. This results in an increase of patients turning to easily accessible online sources when seeking information about their condition. Considering that nowadays the internet has become a critical tool in patient decision-making [9], the need for credible and reliable information is of utmost significance. In this regard, it is essential not only to provide materials with high-quality content but also to ensure they are easily comprehensible.

The term “health literacy” refers to an individual's ability to understand and effectively use health information [10, 11]. While online patient education resources play a significant role in shaping patient decisions, their readability often exceeds the general public’s health literacy levels [12–15]. This gap is concerning, as these resources influence patients' expectations prior to treatment, their satisfaction afterward, and, in cases of long-term care, the quality of patient-doctor collaboration.

The main objective of this study was a multilingual analysis of the comprehensibility of online patient education materials (PEMs) on urolithiasis and an assessment of their availability in different languages. An additional objective was to compare the level of readability of these materials depending on the source.

MATERIAL AND METHODS

Search algorithm and data acquisition

This manuscript follows an already established study protocol [16–18]. First, in order to obtain search inputs, the phrase “kidney stones” was translated using Google Translate services into all official European languages. Subsequently, initial searches were conducted. Each phrase was looked up using Google Search Engine. The queries were conducted using Google Chrome browser in Incognito mode to exclude potential confounding effects of the authors' search history. For each search, the first 50 records have been retrieved and further verified.

At first, sites that were not functioning and ones that required the creation of an account, payment, or download of its content were excluded. Duplicate entries were ruled out. Next, each record has been assessed for its relevance. Using Google Translate add-on for entire website translation authors rejected results that did not contain medical information. Personal blogs, internet forums and websites dedicated to alternative treatment methods were not analysed. Due to the inability to assess the readability metrics we excluded videos and infographics. Moreover, we did not include medication or supplement advertisements, together with resources addressing animal owners and veterinary medicine professionals. Lastly, since this study aimed to determine the readability of materials dedicated to patients, scientific articles and literature addressed to healthcare professionals were excluded. The remaining articles were subject for detailed analysis.

Definitions and source classification

To enhance clarity and provide a more comprehensive understanding of the conducted analysis, we provide detailed definitions outlining the criteria used to classify included PEMs. The materials included in the analysis were grouped by their language of origin and the provided definitions, facilitating a thorough examination.

Commercial publisher

This category includes materials created by websites that do not offer products or services directly. However, these sites have clear indicators of other monetisation methods, such as advertisements or the option for a paid subscription.

Medical journal

Electronic patient material was classified under the Medical Journal category if it met the following criteria: it was published on a medical or scientific journal website, the content was not intended for healthcare professionals, and the article was not scientific in nature.

Medical service provider

This category includes PEMs issued by urology clinics, group practices, individual practitioners, or portals offering specialists' consultation services. It also contains materials prepared by diagnostic centres or dietitians.

Foundations

Online materials were considered suitable for this category, if a domestic or international scientific organisation prepared them, for example, European Association of Urology or National Kidney Foundation. Furthermore, the website could not provide any service or product nor any method of income generation, other than donations.

Retailers

Evaluated articles published by websites directly selling medication or supplements, alongside websites connected with physical pharmacies fell under the Retailers category.

Pharmaceutical companies

Patient materials with solely educational purposes issued by companies producing medication or instrumentation used for the treatment or prevention of urolithiasis were classified in this section. If the website offered a purchase option, it was classified as an “Retailer”.

Sources providing reliable information about urolithiasis that could not be assigned to the aforementioned were categorised as “Other”.

Readability assessment

Numerous statistical measures are available for assessing the readability of the analysed materials. However, most of these methods have been validated solely in English, which restricts their applicability for comparing results across different countries due to linguistic constraints. Consequently, the only statistical method suitable for evaluating materials in all official European languages was the calculation of the LIX score. Results interpretation was conducted in accordance with the scale proposed by Anderson [19]. Accordingly, scores below 20 are classified as very easy to comprehend, scores below 30 as easy, scores below 40 as somewhat hard, scores below 50 as hard, and scores below 60 as very hard to comprehend.

The content of the included websites was copied into Microsoft Word (version 16.89.1) using the paste as plain text function. Irrelevant parts of PEMs, such as authors’ information, affiliations, advertisements, figures, hyperlinks, disclaimers, and contact information, were removed. Prepared materials were saved as separate files using the “Save as Plain Text” feature. Each text was subsequently copied and pasted into the online LIX calculator at https://haubergs.com/rix. The calculator computes the LIX score, the number of words, the number of sentences, and the average number of words per sentence. All metrics were saved for analysis. Each step of the described analysis has been conducted by two individual researchers. The obtained results were compared, and in the event of discrepancies, the data were re-evaluated.

The obtained LIX scores were organised according to the provided definitions. Their distribution was assessed using IBM SPSS software version 27.0.1.0, and descriptive statistics were calculated. The individual groups were compared using appropriate statistical tests, with statistical significance set at a p-value of <0.05.

Bioethical standards

Due to the nature of the study, the consent of the ethics committee was not required.

RESULTS

Prevalence and inclusion rate

A total of 723 articles were analysed for readability. The English term yielded the highest number of results, followed by Spanish and Portuguese. Notably, the number of hits obtained in English was 13 times greater than that for Spanish, underscoring the significant dominance of English-language materials. In contrast, the Finnish term garnered the fewest hits (5,840), followed by Irish and Estonian, which had 16,800 and 18,400 hits, respectively. The highest inclusion rate of 90% was observed for Bulgarian, Dutch, and English, with 45 articles included from the initial 50 results. Conversely, the Estonian and Irish searches exhibited the lowest inclusion rates at 22% and 10%, respectively. Table 1 provides a detailed overview of the number of websites included in the analysis, the search queries used, and the total number of hits obtained.

Table 1.

Overview of search terms, total number of hits and ratio of included materials

Language	Search term	Total no. of hits	Included PEMs, n (%)
Bulgarian	камъни в бъбреците	630,000	45 (90)
Croatian	bubrežni kamenci	69,600	36 (72)
Czech	ledvinové kameny	79,600	25 (50)
Danish	nyresten	58,700	17 (34)
Dutch	nierstenen	155,000	45 (90)
English	kidney stones	99,900,000	45 (90)
Estonian	neerukivid	18,400	11 (22)
Finnish	munuaiskiviä	5,840	19 (38)
French	calculs rénaux	1,520,000	36 (72)
German	Nierensteine	1,180,000	35 (70)
Greek	πέτρες στα νεφρά	125,000	43 (86)
Hungarian	vesekövek	40,800	26 (52)
Irish	clocha duáin	16,800	5 (10)
Italian	calcoli renali	1,140,000	39 (78)
Latvian	nierakmeņi	90,900	23 (46)
Lithuanian	inkstų akmenys	76,200	39 (78)
Maltese	ġebel fil-kliewi	34,900	22 (44)
Polish	kamienie nerkowe	90,000	38 (76)
Portuguese	pedras nos rins	1,840,000	35 (70)
Romanian	pietre la rinichi	360,000	39 (78)
Slovak	obličkové kamene	56,700	25 (50)
Slovenian	ledvični kamni	26,400	22 (44)
Spanish	cálculos renales	7,690,000	34 (68)
Swedish	njursten	106,000	19 (38)

No. – number

Readability by origin

Out of the 723 materials analysed, only one had a LIX score below 30, classifying it as “easy” to read. All other analysed PEMs had LIX scores exceeding 30. Overall, the English articles performed the best, being the only language group with a mean LIX score below 40, which marks the threshold between “somewhat hard” and “hard” to read. Following English, the best results were seen in materials written in Dutch (42 ±6), Swedish (44 ±6), and Danish (45 ±5), though these scores were still notably higher, placing them in the “hard to read” category. By contrast, Finnish (68 ±6), Lithuanian (65 ±6), and Hungarian (63 ±5) PEMs had the worst outcomes, with LIX scores that far exceeded the threshold of 50, classifying them as “very hard to read” and among the most difficult to comprehend. Figure 1 presents the mean LIX score values for the PEMs, categorised by their language of origin.

Figure 1.

Mean LIX score values for educational materials classified by language of origin.

Complexity and length

Materials in Bulgarian (87 ±77), German (79 ±47), and English (79 ±61) recorded the highest average sentence counts, while French (1,443 ±1,042), Bulgarian (1,389 ±1,179), and Romanian (1,371 ±1,197) had the highest average word counts per article. The highest words-per-sentence ratios were observed in Italian (22 ±4), Irish (21 ±4), and Greek (20 ±4), whereas Finnish (11 ±3), Lithuanian (13 ±2), and Dutch (13 ±3) had the lowest ratios. Interestingly, despite the low words-per-sentence ratios, which would suggest conciseness and high readability, Finnish and Lithuanian are among the languages with the highest LIX scores. Further details on word counts, sentence counts, and words-per-sentence ratios are provided in Table 2.

Table 2.

Lix scores, word counts, sentence counts, and words per sentence ratios by language of origin

Language	Lix score	No. sentences	No. words	Words/sentence ratio
Bulgarian	54 ±5	87 ±77	1,389 ±1,179	16±2
Croatian	61 ±6	50 ±32	805 ±481	17 ±4
Czech	54 ±5	74 ±50	979 ±686	15 ±4
Danish	45 ±5	55 ±37	797 ±475	16 ±2
Dutch	42 ±6	67 ±60	849 ±757	13 ±3
English	38 ±6	79 ±61	1,244 ±900	17 ±3
Estonian	62 ±4	40 ±13	564 ±177	14 ±3
Finnish	68 ±6	33 ±30	360 ±307	11 ±3
French	56 ±5	76 ±54	1,443 ±1,042	19 ±3
German	53 ±4	79 ±47	1,066 ±623	14 ±2
Greek	55 ±6	57 ±50	1,117 ±1,101	20 ±4
Hungarian	63 ±5	51 ±47	712 ±640	15 ±3
Irish	53 ±10	50 ±62	1,098 ±1,508	21 ±4
Italian	61 ±5	63 ±53	1,339 ±1,093	22 ±4
Latvian	61 ±7	52 ±34	765 ±459	16 ±4
Lithuanian	65 ±6	55 ±37	692 ±379	13 ±2
Maltese	62 ±6	31 ±23	446 ±273	15 ±4
Polish	62 ±4	71 ±36	1,014 ±503	15 ±3
Portuguese	49 ±5	48 ±24	871 ±426	19 ±3
Romanian	56 ±6	77 ±89	1,371 ±1,197	19 ±3
Slovak	54 ±6	59 ±47	775 ±538	14 ±3
Slovenian	55 ±6	53 ±52	801 ±717	16 ±3
Spanish	51 ±5	70 ±56	1,224 ±955	18 ±4
Swedish	44 ±6	78 ±49	1,122 ±710	14 ±4

No. – number

Readability by category

A subgroup analysis based on the classification of PEM sources revealed that medical journals have the highest average LIX score (62 ±7). In contrast, materials from foundations and “other” category displayed lower LIX scores of 51 ±11 and 51 ±8, respectively, suggesting that these articles may be more accessible to a broader audience. Additionally, pharmaceutical companies produce patient-directed materials with the highest average counts, averaging 77 ±79 sentences and 1,307 ±1,322 words. Conversely, articles published by foundations tend to be shorter, with averages of 58 ±66 sentences and 891 ±1,019 words, indicating a more concise and focused communication style. However, no statistically significant differences were observed among these groups. Figure 2 depicts mean LIX score in regard to the allocated category. This is followed by Table 3, which provides detailed metrics for each subgroup analysed.

Figure 2.

Mean LIX scores by allocated category.

Table 3.

Detailed metrics by the category of PEMs

Source	Lix score	No. sentences	No. words	Words/sentence ratio
Commercial publisher	57±9	63 ±48	1,000 ±885	16 ±4
Medical journal	62 ±7	72 ±54	1,179 ±1,020	16 ±5
Medical service provider	53 ±10	62 ±53	976 ±752	17 ±4
Foundations	51 ±11	58 ±66	891 ±1,019	15 ±3
Retailers	55 ±7	72 ±45	1,079 ±651	16 ±4
Pharmaceutical companies	56 ±6	77 ±79	1,307 ±1,322	16 ±4
Other	51 ±8	69 ±27	1,028 ±430	15 ±3

No. – number

DISCUSSION

The internet has become one of the most common sources for information, especially when it comes to medical knowledge [20]. This is also true for patients suffering from kidney stones. However, stone formers often encounter various informational pitfalls. Although kidney stone disease may seem straightforward, it can stem from a range of issues, both metabolic and anatomical [1–5]. As a result, not all information found online applies to every individual case. This is further complicated by the spread of outdated information that contradicts current medical knowledge and is often presented in language that is difficult for non-medical professionals to understand [21].

The evaluation of the readability of Google-searched materials related to kidney stones showed that the level of comprehension of the texts exceeds what would be expected for materials designed for a broad readership. None of the tested languages achieved a LIX score at the “easy to read” level. Though English, according to the results, has the highest overall comprehensibility and favourable text structure; it is still classified as “a little hard to read” by the LIX score. Higher LIX ratings were assigned to articles written in Dutch, Swedish, Danish, and Portuguese, marking them as “hard to read”. The articles in the remaining languages were classified as “very hard to read” due to their LIX ratings being higher than 50. Texts with such a high score are only comprehensible to readers with greater education. This is a challenge given that only approximately 30% of Europeans have a tertiary education [22]. The complexity of these articles undermines their primary goal: to provide inclusive access to information.

Although this study focused exclusively on urolithiasis, it is important to recognise that this issue is not limited to urology. Similar findings regarding poor readability of online materials have been observed in dermatology, gynaecology, and ophthalmology [16, 17, 23, 24]. The lack of easily accessible and understandable resources negatively impacts the treatment process. Additionally, difficult to read PEMs further contribute to healthcare disparities, as individuals with higher education and income are more likely to access and comprehend such information [25–27]. Unfortunately, this issue is exacerbated by those profiting from misinformation and false claims. A striking example identified in our analysis is the commercial sale of seahorse extract, falsely promoted as a preventive measure for kidney stones, which ranks among the top search results in Bulgarian. Lastly, social media have powerful impact over the younger generations. Unfortunately, they often present a skewed reality, further promoted by various media algorithms. Patients’ perspective focuses mainly on negative and adverse events after the procedures, whereas the professional approach is often commercially oriented [28, 29]. As a result, misinformation prevails.

Publications like this are vital as they clearly highlight the problem while also identifying potential factors for improvement. In this case, a primary step would be to create materials that avoid complex vocabulary and paraphrase poorly evaluated sentences by statistical methods. Support from artificial intelligence could prove invaluable in this endeavour [30, 31]. Naturally, not every patient has the skills needed to effectively use the available large language models. In fact, these models often fail to generate accurate results [32]. However, content creators can leverage these tools to enhance readability without sacrificing the essence of their knowledge.

This study makes a significant contribution to the field by providing the first comprehensive analysis of urological PEMs across all 24 official European languages. It also presents a standardised approach to evaluating these materials based on their source. Nevertheless, our publication has certain limitations. First, we did not conduct a factual analysis of the content. The linguistic barrier and the absence of statistical methods suitable for such comparisons rendered this analysis unfeasible. We believe that thorough content analyses should be undertaken for each language individually, ideally by scientists and specialists in the field of endourology who are native speakers of the assessed languages. Only this approach can yield an adequate evaluation of the examined content. Second, while we analysed a substantial number of articles, our search was limited to results from the Google search engine. However, it is noteworthy that over 90% of internet users globally rely on this search engine, which lends some credibility to our findings [33]. Lastly, the readability assessment was conducted using only one statistical test. Unfortunately, this is the only measure validated in multiple languages [19, 34–36]. However, in studies that have examined the readability of PEMs using various tests, the results consistently align across different methodologies, demonstrating that the LIX score alone is a sufficient measure of readability [37].

CONCLUSIONS

The analysed online materials on kidney stones are overly complex for the intended audience, with only a portion of the English materials deemed acceptable. This complexity impedes patient understanding and hinder the treatment process. Simplifying the language and structure, along with leveraging artificial intelligence for content development, could help bridge this comprehension gap. Future research should focus on particular languages and their credibility to ensure the accuracy of the information provided. Improving patient awareness regarding their condition can improve adherence to healthy behaviours, ultimately contributing to a reduction in both the incidence and economic burden of kidney stones.

Funding Statement

FUNDING This research received no external funding.

CONFLICTS OF INTEREST

The authors declare no conflict of interest.

ETHICS APPROVAL STATEMENT

The ethical approval was not required.