Readability and understandability of andrology questionnaires

Abstract

Objective

Medical questionnaires, which enable collection, comparison and analysis of appropriate data as a means of written communication between a patient and a doctor, must be easily readable, and understandable. Here, we measure the readability and understandability of questionnaires used in andrology and examine the relationship between the educational status of the patients and the understandability of the forms.

Material and methods

Seven questionnaires (SHIM, AIPE, IIEF, MSHQ-EjD, PEDT, NIH-CPSI and IPSS) used to diagnose andological diseases were selected from the European Association of Urology guidelines. The number of syllables per word, the number of words in a sentence, and the average word and sentence lengths were calculated for each Turkish validated form. Readability scores were calculated, and closet tests were used to measure the understandability of the texts.

Results

Three hundred and twenty-seven male volunteers participated in the study. Two hundred and sixteen of the participants (66%) had a high school or college education. The readability level of the seven forms was determined to be “Difficult” or “Very Difficult,” and at least a high school education level was required to understand the forms. As education level and monthly income increased, the understandability of the forms increased; as the readability of the forms became more difficult, their understandability decreased (p<0.001).

Conclusion

The readability levels of questionnaires used in andrology are well above the reading level of Turkey. Health providers can help patients to fill out forms to increase doctor-patient communication.

Introduction

Inquiry forms (“questionnaires”) are lists of questions prepared for patients to obtain specific information. They enable the collection, comparison and analysis of appropriate data as a means of written communication between a patient and a physician. In order for the form to be effective, the patient should be able to read the form and interpret it correctly. Readability is the degree of convenience for a reader to understand a written text.^[1] The readability of a text is related to the degree of difficulty of the words and the sentence structures used. On the other hand, the understandability of a text is directly related to the readability of the text and the education level of the reader.^[2]

Approximately 47% of the adult Americans have problems understanding complex healthcare information given to them by health care providers.^[3] The elderly, poorly educated individuals and individuals with inadequate mother tongue skills were found to have a poor level of reading comprehension.^[4] In the report prepared by the Organization for Economic Cooperation and Development in 2016 that compared adult skills in member states, Turkey ranked 33^rd in reading comprehension among 35 member states; Turkey’s reading comprehension score is below the average.^[5] In Turkey, there are many questionnaires that have already been validated in Turkish in the field of andrology. These forms, which are often read and filled out by the patients themselves, enable urologists, and andrologists to measure and evaluate diseases.

The aim of this study was to measure the readability and understandability of the questionnaires used in andrology and to examine the relationship between the comprehensibility level of the forms and the educational status of the readers. In our literature review conducted both in English and Turkish, no study on the readability and comprehensibility of andrology questionnaires was found.

Material and methods

This study was carried out between June 1, 2017 and February 5, 2018 with approval of the Ethics Committee of Okmeydanı Training and Research Hospital of University of Health Sciences (11.04.2107-843). European and American urology guidelines were scanned automatically using the key words “questionnaire,” “inquiry form,” “survey” and “instrument.” Thirty inquiry forms used in the guidelines were identified, including 10 in the field of Andrology. Because the study was planned to be conducted with male volunteers, two forms (the Premature Ejaculation Profile-Women and the Female Sexual Function Inventory (FSFI)) were excluded from the study. The Sexual Health Inventory for Men (SHIM), the Arab Index of Premature Ejaculation (AIPE), the International Erectile Function Form (IIEF), the Male Sexual Health Ejaculatory Dysfunction Questionnaire (MSHQ-EjD), the Premature Ejaculation Diagnostic Tool (PEDT), the National Institute of Health Chronic Prostatitis Symptom Index (NIH-CPSI) and the International Prostate Symptom Score (IPSS) with Turkish validations were included in the study.

Readability analysis

First, the questionnaires were loaded into Microsoft® Word® 2007 software (Microsoft, Washington) to check their spelling and grammar. The readability scores of the forms were calculated using the “Çetinkaya-Uzun Readability Formula”.^[6] The average word and clause lengths of the texts were calculated for use in this formula. Average word length was calculated by dividing the total syllable count by the total word count, and the average sentence length was calculated by dividing the total word count by the total clause count. Readability scores were calculated separately for each text using the following formula: Çetinkaya-Uzun Readability Score = 118.823-(25.987 × average word length)-(0.971 × average clause length) (Table 1).

Table 1.

Readability levels of questionnaires

Questionnaires	Word Count	No. of Sentences	No. of Syllables	Average Word No	Average Sentence Length	Readability Score
Sexual Health Inventory for Men (SHIM)	43	5	139	3.23	8.6	26.53
Arabic Index of Premature Ejaculation (AIPE)	65	7	199	3.06	9.3	30.27
International Index of Erectile Function (IIEF)	178	15	460	2.58	11.9	40.22
Male Sexual Health Questionnaire Ejaculatory Dysfunction (MSHQ-EjD)	38	4	129	3.39	9.5	21.50
Premature Ejaculation Diagnostic Tool (PEDT)	40	5	127	3.18	8.0	28.42
National Institutes of Health-Chronic Prostatitis Symptom Index (NIH-CPSI)	82	9	243	2.96	9.1	33.07
International Prostate Symptom Score (I-PSS)	69	7	178	2.58	9.9	42.16

Çetinkaya-Uzun Readability Scores vary between 0 and 100; higher scores indicate higher readability and lower scores indicate lower readability. Distribution of the readibility scores among age groups was as follows: 6–10 years, 90–100 pts; 10–13 years, 70–89 pts; 13–16 years, 50–69 pts; high school graduates, 30–49 pts; university graduates, 0–30 pts.

Understandability analysis

Ensuring accurate and honest responses to the questions on patients’ questionnaire forms is of great importance for identifying the degree of illnesses. Comprehension is a preliminary condition for patients answering questions properly. The primary factors affecting the understandability of a narrative are word length (i.e., the number of syllables) and the frequency of sentences. When we look at formulas developed to determine the readability levels of English texts in particular, we can see that they are based on average word and sentence length.^[7]

The understandability test was conducted with male volunteers older than 18 years who were able to read and write in Turkish, were not mentally retarded and who had at least a primary school education. In order to determine the understandability of the questionnaires, the closet test was applied. To test the understandability of each form, the fourth word of the form was left blank, and volunteers were asked to fill in the blank with the most appropriate word. An explanatory text was prepared to inform the volunteers about the study. This explanatory text stated that the purpose of this application was not to test the subjects but to evaluate the questionnaires, that there was no time restriction to fill in the blank with words and that the subjects were required to read the entire text first and fill in the blanks with the words they thought were appropriate for each blank. When the filled-in words were evaluated, words that were the same or synonymous with the extracted word were accepted as correct. Spelling mistakes in the words written by the participants were disregarded, and these words were accepted as correctly spelled. The number of words that the participants filled in correctly in a given text was taken as the raw score and evaluated on a 100-point scale.

Statistical analysis

Mean, standard deviation, median, frequency, and ratio values were used in the descriptive statistics of the data. The distribution of the variables was measured using the Kolmogorov-Smirnov test. The Mann-Whitney U test was used to analyze the quantitative variables. The comparisons between the groups and the understandability scores of the forms were analyzed using the Kruskal-Wallis method, and all of statistical analyses were completed using IBM Statistical Package for the Social Sciences 22.0 software (IBM SPSS Corp.; Armonk, NY, USA).

Results

Table 2 lists the demographic characteristics of volunteers participating in the study. Three hundred and twenty-seven male volunteers with a mean age of 40±14 years participated in the study, and 45% of the participants were between the ages of 35 and 45. Two hundred and sixteen (66%) participants were high school or university graduates, 66% of them were married and only 17% were retired.

Table 2.

Demographic characteristics of volunteers

		Min–Max	Median	Mean±SD/n-%
Age (years)		18.0–81.0	38	40±14
Age	18–30			87-26.6%
	31–45			147-45.0%
	46–60			60-18.3%
	≥60			33-10.1%
Education level	Primary S.			75-22.9%
	Secondary S.			36-11.0%
	High S.			81-24.8%
	University			135-41.3%
Marital status	Single			111-33.9%
	Married			216-66.1%
Working status	Working			270-82.6%
	Retired			57-17.4%
Monthly income		0–9000	2300	2753±1643
Monthly income	0–1500 TL			60-18.3%
	1501–3000 TL			174-53.2%
	3001–5000 TL			66-20.2%
	≥5001 TL			27-8.3%
Understadability scores of questionnaires		0–100	31	35.6±12.1

1 Euro= 6.2 TL, 1 USD=5.30 TL

Table 3 lists the readability scores of the questionnaires and the educational levels corresponding to the scores. The average readability score of these seven forms was 31.4. The readability scores of the International Prostate Symptom Score (IPSS), the International Erectile Function Form (IIEF), the Arabic Index of Premature Ejaculation (AIPE) Form and the National Institute of Health Chronic Prostatitis Symptom Index (NIH-CPSI) were between 30 and 49, and the corresponding educational status was high school and above. It was determined that the readability levels of the remaining questionnaires were “Very Difficult” and corresponded to a university education level.

Table 3.

The readability scores of the questionnaires and the educational levels corresponding to the scores

Questionaire	Readability score	Readability level	Education level
Sexual Health Inventory for Men (SHIM)	26.53	Very difficult	University
Arabic Index of Premature Ejaculation (AIPE)	30.27	Difficult	High S.
International Index of Erectile Function (IIEF)	40.22	Difficult	High S.
Male Sexual Health Questionnaire Ejaculatory Dysfunction (MSHQ-EjD)	21.50	Very difficult	University
Premature Ejaculation Diagnostic Tool (PEDT)	28.42	Very difficult	University
National Institutes of Health-Chronic Prostatitis Symptom Index (NIH-CPSI)	33.07	Difficult	High S.
International Prostate Symptom Score (I-PSS)	42.16	Difficult	High S.

Table 4 lists the understandability levels of the forms according to the demographic characteristics of the participants and the readability scores of the forms. The questionnaires were divided into two groups according to their readability scores: “Difficult” and “Very Difficult”. The median understandability scores of the forms in the “Very Difficult” and “Difficult” groups were 22 and 34 pts, respectively, and a significant difference was found between the mean understandability scores of the two groups (p<0.001). We found that as the participants’ education level and their monthly incomes increased, the understandability scores of the forms also increased (p<0.001). As the readability of the forms became more difficult and the educational level of the volunteers decreased, the understandability of the forms decreased (p<0.001) (Figure 1).

Table 4.

Comparison of groups

		Understandability scores of Questionnaires
		Min–Max	Median	Mean±SD	p
Age (years)	18–30	10–86	84	73.3±27.6	0.000K
	31–45	14–92	71	62.5±31.7
	46–60	0–78	23	38.8±32.3
	≥60	0–45	15	14.4±11.4
Education Level	Primary S.	0–45	17	17.0±10.4	0.000K
	Secondary S.	0–48	40	46.5±32.6
	High S.	12–76	60	54.9±29.1
	University	18–88	74	81.3±22.5
Marital status	Single	0–100	78	65.7±31.6	0.001m
	Married	0–100	54	51.3±34.8
Working status	Working	0–100	74	61.9±33.0	0.000m
	Retired	0–92	19	29.4±27.3
Monthly income	0–1500 TL	0–82	26	41.1±32.0	0.000K
	1501–3000 TL	0–76	42	47.6±33.1
	3001–5000 TL	18–88	86	64.2±25.2
	≥5001 TL	44–100	74	73.5±5.6
Readability of questionnaires	Difficult	8–100	34	31.5±6.5	0.000m
	Very difficult	6–100	22	20.5±3.4

^KKruskal-Wallis,

^mMann-Whitney U test.

1 Euro= 6.2 TL, 1 USD=5.30 TL

Figure 1.

The association between education levels of volunteers and understandability scores of questionnaires

Discussion

The aim of this study was to measure the readability levels of inquiry forms used for diagnosis, follow-up, and scientific studies in the field of andrology. We also sought to understand how well these forms are understood by the patients. Our findings revealed that the selected questionnaires are not suitable for patients because the levels of their readability and understandability are generally characterized as difficult. Since it is difficult to understand the questions in these forms, it is hard to make accurate diagnosis based on the responses given to the questions.

In our study, the average readability scores of the forms were found to be 31.4. We determined that these scores corresponded to at least a grade 12 education level. According to 2010 data, the average duration of education for Turkish individuals aged 15 and older is 7.11 years, and it falls to 6.33 years among women.^[8] Therefore, the readability levels of the forms are much higher than the average education level of Turkish people. Breese et al.^[9] studied the readability and understandability of patient consent forms in the United States and found that the readability levels of preoperative consent forms corresponded to grade 12. This study also concluded that the average level of education of United States citizens was 7 years and that 40% of the patients were consenting to forms without fully understanding their contents.

Today, readability is a universal concept of intense focus for linguists.^[10] Medical forms need to be written in simple, plain and understandable language so that patient status can be properly questioned. Understanding what is written on a form is a prerequisite for making correct decisions for doctors. Gazmararian et al.^[4] found that low socioeconomic status, elderliness and low income were negatively correlated with reading skills. Our results were consistent with those of Gazmararian et al.^[4] In particular, we determined that individuals with low levels of literacy had less knowledge about the state of their illnesses, so less likely to use health care resources, including preventive care, and less likely to comply with the treatment regimens. These individuals also have greater difficulty communicating effectively with healthcare professionals.^[11] Inquiry forms play a vital role in communication between a patient and a clinician. The information that is obtained via forms can be used to individualize treatment plans and to ensure that treatment decisions are made taking into account the general goals and objectives of each individual. For these reasons, if the readability level of inquiry forms and questionnaires is in accordance with the patients’ educational status, then patient-doctor communication can be increased.

Inquiry forms are used for clinical and research purposes, and measurement characteristics such as reliability and validity must be assessed using standardized criteria to determine whether a form has a high diagnostic ability.^[12] At the same time, language suitability and readability of forms translated into Turkish should be assessed for reliability and validity. In particular, an evaluation of readability after the validation of the forms in Turkish will increase reliability and validity. As noted in our study, the “Çetinkaya-Uzun Readability Formula” is a suitable way of evaluating readability for forms translated into Turkish.

The majority of studies have concluded that there is a relationship between education level and comprehension ability of the individuals.^[2] Readability is an important factor in analyzing the understandability of medical texts, but it is not sufficient by itself. In particular, medical forms consisting of words with long syllables have reduced readability, and these forms are accordingly identified as difficult texts.^[13] Furthermore, medical terms in foreign languages that are frequently repeated throughout texts are among the primary aspects preventing comprehension. For these reasons, relying on readability scores alone may not be the most accurate method to assess the comprehensibility of texts. However, readability scores can provide important information about the structural difficulties of the texts.

There are some limitations to this study. First, the forms used infrequently in clinical research were not taken into consideration. Second, the volunteers participating in the study were not patients, and they accordingly did not have a good grasp of the subjects of the texts. Also, only a single formula was used for the readability analysis; analyses of forms and comparison of readability scores using different formulas were not performed.

Inquiry forms or questionnaires are frequently used for diagnosis, treatment and scientific research purposes in andrology. However, this study showed that the readability levels of these forms are well above the reading level of most Turkish citizens. We found that there was a negative relationship between the readability levels of the forms and their understandability. Particularly for patients with low levels of education and socioeconomic status, the understandability of forms is further reduced. As a result, receiving help from a healthcare provider while filling in questionnaires will boost doctor-patient communication levels for patients with low levels of education.