To understand or not to understand: This is the problem

Abstract

Background/Aims

The aim of the study was to evaluate the level of readability of the informed consent used before colonoscopy in a university hospital by different methods, and to investigate related sociodemographic variables.

Materials and Methods

This descriptive, cross-sectional study included 211 subjects aged over 18. Three different measurement tools were used to evaluate the level of readability of the informed consent: The Cloze Readability Procedure, the Flesch’s readability formula adapted to Turkish by Atesman, and the readability formula developed by Cetinkaya and Uzun.

Results

The readability scores of the text, calculated according to Atesman’s formula, the Cetinkaya-Uzun, and Cloze Readability Procedure were 50.183 (the average level of difficulty), 31.021 (frustration level), and 26.68 (frustration level), respectively. The informed consent was found to be appropriate for individuals educated at the grade level 10 and above. The following participants were found to have significantly higher scores: educated above high school; reading books, magazines, digital media, and printed media every day or every other day; and previously informed about colonoscopy.

Conclusion

Within the framework of a colon cancer screening program conducted by the Public Health Institution of Turkey, individuals who tested positive were subjected to the colonoscopy procedure, if necessary. This increases the importance of the informed consent form used prior to the colonoscopy procedure. The patients and/or patients’ relatives who are at the frustration level of reading ability would have problems in reading and comprehending the text. The Cloze Test Procedure might be used in examining problematic consent forms.

INTRODUCTION

Effective communication is the cornerstone of an effective therapeutic relationship (1). Even if the diagnosis is accurate and the treatment is technically right, these misunderstandings cause disappointment among physicians as well as patients, leading to frustration, dissatisfaction among patients, reduced medical effectiveness, conflicts, and lawsuits.

Shift from paternalism toward patient autonomy in health care increased the autonomy and participation of patients in the medical decision-making process (2).

The principle of autonomy in current medical practice is expressed through the doctrine of informed consent. The main purpose is to inform the patient and ensure that he or she understands the information (3). Therefore, informed consent requires not only that the patient is willing to make a decision on the matter, but also that the information is disclosed to the patient and that it is comprehensible (4).

Informed consent is different from the signature of a patient put on a form indicating that the patient has approved the procedures to follow. The forms are expected to be sufficiently informative for and comprehensible by patients at the grade levels 3–7 (5). Most of the studies conducted in our country to determine the level of readability are for course books (6,7). There are only a few studies measuring the level of readability of the informed consent forms, and patient education materials prepared for patients in our country indicated that written patient education materials are not easy to read and the level of readability of health education materials used in primary health care are higher than the grade level 6 (8–11).

The Cloze Readability Procedure (Cloze Procedure) and readability formulas are widely accepted approaches in the literature on measuring the level of text readability and classifying them according to their difficulty levels. The Cloze Procedure is an objective technique of defining and classifying, based on the idea that readers have the ability to fill in the missing word in the text in accordance with their reading skill (12,13). This method can be used with written materials in almost every language since it is not based on a specific language. Another commonly used approach to estimate the level of readability is the readability formulas. When formulas are evaluating the text independently from the patients, the Cloze Procedure gives an average result according to the patients understanding. The reliability of these two different kinds of measurements depends on their similarity, which is not investigated in literature.

The purpose of this study is to evaluate the level of readability of the informed consent form used at a university hospital before colonoscopy using the Cloze Procedure and readability formulas to compare the results and investigate the relation between the scores of the Cloze Procedure and the sociodemographic variables affecting readability.

MATERIALS AND METHODS

Study population

This descriptive, cross-sectional study was conducted at a university hospital in the city of Izmir. The participants were selected through simple random sampling among individuals older than 18 years, and their relatives, who applied to the polyclinics of the hospital for various reasons. Individuals who previously read the same copy of the “informed consent form” used in the present study, illiterate individuals, those suffering from health issues preventing them from reading and comprehending what they have read (such as visual impairment, drug abuse, dyslexia, and mental disabilities), individuals with a poor general medical condition, and those who cannot sufficiently communicate were excluded from the study.

A questionnaire was administered to participants through a face-to-face interview survey. The questionnaire consisted of two parts. The first part is the Personal Information Form, which was prepared by the researchers to collect sociodemographic information. The second part was prepared in accordance with the Cloze Procedure and included a text consisting of 300 words from the colonoscopy informed consent form. The study, approved by the ethics board of the Dokuz Eylül University, provided the participants with information about the study before the administration of survey, and only the volunteers participated in it.

Data collection tools

A two-part study was constructed. In the first part, the Flesch’s readability formula adapted to Turkish by Atesman and the readability formula developed by Cetinkaya and Uzun (14, 15) applied to the informed consent form to determine the level of readability that allowed a reader-free evaluation at the beginning.

The Flesch’s Readability Formula adapted to Turkish by Atesman

This is the first formula (1997) adapted to Turkish from the Flesch’s formula called Reading Ease to determine the level of readability of Turkish texts (Table 1) (14).

Table 1.

Classification of the level of readability of Turkish texts in accordance with the reading scores in the readability formula by Atesman

Readability score	Level of readability
90–100	Very easy
70–89	Easy
50–69	Average
30–49	Difficult
1–29	Very difficult

Readability Formula by Cetinkaya and Uzun

Cetinkaya and Uzun (15) developed a formula to measure the readability of Turkish texts in 2010 (Table 2). The Flesch’s readability formula adapted to Turkish by Atesman does not provide information about the suitability of the text for the individual; it only gives an idea about the structural difficulty level of readability of the text. Cetinkaya and Uzun’s readability formula, in contrast, makes it possible to match the reading material to the reading level of the target reader group.

Table 2.

The Level of readability and education according to the readability scores of Turkish texts in Cetinkaya and Uzun’s formula

Readability score	Level of readability	Level of education
51 and above	Independent reading level	Grade levels 5, 6, and 7
35–50	Instructional reading level	Grade levels 8 and 9
0–34	Frustration level	Grade levels 10, 11, and 12

The second part involved assessment of the level of readability for the same informed consent by using the Cloze Procedure, which showed participants’ differences.

The Cloze Procedure

The Cloze Procedure, used to determine the level of readability of the informed consent form and the individual’s capability to understand the text, is based on Gestalt completion, the ability of the mind to complete incomplete words, images, and opinions (12). The Cloze Procedure answers two questions: first, the Cloze scores indicate the participants’ skills in reading and comprehension. Second, the scores indicate the level of readability of reading materials. A Cloze score greater or equal than 60 means “the text is comprehensible by the reader,” a score greater or equal than 40 and lower than 60 means “the text can be comprehended with some help,” and a score lower than 40 means “the text is incomprehensible by the reader” (Table 3) (16). A section of the text consisting of 300 words is selected for this procedure. The first and last sentences of this section are omitted, and every fifth word starting from the second sentence is deleted. The procedure ends after 50 words. The participants are asked to fill in the blanks in the testing material without a time limit. Each blank filled by the participant is counted in a way that synonyms are not accepted as being correct. The test scores (out of 100) are obtained by multiplying the total number of correct words by 2 (13).

Table 3.

The level of comprehension in accordance with the cloze procedure

Correct answers	The level of comprehension
≥60 to ≤100	Independent reading (understood by the reader)
≥40 to <60	Instructional reading (needing the support of an educator)
>0 to <40	Frustration reading (not understood by the reader)

Measures and statistical analysis

The Flesch’s readability formula adapted to Turkish by Atesman

$$\text{Readability\hspace{0.17em}Score}=198.825-40.175\times \text{X}1-2.610\times \text{X}2,$$

where X1=Average word length in syllables and X2=Average sentence length in words.

The readability formula by Cetinkaya and Uzun

$$\text{Readability\hspace{0.17em}Score}=118.823-25.987\times \text{Average\hspace{0.17em}word\hspace{0.17em}length\hspace{0.17em}}-0.971\times \text{Average\hspace{0.17em}sentence\hspace{0.17em}length}$$

Both formulas were applied to the first 100 words from the beginning of the text. If the sentence was not finished after 100 words, words until the end of the last sentence were added over 100 words. The total numbers of syllables, words, and sentences until that point were counted.

After determining the reading level of the material by formulas, participants’ data were analyzed using the Statistical Package for Social Sciences version 22.0 software (IBM Corp.; Armonk, NY, USA). Sociodemographic variables are presented as number and percentage distribution. Cloze scores’ continuous variables were reported as means with standard deviations. While evaluating the variation in the level of readability in accordance with the demographic data by the Cloze Readability Procedure, we used the independent samples t-test for continuous variables and one-way analysis of variance (ANOVA). The post hoc Tukey and Tamhane’s T2 tests were used to determine which groups cause the difference among groups. We accepted p<0.05 as statistically significant.

RESULTS

The level of readability according to the formulas

The number of sentences analyzed to determine the level of readability of the text was 9, the number of words was 105, and the number of syllables was 309. The average sentence length was 11,666 words, and the word length was 2,942 syllables.

The readability score of the text, calculated according to Atesman’s formula, was 50,183, which indicates an “average level of difficulty” according to Atesman. The readability score of the text, calculated according to Cetinkaya and Uzun’s formula is 31,021. According to the classification made by Cetinkaya and Uzun, the text is at the “frustration level,” and its level of readability is grade level 10 and above.

The level of readability according to the cloze procedure

The average number of correct answers of the participants in the Cloze Procedure was 13.34, and their average Cloze score was 26.68 out of 100. The average Cloze score of the participants was at the “frustration level.” The Cloze score distribution is shown in Table 4.

Table 4.

The cloze procedure results

Level of readability according to the cloze procedure	n	%
≥60 to ≤100 Independent reading (understood by the reader)	2	0.9
≥40 to <60 Instructional reading (needing the support of an educator)	33	15.6
>0 to <40 Frustration reading (not understood by the reader)	176	83.4

The level of comprehension according to the participants’ sociodemographic characteristics

Of the 216 subjects interviewed, 5 were excluded from analysis because they did not meet eligibility criteria for the project, leaving a usable sample of 211. Those 5 subjects could not complete the questionnaire reliably due to insufficient communication (n=3) and visual impairment (n=2). The sample of this study consisted of 211 participants between the ages of 19 and 87. The average age of the participants was 48.97±13.4.

Table 5 shows the results of descriptive statistics, independent samples t-test, and one-way ANOVA analysis indicating whether there is a difference among their Cloze scores according to the participants’ sociodemographic characteristics.

Table 5.

Cloze readability procedure results by sociodemographic variables

Sociodemographic variables	n (%)	Mean cloze score±SD	p	Post Hoc Test***
Age groups**
19–24	16 (7.6)	31.75±13.32	0.370
25–49	93 (44.1)	26.73±14.71
50–70	89 (42.2)	25.39±13.57
Above 70	13 (6.2)	28.77±11.18
Gender*
Female	101 (47.9)	26.63±14.58	0.969
Male	110 (52.1)	26.71±13.43
Education**
Literate, primary, secondary (1)	60 (28.4)	20.20±13.81	0.000	1–2
High school (2)	82 (38.9)	26.02±12.79		1–3
Undergraduate/graduate degrees (3)	69 (32.7)	33.07±12.74		2–3
Income level**
Income is less than expenses (1)	76 (36.0)	21.74±12.27	0.000	1–2
Income is equal to expenses (2)	109 (51.7)	29.72±14.31
Income is more than expenses (3)	26 (12.3)	28.31±13.51
The place they lived the longest*
Rural	51 (24.2)	25.92±12.54	0.660
Urban	160 (75.8)	26.91±14.41
Frequency of reading books**
Every day or every other day (1)	35 (16.6)	36.23±10.04	0.000	1–2
Once a week (2)	32 (15.2)	25.50±12.93		1–3
Once a month or less frequently (3)	144 (68.2)	24.61±14.10
Frequency of reading magazines, digital media, printed media**
Every day or every other day (1)	117 (55.5)	28.89±13.24	0.002	1–3
Once a week (2)	41 (19.4)	27.66±14.32
Once a month or less frequently (3)	53 (25.1)	21.02±13.91
Previously informed about colonoscopy*
Yes	39 (18.5)	30.97±12.14	0.021
No	172 (81.5)	25.70±14.19

^*Independent samples t-test was applied;

^**One-way analysis of variance was applied;

^***Post hoc test: Tukey and Tamhane’s T2 tests were applied

Comparison of different readability methods

The results of the different readability measurements used in the study are shown in Table 6.

Table 6.

Results of readability level assessment

Methods	Score	Assessment
Atesman’s formula	50.183	Average level
Cetinkaya and Uzun’s formula	31.021	Frustration level (Grade 10, 11, and 12)
Cloze Readability Procedure	26.68	Frustration reading (not understood by the reader)

DISCUSSION

This study examined the level of readability of an informed consent form used before the colonoscopy procedure at a university hospital. We found that the average length of sentences was 11,666 words, and the average length of words was 2,942 syllables. Likewise, Kaya et al. (17) conducted a study with 20 patient education materials and found the average sentence length to be 10.57 words and the average word length to be 2.77 syllables. When the range between the easiest text and the most difficult text was examined, the average sentence length according to Atesman’s formula for Turkish is in the range between 4 and 30 words. The average word length is between 2.2 and 3.0 syllables. Considering the average word and sentence lengths, we see that the average word length in this study is close to “the most difficult text,” while the average sentence length is below the average reported by Atesman. Sentences with a lower number of words are easy to understand, whereas sentences with a higher number of words are difficult to understand. It is emphasized that a text should consist of short sentences and, more importantly, short words (18). Simple and short syllables are easier to understand in Turkish (19). The long sentences in the informed consent form used in this study caused the readability score to fall. Thus, the informed consent form, which is an informative and promotional educational material, is difficult to read in its current form.

This study was conducted to evaluate the level of readability of the informed consent form using the Cloze Procedure and readability formulas and to ascertain the relation between the Cloze scores and sociodemographic variables affecting readability. We concluded that the level of readability of the subject text is “average” according to a formula adapted to Turkish and “frustration” according to the formula specifically developed considering the grammatical structure of Turkish. According to Cetinkaya (15), for an individual to easily understand a text at the instructional reading level and comprehend the given message, he or she should receive education for at least 8–9 years. Moreover, an individual needs to receive at least 10–12 years of education to be able to understand a text that is at the frustration level. In the United States, adults cannot read education materials at the grade level above 8; however, most patient education materials are prepared at the high school level (20). Ertem-Vehid et al. (9) conducted a study on an informed consent form and found that these forms are not appropriate for individuals who received less than 8 years of education. The text we used in this study was at the “frustration level” and was found to be appropriate for individuals who are at the grade level 10 and above. However, more than a quarter of the participants (28.4%) received less than 9 years of education. While preparing informed consent forms, related authorities should pay attention to make these documents understandable by the part of the population with the lowest level of education.

There is a direct relationship between the syllable structure and sentence length of the text and reading formulas. The fact that the level of readability of this text was “average” according to the Flesch’s readability formula adapted to Turkish by Atesman indicates that a formula developed for a different language would produce different results in defining the level of readability of Turkish texts, even if it is adapted in accordance with the structure of the Turkish language. However, the Cloze Procedure is a method independent of factors such as the sentence length and the number of syllables, and it was not developed for a specific language. The fact that most participants (83.4%) in this study received a score less than 40% indicates that this text is very difficult for the participants to read and understand, the level of readability of this text is over the comprehension level of the participants, and the participants cannot read and understand this informed consent form on their own. Studies conducted in our country to determine the level of readability of patient education materials and the informed consent forms have only used readability formulas. There is no study in Turkish that uses the Close Procedure to determine the level of readability of patient education materials and informed consent forms to which we could compare our results. However, considering the Cloze scores we obtained, we see that there are a lot of similarities between our results and those by Kusec et al. (16), who found that 93.3% of the participants in Zagreb with a low level of education and 73.3% of the participants with a high level of education were unable to comprehend the consent form, which means that the present study was consistent with the example of Zagreb.

Considering the results of this study, we see a significant difference between the Cloze scores of participants based on their educational profile; the frequency of their reading books, magazines, digital media, and printed media; and whether they were previously informed about colonoscopy. According to McKenna et al. (21), the reading ability develops as students become more qualified, learn more information than they can use in life, and as they get older. In a study conducted by Griffin et al. (22) on participants with an age range of 65 and above using the Close Procedure, younger participants had higher scores than older ones; however, this might result from the fact that cognitive functions deteriorate with age. In our study, young adults with an age range between 19 an24 had the highest average Cloze score; however, there was no significant relationship between the age of participants and their Cloze scores. The participants of our study who were at least 18 years old read the informed consent form, which was prepared for individuals aged 18 and above. We may have failed to find a significant difference between the age and the Cloze scores of the participants since the study was prepared for the target age group. The Cloze scores, which increase with the increase of the participants’ level of education, also increase as the participants’ language skills and their level of comprehension increase. The fact that the number of correct answers in the Cloze Procedure increases with the level of education shows that the informed consent forms should be prepared bearing the lower grade levels in mind. However, we should also keep in mind that even if an individual has a high level of education, if he or she is not familiar with the medical literature, he or she might misunderstand some medical terms. Therefore, consent forms should avoid medical terms as much as possible. We found that the rate of medical terms in the text used for this study is 8% (4 words). Our study shows that gender does not have a significant effect on the reading skill, which is consistent with the findings of Griffin et al. (22). Moreover, we found that participants whose income is less than their expenses have significantly lower Cloze scores than the participants whose income equals their expenses. Studies conducted by Sallabas (23) and Kovacioğlu (24) indicate a significant positive relationship between the attitudes toward reading and the level of reading comprehension. The study conducted by Bas (25) on attitudes toward reading shows that there is a positive relationship between a monthly income and the level of positive attitude. This study did not find a significant difference among participants’ Cloze scores based on where they lived the longest. We think that the fact that there are better opportunities in the city centers compared to the peripheral settlements have an effect on the reading habits of the participants. If we had treated villages as a separate unit of settlement in this study, the numbers in the samples would have changed, and this might have caused a variation among the Cloze scores. We discovered that the reading comprehension level of those who read every day or every other day is higher than those who have a lower frequency of reading books, magazines, digital media, and printed media. Koksal et al. (26) showed that as the frequency of reading books and newspapers increased among students in Grades 4 and 5, the total number of words and the total number of different words they used in informative and narrative texts increased. These results prove that the vocabulary knowledge of readers is one of the effective factors determining their performance of reading comprehension in that language (27). The problem is that there is a consensus in Turkey that many citizens lack good reading habits. As an indication of this consensus, in a case study conducted in Turkey in 2011, it has been shown that about 31% of the participants do never read a book, and 44% read less than 10 books per year (28). The present study showed a significant difference between whether the participants were previously informed about the colonoscopy procedure and their Cloze scores. In other words, we can say that those participants who were informed about the colonoscopy procedure, even though it was not with the same form, understood the informed consent form better than those who were not informed.

The major limitation of this study is the sample selection bias. Since participation was based on willingness, individuals who thought they could fulfill the assigned task accepted to participate the study. It is possible that the sample selected for this study is more qualified in reading and comprehension than the individuals who did not participate.

There might be selection bias in the selection of the informed consent form as well. Our study is based on a single informed consent form that is frequently used at a university hospital and signed by patients before the colonoscopy procedure. Therefore, it is difficult to generalize the results of our study to the whole country. It would be more helpful to conduct a more comprehensive study in the future to have an idea about the general situation in the whole country.

In conclusion, our study indicates that patients and/or patients’ relatives at the frustration level of reading ability would have problems in reading and comprehending this text. In our country, within the framework of a colon cancer screening program conducted by the Public Health Institution of Turkey, after the screening test is applied to individuals within the age range of 50–70 years, individuals who tested positive in the test are subjected to the colonoscopy procedure if necessary. This increases the importance of the informed consent form used prior to the colonoscopy procedure. This fact was considered at the beginning of this study, while selecting patient education materials. In this respect, forms need to be reviewed and revised.

Although all three methods employed in our study brought about comparable results, the results obtained through the formula developed by Cetinkaya and Uzun and the results of the Cloze Procedure overlap. Therefore, the Close Procedure might be used in examining problematic consent forms and informative materials after the documents are evaluated using the formulas.