Development, validation, and application of a novel tool to measure disease-related knowledge in patients with inflammatory bowel disease

Abstract

Background/Aims

The Crohn’s and Colitis Knowledge (CCKNOW) score does not reflect updated knowledge relating to inflammatory bowel disease (IBD). The aim of this study was to develop, validate, and apply a novel tool to measure disease-related knowledge in IBD patients.

Methods

A questionnaire composed of 24 items regarding knowledge of IBD was developed: Inflammatory Bowel Disease Knowledge (IBD-KNOW). Discriminate ability of IBD-KNOW was validated in three occupational groups (14 doctors, 20 nurses, and 19 clerks). The CCKNOW and IBD-KNOW were administered to IBD patients. Factors affecting the level of IBD-related knowledge were analyzed.

Results

The median Inflammatory Bowel Disease Knowledge (IBD-KNOW) score was significantly different among the three groups for validation (22 doctors, 20 nurses, and five clerks; p < 0.001). The IBD-KNOW showed excellent internal consistency (Cronbach α = 0.952) and high correlation with CCKNOW (Spearman ρ = 0.827, p = 0.01). A total of 200 IBD patients (120 Crohn’s disease, 80 ulcerative colitis) completed questionnaires. Multivariate analysis showed that a higher IBD-KNOW score than the median was associated with hospitalization history (odds ratio [OR], 2.625; p = 0.003), high education level (OR, 2.498; p = 0.012), and information acquired from patient organization (OR, 3.305, p = 0.035).

Conclusions

The IBD-KNOW demonstrated excellent test characteristics. Hospitalization history, education level, and information acquired from patient organization play an important role in correct IBD-related knowledge.

INTRODUCTION

Inflammatory bowel disease (IBD) is a chronic incurable disease, and patients have to manage this disease for their entire lifetime. In addition, the natural course of this disease is sometimes very unpredictable, and a variety of complications can be developed while the disease duration gets longer [1]. Therefore, adaptive coping strategies for the disease are essential. Each patient’s correct understanding of IBD is helpful to achieve this goal [2]. Together with this, a study reported that higher levels of knowledge reduce health care costs in patients with IBD [3]. On the contrary, incorrect understanding of the disease in patients with IBD causes several problems. For example, most patients with IBD believe that a certain food causes the recurrence of the disease, thus avoiding it. This incorrect knowledge decreases the quality of life in patients with IBD [4]. In addition, it was reported that the lack of knowledge about medication in patients with IBD decreases treatment compliance, and it is related with disease recurrence [5]. Therefore, education programs to correct inappropriate knowledge on IBD are important. Tools to measure the level of disease- related knowledge objectively are also necessary to evaluate the effectiveness of education programs.

At present, two questionnaires are known as tools to measure the degree of general knowledge of the disease in patients with IBD. One is a questionnaire that was developed by Jones et al. [6] in 1993, and this has been rarely used because the evidence documenting its use is not enough. The other one is the Crohn’s and Colitis Knowledge (CCKNOW) score, which was developed by Eaden et al. [7] in 1999, in the United Kingdom, and this has been used widely. The CCKNOW is composed of a total of 24 items evaluating five domains: general knowledge, anatomy, medication, diet, and complication. The CCKNOW has the following limitations. First, more than 15 years have already passed since it was developed. Therefore, it cannot reflect updated knowledge relating to IBD treatment, which has changed rapidly. For example, the CCKNOW includes an item regarding elemental diet. However, meta-analysis showed that elemental diet is less effective than corticosteroids for inducing remission in Crohn’s disease [8]; therefore, it is not mainly used in treating Crohn’s disease in adults. In addition, items regarding biologics such as tumor necrosis factor alpha inhibitors, which have been widely used since the last decade, are not included. Therefore, revalidation of the questionnaire itself may be required [9]. Second, although the targets of CCKNOW are patients with Crohn’s disease and ulcerative colitis, the ratio of items related with knowledge for Crohn’s disease is higher than that of items specific for ulcerative colitis (25% vs. 16.7%). The CCKNOW score reported in patients with Crohn’s disease is higher than that of patients with ulcerative colitis in several studies [10,11]. It is not easy to differentiate whether the above-mentioned finding reflects the different levels of disease-related knowledge between patients with Crohn’s disease and ulcerative colitis, or this result is due to the basic limitation of the CCKNOW questionnaire itself because CCKNOW has an uneven distribution of disease-specific items. Third, the ratio of items of which the correct answer is “true” is too high in CCKNOW (91.7%, 22/24 items), which can also cause bias.

Given this background, we set out to develop and validate a novel tool to measure disease-related knowledge in patients with IBD. Subsequently, we applied this tool to patients with IBD, and we evaluated what are the predictive factors for the high level of disease-related knowledge in patients with IBD.

METHODS

This study was carried out in three phases: questionnaire development, validation, and clinical application to patients with IBD. This study was approved by the Institutional Review Board of Seoul National University Bundang Hospital (B-1506-304-306), and all patients provided informed consent.

Development of questionnaire

Two IBD specialists (H.Y. and S.K.Y.) developed questionnaire items regarding knowledge of various aspects of IBD (anatomy, function, epidemiology, diet/lifestyle, general knowledge, medication, complication, surgery, reproduction, and vaccination). After several discussions, a draft questionnaire composed of 51 items was constructed. This first draft questionnaire was administered to 44 IBD patients (22 with Crohn’s disease and 22 with ulcerative colitis) in the IBD clinic of two hospitals (Seoul National University Bundang Hospital and Asan Medical Center). We excluded items wherein the correct answer rate was less than 5% or more than 95% because these items were considered to have no discriminate ability. We also removed items wherein the correct answer was likely to be controversial and items that were nonfundamental. After refining the phrasing, a second draft questionnaire composed of 25 items was completed. This questionnaire was tested on another 40 patients with IBD (21 with Crohn’s disease and 19 with ulcerative colitis) in the aforementioned two hospitals. One item was additionally excluded by reviewing the results of the second pilot study. A final questionnaire with a total of 24 items was completed: Inflammatory Bowel Disease Knowledge (IBD-KNOW). The items were translated in English by a professor (J.H.C.) who is fluent in both Korean and English to make the international version of this questionnaire. During translation, he discussed with one (H.Y.) of the original developers several times to preserve the meaning of the original items. Afterward, the English version was back translated in Korean by a nurse who is working at an international healthcare center. The original developers checked meaning alterations by comparing the items in Korean and English. Lastly, we had the items corrected by a professional English language editing company (Editage, Seoul, Korea). The final international version and Korean version of the IBD-KNOW is attached as Appendixs 1 and 2, respectively.

Validation of questionnaire

Discriminate ability of the IBD-KNOW was validated in three occupational groups with different levels of IBD-related knowledge (14 gastroenterology fellows, 20 ward nurses, and 19 administrative staffs in Seoul National University Bundang Hospital). To avoid bias from preknowledge for the aim of this study, we did not explain the aim of this study specifically to the members of three groups. We just explained that we were developing new questionnaire to evaluate disease-related knowledge for IBD patients. We contacted each group separately for them not to know they would be compared to each group. We administered both CCKNOW and IBD-KNOW questionnaires in these subjects; we used the already validated Korean version of the original CCKNOW [11]. When people answer a questionnaire with several items, they have a tendency to lose concentration and answer more incorrectly toward the end. Therefore, to avoid bias from this event, we randomly administered the CCKNOW first to half of each group and the IBD-KNOW first to another half of each group. The total score of the questionnaire was calculated with a scoring system of one point for each correct answer. The internal consistency of the IBD-KNOW was evaluated using Cronbach α [12]. We evaluated the readability of the international version of the IBD-KNOW by calculating the Flesch-Kincaid reading score [13]. We used free online tool to calculate the Flesch-Kincaid reading score (https://readability-score.com/text/).

Clinical application of the questionnaire to patients with IBD

The CCKNOW and IBD-KNOW questionnaires were administered to consecutive patients who visited the IBD clinic in three tertiary referral hospitals (Seoul National University Bundang Hospital, Asan Medical Center, and Ewha Womans University Mokdong Hospital). The questionnaires were distributed to the patients by doctors or nurses in person. Similar to the validation phase, we randomly administered the CCKNOW and IBD-KNOW first to half and another half of the subjects, respectively. At the end of the main questionnaires, two simple questions regarding awareness of IBD and source of IBD-related information were additionally administered to each subject.

The data quality of each questionnaire was evaluated by calculating the ratio of items where answer was not checked or more than one choice was checked to total items. The same questionnaires were re-administered to 18 patients 2 to 4 weeks after the initial survey to evaluate test-retest reliability [14].

We evaluated patient factors affecting the level of IBD-related knowledge. First, we defined “high IBD-KNOW score” as higher than the median. Then, we performed univariate and multivariate analyses to search factors related with high IBD-KNOW score.

Statistical analysis

SPSS version 18.0 (SPSS Inc., Chicago, IL, USA) was used for statistical analysis. Continuous variables were analyzed using Wilcoxon signed-rank or Kruskal-Wallis test. In cases where there was a significant difference by Kruskal-Wallis test, pairwise comparisons were additionally done using Mann-Whitney U test. The chisquare or Fisher exact tests were used to analyze categorical variables. Simple association was assessed with the use of Spearman rank-correlation analysis. Multiple logistic regression analysis was used to identify possible covariates as significant predictors for high IBD-KNOW score. Covariates that showed a significant association in univariate analysis were subjected to multiple logistic regression analysis. All results were considered statistically significant when p values were less than 0.05.

RESULTS

Validity of the IBD-KNOW

Fig. 1 shows the level of IBD-KNOW and CCKNOW scores among the three occupational groups for validation. The median IBD-KNOW score was significantly different among the three groups (22 gastroenterology fellows, 20 ward nurses, and five administrative staffs; p < 0.001). Post hoc analyses also showed that the level of the IBD-KNOW was significantly different between each group (p < 0.001 in all comparisons). The median CCKNOW score was also significantly different among the three groups (16.5 gastroenterology fellows, 16 ward nurses, and 5.5 administrative staffs; p < 0.001). However, in pairwise comparisons, there was no difference in the level of CCKNOW between fellows and nurses (p= 0.191). The IBD-KNOW showed excellent internal consistency (Cronbach α = 0.952) and high correlation with the CCKNOW (Spearman ρ = 0.827, p = 0.01) (Fig. 2). The Flesch-Kincaid reading score of the CCKNOW and IBD-KNOW was 4.1 and 4.0, respectively.

Figure 1.

Level of the (A) Inf lammatory Bowel Disease Knowledge (IBD-KNOW) and (B) Crohn’s and Colitis Knowledge (CCKNOW) scores among the three occupational groups for validation.

Figure 2.

Correlation between the Inf lammatory Bowel Disease Knowledge (IBD-KNOW) and Crohn’s and Colitis Knowledge (CCKNOW) scores among the three occupational groups for validation.

Clinical application of the IBD-KNOW

Between October 2015 and November 2015, a total of 200 patients with IBD (120 with Crohn’s disease and 80 with ulcerative colitis) completed both the IBD-KNOW and CCKNOW. Table 1 shows demographic characteristics of the study population. Men composed approximately three-fourths of the population, and the median age was 34 years old. Table 2 shows the results of adjunctive questions that were administered to patients with IBD together with the main questionnaires. Only 24.6% of the patients had known of IBD before diagnosis. Except for doctors, the most important source of information about IBD was the internet; 72.5% of patients acquired necessary information relating to IBD via internet.

Table 1.

Demographic characteristics of the study population (n = 200)

Characteristic	Value
Male sex	149 (74.5)
Age, yr	34 (16–83)
Type of IBD
Crohn’s disease	120 (60.0)
Ulcerative colitis	80 (40.0)
Disease duration, mon	48 (1–360)
Smoking statusa
Never or past smoker	148 (78.3)
Current smoker	41 (21.7)
History of IBD-related hospitalizationa	105 (55.0)
History of IBD-related operationa	43 (21.7)
Family history of IBDa	12 (6.3)
Medication history
5-Aminosalicylates	189 (94.5)
Immunomodulators	136 (69.0)
Corticosteroids	125 (63.1)
Biologics	61 (30.5)
Education levela
High school or less	58 (29.1)
College/University or more	141 (70.8)

Values are presented as number (%) or median (range).

IBD, inflammatory bowel disease.

^aA small amount of data was missing.

Table 2.

Awareness of IBD and source of acquired IBD-related information

Variable	No. (%)
Have you ever heard of IBD before diagnosis?a
Yes	49 (24.6)
No	150 (75.4)
Source of information about IBD (multiple selection is possible)
Doctor	154 (77.0)
Nurse	19 (9.5)
Internet	145 (72.5)
Books	19 (9.5)
Patient organization	22 (11.0)

IBD, inflammatory bowel disease.

^aA single data was missing.

The data quality of IBD-KNOW was significantly better than that of CCKNOW (the rate of unqualified answers: 0.3% [16/4,800] vs. 0.7% [34/4,800], p = 0.015). The test-retest reliability of the IBD-KNOW and CCKNOW was not significantly different (71.3% vs. 72.9%, p = 0.569).

The mean score of the IBD-KNOW and CCKNOW was 13.3 and 9.4, respectively. Fig. 3 shows the correct answer rate of each domain of the IBD-KNOW questionnaire. The mean correct answer rate of the total 24 items was 55.7%. The correct answer rate of the reproduction domain was the lowest (36.9%); in the subgroup analysis, the correct answer rate of reproduction domain of male and female was 31.0% and 53.9%, respectively. The correct answer rate of each item of the IBD-KNOW is attached as Appendix 3. The single item wherein the correct answer rate was the highest and lowest was regarding smoking in Crohn’s disease (85.4%) and indication of biologics (28.0%), respectively.

Figure 3.

Correct answer rate of each domain of the Inflammatory Bowel Disease Knowledge (IBD-KNOW) questionnaire.

Table 3 shows univariate and multivariate analyses of predictive factors for a high IBD-KNOW score. Among several demographic factors, disease duration more than 5 years, IBD-related hospitalization history, high education level (college/university or greater), and information acquired from patient organization were significantly associated with a high IBD-KNOW score in the univariate analysis. The multivariate analysis showed that hospitalization history (odds ratio [OR], 2.625; 95% confidence interval [CI], 1.400 to 4.923; p = 0.003), high education level (OR, 2.498; 95% CI, 1.220 to 5.116; p = 0.012), and information acquired from patient organization (OR, 3.305; 95% CI, 1.090 to 10.017; p = 0.035) were predictive factors for a high IBD-KNOW score after adjusting for other variables.

Table 3.

Univariate and multivariate analyses of predictive factors for high IBD-KNOW score^a

Variable	Patients with high IBD-KNOW score	Univariate analysis	Multivariate analysis
		p value	OR (95% CI)	p value
Disease duration, yr		0.016		0.232
< 5	42 (37.5)		1
≥ 5	48 (54.5)		1.471 (0.782–2.768)
History of IBD-related hospitalization		< 0.001		0.003
No	27 (31.4)		1
Yes	62 (59.0)		2.625 (1.400–4.923)
Education level		0.001		0.012
High school or less	16 (27.6)		1
College/university or more	74 (52.5)		2.498 (1.220–5.116)
Patient organizationb		0.001		0.035
No	73 (41.0)		1
Yes	17 (77.3)		3.305 (1.090–10.017)

Values are presented as number (%).

IBD-KNOW, Inflammatory Bowel Disease Knowledge; OR, odds ratio; CI, confidence interval; IBD, inflammatory bowel disease.

^aHigh IBD-KNOW score was defined as higher IBD-KNOW score than the median.

^bSource of acquired IBD-related information.

DISCUSSION

The score difference was evident among the three occupational groups for validation of the IBD-KNOW in the present study. However, the CCKNOW showed no discrimination between nurses and doctors regarding the level of IBD-related knowledge. Based on the validation result during its publication in 1999, the median score of junior doctors, staff nurses, and ward clerks for the 24-item CCKNOW questionnaire was 22.0, 16.0, and 5.0, respectively [7]. That is, the CCKNOW score of nurses and general people in the present study is surprisingly similar to that of nurses and general people in the original study of Eaden et al. [7]. Therefore, we may say that the big discrepancy in the score of the doctors between the original and present studies (22.0 vs. 16.5) caused the loss of discrimination of CCKNOW between doctors and nurses in the present study. We could identify items wherein the correct answer rate was very low in the doctors (data not shown) by analyzing the correct answer rate of each item. The item wherein the correct answer rate was the lowest (6%) was the following: “During a flare up of inflammatory bowel disease, the platelet count in the blood rises.” We presume that because the rise of platelet during inflammation is clinically not an important phenomenon, the junior doctors who attended the present study did not know well about this item. Because elemental diet and sulfasalazine are also treatment options wherein its use is relatively decreasing recently, the correct answer rate regarding these items might be low (25% and 38%, respectively). In addition, because the items regarding genetics and growth of children have some basis for misunderstanding, the correct answer rate might be low (38% and 19%, respectively). These results imply that CCKNOW has some problems in measuring clinically important knowledge on IBD at the present. It was reported that patient understanding of IBD was no better in 2013 than in 1999 in the United Kingdom [15]. Based on the findings of the present study, we suspect that they evaluated the patients in the present using the knowledge in the past. On the contrary, we included items regarding updated trends of IBD treatment such as biologics in the IBD-KNOW questionnaire [16]. In addition, we included an item regarding vaccination in the IBD-KNOW because this has been recently addressed as one of practices to improve quality of care in IBD [17].

In the present study, the data quality of the IBD-KNOW was significantly better than that of CCKNOW. This might be because all items of the IBD-KNOW are composed of three choices “true,” “false,” and “don’t know,” which are different from CCKNOW wherein some items have multiple choices of up to five. In summary, the IBD-KNOW showed better discrimination and data quality than the CCKNOW. The test-retest reliability of the IBD-KNOW was not different than that of the CCKNOW. In addition, the IBD-KNOW showed excellent internal consistency and high correlation with the CCKNOW. The Flesch-Kincaid reading score of the IBD-KNOW was similar to that of the CCKNOW. Therefore, we think that the competence of the IBD-KNOW as an alternative to the CCKNOW is enough.

For patients with IBD, the median score of the CCKNOW has been known as 8.0 to 11.5 in the developed countries [3,10,15,18]. The median score of the CCKNOW was as low as 4.7 to 6.9 in the developing countries [19,20]. In the present study, the mean score of the CCKNOW in patients with IBD was 9.4; this result is very similar to that of another Korean study that was performed several years ago (mean, 9.5) [11]. It was reported that half of the patients who answered the CCKNOW felt that the questions were too difficult [21]. On the contrary, the mean score of the IBD-KNOW was 13.3. Therefore, although we could not perform a survey comparing the IBD-KNOW with CCKNOW regarding the feeling of item difficulty, we think the IBD-KNOW is less difficult than the CCKNOW. The absolute difficulty of the IBD-KNOW was also appropriate; the mean correct answer rate was 55.7% (range, 28.0% to 85.4%). Among 10 domains of the IBD-KNOW, the correct answer rate of the reproduction domain was the lowest at 36.9%. This result also corresponds well to the previous report that the subjects of fertility and implications for pregnancy are particular areas of deficit in the realm of IBD-related knowledge [9]. However, because only two items regarding reproduction are included in the IBD-KNOW, we recommend using more focused questionnaire in evaluating the level of reproduction-related knowledge specifically in IBD patients [22].

Education for IBD patients improves medication adherence. Treatment adherence correlates with the disease- related knowledge of patients [14]. That education increases the score of the CCKNOW is also well known [23]. In the present study, information acquired from patient organization was a predictor for high IBD-KNOW score in the multivariate analysis. The results of the original CCKNOW study by Eaden et al. [7] suggested that patients who are patient organization members achieved significantly higher scores than that of nonmembers. In addition, the patient organization members showed higher adherence in IBD treatment [24]. Taken together, we suggest that physicians encourage patients with IBD to join patient organization and conduct education programs intensively through patient organization; it would be a good strategy to increase disease-related knowledge and patient’s adherence to treatment.

In the present study, only 24.6% of the patients had known of IBD before diagnosis. This is very similar to the results of the European multi-country study (23%) [25]. Similarly, because several patients have lack of prior knowledge regarding IBD, they are usually embarrassed and eager to acquire the information for IBD when their diagnosis was confirmed. In the present study, except for doctor, the most important source of information on IBD was the internet. This result is also consistent with prior studies [2,25]. Information acquired from internet was not related with the high IBD-KNOW score in the present study. This implies that although the internet is an easily accessible tool for disease-related knowledge, the information in the web is largely incorrect [26]. Therefore, we suggest concentrating in online educational courses, especially in connection with patient organization to enhance IBD-related knowledge.

This study has several limitations. First, during the development of IBD-KNOW questionnaire, we tested candidate items in IBD patients in two tertiary referral hospitals. The final version of IBD-KNOW was validated in other IBD patients in the three hospitals including the above-mentioned two hospitals. Therefore, the clinical characteristics of validation cohort may very similar to those of development cohort. Therefore, IBD-KNOW questionnaire should be validated with external cohort, who may have different clinical characteristics compared with internal cohorts. Second, although we paid close attention to prevent meaning alteration during the making of the international version of the IBD-KNOW, validation of the international version is necessary. We anticipate that this task would be performed in patients with IBD who use English in the near future. Third, although we tried to make the number of items related with Crohn’s disease and ulcerative colitis equal in the IBD-KNOW, the number of items related with these two diseases is slightly different (one more item related with Crohn’s disease). However, we succeeded in adjusting the ratio of items wherein the correct answer is “true” and “false” exactly half and half.

In conclusion, the IBD-KNOW, a novel assessment tool of IBD-related knowledge, demonstrated excellent test characteristics. Hospitalization history, education level, and information acquired from patient organization play an important role in correct IBD-related knowledge.

KEY MESSAGE

1. The IBD-KNOW (Inflammatory Bowel Disease Knowledge), a novel assessment tool of inflammatory bowel disease (IBD)-related knowledge, demonstrated excellent test characteristics compared with outdated Crohn’s and Colitis Knowledge score.

2. Hospitalization history, education level, and information acquired from patient organization play an important role in correct IBD-related knowledge.

Appendix 1. International version of IBD-KNOW (Inflammatory Bowel Disease Knowledge)

kjim-2017-104-app1.docx ^{(16.4KB, docx)}

Appendix 2. Korean version of IBD-KNOW (Inflammatory Bowel Disease Knowledge)

kjim-2017-104-app2.docx ^{(64.7KB, docx)}

Appendix 3. The correct answer rate of each item of the IBD-KNOW (Inflammatory Bowel Disease Knowledge)

Item		Correct answer rate
1.	The terminal ileum is the last part of the small bowel. It is located in the right lower abdomen.	40.2
2.	The rectum is part of the colon. It starts approximately 15 cm from the anus and finishes at the anus.	68.5
3.	The function of the colon is to absorb nutrients.	44.7
4.	People can survive without the colon, but not without the small bowel.	36.7
5.	Specific foods to be avoided in inflammatory bowel disease are well known.	44.7
6.	Smoking cessation is important to prevent worsening of Crohn’s disease.	85.4
7.	Risk of inflammatory bowel disease increases with family history of this condition.	46.0
8.	Inflammatory bowel disease can develop in all age groups, but is more frequent at younger ages.	67.4
9.	Anemia may develop if severe inflammation persists.	74.2
10.	Crohn’s disease can occur anywhere in the digestive tract, from the mouth to the anus.	84.9
11.	Ulcerative colitis rarely involves the rectum.	46.2
12.	Inflammatory bowel disease can involve organs other than the bowels.	48.2
13.	Inflammatory bowel disease is considered cured if symptoms do not recur after a few years.	68.0
14.	Inflammation in the bowels may persist even if the symptoms improve after treatment initiation.	80.5
15.	Long-term steroid administration is advised to reduce inflammation recurrence.	66.0
16.	Constant blood monitoring is indicated for patients who are on immunosuppressive agents, such as azathioprine, because their white blood cell count may decrease.	62.0
17.	Biological agents are mainly used in patients with mild symptoms.	28.0
18.	Suppository or enema is used to treat cecal inflammation in patients with ulcerative colitis.	36.5
19.	Patients with inflammatory bowel disease for 8–10 years should have colorectal cancer screening.	70.0
20.	Permanent colostomy is performed if surgery is indicated for patients with ulcerative colitis.	40.5
21.	Patients with Crohn’s disease of the small bowel may be cured after surgery.	57.3
22.	Patients with inflammatory bowel disease should stop all the medications when considering pregnancy.	42.2
23.	Most patients with inflammatory bowel disease are advised cesarean section delivery.	31.5
24.	Immunocompromised patients with inflammatory bowel disease should avoid any kind of vaccination.	66.5