Assessing Patients’ Knowledge on Chronic Lymphocytic Leukemia: Validation of the ERIC CLL Knowledge Questionnaire in Greece

Chronic lymphocytic leukemia (CLL) is a chronic malignancy of B lymphocytes with an overall crude annual incidence rate in Europe of 4.9 per 100,000 and more than 11,000 new diagnoses/year.¹ Approximately, 80-85% of all patients with CLL are asymptomatic at the time of diagnosis, which is often made after a routine blood test raises suspicion by revealing an increased lymphocyte count. Evidence suggests that the “invisibility” of CLL makes it difficult for patients to accept the diagnosis of an incurable malignancy, thus often requiring more information about their condition, its management, and the way it might affect their life in the future.²

Sufficient knowledge is a prerequisite for successful self-management which encompasses “the individual’s ability to manage symptoms, treatment, physical and psychosocial consequences and life changes inherent in living with a chronic condition.”³ Therefore, assessing patients’ knowledge of their disease and its management in a quantitative way is critical to identifying gaps and enhancing health education through the design of appropriate psychoeducational interventions, applied at both a population and an individual level.

The current article describes the development, psychometric analysis, and pilot testing of the CLL Knowledge Questionnaire developed by the European Research Initiative on CLL (ERIC) within the context of its CLL patient empowerment program. The ERIC CLL Questionnaire is a self-administered instrument used to assess patient knowledge with regard to hematologic markers, CLL-specific disease manifestations and issues, CLL self-management, and CLL treatment-related adverse events.

The first version of the CLL Knowledge questionnaire was developed by a multidisciplinary panel of scientists involved in CLL research and consisted of 31 questions. The main areas of significance were identified, and the pool of multiple choice questions was enhanced accordingly with items assessing knowledge on (1) hematologic markers (4 items); (2) CLL-specific disease manifestations and issues (12 items), (3) CLL self-management (8 items), and (4) CLL treatment-related adverse events (7 items). The questionnaire was initially developed in English. It was then translated into Greek by 2 separate researchers, experts in the field and fluent in both languages. The two resulting Greek versions were subsequently consolidated into 1 final version by a third researcher within the research team. This original 31-item version of the questionnaire was given to a small number of patients with CLL (n = 5) who were contacted via the Greek CLL Patient Organization in order to check for comprehensibility, ease, and timing of completion. All patients completed the questionnaire unaided. After approval by the Ethics Review Committee of Centre for Research and Technology Hellas, 55 participants were sampled from 3 groups namely: (1) 21 hematologists in training (38.1%), (3) 17 other healthcare professionals (30.9%), and (3) 16 members of the general public with diverse professions (29%).

The psychometric properties of the questionnaire were assessed by examining the following parameters:

• – Content and face validity: The knowledge questionnaire was developed by a panel of physicians and other healthcare professionals with expert knowledge on CLL. The hematologists of the panel were all members of the ERIC and involved in the care and management of patients with CLL. The development process went through many iteration cycles of rephrasing, merging, omitting, or adding items in order to ensure each item’s clarity and importance.

• – Item analysis: Knowledge scores were generated by summing up the number of correct responses for each participant. Thus, responses ranged from 0 to 31 with 31 being the highest score corresponding to 31 correct items. (1) Index of difficulty: This index is calculated by the proportion of participants answering the item correctly and calculated by the number of correct responses over the total number of responses. This index provides an assessment of all questionnaire items and their appropriateness for inclusion in the scale. In order to rate items in terms of difficulty, the following thresholds (set at the 1st and 3rd quartile) were adopted: (i) (0-0.25) was interpreted as difficult, (ii) (0-25-0.75) was interpreted as right difficulty, (iii) (0.75-1.0) was interpreted as easy. (2) Index of discrimination: The discriminating power of an item lies in whether it can discriminate between the knowledgeable and the nonknowledgeable.⁴ This can be achieved by examining the relationship between how well participants did on the item and their total knowledge score. In order to ascertain each item’s index of discrimination, knowledge scores were analyzed by performing quartile analysis, and 2 groups were defined corresponding (1) to participants who achieved the higher 25% scoring (> 29 or equal) and (2) participants who achieved the lowest 25% scoring (< 17 or equal). Based on these groups, the D-value, a parameter of item discrimination, was computed for each item by subtracting the proportion of respondents answering correctly in the lowest quartile from the proportion of respondents answering correctly in the highest quartile. In order to rate items in terms of their discriminatory power, the following thresholds were adopted: (i) (0.40-1.0) was interpreted as a very good item, (ii) (0.30-0.40) as a good item, and (iii) (0.20-0.30) as a fair item and (0.09-0.20) as a poor item. Both indices were calculated for each of the 31 items included in the original set of the knowledge questionnaire and constituted the primary criteria for item selection. Following the application of these criteria, 4 items were excluded from the original 31 items leading to the final subset of 27 items that range in difficulty and maximize discrimination (see Supplemental Digital Content, Figure 1, http://links.lww.com/HS/A139 and see Supplemental Digital Content, The Methodology of Item Selection, http://links.lww.com/HS/A140)

• – Readability Analysis: The final 27-item version was passed through a readability tool (Flesch Readability Index), which assesses readability based on the average number of syllables per word and the average number of words per sentence. Scores range from 0 to 100. The higher the score the greater the number of people who can understand the document. The CLL knowledge questionnaire yielded a satisfactory result of 70-80 which is interpreted as “fairly easy to read.”⁵

• – Reliability Analysis: The internal consistency was assessed using Cronbach’s alpha coefficient. This reliability estimate measures the degree to which a set of items assesses a single construct. If the Cronbach’s alpha can increase by the omission of an item, then it is advisable to exclude said item. Internal consistency estimates range from 0.0 to 1.0 while estimates of > 0.7 are considered satisfactory. The reliability analysis was performed in the subset of the chosen 27 items revealing that the Cronbach’s alpha exceeded the 0.9 index (a = 0.94).

• – Criterion-related validity: The three different groups of participants were expected to markedly differ in CLL relevant knowledge (ie, high knowledge levels, medium knowledge levels, low knowledge levels, respectively). The questionnaire’s ability of discrimination between the different groups was assessed by comparing the performance of the 3 groups. The comparison was performed with one-way, between subjects’ analysis of variance and showed that there were statistically significant differences in the mean CLL knowledge scores between the 3 groups (F = 50.30; P < 0.001). Post hoc comparisons using the Tukey’s honestly significant difference test indicated that the mean score for the first group (mean = 24.52, SD = 1.77) was significantly different to the second group (mean = 19.82, SD = 2.9) (P = 0.005) as well as the third group (mean = 10.00, SD = 7,22) (P < 0.001). The mean scores of the second and third groups were also significantly different (P = 0.005) (see Supplemental Digital Content, The CLL Knowledge Questionnaire, http://links.lww.com/HS/A141).

The development, testing, and analysis of the questionnaire were designed after drawing on the methodology used in the development of knowledge questionnaires in other disease areas such as asthma, kidney disease, arthritis, and multiple sclerosis.^6–9 The newly developed CLL knowledge questionnaire demonstrated good reliability and validity and can be utilized in the assessment of CLL knowledge of patients with CLL in clinical practice. Specifically, the first 3 sections can be used with treatment-naive patients, while all 4 sections can be used with patients who are receiving or have received treatment. As the CLL knowledge questionnaire was developed by ERIC members from different countries (ie, Greece, Italy, Sweden), it is not context-specific and rather aims at a wide, international audience.

Further work can test the ERIC CLL knowledge questionnaire on an experimental basis to assess change in knowledge states after the implementation of psychoeducational interventions aiming to empower CLL patients and the potential effects of such interventions on patients’ psychologic well-being over time. Furthermore, the ERIC CLL knowledge questionnaire needs to be thoroughly tested in other ERIC member states in the native language and specific sociocultural context in order to evaluate its generalizability. That said, the herein presented CLL knowledge questionnaire constitutes a useful tool for disease management with the potential to reflect the positive effect of empowerment on CLL patients’ health status and might be seen as a concrete step towards their active involvement in their care.

Acknowledgments

We would like to thank the Greek Chronic Lymphocytic Leukemia (CLL) Patient Organization and the participants of this study who gladly gave their time to assist the research team with the development of the CLL Knowledge Questionnaire. Furthermore, Anna Vardi, hematologist who kindly assisted with the forward and back translation of the questionnaire.

Sources of funding

The study was supported in part by unrestricted grant support by Janssen Europe, the Middle East and Africa, Janssen-Cilag Hellas Société Anonyme, and Gilead Sciences Hellas.

Disclosures

The authors have no conflicts of interest to disclose.