Development and validation of education materials to reduce childhood blindness due to retinopathy of prematurity

Abstract

BACKGROUND:

Timely and appropriate follow-up appointments for infants at risk for retinopathy of prematurity (ROP) are very important to prevent blindness. Caregivers are important members of the ROP team, and their involvement is essential in ensuring optimal visual outcomes. This paper aimed to develop health information materials on ROP by a systematic process for better comprehensibility by the target audience of low literacy.

MATERIALS AND METHODS:

It was a methodological study conducted at the neonatal intensive care unit of a tertiary care hospital, North India. The development and validation of educational materials was conducted in six steps. The study focused on both the knowledge of the target audience and on the validation of the educational materials by experts and caregivers of ROP eligible infants.

RESULTS:

Most of the items (content, language, layout, motivation, and cultural appropriateness) were in either a suitable or adequate category. Only one item illustration was in the nonsuitable category. The mean final score of the leaflet after revision by experts was 9 (maximum score = 10). Regarding readability, The Flesch Reading Ease Score, Flesch-Kincaid Grade Level, and gunning fog index were found to be 72.5, 7.4, and 6.2, respectively. The leaflet was found to be suitable for the seventh grader. The mean knowledge score of the parents was 4 (maximum score-5).

CONCLUSION:

The study showed satisfactory acceptance of the developed ROP information materials by caregivers and experts. A similar approach could be adopted for the development of other health information materials.

Introduction

Retinopathy of prematurity (ROP) is one of the major causes of childhood blindness that primarily affects premature infants.[1] In 2012, worldwide, an estimated 50,000 children got blind from ROP.[2] This global and national burden of ROP is set to increase tremendously in near future with further improvements in the survival of sick preterm and low birth weight babies.[3,4] However, it can be prevented and treated, if detected on time through follow-up screenings.

The ROP team in the hospital includes neonatologists, paediatricians, ophthalmologists, and nurses who are all involved in the care, screening, and treatment of a baby with ROP. However, the role of caregivers must not be underestimated.[5] Studies have shown that parents who are more aware of their child's medical condition, and who are engaged in their care while in the neonatal intensive care unit (ICU), have more positive attitudes and are more likely to bring their infant back for follow-up.[6,7] Good communication is very important for developing relationships with parents. Verbal communication should be supported by written materials. Literature has shown that written information increases awareness, knowledge, recall, compliance, and satisfaction of patients and caregivers.[8]

It should also be emphasized that all the time and effort spent on generating the education material is worthless if the target audience cannot understand it. The information which is not comprehensible could lead to an increase in anxiety, and this may cause them to move away from the importance of the issue and neglect it. Therefore, this article aims to give a comprehensive description of the process of construction and validation of health information materials regarding prevention and timely screening for ROP for caregivers. These materials can be used as reinforcement by neonatologists, paediatricians, nurses, and other auxiliary health-care workers during verbal communication with the caregivers. A similar methodological approach can be used for the evaluation of the health education materials, in general, to augment verbal communication.

Materials and Methods

Study design and settings

It was a methodological study conducted in 2018 at the neonatal ICU of a tertiary care hospital, North India.

Study participants

The study focused on both the knowledge of the target audience (caregivers of premature infants) and on the validation of the educational materials by experts and target audience.

Data collection tools and technique

The development and validation of educational materials was conducted in six steps:[9]

Review of literature regarding retinopathy of prematurity

A comprehensive review was done on all aspects of ROP which are important from caregivers’ perspective starting from the relation of ROP with prematurity, risk factors, need for timely screening and treatment, screening procedure, complications, and prevention. An interview schedule was developed for caregivers of infants who were eligible for ROP screening based on guidelines.[10]

• Gestation <35 weeks AND/OR

• Birth weight <2000 g AND/OR

• Gestation 35–36 weeks with the presence of risk factors for ROP.

Identification of needs/gaps in the knowledge of caregivers through interviews

A total of forty caregivers were interviewed by pediatric nurses in tertiary care. They were inquired about their awareness of ROP and the information they want to know about this sight-threatening disease. For open-ended questions, the answers of caregivers were recorded verbatim in the same language. The data were analyzed utilizing content analysis from which the researchers highlighted snippets of interest and grouped them into categories. Based on the results, a rough draft of the educational leaflet on frequently asked questions and a poster on ROP for display in waiting areas was designed with relevant illustrations in English.

Assessment of the readability of the developed materials

Centre for Disease Control has recommended that for better comprehension readability of patient education materials should not be higher than sixth- to eighth-grade level.[11] We used the Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), and Gunning fog index for the evaluation of readability and comprehensibility as they have extensively been used within the health-care literature with evidence of their validity and reliability.[12] To calculate these readability scores, the online readability text consensus tool was used.[13] This tool analyzed the text and calculated the number of sentences, words, syllables, and characters in the sample. The accuracy of the online method has been confirmed by the prior comparison of automated and manual calculations.[14]

Validation of the educational material by experts

The ROP leaflet was subsequently validated by a panel of experts including four neonatologists, two ophthalmologists, two pediatric nurses, and two public health professionals who had experience in the development of health educational materials. The evaluation focused on the coherence, adequacy, clarity of the information, layout, motivation, as well as quality of the illustrations on a three-point Likert scale based on Suitability Assessment of Materials (SAM).[15] The SAM has been validated and successfully used in prior studies on printed health information. The SAM consists of six evaluation criteria: content (behavior information to help solve their problem), language (common, explicit words are used), illustration (simple drawings/sketches are used), layout (type size is at least 12 point, no ALL CAPS for long headers or running text), motivation (complex topics are subdivided into small parts so that readers may experience small successes in understanding or problem-solving), and cultural appropriateness (images and examples present the culture in positive ways). The final scores should be equal to or >60%, to consider the material adequate.[16] Space was available on the questionnaire for open comments about the materials.

Translation into the local language (Hindi and Punjabi)

Since our target audience mainly includes caregivers with low literacy, the leaflet was then translated into local languages (Hindi and Punjabi) by a team of researchers for better comprehension. The translated versions were again circulated among the panel of experts and recommendations were incorporated.

Legitimating and testing of the educational material by caregivers of retinopathy of prematurity eligible babies

An assessment of leaflet was then performed among ten caregivers of ROP eligible babies (5 each for two languages). It included purpose, design, language, appearance, and motivation. The quantitative data analysis was done by 'IBM SPSS Statistics for Windows, version 23 (IBM Corp., Armonk, N.Y., USA).

Ethical considerations

This study was done as a part of the project entitled “Reducing ROP by improving the quality of neonatal care in special newborn care units” approved by the Institute Ethical Committee (IEC-07/2015-251). The written informed consent of the participants was taken from every respondent before data collection.

Results

A total of forty caregivers, with a majority (n = 35, 87.5%) being mothers, were interviewed (20 on the day of their infant's first ROP screening at neonatal ICU and 20 on the day of discharge of their infant) at tertiary care hospital. The age group of caregivers ranged from 18–32 years with around half (47.4%) had two children. Eleven of them were graduate/postgraduate with one-third (37.5%) having high school certificates. More than half (55%) belonged to lower-middle socioeconomic status. The birth weight of their infants ranged from 600 to 2000 g.

The caregivers of all the infants had already undergone their first ROP screening; even then, only one of them was aware of this sight-threatening disease. She searched about that on the internet. They only knew that their babies had eye examinations and were being called for the next eye examination but were not aware of the purpose and importance of that examination. When they were asked about their views what they would like to know from a health-care provider, the following themes arose:

• Relationship between prematurity and childhood blindness

• Risk factors

• Screening procedure and its duration

• Importance and frequency of follow-up examination

• Prevention of ROP

• Treatment of ROP

• When and where to go?

Some of the excerpts from the interview

“Do all premature children get blind?”

“How will I know whether my child has ROP?”

“My child has born in seven and half months (premature) so I wanted to know how can I save my child from this sight-threatening problem.”

“Can this examination be done at a hospital near my house?”

“How many times, I have to come for screening? How much time it will take?”

“Will this examination cause pain to my baby?”

“If my child has ROP, would he be blind now? Can his sight be saved?”

Based on the literature and caregivers’ views, a leaflet and poster on ROP were developed in English taking into account the checklist for quality Patient Information Leaflets.[17]

An item-by-item analysis of the tool revealed that most of the items (content – 63.3%, language 62%, illustrations – 60%, and cultural appropriateness – 65%) were in the adequate category [Figure 1]. Only two subitems in illustrations; relevance and captions were found in the nonsuitable category. The experts also proposed some suggestions such as first ROP screening at 1 month uniformly instead of 4 weeks, colors of boxes, important text to be bold, use of pictures with good resolution, time of follow-up examinations, and language editing. One of the experts also recommended adding case studies to make caregivers aware of the gravity of the condition.

Figure 1.

Validation of retinopathy of prematurity leaflet by experts under various parameters of SAM

The leaflet was modified as per the experts’ comments, and two case studies were also added to engage and motivate the targeted audience for timely screening and follow-up of ROP. The leaflet was again distributed among the experts and the mean final score by experts on the recommendation of this leaflet was 9 (maximum score = 10).

Regarding readability, The FRES, FKGL, and gunning fog index were found to be 72.5, 7.4, and 6.2, respectively. The leaflet was found to be suitable for seventh grader. The revised English version was translated into the local language (Hindi and Punjabi) and was again circulated among experts for their feedback. The required language corrections were done and were validated and tested among ten caregivers. All of the caregivers were satisfied with the layout of the leaflet and found case studies very useful and motivating [Table 1]. Knowledge assessment of caregivers regarding ROP was done by asking five questions from the leaflet, and the mean score was 4 out of 5. The caregivers appeared to understand the leaflet and poster, achieving the goals of the study.

Table 1.

Validation of retinopathy of prematurity leaflet by caregivers

Items	Suitable (%)	Adequate (%)
Content	80	20
Language	80	20
Illustrations	70	30
Layout	100	0
Motivation	90	10

Discussion

The prevention, detection, and treatment of ROP are a team responsibility. Caregivers are the important members of this team, and their involvement is essential in ensuring optimal visual outcomes. The results of this study indicated that even on the day of discharge whereby, most of the babies have already undergone at least ROP one screening, almost all caregivers were unaware of ROP. They only knew that eye examination was done, but the purpose for that was not clear. This lack of parent awareness results in reduced follow-up ROP screenings and delayed reporting, which is crucial for ROP prevention and treatment. The possible reasons are that the hospital had a simple policy for recommending ROP screening, as a piece of written advice to the parents of premature infants on discharge card, typically containing sentence on a date, time, and place of ROP examination. Second, in the public sector, tertiary care hospital with a very high neonatal ICU admission rate with crunched hospital staff, there are limitations on the consultation time. Third, there is no effective well-written information available that could be given to the caregivers. Thus, providing caregivers with well written and validated information about the ROP can increase their participation and would reduce this preventable childhood blindness. This has been confirmed by a study, wherein giving written information to parents in terms of consequences of ROP increased the number of infants being examined during the specified time.[7] Moreover, the availability of leaflets in local languages will further enhance its reach and comprehensibility to the grass-root level. This is in line with the previous studies which have shown that providing patients with well-designed information leaflets improves patients’ acceptance and satisfaction.[18,19]

The main way to improve caregivers’ understanding is by seeking their active participation in the development of information materials for them. Therefore, the study explored and incorporated their knowledge, gaps, and views to develop ROP materials for them. In addition to increasing text coherence, we also integrated illustrations and text as research has shown that information is better comprehended when these two are combined.[20] A similar methodological approach has also been used in a study to develop a health educational package for preemie moms in the care of their baby after hospital discharge.[21] Along with the design of the information material, there is an equal need for its comprehensibility, especially by the target audience of low literacy. Studies have shown that health education materials are written at a higher reading level suited to the average reader.[22,23,24] The materials in a study were developed using simple language, made them suited for the average reader (seventh grader).

We agree that there is no substitute for good verbal discussion, but written materials can play an important part in supplementing and reinforcing information, as long as they conform to the highest standards of scientific accuracy, comprehensibility, and relevance.[25] We followed the standardized checklist for quality patient information leaflets and included all parameters including the date of last update, references to sources of the information, number, and address of contact person along with other requirements.[17]

Two short case stories were also added on the recommendation of experts, wherein the positive and negative outcomes of timely ROP screening and treatment were presented. This is in concordance with other studies.[26] Storytelling has its ability to present information couched within a personal account that engages the reader and validates their own experiences. There is evidence that memory of information may be enhanced when presented in narrative form.[27]

The study is the first of its kind in the application of a systematic process for the development of information materials for ROP for caregivers, the involvement of both consumers and producers of information materials, use of readability formulas for better comprehension, and testing of materials with the targeted audience.

Limitations and recommendation

The knowledge level was measured shortly after participants had read the leaflet. Second, all participants were from a single tertiary care hospital, which may affect the evaluation of this educational material, as their opinions may reflect the social characteristics of the population seen in this hospital. However, since the hospital is one of the renowned public sector hospitals, there is an equal possibility of the presence of all sections of the society. We recommend that a similar methodological approach can be adopted by health-care professionals to develop other printed and online health information materials to increase the knowledge of patients/caregivers.

Conclusion

The ROP information material design and validation involved scientific knowledge, teamwork, and consideration of the audience receiving the material. The study showed satisfactory acceptance of the written ROP information materials by caregivers and experts. This leaflet is expected to be an effective tool for communication with caregivers that would help increase ROP screening and follow-up and reduce the burden of childhood blindness.

Financial support and sponsorship

The study was funded by Queen Elizabeth Diamond Jubilee Trust, UK.

Conflicts of interest

There are no conflicts of interest.