Web-based learning strategies in combination with published guidelines to change practice of primary care professionals

Abstract

Background

Published clinical guidelines need to be combined with effective educational interventions to produce change in practice. Problem-based learning (PBL) groups are effective; however, research studies have not looked at change in practice following PBL that is independent of group interaction.

Aim

To determine whether clinical guidelines combined with modified PBL e-learning modules, in which dialogue between learners is replaced by interaction with a computer, could increase knowledge, result in changes in practice, and be acceptable to participants as a mode of modifying practice.

Design of study

Quantitative and qualitative methods.

Setting

Web-based application.

Results

Online PBL modules were studied by 45 primary care clinicians. Replacement of group functions by interaction with the computer was found to be an acceptable form of learning, and resulted in reported changes in practice among GPs choosing this method of study. This study shows that changes were independent of demonstrable knowledge gains.

Conclusion

Online PBL modules could complement other arms of guideline-implementation strategies. More work is needed to determine which groups would benefit most.

INTRODUCTION

Evidence-based practice (integrating individual clinical expertise with the best available external clinical evidence from systematic research,¹ to inform discussion with individual patients) is expanding due to pressure from clinicians, managers, and lawyers; however, there is good evidence that unsolicited distribution of guidelines, or posting these on the internet does not lead to changes in practice.^2–4 Guideline distribution needs to be combined with effective educational interventions and implementation plans to change practice.^3–5

For common primary care conditions, particularly those in which a number of team members are involved in management, the gold standard for learning about guidelines would be learning within the organisation, as this leads to agreement between colleagues on acceptable standards and learning about how to achieve those standards,² as well as discussion about overcoming barriers to change. However, this is time consuming, and it is not feasible for all guidelines to be discussed in this way; other forms of educational intervention, suitable for individuals, need to be offered if change is to be achieved. Any implementation plan should include consideration of how learning can be accessed by all relevant clinicians: those working in one practice, locums, out-of-hours doctors, remote and rural clinicians, and those whose domestic circumstance make attending meetings problematic.

E-learning has been shown to be increasingly popular.⁶ E-learning packages may take many forms: hypertext-based packages, such as online books, exercises where skills are practised until mastered, simulations where students have to apply knowledge, and intelligent tutoring to facilitate mastery of procedures;⁷ the key is that all should facilitate learning based on sound educational principles.

E-learning packages have the advantages that they can be used efficiently at any time, and anywhere; learners can work at their own pace, but can also receive immediate feedback on performance.⁸ They offer important benefits for remote and rural practitioners, and those with other commitments that make attending traditional continuing medical education (CME) events difficult. Fordis et al demonstrated that online CME, designed to provide the same stimuli and opportunities for case-based consideration with immediate feedback as a live session, could be equally effective in bringing about change in practice.⁹

Problem-based learning (PBL) is an educational strategy that is based on cognitive educational theory, and encourages development of problem-solving and judgment-making skills.¹⁰ Traditional forms have been shown to be effective in changing practice when used with guidelines.¹¹ Traditional PBL involves group discussion, which has been thought to lead to clarification of learning needs and elaboration of learning.¹² Problems have been combined with email discussion groups, showing mixed results;^13,14 however, these courses are time consuming and require very considerable commitment on behalf of participants.

More recently, software such as Moodle has been developed.¹⁵ Moodle is designed to facilitate online live discussion between students and teachers, forums for discussion not in real time, and workshops where students can collaborate in producing work. In theory, this would be ideal for internet-based PBL programs, but again this requires very considerable time commitment for GPs to take part in asynchronous discussions, or workshops, agreement on times if live discussion is used, and major time commitment for teachers.

How this fits in

The unsolicited distribution of clinical guidelines does not generally result in change in practice; however, in combination with face-to-face problem-based learning (PBL) it is known to be an effective method in supporting change. This study has shown that the interactive elements of traditional PBL can be replaced with other means of producing the benefits of interaction, resulting in changes in reported practice among some GPs.

This project investigates whether combining guidelines with PBL e-learning modules, in which interaction between the participant and his/her peers is replaced with interaction between the participant and the computer, could increase knowledge, result in change in practice, and be acceptable to participants.

METHOD

Three problem-based e-learning modules were developed.

Topic selection

Three topics, chronic kidney disease stage 3, coeliac disease, and urinary tract infections were chosen as there is evidence that current and best practice often vary, this variation has significant health implications, and national guidelines are available (Table 1).

Table 1.

Reasons for choice of guideline topics.

Topic for module	Reasons	Guidelines available
Chronic kidney disease stage 3 in adults	Recently introduced into the Quality and Outcomes Framework (QOF) for the GP contract.16	Chronic kidney disease in adults: UK guidelines for identification, management and referral.20
	New laboratory reporting recently introduced in UK.
	Personal communication from GPs, nurse practitioners, and nephrologists suggest that optimal management in primary care has not yet been achieved.
	As the introduction of laboratory testing to detect chronic kidney disease stage 3 is recent in the UK, publications on compliance with guidelines are lacking
Coeliac disease in adults	Evidence suggests it is under-diagnosed.21	Guidance for the management of patients with coeliac disease.22
	Many practices do not have structured follow-up for patients with coeliac disease.	The management of adults with coeliac disease in primary care.23
	Significant health gains would result from developing practice monitoring protocols
Urinary tract infection in adults	An informal significant event analysis, performed at a meeting of GPs and practice nurses, demonstrated that the majority of participants were not following guideline recommendations; the majority were over-prescribing.	Management of suspected bacterial urine infection in adults.24
	At least one Scottish laboratory is seeing a year-on-year increase in catheter specimens of urine (personal communication)

Module development

The e-modules were developed using standard PBL format. Participants were posed a short series of extended matching questions (EMQs), where participants are offered a much greater number of choices than in standard multiple choice questionnaires; they were then presented with stimulus material. At this point in traditional PBL, groups participants would stop and discuss their learning needs and plan to meet them; in the current study, participants were asked to write down their learning needs before moving on and they were then given a list of suggested learning needs with which to compare their own.

Participants were then asked to use the national guidelines or other published material, which they could download from links in the program, to answer their learning needs. At this stage in traditional PBL groups, participants discuss the results of their information gathering, helping to consolidate and organise learning. In the current study, participants were asked to complete a management plan for the condition studied, in the hope that this would perform the same function, additionally providing an aide mémoire.

To cover all learning objectives, further stimuli were presented, reflected upon, and needs answered using the guidelines until the whole management plan for the condition could be completed. Participants repeated the EMQ, this time provided with the answer and some additional feedback from published work, before completing an evaluation and printing a completion certificate.

Evaluation

Knowledge change was assessed by using a paired Student's t-test to test the null hypothesis that no change in knowledge had occurred between EMQ scores at the start and finish of the modules.

Acceptability and intention to change practice were evaluated by a short questionnaire at the end of the module which was submitted electronically. Two specific questions about whether they intend to change their practice were included for each of the three topics: chronic kidney disease stage 3, coeliac disease, and urinary tract infections. These questions were chosen because evidence suggests that many practitioners need to make changes to their practice specifically regarding these conditions; for example, one of the questions for urinary tract infections asked: ‘Do you think that doing this module will result in a reduction in your antibiotic prescribing?’.

Acceptability and changes in practice since completing the modules were evaluated by performing semi-structured interviews with participants between 3 and 6 months after module completion. All participants were invited, by email, to agree to telephone interview, and non-responders were issued a second invitation.

Face and content validity

An expert in the field, a GP educationalist, a practice nurse educationalist, and a non-medical educationalist reviewed each module, providing feedback on factual aspects, coverage of the topic, educational aspects, web design, predicted acceptability, and appropriateness for the target audience.

Recruitment of participants

GPs and practice nurses were invited to participate in free continuing-professional development. Invitations were by NHS contact group emails, forwarded by appraisal administrators, and some associate advisers and practice nurse advisers across Scotland.

RESULTS

Results of online evaluations

Fifty-eight modules were completed by 45 participants; 43 of the participants were GPs. The probability that no change in their knowledge had occurred, and participants' perceptions of the effect of studying the module as reported by completion of the second EMQ test and the questionnaire at the end of the module, are shown in Table 2.

Table 2.

Results of online evaluations for participants completing a module.

	CKD stage 3	CD	UTI
Participants, n	23	17	18
Mean improvement in EMQ score (%)	0.009	20.6	12
P-value	0.998	0.014	0.062
Participant found the module helpful, n	11	14	17
Participant would do another module of this type, n	14	15	15
Participant thought doing module would result in both suggested changes in practice, n	13	14	10
Participant thought doing module would result in one of the suggested changes in practice, n	18	16	15
When asked if practice was reinforced or change required, n replying ‘change’	18	12	14

CD = coeliac disease. CKD = chronic kidney disease. EMQ = extended matching question. UTI = urinary tract infection.

Results of interviews

Ten interviews were conducted with participants who had studied 19 modules between them. The following themes were identified.

Concept

Nine of the ten interviewees had found the modules useful and enjoyable; they particularly liked the options of studying where and when they wished, and the fact that topics were interesting and frequently encountered. Three interviewees specifically commented on ‘How practical the modules were’. Other comments were ‘Good way to learn, I remember things best if I can relate them to a patient’; ‘Made me work smarter’; ‘A good method of learning’; ‘Good fun, good idea’; ‘Not as hard work as I expected — refreshing’. One participant ‘preferred BMJ Learning structure’, where information is provided in small chunks, but also admitted that ‘I probably did not commit sufficient time to [the chronic kidney disease module]’.

Structure

Participants felt that the modules were ‘Clear concise and easy to follow’; the EMQs were successful in provoking cognitive dissonance, particularly for the participant who said ‘Questions at the start made me realise that I did need to read the guidelines; I thought UTIs was bread and butter and I would know it all’. The patient-orientated approach was specifically appreciated by many interviewees.

The majority of participants did not actually write down learning needs as requested; however, one participant was ‘Surprised to realise that I did not get them [learning needs] all at once’; 37% of the modules resulted in a management plan being written down. Those produced a management plan found it useful: comments included ‘Flow chart useful, have referred to it since’; ‘Keep flow chart on my desktop’; ‘Keep management plan in my bag’.

Knowledge tests

Participants found the EMQ useful in stimulating dissonance: ‘Useful to test oneself, does not really matter what style of questions’; and ‘Stimulating questions’. However, many found it annoying that their scores had not improved. The format was unfamiliar to them, and enough attention was not given to studying questions. Specific comments were: ‘Didn't read them carefully enough’; ‘Had to read them carefully’; ‘Skimming questions may have been an issue’.

Change in practice

Fifteen of the 19 modules studied resulted in participants feeling that practice needed to change, and in 15 cases participants felt that it had changed following the module. Specific changes reported included reduced urine testing, reduced and more-discriminating prescribing, change in preferred antibiotics in urinary tract infections; more-frequent testing for coeliac disease, and greater awareness of the implications of test results; and improvement in the practice chronic kidney disease protocol. Increased confidence was particularly noted by those using the chronic kidney disease module, but there were few changes as participants had already made changes to their management of low estimated glomerular filtration rate before studying.

Several participants recognised that to bring about change they needed to involve their colleagues. No barrier to change other than colleague cooperation was revealed.

DISCUSSION

Summary of main findings and comparison with existing literature

There is not sufficient information to draw any conclusions about the effectiveness or acceptability of the modules in nurses or nurse practitioners; any conclusions drawn will only apply to GPs. Also, as it was impossible to collect data from clinicians visiting the site who did not complete a module, there is no information from those who chose not to start or not to finish a module. Findings are therefore limited to those who chose and worked through a module.

Evidence suggests that to give the modules and guidelines presented here the very best chance of resulting in change in practice, they should be studied by teams together, who would accept the recommendations, discuss barriers, plan implementation, and monitor adherence; this is not feasible in British primary care because each module would require a session of protected time. However, if key individuals study and then are able to share their learning with their teams, discuss, and introduce plans for change, effective change will occur.

In planning educational interventions, thought should be given to clinicians who find attending conventional educational sessions problematic. This study has been able to show that, for some clinicians, modification of traditional PBL structure to allow individuals to interact with a computer rather than colleagues is acceptable, and can result in reported change in practice. The results suggest a high level of acceptability among practitioners choosing this type of resource. Practitioners liked the fact that they could study where and when they wished, they liked the patient-orientated approach, found the topics interesting, and appreciated the ease of use of the modules.

Participants had been asked to stop, consider, and write down learning needs, and complete a written management plan. The majority of participants did not actually write, thus gains appear to be independent of actually writing in some of this group of clinicians.

Most participants reported intention to change practice; learners who studied urinary tract infections and coeliac disease frequently reported that they had changed their practice; others had not actually changed because they were not working in a situation where they currently managed the problem themselves. Some learners studying chronic kidney disease said they had not changed practice because they had studied and developed a management plan for this condition before studying the module; however, they did gain confidence from their study. Financial incentives can help reduce resistance to implementation in relation to hypertension treatment within the Quality and Outcomes Framework in the UK;^16,17 this study suggests that the same applies to chronic kidney disease.

Change in practice was confirmed by all participants interviewed on urinary tract infection and coeliac disease modules reporting examples of change; these were particularly numerous and varied for the urinary tract infection module, possibly reflecting the frequency with which practitioners deal with urinary infections.

Participants did not identify barriers to implementing change on a personal level, but two felt that colleagues studying the modules would help their practice team to implement change. Carlsen and colleagues recently published a meta-synthesis of studies of GPs' attitudes to guidelines, and identified a number of barriers to implementation.¹⁸ These include questioning of the guideline; the fact that patients are often more complicated than they appear in guidelines, and have their own views which can be impediments; fear of affecting the doctor–patient relationship; fear of ‘missing a diagnosis’ in guidelines that seek to limit investigations; time, skills, and resource limitations; and complex guidelines format. Most of these issues were not reflected in the current study; this may in part be a function of the PBL format prompting participants to consider these issues.

Practical issues was the only category represented as a barrier in this study, and these practical issues related to personnel. Some participants had shared learning with their teams; this is pleasing but may say more about the individuals, and the nature of their teams, than about the modules. However, it can be concluded that for participants who were prepared to share learning with their teams, the modules provided a useful stimulus to team learning.

It was not possible to demonstrate an increase in knowledge except possibly in the coeliac disease module; this is surprising in the light of acceptability, change data, and annoyance of interviewees on not improving. It may be that EMQ tests are less reliable in this group of learners than in students¹⁹ (in whom most evaluations of EMQs have been undertaken), and difference may be a function of familiarity and of the difference between busy professionals seeking to improve themselves, and students seeking to pass important examinations.

Limitations of the study

Agreement to be interviewed was obtained by only 10 doctors or 22% of participants. All participants were invited to participate in interviews, but only volunteers could be interviewed, so interviewees were self-selecting; this is likely to result in a positive bias for the modules and means that it is only possible to draw conclusions that some participants achieved the reported advantages. Therefore further work is needed to verify any conclusions. Sufficient data were not obtained to allow any conclusions to be drawn about use of modules by nurses in primary care, and further work should address these groups. Additionally, there is no information about clinicians who chose not to use this method of learning; conclusions can only be drawn for participants choosing this method of study.

This study lacks hard evidence of change. Widespread geographical locations of participants made this information impossible to collect. Reported change is never as reliable as hard evidence; however, interviews provided the opportunity to request that participants describe changes which provided additional collaboration and the opportunity to explore attitudes and any barriers to change.

EMQ tests were used because of reported benefits; however, these benefits had not been substantiated in this type of learner, multiple choice questions would provide a more-familiar and easier-to-use format for further studies.

Implications for future research

This small study has demonstrated that combining national guidelines with online PBL modules can result in reported change in practice in GPs selecting this form of study. E-learning replaced group interaction with an alternative method of stimulating recognition of needs, consolidation, and elaboration of learning. Further work is needed to determine whether similar resources should be provided in association with the release of national guidelines,³ and to clarify the characteristics of audiences expected to benefit most from problem-based e-learning.

Funding body

NHS Education Scotland (project number 60011)

Ethical approval

Not applicable

Competing interests

The authors have stated that there are none

Discuss this article

Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss