Effects of e-learning in a continuing education context on nursing care: a review of systematic qualitative, quantitative and mixed studies reviews (protocol)

Abstract

Introduction

Continuing education (CE) is imperative to the future of professional nursing. The use of e-learning by registered nurses for CE is spreading. A review of systematic reviews will be conducted to develop a broad picture of the effects of e-learning in a CE context on nursing care.

Methods and analysis

Systematic qualitative, quantitative and mixed studies reviews published in English, French or Spanish from 1 January 2006 will be included. The outcomes of interest will be extracted and analysed inductively and deductively from the Nursing Care Performance Framework; some themes include nursing resources, nurses’ practice environment, processes, professional satisfaction, and nursing sensitive outcomes. Three reviewers will independently screen first the title and abstract of the papers, and then the full texts in order to assess eligibility. Two teams of two reviewers will extract the selected reviews’ characteristics and data. The results from various types of reviews will be integrated using a data-based convergent synthesis design. We will conduct a thematic synthesis and transform all quantitative and mixed data into qualitative data.

Ethics and dissemination

Ethics approval is not required for review of systematic reviews. We will summarise evidence concerning the negative, neutral and positive effects of various forms of e-learning on different aspects of nursing care. If we find gaps in the literature, we will highlight them and suggest ideas for further research. We will also focus on positive effects and present, if possible, the components and characteristics of e-learning interventions that were found to be successful. We will present this protocol and results in international conferences in nursing, medical, and health informatics domains. We will also submit the results of our work for peer-review publication in a journal indexed in the international bibliographic database of biomedical information.

Strengths and limitations of this study.

• Review of systematic qualitative, quantitative and mixed studies reviews is an innovative and emerging type of research synthesis. The inclusion of reviews using multiple research designs and a diversity of data is justified by the possibility of broadening the repertoires of effects of e-learning on nursing care.

• To the best of our knowledge, this is the first review of systematic reviews that uses the Nursing Care Performance Framework to draw a broad, multidimensional and systems-based perspective on the dimensions and indicators of nursing care that can be impacted by e-learning interventions.

• Review of systematic reviews is still in its infancy regarding reporting, assessment of methodological quality, risks of bias and quality of evidence, especially for the qualitative and mixed studies reviews.

• One of the limits of reviews of systematic reviews is the lack of granularity of information provided by the review authors.

Introduction

Continuing education (CE), a term often used interchangeably with continuing professional development, lifelong learning and staff development,¹ is an imperative for the future of professional nursing.² In many countries, CE is mandated by professional or regulatory bodies, which encourages nurses to participate in these activities.³ CE is an opportunity to acquire knowledge, improve performance, support growth and development as a nursing profession, expand the nursing role and introduce, develop and advance professional competencies/skills.^{3 4} Ultimately, CE is intended to improve quality of care and patients’ health status due to changes in healthcare provider practice.⁵

Nurses may engage in CE activities for myriad reasons; some seek opportunities voluntarily, whereas others complete CE credits for specialisation or licensure. While there is a breadth of nursing-specific CE activities, nurses searching for CE may face many barriers in terms of work schedule/commitments, lack of support (from coworkers, employers and organisation), geographic distance, time away from work and activity cost.^6–8 The use of electronic (eg, computer and web-based) and mobile devices (eg, smartphones and tablets) to support learning (ie, e-learning and m-learning) is a promising avenue to face these challenges.

e-Learning is an umbrella term that encompasses various concepts and technologies related to learning, such as distance, digital, electronic, online, web based and mobile learning.⁹ For this work, we will use ‘e-learning’ as the terminology entailing a variety of electronic, digital or mobile devices used to support learning. e-learning has many advantages; it reduces travel time, is flexible and accessible, can be cost-effective and can allow learners to learn at their own pace and from the place of their choice.^{10 11} Furthermore, e-learning has the potential to provide tailored content and instructional methods based on the individual needs of learners and can present a variety of multimedia components such as text, audio, still and motion visuals to support acquisition of knowledge and skills.¹⁰ Even if there is no strong evidence to prove that e-learning is superior to traditional learning, results of systematic reviews (SRs) support that this is an effective alternative way to learn.^11–13 Moreover, it has positive impacts on nurses’ knowledge, skills, level of self-efficacy and satisfaction.^{13 14}

However, e-learning is not a panacea.¹⁵ Learners can encounter barriers, like skill requirement for using a particular device, low level of technological literacy, loss of time when the system/device does not work properly or the reduction of social contact compared with face-to-face learning.^{16 17} Clark and Mayer¹⁰ summarised drawbacks surrounding e-learning, including too many multimedia components interacting at the same time, a lack of features that promote learning, a loss of an exploratory learning environment and a lack of guidance for learners. The authors highlighted an interesting point: learning is better supported by effective instructional methods than by delivery medium (eg, virtual classroom and face-to-face classroom). Furthermore, we have to keep in mind that the process of knowledge translation into clinical practice is embedded in a complex and challenging phenomenon, which can be influenced by various elements such as: the nature of knowledge to be transferred, the expected outcomes of the educational intervention, the way the knowledge is transferred (eg, the instructional methods/implementation strategies, the use of tailored and individualised educational approaches and the medium) and the target audience.¹⁸

E-learning technologies have been studied extensively in nursing, especially for students in an academic context, as supported in a review of SRs (n=22).^{9 12} The results of this review did not lead to robust evidence of the superiority of e-learning over traditional learning, nor did they conclude which technology or medium of e-learning best influenced the acquisition of skills and knowledge for nursing students at undergraduate and postgraduate levels. However, e-learning was shown to reduce the cost related to education and save time for students and lecturers. To the best of our knowledge, there is no review of SRs that focuses on e-learning in a CE context for registered nurses (RNs).

Objective

The objective of this review of SRs is to systematically summarise the best evidence that comes from systematic qualitative, quantitative and mixed studies reviews (MSRs) regarding the effects of e-learning in a nursing CE context on nursing care (ie, resources, services and patients’ outcomes). We used the terminology ‘review of systematic reviews’ because it describes the concept in a simple and specific manner. Other terms are less specific, such as ‘overview’, which can be used in a generic way.¹⁹

To meet this objective, we will use a process of data conceptualisation by mobilising both inductive (data driven) and deductive (theory driven) approaches iteratively or simultaneously to guide all the methods and analysis processes. We will be open-minded to allow the emergence of new concepts, but we will also use concepts from an existing framework, the Nursing Care Performance Framework (NCPF),²⁰ as a tool to extract, synthesise and interpret data. The NCPF is useful to define an important concept of this review, namely, ‘nursing care’.

Why it is important to do this review of SRs

• The context of nursing education in an academic setting versus in a workplace setting as a CE opportunity is different. Inexperienced undergraduate students learn a large repertoire of clinical competencies over a short period of time, during their schooling period, whereas practicing nurses engage in a CE context to maintain and reinforce their clinical expertise over the long-term.

• Knowledge synthesis at the third level of research (ie, review of SRs) about the effects of e-learning already exists in an academic context, but there is not one exclusively on nursing workplace and CE.

• To complement existing nursing knowledge, we believe that it could be useful to use a review of SRs with an exploratory lens, as suggested by Caird et al.²¹ The synthesis it provides is ideal for identifying existing e-learning interventions used by RN in their workplace settings and possible outcomes of interest (based on the NCPF) and their effects (ie, positive, no effect or negative effects). NCPF has never been used as a framework to extract and analyse data for educational interventions among nurses.

Nursing Care Performance Framework

The NCPF²⁰ will be used to conceptualise how e-learning interventions could influence nursing care and impact health outcomes. This is an organisational model, originally composed of 3 subsystems, 14 dimensions and 51 indicators, from which we have juxtaposing elements of the actual scope of nursing practice²² as well as findings from our previous work²³ carrying out the impact of information and communication technologies (ICTs) on nursing care. Figure 1 presents the adapted version of the NCPF, which represents the range of possible outcomes for which data will be sought in this review of SRs.

Figure 1.

Adapted version of the Nursing Care Performance Framework, which represents the range of possible outcomes for which data will be sought in this review of SRs.

The NCPF demonstrates how the interplay of three nursing subsystems (structure, services and patients’ outcomes) can operate to achieve three key functions: (1) acquiring, deploying and maintaining nursing resources (structure); (2) transforming nursing resources into nursing services (processes) and (3) producing changes in patients’ conditions in response to the nursing services provided (‘nursing-sensitive outcomes’ or patients’ outcomes).

The first function refers to the human and material resources needed to provide effective nursing care, such as nursing staff supply, working conditions, staff maintenance and economic sustainability. The first way e-learning could influence nursing care is by considering it as a resource (ie, the first subsystem of the NCPF). We could pay attention to these elements when we extract data from SRs: exploring whether the availability of e-learning in healthcare settings impacts the quality of life at work for nurses and if e-learning acts as facilitator/motivator to enhance nurses’ working conditions or serve as a barrier that inhibits them. Another question could be: to what extent can e-learning create favourable conditions that attracts nurses and reinforces stability in the workforce?

The second function encompasses nursing services (ie, the second subsystem of the NCPF), which are defined in various dimensions: nurses’ practice environments (eg, nurse autonomy and collaboration), nursing processes (eg, assessment, care planning and evaluation, and problems and symptom management), nurses’ professional satisfaction and patient experience. E-learning can be viewed as a resource that has the potential to influence all dimensions of nursing services at different levels. E-learning can be seen as way to support nursing work and create a professional practice environment for nurses by, for instance, facilitating collaborative practice. eElearning could impact what nurses do, for instance, nursing interventions (processes), or the ability of nurses in using their competencies to provide healthcare. Resulting from these two dimensions, e-learning could influence nurses’ professional satisfaction in terms of quality of care provided, satisfaction or dissatisfaction of nurses using e-learning and/or patient experience.

The desirable end result of the interactions between nursing resources and nursing services is to improve patients’ conditions. The third function is then described as the positive changes that can be detected among patients (also called ‘nursing-sensitive outcomes’, ie, the third subsystem of the NCPF). As other models used in the learning domain,^{5 24} we could speculate that if e-learning changes nursing resources and nursing services, patients’ outcomes could be potentially affected. Examples of indicators in the NCPF are: patient comfort and quality of life, risk outcomes and safety, empowerment and functional status.

The NCPF has been chosen to fit in the scope of this review of SRs for many reasons: (1) it was useful as an extraction and analytical tool in previous work;²³ (2) it offers a broad, multidimensional and system-based perspective on the dimensions and indicators of nursing care that can be impacted by e-learning interventions; and (3) it can highlight many indicators that could be relevant to document and measure ways in which nursing care performance is impacted by CE.

Methods

The protocol of this review of SRs has been registered at the International prospective register of systematic reviews (PROSPERO), with registration number CRD42016050714. We used the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) checklist to guide the elaboration of this protocol (see online supplementary appendix 1).²⁵

Design

We will conduct a review of systematic qualitative, quantitative and MSRs which is, to the best of our knowledge, an innovative and emerging type of research synthesis. The inclusion of SRs using multiple research designs is justified by the possibility of broadening the repertoires of effects of e-learning on nursing care.

As underlined by Lunny et al,²⁶ methods to conduct, interpret and report review of SRs are in their infancy. To the best of our knowledge, no unified and integrated tool allows a comprehensive reporting of a review of systematic qualitative, quantitative and MSRs. We will follow the general methods for Cochrane reviews²⁷ and other relevant works in this domain^{26 28 29} to conduct and report the review of SRs.

Eligibility criteria

The scope in this review of SRs is formulated using PICOS (participants, interventions, comparisons, outcomes and study design).^{30 31}

Type of reviews

We will include all types of systematic qualitative, quantitative and MSRs that evaluate the influence of e-learning used by nurses on nursing care in a CE context that have been published in French, English or Spanish from 1 January 2006.

Publication type

To be included, the reviews have to be ‘systematic’:³²

• clear and unambiguous;

• include a type of research and one or a combination of method(s);

• have specific research question(s), precise inclusion criteria, a comprehensive search strategy, a quality appraisal process and a rigorous synthesis.

The systematic qualitative, quantitative and MSRs must be published in peer-reviewed journals. Reports that outline a systematic methodology are included. We will exclude grey literature (eg, conference proceedings, trial registries and dissertations) and non-SRs such as literature reviews.

Population

We will include RNs according to the professional legislation of each country. Reviews that target RNs and other health professionals (eg, physicians) will be included as long as it is possible to differentiate nurses and to extract these participants’ data. Patients receiving care from qualified RNs through the medium of e-learning will be part of this work, as long as nursing-related outcomes are discussed. We will exclude undergraduate nursing students in an academic context.

Intervention

All types of e-learning delivered through different devices are targeted. Blended learning interventions will be included as long as they have an ‘electronic’ or ‘digital’ component. Any types of simulation, including with a ‘physical’ mannequin (eg, high-fidelity simulation and technology-enhanced simulation) will be excluded. However, simulation could be included if it is done through virtual reality (ie, in an electronic learning environment).

Comparisons

We will include these types of comparisons: face-to-face learning, any other e-learning intervention and blended learning.

Outcomes

The outcomes will include but are not limited to the three subsystems (ie, nursing resources, nursing services and nursing sensitive outcomes), dimensions (eg, working conditions, time and efficiency, nurses’ practice environment, nursing processes, professional satisfaction and nursing sensitive outcome) and indicators (eg, learning, nurse–patient relationship and knowledge access) showed in the adapted version of the NCPF in figure 1.

Definitions and/or examples of components are presented (see online supplementary appendix 2) related to each outcome of interest. The purpose is not to provide ‘standardised’ definitions but to offer a guidance for the data extraction process. No ‘standardised’ definition is available for the outcome of interest based on the fact that included SRs may have: diversity in terms of the nature of data (qualitative, quantitative and mixed), heterogeneity in e-learning interventions and various possible outcomes. Furthermore, the data synthesis approach is abductive. This means that we will use the NCPF as a starting point to extract the data and analyse them, but we will let new data emerge from the reviews. If stable and fixed definitions are provided, the inductive part can be compromised.

The main outcomes of interests are those targeting the effects of e-learning on nursing resources and services. Then, if the outcomes belonging to these dimensions are found in the SRs, patients’ outcomes will be extracted.

We will exclude SRs that focus only on patients’ outcomes without discussing nursing resources or services. At least one nurse-related outcome need to be present in order to include a publication. Determinants of e-learning use (eg, intended use) without reporting ‘actual use’ of e-learning will also be excluded.

Search methods for the identification of systematic reviews

Publications will be searched through general health sciences (PubMed and Embase), nursing (CINAHL) and Joanna Briggs Institute electronic databases. Structured search strategies will be developed using the thesaurus terms of each database and using free text, targeting the ‘title’ and ‘abstract’ fields. The strategies will be adapted to the other databases. The search strategy will be developed by the research team and validated by a health information specialist. The results of each database search will be collected in a single reference database, and duplicate citations will be removed. An example of the search strategy in PubMed is presented (see online supplementary appendix 3). This strategy will be adapted and refined according to the specificities of the databases. Furthermore, to obtain additional SRs, we will hand search for relevant ones, contact authors to find other relevant works in this domain and will consult reference lists of included SRs.

Data collection and analysis

Selection of systematic reviews

The research team will use DistillerSR, a web-based SR software from Evidence Partners (Ottawa, Canada), to perform the overall tasks related to the realisation of a review of SRs. Citations retrieved from the searches will be imported into a reference management software such as Endnote. The database containing all the references will then be imported in DistillerSR. Three reviewers (GR, JPG and EH) will independently screen the title and abstract of the papers in order to assess their eligibility. Each paper will be reviewed twice. The reviewers will compare their results and discuss them in case of discrepancies. If a consensus cannot be reached, arbitration with a third review author will be required. After the first round of screening, full text copies of publications that meet the pre-established inclusion criteria will be retrieved. In cases when the information regarding the eligibility of a review is limited or incomplete (eg, when only an abstract is available), we will contact authors to request the full text or further details. We will use the PRISMA flow diagram to show the overall process of reviews selection.³³

Data extraction and management

The coding process will be done by four independent reviewers (GR, JPG, EH and JBP). We will use the NCPF to code, organise and classify the data according to the three subsystems (ie, resources, services and outcomes), the dimensions and the indicators. This is the deductive part of the synthesis. Additional codes will be generated inductively by the four reviewers from the text of the articles without fitting them into the existing model. The four reviewers will begin by coding a set of the same three articles independently in order to ensure consistency during the coding and data extraction process. The independently developed frameworks or ‘coding plan’ will then be compared and combined into a single integrated framework.²⁶ Any conflict arising through this extraction process will be discussed between the four reviewers. After a general agreement on coding and data extraction, the remaining articles will be divided equally between two teams of two reviewers.

The four reviewers will summarise general characteristics about SRs: purpose, type of review (qualitative, quantitative or mixed), examples of topics covered, number of studies included, target populations, search dates and context (eg, mandatory CE and workplace). Details about e-learning interventions, comparisons and outcomes will also be extracted as follows: examples of e-learning interventions, devices or media used, examples of educational strategies and material, theory used to develop and evaluate interventions (eg, learning theory and behavioural change), examples of comparison interventions, dimensions and indicators based on adapted version of NCPF, effects of e-learning as reported by authors and nature of the effects (qualitative, quantitative or mixed). Any disagreements arising during the data extraction process will be resolved by discussion and consensus involving the two reviewers or will involve a third review author if needed.

Methodological quality assessment of included systematic reviews

In this review of SRs, we will include different designs. The array of underlying types of SRs combining qualitative, quantitative and mixed method evidence can render reporting and assessing the quality of reviews of SRs more complex. At the time of this review of SRs, we found no reporting guidelines on assessing methodological quality of qualitative and MSRs.

One of the most commonly used tools for authors of quantitative SRs using a randomised controlled trial design is the Assessment of Multiple Systematic Reviews (AMSTAR).^{34 35} AMSTAR is an 11-item checklist from which reviewers assign one point when the criterion is met. Quality is characterised at three levels: 8–11 is high quality (ie, minor or no methodological limitations), 4–7 is medium quality (ie, moderate methodological limitations) and 0–3 is low quality (ie, major methodological limitations).³⁶ AMSTAR items provide an assessment of methodological criteria such as the comprehensiveness of the search strategy and whether the quality of included studies was evaluated and accounted for.³⁷ Although AMSTAR has limitations (eg, inappropriateness of applying some criteria to MSRs and qualitative reviews), as underlined in previous work,²³ the four reviewers (GR, JPG, EH and JBP) will apply the tool to all SRs in order to use the same criteria for quality assessment.

Risks of bias and quality of evidence

Others challenges encountered for authors of reviews of SRs are the assessment of limitations (risk of bias) as well as the quality of evidence in SRs³⁸ A tool has been recently published, named ROBIS, to assess or avoid the risk of bias in SRs.³⁸ It has been developed for guideline developers and authors of reviews of SRs. Three steps can be filled in when using the tool: (1) assessment of relevance (optional) between a review question and its fit/match with the review of SRs question, (2) identification of research steps where bias may be introduced into the SR process (ie, eligibility criteria, identification and selection of SRs, data collection and review appraisal, and synthesis and findings) and (3) overall judgement of risk of bias. Bias appears if limitations in the design, conduct or analysis of a review alter the results. Two reviewers will then assess independently the risk of bias with ROBIS tool and will compare their results.

We found no tool or guidance to perform the quality of evidence assessment for authors of reviews of SRs. The Grades of Recommendation, Assessment, Development and Evaluation (GRADE) has been largely adopted as a tool to judge the overall quality of evidence for each individual outcome (ie, consideration of within-study risk of bias, directness of evidence, heterogeneity, precision of effect estimates and risk of publication bias) in the context of quantitative primary studies, especially those using experimental or quasiexperimental designs.^{39 40} When the unit of analysis is SRs and not primary studies, it is not always possible to extract GRADE ratings because data can be missing, not reported adequately or reported in different ways across the SRs. The use of a tool to assess the quality of evidence has to be modified for use in reviews of SRs.⁴¹ Recently, two tools have been published to assess both the confidence in qualitative review findings (methodological quality or dependability) and the potential influence of study quality on the review findings: confidence of synthesised qualitative findings, named ConQual,⁴² and Confidence in the Evidence from Reviews of Qualitative research, called CERQual.⁴³ They both aim to provide a qualitative equivalent to the GRADE approach and both present a final ranking,⁴⁴ but they are not currently considered as gold standard. We found no tool to assess the quality of evidence in MSRs. In this review of SRs, we will report the assessment of quality of evidence and risk of bias performed by original systematic quantitative, qualitative and MSRs authors who used GRADE, ConQuaL, CERQual or other approaches. In other words, only the quality indicators used by the authors of the original SRs will be reported, and no additional evaluation will be done.

Finally, another element to consider in a review of SRs is the risk of biased results caused by the repetition of primary studies that are included more than once (ie, overlaps) across the SRs.⁴⁵ It is important to calculate the actual degree of overlap in reviews of SRs with the corrected covered area method in order to report these overlaps properly.⁴⁵ As suggested by Studziński et al,⁴⁶ one reviewer will generate a matrix that will cross-link the SRs (columns) with primary studies included in the reviews (rows), and a second reviewer will check the matrix.

Data synthesis

An important challenge of data synthesis is the integration of the systematic qualitative, quantitative and MSRs.⁴⁷ In order to integrate the results from various types of SRs, we will perform a qualitative thematic synthesis using a data-based convergent synthesis design.^{48 49} We will qualify quantitative data, as we did in our previous work.²³ Qualifying the quantitative data means that we will use a textual and narrative approach to name and qualify the effect. We will then categorise the quantitative effect under a specific theme (eg, knowledge use). Within this theme, subthemes may be created to make a distinction between qualitative, quantitative and MSRs’ findings. Aromataris et al⁵⁰ suggest to present overall effect estimates, numerical data and overall synthetised qualitative findings extracted from each review in a tabular presentation of findings. Under a theme, subthemes could be divided by type of review (ie, qualitative, quantitative or MSRs) to keep the details, and then, an integrated synthesis could be conducted to summarise the effects.

However, if the results of the SR demonstrate that e-learning leads to a significant increase in knowledge, instead of reporting the p- Value, we will qualify the result: positive effect of e-learning on knowledge level. Frantzen and Fetters⁴⁷ call this approach ‘transformation’, in which quantitative data are transformed into qualitative data. We will also organise the results into themes and subthemes according to the specific dimensions of nursing care (eg, practice environment, nursing processes, professional satisfaction and nursing-sensitive outcomes) and their corresponding indicators. Even if this is an uncommon approach, we do believe that this way of synthetising will allow us to keep the richness of the results.

In order to transform all quantitative and mixed data into qualitative data, we will employ Thomas and Harden’s approach.⁵¹We will follow these three steps: (1) coding relevant extracts of each SR line by line, (2) developing descriptive themes and (3) generating analytical themes. This might lead to an adapted version of the NCPF cited earlier. The thematic synthesis will be done in an inductive and deductive way (ie, abductive), which means that some themes will be organised based on the NCPF,^{23 24 52} while others will emerge inductively.

Conclusion

Results of this review of SRs could be used to understand the dimensions of nursing care that have the potential to be supported, enhanced or constrained by the use of e-learning to sustain CE activities among nurses. This review of SRs is a continuation of previous work that has been done about the impacts of various types of ICTs (excluding e-learning interventions) on nursing care.²³ Some reviews on e-learning used by nurses or nursing students target specific outcomes, especially knowledge, attitudes, barriers and facilitators, skills and satisfaction regarding the use of e-learning.13 14 53 54 By using the NCPF to organise, extract and analyse the data, this review of SRs could provide a good starting point to deepen our understanding regarding the dimensions and indicators of nursing care that can be impacted by e-learning. With the growing presence of digital devices in nursing care systems, we think it is important to document the interaction of e-learning and nursing care dimensions and indicators. We believe that if we better understand the effects of these e-learning interventions, we can deploy strategies to facilitate their implementation and integration into nursing care, nursing research, management and education. Consequently, we can overcome their negative effects and optimise positive ones in order to use them to their full potential as tools to support nursing practice and, ultimately, improve patient outcomes.