Abstract
Aim and objectives
To develop a suite of metrics and indicators to measure the quality of children's nursing care processes. The objectives were to identify available metrics and indicators and to develop consensus on the metrics and indicators to be measured.
Background
The Office of Nursing and Midwifery Services Director, Health Service Executive, in Ireland established seven workstreams aligned to the following care areas: acute, older persons, children's, mental health, intellectual disability, public health nursing and midwifery.
Design
A comprehensive design included stakeholder consultation and a survey with embedded open‐ended questions.
Methods
A two‐round online Delphi survey was conducted to identify metrics to be measured in practice, followed by a two‐round online Delphi survey to identify the associated indicators for these metrics. A face‐to‐face consensus meeting was held with key stakeholders to review the findings and build consensus on the final metrics and indicators for use. A STROBE checklist was completed.
Results
A suite of eight nursing quality care process metrics and 67 associated process indicators was developed for children's nursing.
Conclusions
By creating a national suite of metrics and indicators, more robust measurement and monitoring of nursing care processes can be achieved. This will enable the provision of evidence for any local and/or national level changes to policy and practice to enhance care delivery.
Relevance to clinical practice
The roll‐out of the metrics and indicators in clinical practice has commenced. This national suite of metrics and indicators will ensure that a robust system of measurement for improvement is in place to provide assurance to Directors of Nursing of the quality of nursing care being provided to children and their families. It supports the value of nursing sensitive data to inform change and improvement in healthcare delivery and to demonstrate the contribution of the nursing workforce to safe patient care.
Keywords: children’s nursing, indicators, metrics, quality, safety
What does this paper contribute to the wider global clinical community?
As far as we are aware, this is the first study internationally to identify process metrics and indicators for children's nursing.
The paper sets out a robust participatory collaborative approach to the development of metrics and indicators for use in clinical practice, increasing the capacity of adoption of the findings.
1. INTRODUCTION
There is a requirement to generate data that provide assurance that national standards of care are being met, and that care delivery is based on best international practice (Cusack, Dempsey Ryan, Kavanagh, & Pitman, 2014; Department of Health, 2016,2017). This required the development and national agreement of a set of metrics and indicators that could be used consistently to measure nursing and midwifery care processes for children's nursing. In 2012, the Nursing and Midwifery Planning Development Units (NMPDU) of three regions in Ireland supported a number of healthcare organisations across nursing and midwifery disciplines to measure and monitor a range of nursing/midwifery care processes. Measures of nursing and midwifery care processes (metrics and their associated indicators) encompass all transactions associated with how care is provided, from the technical delivery to the interpersonal relationships of care. Following increasing demand for a national suite of metrics, the Office of Nursing and Midwifery Services Director (ONMSD) established seven workstreams in the areas of acute, older persons, children's, mental health, intellectual disability, public health nursing and midwifery. The vision of the ONMSD is to lead and support the development of capacity and capability of nurses, midwives, healthcare assistants and maternity care assistants to achieve excellence in care delivery (ONMSD, 2016). The ONMSD has identified Quality Care‐Metrics as a key innovation to support care provision aligned to evidence‐based practice and standards, enabling front‐line teams to measure nursing and midwifery interventions/care processes for both assurance of care processes being delivered and continuous quality improvement. Quality Care‐Metrics are key performance process measures for nursing and midwifery care and are defined as “Quantifiable measures that capture quality in terms of how (or to what extent) nursing and midwifery care is being done in relation to an agreed standard” (Health Service Executive (HSE), 2018).
This paper presents the development of the nursing metrics for acute children's nursing in Ireland. The aim was to develop a suite of Nursing and Midwifery Quality Care‐Metrics, and their indicators, which can be used to measure the quality of children's nursing care processes in acute services. The specific objectives were to identify the metrics and indicators currently in use, in children's nursing, nationally and internationally; and to develop consensus on the metrics and indicators to be measured.
2. BACKGROUND
A systematic literature review was conducted across all of the workstreams to identify quality care process metrics and associated indicators for nursing and midwifery. A comprehensive search methodology was developed for published literature, which was used across routine scientific database searches. This search was supplemented with searches (a) for relevant clinical practice guidelines and (b) of professional body websites. Eight databases were systematically searched including PubMed, Embase, PyscINFO, ASSIA, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Cochrane Database of Systematic Reviews (CDSR), Cochrane Central Register of Controlled Trials (CENTRAL) and Database of Abstract of Reviews of Effects (DARE). Data extraction was conducted by two reviewers using a purposefully designed data extraction tool. Twenty studies were identified as relevant to children's nursing in this review. A further 23 documents were identified from grey literature relevant to children's nursing. From the collective literature, 13 metrics were initially identified (White et al., 2018). The metrics included made a clear reference to nursing care processes and identified a specific quality process in use or proposed use.
Of these 13 metrics, five already existed in practice in Ireland: medication management; nursing care plan; vital signs; invasive medical devices; and discharge planning. The remaining eight metrics included nutrition; infection control; safeguarding, privacy and dignity; pain management; environment; nursing skills mix; patient/family experience; and early identification of adverse events. A workshop was held to facilitate examination of these metrics by the Workstream Working Group for children's nursing. This included a presentation by the research team and discussion of the relevance of each metric to Irish nursing processes, from the perspective of the senior clinical nursing colleagues present. This group comprised of the Chief Director of Nursing, Children's Hospital Group, Directors of Nursing, Assistant Directors of Nursing, Nurse Practice Development staff, Clinical Nurse Managers, Clinical Nurse Specialists and a parent representative. The metric titled nursing skills mix was removed as it was not deemed to be a process metric and the metric on adverse events was merged with vital signs, leaving a total of 11 metrics and associated indicators for consideration in the survey component of the study.
3. METHODS
A consensus study involving a modified Delphi technique was used to identify metrics and indicators for inclusion in the national Children's Nursing Services Quality Care‐Metrics. The Delphi technique, developed by Dalkey and Helmer (1963), is a widely accepted iterative process for achieving a convergence of opinion on a specific topic from experts within the discipline (Hsu & Sandford, 2007). We conducted a national online Delphi survey for children's nursing services, consisting of two rounds of data collection and analysis to reach consensus on metrics, and two rounds of data collection and analysis to reach consensus on indicators. This was followed by a consensus meeting of the Children's Workstream Working Group. A STROBE checklist was performed and is included as a supplemental file (see Supplemental File S1).
3.1. Participant recruitment
The target population was all qualified nurses working in children's nursing services in Ireland.
At the time of the survey, there were an estimated 2, 200 nurses working in children's services nationally. This includes all part‐time, contract and casual staff. Nurses were excluded if they were students, on a temporary contract with the hospital, or if they were casual staff employed by a nursing agency. With the support of the ONMSD, the survey was advertised nationally via Directors of Nursing within the clinical services. Senior clinical managers distributed information to nurses in their respective clinical areas, inviting those who wished to participate to email the research assistant their contact details, including their email address. Additionally, NMPD project officers attended national nursing conferences and local meetings in clinical services advertising the study and seeking recruitment. All potential participants had an opportunity to contact the research team directly to seek further information about the survey prior to making a decision to participate.
3.2. Procedure
There were four surveys in total in this study. The first two surveys (rounds I and II) were used to reach a consensus on the metrics for use. The third and fourth surveys (rounds III and IV) were used to reach a consensus on the indicators for use. For the first survey (round I), an invitation email was circulated to 337 potential participants who had expressed interest in the study. A link to an online survey was then circulated to this group. This survey link was available to staff for completion over a two‐week period. In this survey, participants were presented with a list of 11 metrics. They were asked to rate the importance of each metric on a 9‐point Likert scale as follows: 1–3 = not important, 4–6 = unsure of importance and 7–9 = important. Participants were invited to add any further “new” metrics that they would consider important or relevant for measuring. This was important to capture any additional metrics that were not found in the systematic review. Participants did not score these new metrics. For the second survey (round II), an online survey link was sent to all participants who had participated in round I. This survey was presented in the same manner. It included all of the metrics from round I that had been identified as important by 70% or more of the participants (Meshkat et al., 2014; Zafar et al., 2012). The mean score of each of these metrics was included for participants to see, and the survey also included the new metrics that were identified by participants in round I. Participants were asked to rate the importance of all of the metrics on the survey on the same 9‐point Likert scale. This survey link was also available to staff for completion over a 2‐week period.
The third and fourth surveys (rounds III and IV) then took place, to reach a consensus on the indicators for each of the metrics. For the third survey (round III), the Workstream Working Group revisited the literature again and examined the indicators which had been found. This again involved a presentation to this group of the indicators found and a discussion of the relevance of each indicator to Irish nursing processes, from the perspective of the senior clinical nursing colleagues present. A total of 85 indicators were identified as suitable for inclusion. In order to encourage participation in this round, the online survey link was circulated to all nurses working in children's services, rather than seeking individual expressions of interest. This survey link was available to staff for completion over a three‐week period as it was during the summer holiday period.
In this survey, participants were presented with a list of 85 indicators. The indicators were presented under the heading of their associated metric. Participants were asked to rate the importance of each indicator on a 9‐point Likert scale as follows: 1–3 = not important, 4–6 = unsure of importance and 7–9 = important. Similar to the survey on metrics, participants were also invited to add any further “new” indicators that they would consider important or relevant.
For the fourth survey (round IV), an online survey link was sent to all participants who had participated in round III. This survey was presented in the same manner. It included all of the indicators from round III that had been identified as important by 70% or more of the participants. The mean score of each of these indicators was included for participants to see, and the survey also included the new indicators that were identified by participants in round III. Participants were asked to rate the importance of all of the indicators on the survey on the same 9‐point Likert scale. This survey link was available to staff for completion over a 2‐week period.
Following completion of the fourth survey, a consensus meeting was held with the Workstream Working Group (listed above), to review the findings from the Delphi surveys and build consensus on metrics and respective indicators. Guidelines for the conduct of this meeting were agreed across all of the seven workstreams, and a judgement framework was used (adapted from Flenady et al., 2016) (Table 1) to agree on the final suite of metrics and indicators to be retained for children's nursing.
Table 1.
Nursing and Midwifery Quality Care‐Metrics/Indicators Evaluation Tool (adapted from Flenady et al., 2016)
| Domain | Evaluation criteria |
|---|---|
| 1. Process focused | The metric/indicator contributes clearly to the measurement of nursing or midwifery care processes. |
| 2. Important | The data generated by the metric/indicator will likely make an important contribution to improving nursing or midwifery care processes. |
| 3. Operational | Reference standards are developed for each metric or it is feasible to do so. The indicators for the respective metric can be measured. |
| 4. Feasible | It is feasible to collect and report data for the metric/indicator in the relevant setting. |
3.3. Ethical considerations
The proposal for the study was reviewed and approved for the study by the Research Ethics Committee, University College Dublin. An information pack including a letter of introduction and participant information leaflet was disseminated to potential participants, advising them of the purpose of the study, the purpose of the particular round of the study, how the data would be used and that confidentiality was assured. Potential participants were informed that participation was voluntary, and were invited to contact the researchers if they required any further information. Informed consent was assumed by completion of the survey.
3.4. Data analysis
Analysis of the surveys entailed examination of the mean scores for each metric or indicator ranked on the 9‐point Likert scale. Consensus on inclusion of a metric or indicator was determined where 70% or more participants scored the metric or indicator as 7 to 9 indicating a higher rating of importance for measurement of children's nursing processes by the survey participants.
4. RESULTS
4.1. Participants
The profiles of the participants in rounds I to IV are presented in Table 2. The majority of those who participated in all rounds were at nursing management grade, predominantly those who were Clinical Nurse Managers (Grade II) and Clinical Nurse Specialists. The results indicate participation from a wide variety of areas of practice. This included intensive care, emergency departments, general medical or surgical units. The category “other” identified that nurses were from the following areas: cardiology; haematology; oncology; respiratory; infection prevention and control; endocrine; metabolics; nephrology; and diabetes care.
Table 2.
Profile of the participants from all rounds
| Characteristic | Round 1 | Round 2 | Round 3 | Round 4 |
|---|---|---|---|---|
| n = 184 | n = 133 | n = 141 | n = 92 | |
| Grade n (%) | ||||
| Staff Nurse | 29 (15.8) | 19 (14.3) | 18 (12.8) | 11 (11.9) |
| Staff Midwife | 2 (1.1) | 1 (0.8) | 1 (0.7) | 0 |
| Public Health Nurse | 1 (0.5) | 1 (0.8) | 0 | 0 |
| Clinical Nurse Manager Grade 1 | 15 (8.2) | 8 (6.0) | 7 (5.0) | 1 (1.1) |
| Clinical Nurse Manager Grade 2 | 37 (20.1) | 22 (16.5) | 40 (28.4) | 28 (30.4) |
| Clinical Nurse Manager Grade 3 | 12 (6.5) | 10 (7.5) | 10 (7.1) | 8 (8.7) |
| Clinical Nurse Specialist | 32 (17.4) | 22 (16.5) | 23 (16.3) | 17 (18.5) |
| Director of Nursing | 3 (1.6) | 2 (1.5) | 2 (1.4) | 0 |
| Assistant Director of Nursing | 14 (7.6) | 12 (9.0) | 6 (4.3) | 8 (8.7) |
| Educator | 27 (14.7) | 22 (16.5) | 23 (16.3) | 11 (11.9) |
| Othera | 10 (5.4) | 13 (9.8) | 11 (7.8) | 9 (9.8) |
| Clinical area n (%) | ||||
| Acute Care | 122 (66.3) | 81 (60.9) | 78 (55.3) | 65 (70.7) |
| Out‐patient and Community Care | 30 (16.3) | 25 (18.8) | 32 (22.7) | 13 (14.1) |
| Education | 19 (10.3) | 19 (14.3) | 23 (16.3) | 8 (8.7) |
| Othera | 13 (7.1) | 8 (6.0) | 8 (5.7) | 6 (6.5) |
This included a number of roles where nurses were in management positions that were not directly related to nursing care delivery, which spanned across areas. Examples include child health development officers, general managers and special projects officers.
4.2. Results—rounds I and II metrics
There was a total response of 184 nurses in round I. One of the 11 metrics (environment, which referred to nurses’ role in noise reduction) did not reach consensus of 70% and was removed after round I (Table 3). The metrics with the greatest consensus in round I were as follows: medication management; vital signs and adverse events; and healthcare‐associated infection prevention. Four additional metrics were identified by participants including palliative care and end‐of‐life care; consent and assent; child and adolescent mental health; and experiences of the child/adolescent. The round II survey was then developed, with the aim of reaching consensus on metrics for children's nursing. Participants were presented with 14 items. This included the 10 items that passed the 70% consensus threshold in round I, together with the mean score of each metric, and the additional four metrics identified by participants in round I. One hundred and thirty‐three nurses responded in round II. All 14 metrics had 70% or greater consensus and all of the metrics were therefore retained. The top five metrics remained the same, with little change in their ranking.
Table 3.
Comparison of results of metrics from rounds I and II
| Metric |
Round I % consensus (mean score) |
Round II % consensus (mean score) |
|---|---|---|
| Medication management | 96.5 (8.78) | 98.5 (8.71) |
| Vital signs and adverse events | 95.4 (8.69) | 97.7 (8.74) |
| Healthcare‐associated infection prevention | 90.2 (8.55) | 91.5 (8.35) |
| Pain assessment and management | 89.0 (8.40) | 96.9 (8.34) |
| Nursing care planning | 87.9 (8.43) | 92.3 (8.32) |
| Patient/family experience | 85.6 (9.95) | 86.9 (8.29) |
| Safeguarding privacy and dignity | 82.1 (8.36) | 86.9 (8.25) |
| Discharge planning | 78.6 (8.33) | 82.3 (8.27) |
| Nutrition | 72.8 (8.11) | 70.0 (8.07) |
| Environmenta | 55.5 (7.96) | |
| Additional metrics identified in round 1 | ||
| Palliative care and end‐of‐life care | 88.5 (8.33) | |
| Consent and assent | 80.0 (7.98) | |
| Child and adolescent mental health | 90.8 (8.39) | |
| Experiences of the child/ adolescent | 77.7 (8.09) |
Did not reach consensus of 70% and was removed after round I.
4.3. Results—rounds III and IV indicators
A two‐round online Delphi survey was then completed to develop consensus on indicators for the metrics. In preparation for this, the metrics from round II and the indicators identified from the literature were examined by the Workstream Working Group, to determine their relevance to the evolving process. It was decided by the Workstream group to retain 11 of the 14 metrics identified in round II: the metric relating to invasive medical devices was incorporated as an indicator under the metric healthcare‐associated infection prevention; the metric consent and assent was incorporated as an indicator under the metric safeguarding, privacy and dignity; and the metrics patient/family experience and experience of the child and family were collapsed to form the metric experience of the child/adolescent and family. A total of 85 indicators were presented to participants, across the suite of 11 metrics. Similar to the survey on metrics, participants were invited to add any further “new” indicators that they would consider important or relevant.
There was a total response of 141 nurses in round III. Four indicators did not reach 70% consensus and were not included in round IV. This included one under the metric medication management (responsibility for the keys is allocated to one registered nurse on a shift‐by‐shift basis [68% consensus]), one under discharge planning (a predicted date of discharge or estimated date of discharge is documented [57.9% consensus]) and two under the metric nutrition (frequency of weight and height measurement [69% consensus]; and appropriate use of nutrition assessment tool [61.9% consensus]). A total of three additional indicators were identified for inclusion in round IV. One was included under the metric on nursing care planning (infection status/alert is recorded [98.8% consensus]), and two were included under the metric on nutrition (correct documentation of fluid intake and output [91.7% consensus] and specific information made available for breastfeeding mothers [81% consensus]). This left a total of 11 metrics and 84 indicators which were presented to nurses in round IV. There was a total response of 92 nurses in round IV. All metrics and indicators presented to nurses in round IV met the threshold of 70% consensus and were retained for discussion at a final consensus meeting (Table 4).
Table 4.
Number of indicators presented per metric in rounds III and IV
| Metric | Indicators presented in round III | Indicators that did not reach 70% consensus | Additional indicators identified in round III | Indicators following round IV |
|---|---|---|---|---|
| Medication management | 26 | 1 | — | 25 |
| Vital signs and adverse events | 5 | — | — | 5 |
| Healthcare‐associated infection prevention | 9 | — | — | 9 |
| Pain assessment and management | 4 | — | — | 4 |
| Nursing care planning | 16 | — | 1 | 17 |
| Child/family experience | 4 | — | — | 4 |
| Safeguarding privacy and dignity | 6 | — | — | 6 |
| Discharge planning | 3 | 1 | — | 2 |
| Nutrition | 6 | 2 | 2 | 6 |
| Palliative care and end‐of‐life care | 2 | — | — | 2 |
| Child and adolescent mental health | 4 | — | — | 4 |
4.4. Findings from consensus meeting
A face‐to‐face meeting was held between the research team and the Children's Workstream Working Group following the conclusion of round IV. The purpose of the Consensus Meeting was to review the findings from the Delphi process and to build consensus on the prioritised metrics and respective indicators. Participants at this meeting were representative of key stakeholders in children's acute services with regard to grade and geographical representation. A parent representative was also present to contribute their experience as a service‐user. In addition to the Workstream Working Group members, additional specialist experts from the field of children's nursing were present to add further clarity and validity to their respective suite of quality care process metrics and indicators. This included Clinical Nurse Specialists in pain management, palliative and end‐of‐life care, infection control, early identification of adverse events and child and adolescent mental health.
Group consensus was measured for each metric and indicator through the process of anonymous electronic voting. This method was used to facilitate the presentation of immediate results. Once again, consensus for mandatory inclusion of a quality care process metric or indicator was pre‐set at 70 per cent. The judgement framework presented in Table 1 was evoked to support decision‐making. Following this meeting, a total of nine metrics and 72 indicators remained for children's nursing. There was a consensus that, in the acute setting of children's services, the specialism of palliative care and end‐of‐life care could not be measured on a day‐to‐day basis. The metric safeguarding privacy and dignity was also not retained. There was consensus that three indicators under this metric were best placed under the metric child and adolescent mental health, and that the remaining two indicators would be brought back to the wider national cross‐discipline steering group for further discussion.
The metrics and their respective indicators had one final review by this group, to align wherever possible the language used across all seven national workstreams. This was to ensure best fit with the “Test Your Care System,” the online web‐based system that is used within the clinical services to collect nursing process data, and to help ensure clarity for the data collectors. Following this, eight metrics and 67 indicators were presented as the first suite of metrics and indicators for children's nursing in Ireland (Table 5). The metric relating to experiences of the child and family was not retained as this information is gathered as part of wider organisational initiatives.
Table 5.
Final suite of metrics and indicators
| Metric | Indicators |
|---|---|
| Medication management |
Security for the storage of medicinal products is managed by the registered nurse All medicinal products are stored in a locked cupboard/locked fridge or within a locked room Where medication trolleys are in use, they are locked and secured as per local organisational policy and open shelves on the medication trolley are free of medicinal products when not in use High alert medicine is identified and stored appropriately, as per local policy There is easy access to an up‐to‐date drug formulary Misuse of Drugs Act (MDA) drugs are checked and signed at each changeover of shifts by registered nursing staff (member of day staff & night staff) Two signatures are entered for each administration of an MDA drug The MDA drug cupboard is locked and security around access to the MDA cupboard is held by a registered nurse Security for the storage of MDA drugs is kept separate to security for other medication The child's prescription documentation includes their legible name and healthcare record number The child's identification band has correct and legible name and healthcare record number/unique identifier The child's allergy status is clearly identifiable on the front page of the prescription chart The child's weight and date of weight are recorded on the front page of the prescription chart The child's locker and bedside/surrounding environment are free of unsecured prescribed medicinal products The generic name is used as appropriate for each medicine prescribed The start date of each prescribed medication is recorded The prescription is written in un‐joined letters The decimal point is clearly marked The correct legible dose of the medication is recorded with correct use of abbreviations The route of medication administration is recorded Prescribed medication not administered have an omission code entered and appropriate action taken The time of medication administrations is as prescribed The minimum dose interval and/or 24‐hr maximum dose is specified for all pro re nata (PRN) medication The prescription has an identifiable prescriber's signature Discontinued medications are crossed off, dated and signed by a person who has prescriptive authority. |
| Nursing care planning |
The child's name, date of birth and healthcare record number/unique identifier are on each page/ screen The child's admission date and time are recorded The child's presenting complaints/reason for admission/ attendance is recorded The child's next of kin/family support details are recorded The child's past medical/surgical history is recorded The child's allergy status is clearly identifiable on relevant nursing documentation All sections of the nursing admission assessment documentation are completed within 24 hr of admission Nursing care plans are evident and reflect the child's current condition Nursing interventions are individualised, dated, timed (using 24 hr clock) and signed Evaluation of the nursing care plan is evident and has been updated accordingly All nursing records are legible and identifiable All nursing entries are in chronological order All abbreviations/grading systems used in the nursing record are from a national or approved list/system All alterations/corrections to the nursing record are as per NMBI guidance Student entries are countersigned by a registered nurse There is evidence of promotion of child and family enablement documented in a communication care plan |
| Discharge planning |
There is documented evidence of discharge planning There is evidence of involvement of the child and family in the discharge plan There is evidence of the provision of postdischarge advice to the child/family |
| Nutrition |
There is evidence of ongoing monitoring of the child's weight There is evidence that child's fluid balance has been assessed and managed Information and support is made available for breastfeeding mothers |
| Healthcare‐associated infection prevention |
The child's infection status/alert is recorded Associated Infection Prevention and Control guidelines are available and accessible There is evidence of appropriate nursing action in the event of a healthcare‐associated infection The child's infection status and any associated risk is communicated to the family and multidisciplinary team There is evidence that a care bundle has been completed for each invasive medical device in use |
| Pain assessment and management |
The child's pain is assessed and recorded using a developmentally appropriate pain scoring tool There is evidence that a pain care plan was initiated There is evidence that the child's pain management is recorded in nursing documentation Re‐evaluation of pain scores are recorded before and after a pain‐relieving intervention |
| Vital Signs Monitoring/PEWS |
The child's baseline physiological observations were assessed, calculated and recorded using the age‐appropriate national PEWS system The child's physiological observations have been reassessed, calculated and recorded using the age‐appropriate PEWS system Any deterioration in the child's condition is documented and there is evidence of adherence to the minimum observation frequency as per age‐appropriate national PEWS guidelines In the event of a deterioration, there is documented evidence of escalation of the child's care and communication to the medical team using the ISBAR as per the age‐appropriate national PEWS escalation protocol There is documentation of the nursing care that has been provided to manage a deterioration in the child's condition (management plan) In the event of infection/sepsis, there is documented evidence of escalation as per national PEWS sepsis/infection protocol |
| Child and adolescent mental health |
A child and adolescent mental health service (CAMHS) plan has been initiated where appropriate There is evidence of appropriate CAMHS referral The child/adolescent and family have been given contact details for advice/follow up with the relevant CAMHS team Evidence for alternatives to clinical holding was explored The reason for the application of clinical holding is documented |
5. DISCUSSION
This study identified a final suite of eight metrics and 67 indicators for children's nursing in Ireland. The metrics and indicators that emerged reflect the focus of safe and effective care delivery by nurses in the acute arena of children's nursing. The metrics that made the final suite reflect global care concerns in caring for children that are sensitive to the influence of nursing practice and that also enable nursing to demonstrate their contribution to safe care delivery. However, during the consensus process a number of interesting decisions were made: consensus not to include the metric environment; and the initial suggestion of an additional metric on palliative and end‐of‐life care and the subsequent decision not to include it in the final suite. Key to understanding these decisions is, in the first instance, that the focus of the study was looking at process metrics. Therefore, the repeated message to participants and workstream members was that the measurable metrics sought from the study need to demonstrate how, or to what extent, nursing care is being provided in a predominantly acute care environment.
The metric relating to the environment referred to creating a healthy atmosphere in terms of noise levels. The removal of this metric at the end of round I, with a consensus of 55.5%, suggests that participants may not identify it as a nursing issue or may not identify it as something that is within their locus of control as the aim of the study was specifically to develop nursing metrics. It is also possible that there remains a gap in knowledge in relation to the impact of the environment on the well‐being of child and family when in the general hospital setting. This may be explained by the fact that the data that do exist on this issue predominantly focus on specialist areas such as oncology and paediatric intensive care units (Kudchadkar, Yaster, & Punjabi, 2014; Lee, Narendran, Tomfohr‐Madsen, & Schulte, 2017). However, there are an increasing number of studies that also explore the impact of noise in general children's services. For example, Stickland, Clayton, Sankey, and Hill (2016) in the United Kingdom found that parents and their children experienced reduced sleep quality during hospitalisation. This was attributed to noise and light and ward schedules; parents stated that it challenged their emotional regulation and parent–child relationships. This was supported in 2018 in a study by Bevan et al. (2019) who examined total sleep time, sleep efficiency, median sound levels overnight in a children's hospital and compared them with those of a child's home environment. They found that children had on average 62.9 min, and parents 72.8 min, per night less sleep in hospital than at home; children and parents reported poorer sleep quality in hospital than at home; and the median sound levels measured for eight of 40 children both at home (34.7 dBA) and in hospital (48.6 dBA) exceeded World Health Organization recommendations of 30 dB. The impact of such noise on the child has been found to impact on behavioural and emotional regulation (Beebe, 2011; Gruber, Cassoff, Frenette, Wiebbe, & Carrier, 2012), physiological consequences in terms of pain sensitivity (Finan, Goodin, & Smith, 2013) and can impact on maternal anxiety and depression (Moore, David, Murray, Child, & Arkwright, 2006). The decision therefore not to include this metric indicates the need for further consideration of the knowledge of nurses of these issues and the need to identify specific areas that nurses can affect change to support addressing this issue for children and their families.
It is also of note that there was an initial addition of a metric palliative and end‐of‐life care, and a subsequent decision not to include it in the final suite of metrics. There has been an increase in the number of children living with life‐threatening conditions, in part owing to technological advances and medical progress, meaning that access to palliative care services is required across extended years (Remedios et al., 2015). In Ireland, recent evidence suggests that there are at least 3,840 children living with a life‐limiting condition (Ling, O'Reilly, Balfe, & Devins, 2015). This is acknowledged nationally in The Report of the National Advisory Committee on Palliative Care (Department of Health & Children, 2001), followed by Palliative Care for Children with Life‐limiting Conditions in Ireland—A National Policy (Department of Health & Children, 2009). However, while the need for expert care for these children in the acute setting was acknowledged by the addition of this metric in round I of the study, it was also evident that staff in acute children's services are increasingly less likely to deliver palliative and end‐of‐life care due to the increasingly robust home nursing and respite care services for these children. This is reflective of patterns in the location of care delivery to these children internationally. While such services in Ireland are developmental in many respects, they are highly valued by children and their families (Brenner et al., 2016) and align with the National Model of Care for Paediatric Healthcare Services in Ireland (Health Service Executive, 2016) which advocates care closer to home. This is consistent with findings internationally where parents favour care in the home for these children, as opposed to in the acute care setting, and they value the compassion, clinical expertise and commitment of a health service that can provide such care (Monterosso, Kristjanson, & Phillips, 2008; Weidner et al., 2011; Widger & Picol, 2008). There was much discussion on the value of ongoing and enhanced education for nurses who care for these children; however, the Workstream Working Group agreed that it would be unlikely that this metric could be measured on a regular basis due to the small number of children requiring this care in acute services and it was agreed to retain it for further consideration when mapping out metrics for the delivery of care to children in the community. The onward development of metrics in the community will need to reflect these changes and will also need to reflect the fact that Ireland is currently initiating substantial changes in the care of children with complex care needs. This is informed by developments in this area at European level identifying the need for enhanced integrated care services for children (Brenner, O'Shea, Larkin et al., 2018; Brenner, O'Shea, McHugh et al., 2018) and supported at national level through the strategic HSE policy imperative SlainteCare which recommends the re‐orientation of the healthcare system towards integrated care consistent with the highest quality of patient safety (Houses of the Oireachtas, 2017).
6. LIMITATIONS
The lower response rate in some rounds may be explained by a busy clinical workload and the fact that the earlier rounds took place during the summer period. It is also possible that there may have been some communication challenges as information on the survey was cascaded through nursing management structures, as staff nurses do not have individual email accounts in the clinical area. An alternative explanation may be that initial enthusiasm diminished over time, as a Delphi study can be slow and time‐consuming. It is also acknowledged that this is a study that had a focus on acute care and the findings do not reflect community child health nursing. This suggests the value of widening this study in future to examine specific issues for this arena.
7. CONCLUSION
The need to deliver greater value and increased efficiency while guaranteeing ever‐higher quality care is placing a requirement on healthcare organisations to provide evidence of the quality and safety of their care. However, quality and patient safety cannot be measured, and improvements cannot be made without reviewing the appropriate data. By creating a national suite of metrics and indicators, more robust monitoring can be achieved which will enable the provision of evidence for any national level changes to policy and practice that may be required to improve care delivery. The collaborative, participatory approach used ensures the relevancy of the developed metrics and indicators, engenders participant ownership, increasing the capacity for adoption of the chosen suite in children's services and heightens the sustainability of metric and indicator use in practice as the nurses and midwives involved in the research process have become advocates for the developed suite.
8. RELEVANCE TO CLINICAL PRACTICE
The new suite of Nursing and Midwifery Quality Care‐Metrics and respective indicators developed for Children's Nursing in Ireland was launched by the HSE ONMSD in June 2018. The relevance of the final set of eight metrics to children's nursing practice has been assured through a rigorous and collaborative process that identified international best practice and incorporated a consensus methodology approach. Implementation of the metrics will be governed by Directors of Nursing that will incorporate both bottom up‐up and top‐down implementation strategies. Key to successful implementation will be the principles of measurement for improvement, and front‐line engagement and ownership to enable the achievement of patient safety and care quality improvement goals.
CONFLICT OF INTEREST
No conflict of interest has been declared by the authors.
Supporting information
ACKNOWLEDGEMENTS
We would like to thank the members of the Workstream Working Group for their time and expertise in the development of the suite of metrics and indicators, and we extend our thanks to the nurses who took the time to respond to the four rounds of surveys in this study.
Brenner M, Browne C, Gallen A, Byrne S, White C, Nolan M. Development of a suite of metrics and indicators for children’s nursing using consensus methodology. J Clin Nurs. 2019;28:2589–2598. 10.1111/jocn.14845
Funding information
The study was funded by the Nursing and Midwifery Practice Development Unit, Health Service Executive, Dublin, Ireland.
REFERENCES
- Beebe, D. W. (2011). Cognitive, behavioral, and functional consequences of inadequate sleep in children and adolescents. Pediatric Clinics of North America, 58, 649–665. 10.1016/j.pcl.2011.03.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bevan, R. , Grantham‐Hill, S. , Bowen, R. , Clayton, E. , Grice, H. , Venditti, H. C. , Hill, C. M. (2019). Sleep quality and noise: comparisons between hospital and home settings. Archives of Disease in Childhood, 104(2), 147–151. 10.1136/archdischild-2018-315168 [DOI] [PubMed] [Google Scholar]
- Brenner, M. , Connolly, M. , Cawley, D. , Howlin, F. , Berry, J. , & Quinn, C. (2016). Family and healthcare professionals’ perceptions of a pilot hospice at home programme for children: A qualitative study. BMC Palliative Care, 15, 89 10.1186/s12904-016-0161-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brenner, M. , O'Shea, M. P. , Larkin, P. , Luzi, D. , Pecoraro, F. , Tamburis, O. , Blair, M. (2018). Management and integration of care for a child living with complex care needs at the acute community interface in Europe. The Lancet Child & Adolescent Health, 2(11), 822–831. 10.1016/s2352-4642(18)30272-4 [DOI] [PubMed] [Google Scholar]
- Brenner, M. , O'Shea, M. P. , McHugh, R. , Clancy, A. , Larkin, P. , Luzi, D. , Blair, M. (2018). Principles for provision of integrated complex care for children across the acute community interface in Europe. The Lancet Child & Adolescent Health, 2(11), 832–838. 10.1016/s2352-4642(18)30270-0 [DOI] [PubMed] [Google Scholar]
- Cusack, E. , Dempsey Ryan, D. , Kavanagh, C. , & Pitman, S. (2014). An evaluation of the development and implementation of a nursing and midwifery metrics system in HSE Dublin north healthcare services. Dublin: Health Service Executive. [Google Scholar]
- Dalkey, N. , & Helmer, O. (1963). An experimental application of the Delphi method to the use o experts. Management Sciences, 9(3), 458–467. [Google Scholar]
- Department of Health . (2016). Position paper one. Value for nurses and midwives in Ireland. Dublin: Office of the Chief Nursing Officer. [Google Scholar]
- Department of Health . (2017). Framework for national performance indicators for nursing and midwifery. Dublin: Department of Health. [Google Scholar]
- Department of Health and Children . (2001). Report of the national advisory committee on palliative care. Dublin: Stationery Office. [Google Scholar]
- Department of Health and Children . (2009). Palliative care for children with life‐limiting conditions in Ireland: A national policy. Dublin: Stationery Office. [Google Scholar]
- Finan, P. H. , Goodin, B. R. , & Smith, M. T. (2013). The association of sleep and pain: an update and a path forward. The Journal of Pain, 14, 1539–1552. 10.1016/j.jpain.2013.08.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Flenady, V. , Wojcieszek, A. M. , Fjeldheim, I. , Friberg, I. K. , Nankabirwa, V. , Jani, J. V. , … Frøen, J. F. (2016). eRegistries: Indicators for the WHO Essential Interventions for reproductive, maternal, newborn and child health. BMC Pregnancy Childbirth, 16(1), 293 10.1186/s12884-016-1049-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gruber, R. , Cassoff, J. , Frenette, S. , Wiebbe, S. , & Carrier, J. (2012). Impact of sleep extension and restriction on children’s emotional lability and impulsivity. Pediatrics, 130, e1155–e1161. 10.1542/peds.2012-0564 [DOI] [PubMed] [Google Scholar]
- Health Service Executive . (2016). The national model of care for paediatric healthcare services in Ireland. Dublin: Health Service Executive. [Google Scholar]
- Health Service Executive . (2018). Nursing and midwifery quality care‐metrics: Children’s services research report. Dublin: Health Service Executive. [Google Scholar]
- Houses of the Oireachtas . (2017). Committee on the future of healthcare. Slainte report. Dublin: House of the Oireachtas. [Google Scholar]
- Hse, C. , & Sandford, B. (2007). The Delphi technique: Making sense of consensus. Practical Assessment, Research and Evaluation, 12(10). [Google Scholar]
- Kudchadkar, S. R. , Yaster, M. , & Punjabi, N. M. (2014). Sedation, sleep promotion, and delirium screening practices in the care of mechanically ventilated children: A wake‐up call for the pediatric critical care community. Critical Care Medicine, 42(7), 1592–1600. 10.1097/CCM.0000000000000326 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lee, S. , Narendran, G. , Tomfohr‐Madsen, L. , & Schulte, F. (2017). A systematic review of sleep in hospitalized pediatric cancer patients. Psycho‐Oncology, 26, 1059–1069. 10.1002/pon.4149 [DOI] [PubMed] [Google Scholar]
- Ling, J. , O'Reilly, M. , Balfe, J. , & Devins, M. (2015). Children with life‐limiting conditions: Establishing accurate prevalence figures. Irish Medical Journal, 108, 93. [PubMed] [Google Scholar]
- Meshkat, B. , Cowman, S. , Gethin, G. , Ryan, K. , Wiley, M. , Brick, A. , Mulligan, E. (2014). Using an e‐Delphi technique in achieving consensus across disciplines for developing best practice in day surgery in Ireland. Journal of Hospital Administration, 3(4), 2589–8. 10.5430/jha.v3n4p1 [DOI] [Google Scholar]
- Monterosso, L. , Kristjanson, L. J. , & Phillips, M. B. (2008). Supportive and palliative care needs of families of children who die from cancer: An Australian study. Palliative Medicine, 22, 59–69. 10.1177/0269216309104060 [DOI] [PubMed] [Google Scholar]
- Moore, K. , David, T. J. , Murray, C. S. , Child, F. , & Arkwright, P. D. (2006). Effect of childhood eczema and asthma on parental sleep and well‐being: A prospective comparative study. British Journal of Dermatology, 154, 514–518. 10.1111/j.1365-2133.2005.07082.x [DOI] [PubMed] [Google Scholar]
- Office of the Nursing and Midwifery Services Director . (2016). Office of the nursing and midwifery services director plan 2016–2018. Dublin: Health Service Executive. [Google Scholar]
- Remedios, C. , Willenberg, L. , Zordan, R. , Murphy, A. , Hessel, G. , & Philip, J. (2015). A pre‐test and post‐test study of the physical and psychological effects of out‐of‐home respite care on caregivers of children with life‐threatening conditions. Palliative Medicine, 29(3), 223–230. 10.1177/0269216314560008 [DOI] [PubMed] [Google Scholar]
- Stickland, A. , Clayton, E. , Sankey, R. , & Hill, C. M. (2016). A qualitative study of sleep quality in children and their resident parents when in hospital. Archives of Disease in Childhood, 101(6), 546–551. 10.1136/archdischild-2015-309458 [DOI] [PubMed] [Google Scholar]
- Weidner, N. J. , Cameron, M. , Lee, R. C. , McBride, J. , Mathias, E. J. , & Byczkowski, T. L. (2011). End‐of‐life care for the dying child: What matters most to parents? Journal of Palliative Care, 27, 279–286. [PubMed] [Google Scholar]
- White, C. , Byrne, S. , Nolan, M. , Brenner, M. , Browne, C. , & Gallen, A. (2018). Developing a national suite of children's quality‐care metrics using consensus methodology. 8th Annual Nursing Research Conference, Temple Street Children’s University Hospital, 20th March.
- Widger, K. , & Picot, C. (2008). Parents’ perceptions of the quality of pediatric and perinatal end‐of‐life care. Pediatric Nursing, 34, 53–58. [PubMed] [Google Scholar]
- Zafar, S. Y. , Currow, D. C. , Cherny, N. , Strasser, F. , Fowler, R. , & Abernethy, A. P. (2012). Consensus‐based standards for best supportive care in clinical trials in advanced cancer. The Lancet Oncology, 13, e77–82. 10.1016/S1470-2045(11)70215-7 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials