Cardiopulmonary resuscitation by Emergency Medical Services in South Africa: Barriers to achieving high quality performance

Abstract

Introduction

Survival rates from out-of-hospital cardiac arrest significantly improve when high-quality cardiopulmonary resuscitation (CPR) is performed. Despite sudden cardiac arrest being a leading cause of death in many parts of the world, no studies have determined the quality of CPR delivery by Emergency Medical Services (EMS) personnel in South Africa. The aim of this study was to determine the quality of CPR provision by EMS staff in a simulated setting.

Methods

A descriptive study design was used to determine competency of CPR among intermediate-qualified EMS personnel. Theoretical knowledge was determined using a multiple-choice questionnaire, and psychomotor skills were video-recorded then assessed by independent reviewers. Correlational and regression analysis were used to determine the effect of demographic information on knowledge and skills.

Results

Overall competency of CPR among participants (n = 114) was poor: median knowledge was 50%; median skill 33%. Only 25% of the items tested showed that participants applied relevant knowledge to the equivalent skill, and the nature and strength of knowledge influencing skills was small. Demographic factors that significantly influenced both knowledge and skill were the sector of employment, the guidelines EMS personnel were trained to, age, experience, and the location of training.

Conclusion

Overall knowledge and skill performance was below standard. This study suggests that theoretical knowledge has a small but notable role to play on some components of skill performance. Demographic variables that affected both knowledge and skill may be used to improve training and the overall quality of Basic Life Support CPR delivery by EMS personnel.

African relevance

• •CPR makes up a vital component of EMS work. That quality has been said to be a reflection of EMS.
• •The burden of cardiovascular disease is increasing in the African setting, along with cardiac arrest.
• •Challenges faced in the low- and middle-income setting to achieving high quality CPR have not been investigated.

Introduction

Sudden cardiac arrest can be defined as “the sudden cessation of cardiac activity so that the victim becomes unresponsive, with no normal breathing and no signs of circulation” [1], and it is considered to be a major cause of death in high-income countries [2], [3]. Cardiovascular disease is responsible for up to 80% of these deaths, and has been established as the leading cause of non-communicable disease (NCD) related deaths worldwide [4]. Historically, low- to middle-income countries (LMIC) have focussed their public health efforts towards their biggest killers, namely communicable diseases [5], [6]. However, in recent years there has been a notable shift in disease burden away from communicable disease in LMICs such as South Africa, with a shift towards NCDs with comparably high death rates to higher income sites [7]. Health systems should develop such that communities have access to appropriate care for the eventual cardiac arrest event [8].

Cardiopulmonary resuscitation (CPR) is the first line treatment and foundation to saving lives following cardiac arrest [2]. Survival from Emergency Medical Services (EMS)-treated out-of-hospital cardiac arrest (OHCA) is significantly improved when the links in the Chain of Survival are implemented in an effective way, with one of the most important aspects being the quality of CPR delivery [9], [10]. Despite this, overall survival rates remain poor. In-hospital survival is somewhat higher (22.3–25.5%) [11] than the out-of-hospital setting (10.8%) [12]. One of the main reasons for the disparity in survival rates has been attributed to the quality of CPR provided [13], with rescuers often performing CPR at sub-optimal standards [14], [15]. Poor quality CPR has been shown to have the same outcome to patients who receive no CPR at all [14], [16], with survival varying between 0% and 3% [17], [18]. Rescuers in high-performance EMS systems understand that their resuscitation efforts depend on the quality of CPR delivered [8].

One of the most important components to achieving high-quality CPR is the way that chest compressions are delivered. Aside from time to initiating compressions directly impacting outcomes, other characteristics of high-quality compressions include: depth, rate, and recoil. Inadequate rates have been associated with poorer return of spontaneous circulation outcomes [19], and survival to hospital discharge rates fall by 30% when compression depth is too shallow [20]. This is concerning since many rescuers perform compressions at rates that are too slow [15] or at a depth that is too shallow [20]. Leaning during compressions is also common among rescuers and limits chest wall recoil, which limits coronary perfusion pressure which influences resuscitation outcomes [21].

Another major determinant of survival from ventricular fibrillation cardiac arrest is time to defibrillation [22]. From collapse until defibrillation occurs, survival rates decline by 7–10% for every minute that no CPR is performed, and 3–4% for every minute whilst CPR is provided by bystanders [23]. Thus, early defibrillation is a powerful predictor of survival from cardiac arrest for those victims who are in ventricular fibrillation/pulseless ventricular tachycardia, and is closely linked to the quality of CPR delivered by rescuers.

Despite excellent CPR training courses being available, knowledge and skills have been shown to decay rapidly: in as little as one month after training [24], [25], [26]. Ideal re-training intervals remain unknown. Skills also tend to decay more rapidly than knowledge [24], and require innovative methods to alter the decay curve of CPR skills. Frequent short-duration interaction with CPR skills have been suggested as a solution, but more evidence is needed to support this [27], [28].

One study determined that knowledge has a small but positive impact on skills [29]. This same study also showed that knowledge of the guidelines specifically influenced two respective skills: rate and compression to ventilation ratio. It was also reported that overall knowledge and skill performance remained poor. Another variable that has been shown to have a significant effect on CPR performance is the experience of the provider. Whilst several studies in the past correlated more experience with better performance [30], [31], other more recent studies correlated greater experience with worse performance [29].

EMS systems in South Africa are based on a three-tiered response system, which is generally comprised of basic (BLS), intermediate (ILS) and advanced life support (ALS) qualified personnel. Respectively, these tiers are represented by the following qualifications: basic ambulance assistant (BAA); ambulance emergency assistant (AEA); ALS remains the same. The use of automatic external defibrillators (AED) by BAA personnel has been delayed compared to most systems worldwide, and was only introduced into their scope of practice in 1999 [32]. The actual use of AED’s by BAA’s has been further delayed due to a lack of consistency in training throughout the country, as well as budgetary constraints which has hindered the placement of these devices in most services [33]. To date, wide variation exists among BAA’s trained to use an AED correctly, with a large proportion still not having access to the use of these devices. Defibrillation has been part of the scope of ILS/AEA qualified personnel for much longer, and is considered standard practice within this tier-level.

The aim of this study was to describe the quality of BLS CPR delivered by ILS qualified EMS personnel in a simulated setting, and to determine potential demographic barriers that contribute towards better or worse performance.

Methods

We undertook a prospective, descriptive analytical study to compare demographic information, theoretical knowledge and psychomotor skills of EMS personnel in one province in South Africa. Non-experimental, observational methods were used. The research was approved by the Human Research Ethics Committee of the University of Cape Town (ref no. 575/2013), as well as by the respective provincial Department of Health under study.

A province outside of the Western Cape was chosen. To reduce the probability of bias, we randomly selected (using Microsoft Excel) 20 participants from each of the seven districts within the province, targeting 140 (18%) of the 578 ILS qualified population from the public sector for enrolment. Selected participants were invited to attend a continuous professional development refresher course on CPR. This invitation was extended to private sector ILS qualified EMS personnel in the respective districts. Upon arrival, participants were offered the opportunity to consent to and participate in the study prior to the commencement of the CPR refresher course.

Demographic data from each participant was collected using a 16-point questionnaire (Appendix A). Theoretical knowledge was tested using a 20-point multiple choice questionnaire consisting of questions used for the BLS for health care providers course (Appendix B). Psychomotor skill performance was measured via offsite video review. Each participant was given the same scenario of an unconscious patient presenting a shockable rhythm, and their performance was rated by BLS certified instructors using the one-rescuer adult BLS CPR skill sheet (Appendix C) [34], which uses a binary (mark if done correctly) marking system. Ventricular Fibrillation was the shockable rhythm displayed for each scenario and was generated through the manikin’s attached electrocardiogram trainer device. This rhythm was displayed on the monitor if the monitor was switched on by the participant.

Each performance was video recorded using four standardised camera angles. This standardised camera placement would enable independent reviewers to optimally view each performance from both a detailed and overall perspective. Camera angles were then edited and synchronised onto one screen with a central timer display for the purpose of review by the independent reviewers.

Performances were then randomised and sent off to three independent reviewers who were certified BLS instructors. The skills sheet used captures data on general CPR characteristics, as well as various characteristics of compressions. The original skills sheet consisted of twelve items. We included an additional eleven steps, hereafter referred to as the ‘modified’ skills sheet, thus totalling 23 items. This was to measure additional skills directly relating to BLS CPR. These additional steps were validated by one of the independent reviewers. The Ambu® Man Defib Trainer manikin that was used displays parameters such as depth of compressions, correct hand position, and correct delivery of ventilations.

Data was transcribed from assessment sheets and captured by the researcher in Microsoft Excel. Statistical analysis of the transcribed data was performed by a biostatistician using SAS Version 9.2. All data was kept in a secure file to ensure confidentiality and anonymity of the participants. To further ensure anonymity, participants’ names were replaced by a unique identification code known only to the researcher. Descriptive statistics were used to describe demographic variables and overall knowledge and skill performance. For each section (demographics, knowledge, and skill), numerical data was summarised by medians and percentiles (IQR) or means and standard deviations (SD). Categorical data was summarised using frequencies and percentages. Results scored by the independent reviewers were tested for both inter and intra-rater reliability using Spearman’s rank correlation and kappa statistics.

Pearson’s correlation and linear regression analysis were used to determine the relationship and effect of numerical demographic information on knowledge and skill. Multiple linear regression analysis was also performed to account for confounding between independent variables. For categorical data, the Kruskal-Wallis test was used to compare values for the different categories of independent variables. Lastly, we aimed to determine the effect that knowledge has on skill using linear regression analysis. A significance level (α) of <0.05 was used throughout the study.

Results

Data collection occurred over a three-week period during November 2013. We were able to reach a total number of 114 public and private participants combined, from six of the seven districts within the province. The seventh district was unreachable due to logistical challenges. Of the intended 140 public participants that we aimed for, we were only able to reach 104 (response rate of 74.3%). This sample made up almost 18% of the total ILS population from the public sector.

Demographic data of participants are displayed in Table 1. Almost half of the participants reported having performed CPR less than twice in the previous year, whilst 100% of participants stated that CPR was the most important skill in their scope of practice.

Table 1.

Demographic factors, training, and experience of participants (n = 114).

Mean age ± SD, years	42 ± 8
Male	90 (79)
Median number of years qualified as a BAA (IQR)*	13 (9–19)
Median number of years qualified as an AEA (IQR)	8 (4–14)
Last CPR training occurred during AEA course	63 (55)
Trained per the current guidelines (2010 AHA Guidelines)	11 (10)

BLS, basic life support; CPR, cardiopulmonary resuscitation; SD, standard deviation; BAA, basic ambulance assistant; AEA, ambulance emergency assistant; IQR, inter-quartile range. ^*This shows that participants held a BAA qualification approximately 5 years prior to them obtaining their AEA qualification. All participants were AEA qualified at the time of the study. Data are presented as n (%) unless otherwise specified.

The overall median result achieved for both knowledge and skill was 50.0% (IQR 35.0–65.0%) and 33.3% (IQR 16.7–58.3%), respectively. Fig. 1 represents both overall knowledge and skill results for the six characteristics that constitute high quality chest compressions. Participants scored above 50% in only two of the six knowledge characteristics, and in only one of the six skill characteristics. Five (4.4%) participants did not perform compressions at all throughout their skills assessment. Of those that did, the median time to initiating compressions was 61 s (IQR 30–93 s).

Fig. 1.

Comparison of knowledge versus skill results for chest compression characteristics (proportions are for correct display of knowledge or skill).

The three independent reviewers were tested for intra-and inter-rater reliability for their respective skills results given. Intra-rater reliability was tested by ensuring that reviewers reassessed their same performances three weeks later. The level of agreement was found to be ‘moderate’ (kappa statistic = 0.59) for the original skills result, which improved to ‘good’ when considering the modified skills sheet which consisted of additional variables (0.74). Inter-rater reliability was tested by comparing the mean percentage scores given by the reviewers for the skills assessment. We noted that one of the reviewers scored significantly different (p < 0.001) and lower from the other two, who were similar (p = 0.98).

The manikin was in a shockable rhythm throughout the scenario unless a shock was delivered at any point. The monitor was already attached for the participant, and Fig. 2 depicts the action that was taken with the monitor. We found that 30.7% (n = 35) of participants did not deliver a shock or did not switch on the monitor throughout the six-minute scenario. Of those that did deliver a shock (n = 79, 69.3%), the median time from arrival on scene to shock delivery was 4 min 12 s (IQR 3 m 2 s–5 m 14 s). Of these who did deliver a shock, only 20.2% (n = 23) did so in what was deemed to be a safe and/or effective manner.

Fig. 2.

Action taken during skill performance with defibrillator for the ventricular fibrillation cardiac arrest scenario (n = 114).

Table 2 represents the analysis of demographic factors, both numerical (age, experience, and years since last trained in CPR) and categorical (sector, gender, place of training, and guidelines trained according to, etc.), which may have influenced knowledge and/or skill performance.

Table 2.

Analytical statistics of knowledge and skill results for the different numerical demographic variables.

Demographic variables	Assessment	Simple linear regression analysis	Multiple linear regression analysis
		Crude coefficient (95% CI)	Adjusted coefficient (95% CI)
Age	Theory	−1.00 (−1.36 to −0.64)^	−1.08 (−1.66 to −0.51)^
	Original Skills	−0.61 (−1.10 to −0.13)*	−0.21 (−0.94 to 0.52)
Years since BAA qualification	Theory	−0.88 (−1.35 to −0.41)^	−0.01 (−0.98 to 0.96)
	Original Skills	−0.67 (−1.30 to −0.04)	0.63 (−0.60 to 1.87)
Years since AEA qualification	Theory	−0.85 (−1.30 to −0.40)^	0.39 (−0.60 to 1.37)
	Original Skills	−0.84 (−1.42 to −0.26)	−1.06 (−2.32 to 0.20)
Time elapsed since last trained in CPR	Theory	−0.96 (−1.45 to −0.47)^	−0.60 (−1.20 to 0.001)*
	Original Skills	−1.11 (−1.68 to −0.53)^	−0.73 (−1.50 to 0.04)

AEA, Ambulance Emergency Assistant; BAA, Basic Ambulance Assistant; CI, confidence interval; ^*p value < 0.05; ^{^}p < 0.001.

When considering the estimates, the factors found to be univariately associated with both decreased knowledge and skill were: greater age, greater experience and the more time elapsed since last trained in CPR. However, when these factors were adjusted for, only decreased knowledge was significantly influenced by greater age. Borderline non-significant trends were also shown for decreased knowledge performance and skill with greater time elapsed since last trained in CPR after adjustment for the other variables in the model.

Categorical factors that influenced knowledge were: the sector of employment (p < 0.001), the place of training (p = 0.001), and the type of guidelines participants were trained according to (p < 0.001). The categorical factors that influenced skill were: the most recent type of CPR training (p = 0.02) as well as the type of guidelines participants were trained according to (p = 0.01).

The remaining demographic factors that appeared to have no influence on knowledge and/or skill performance were: gender, the number of times CPR was performed in the previous year and urban vs. rural setting.

When considering the influence that knowledge had on skill performance, a significant weak positive correlation of r = 0.40 was observed between the two variables (p < 0.001). Using the original skills performance percentage as the dependent variable and the theory percentage as the independent variable, the regression model was significant (p < 0.001) and the coefficient of determination (r²) was 0.157. Thus 15.7% of the change in the original skills percentage was due to the change in the theory percentage and 84.3% was due to other variables. This result shows that knowledge of the participants does play some – albeit a small – role in their skill performance. Other possible factors, such as the demographic factors, may play a larger role in skill performance.

Discussion

This is the first study to describe the quality of CPR delivery by EMS personnel in Africa. There is a paucity of research studying the relationship between theoretical knowledge and psychomotor skills of CPR. To our knowledge, the only studies similar to this are of emergency medical technicians in the pre-hospital setting [29], and hospital staff and medical students in the hospital setting [35]. These results were similar in that knowledge of the guidelines had a significant effect on some components of CPR performance, but overall knowledge and skill remain poor. Brown showed knowledge components that affected skill were rate and compression to ventilation ratio, whilst Burkhardt similarly showed knowledge to have an effect on rate and hand placement [29], [35]. Our study showed additional components where knowledge had a significant effect on skill. Despite these theoretical components having a significant effect on their respective skill components, results remained dismal. Correct compression rate was also correlated to knowledge which supports prior findings [35], however only 8% achieved this.

The World Health Organization believes that “the state of cardiac arrest and trauma resuscitation is a good reflection of the standard of prehospital and emergency care in any health system” [36]. Since survival from cardiac arrest relies on the quality of CPR delivered [13], these results reflect the standard of care delivered by this EMS population. Additionally, poor results were achieved for components that directly related to clinical outcomes, namely, time to initiating compressions and time to defibrillation. Aside from a minority of participants who never performed compressions nor defibrillation throughout the skills assessment, the median times of those who did were unacceptably slow. Time from arriving at the patient’s side to initiating compressions should take no longer than 20 s [2] and time to defibrillation should be no more than 45 s [37]. The median time to initiating compressions was three times slower and defibrillation was almost six times slower. When not performed within these stipulated times, survival is significantly reduced [38].

An important factor, similar to prior studies, must be highlighted when considering the reasons other than poor knowledge causing poor skills results. Knowledge appeared to only have a small influence on skills (15.7–27.8%). The application of the skill of CPR has also been shown to be dependent on the rescuer’s training, experience and confidence [39]. In our study, some demographic factors also appeared to have an influence on both knowledge and skill.

We could not find literature on knowledge or skills worsening with advancing age. In the clinical setting, it is often assumed that older and more experienced clinicians possess greater skill proficiency. Contrary to this popular belief, others arrived at the same conclusion – as age increased, knowledge results worsened [29]. However, senior EMS personnel generally coordinate the CPR event rather than perform the skill itself [28]; nonetheless, our results are the same when it comes to increasing age. Another issue that this population faces is prolonged periods without re-training in CPR. The evidence is clear that this leads to worse performance, and similarly for not keeping up with current guidelines [40]. Another major barrier is that there are no known continuous quality improvement systems in place to ensure the delivery of high-quality CPR to the out-of-hospital cardiac arrest victim. Systematic continuous quality improvement systems have been shown to optimise outcomes in numerous health conditions [41], [42], [43]. Without such a system in place these results are to be expected, as are survival rates that reflect such performance quality.

It would not be possible to compare knowledge between public and private sector due to the smaller number of private participants. There are no other studies comparing these two sectors. The place where participants received their instruction also appeared to influence knowledge. This is consistent with other literature which highlights the quality of the instruction influencing knowledge retention [44], [45]. The majority of these participants (90%) reported not being trained to the latest guidelines, which is most likely the biggest contributing factor to poor knowledge and skills results. Staff practicing to outdated guidelines can only lead to poor adherence of the current guidelines. Additionally, prolonged periods of time between training intervals will cause both poor knowledge and skill retention of BLS CPR [13].

This study had several limitations. First, a sample size or power estimation was not done prior to the study commencing. The sample size was chosen on logistical grounds. However, retrospectively, it should be noted that where statistical hypothesis testing was done, the statistical significance appears to match the clinical significance of the result, indicating that there was sufficient power for these analyses from the sample size. For instance, where the correlation coefficient was of a clinically relevant size, the statistical significance was also <0.05. And where the correlation coefficient was not of clinically relevant size, the p value was also not statistically significant. This suggests that no type I or II errors were made in the interpretation of the results, and that the power level was satisfactory. There might however have been small subgroups in this study which would have benefitted from a larger sample size, and this has been noted as a limitation of the study. We represented ILS qualified EMS personnel in six of the seven districts and have no reason to believe that we had significant selection bias which would detract from the results.

A high-fidelity CPR manikin would have provided a more comprehensive and precise measurement of the CPR skill performance. Due to funding constraints, we opted to use a more readily available low-fidelity manikin that captures the basic characteristics of CPR, and relies on observation with basic measurable features (adequate ventilation and chest compression parameters). Whilst this method of observation is less ideal, it is still considered to be standard practice and can be effectively replicated in other similar settings that are under resourced.

There appeared to be discrepancy between scores given by the independent reviewers for skills results. The level of agreement was statistically considered to be ‘moderate’ to ‘good’, and is not likely to have detracted from the overall substandard skill performance. This data set was collected in November 2013, and reflected the knowledge and skill performance of the sample at the time. Whilst this data set may be dated, it does provide us with the first snapshot view of CPR performance by EMS personnel in South Africa.

Conclusion

The simulated CPR performance of EMS participants was well below that required to achieve optimal survival rates in victims of cardiac arrest. The clinical impact of this requires further investigation in the OH setting. Knowledge was shown to have a small effect on skill performance, whilst certain demographic factors appeared to have a larger influence. These factors shown to negatively influence performance should be strongly considered when developing methods to improve knowledge and skill retention, and ultimately skill performance. The lack of ongoing training and continuous quality improvement systems contributes to worse performance, and implementation of these systems is recommended. Training sessions should place more emphasis and time on practicing the skill of CPR, and less so on the theoretical knowledge component. The combination of these systems and training methods is likely to have a positive clinical impact for the out-of-hospital cardiac arrest victim attended to by EMS.

Dissemination of results

The results of this study have been shared with EMS stakeholders in the province under study at the Emergency Care Society of South Africa conference during a presentation. Practical implementation strategies and ideas were recommended for their use.

Author contributions

JV was the original author, and drove the entire process from conception to completion. LW affirmed the original idea, and made contributions along the way, especially towards the drafting of the final write-up. RA gave constant input and direction from the beginning through to the end of the write-up, along with face-to-face time with complex areas. RB contributed research and statistical knowledge at the initial concept of the idea, methodology development assistance, as well as expert advice into data analysis.

Conflict of interest

This study was partly funded by the Division of Emergency Medicine, University of Cape Town. LW is an editor of this journals. He was however not involved in the editorial workflow of this paper. The authors declare no further conflicts of interest.