Abstract
Background
Adverse events (AEs) epidemiology is the first step to improve practice in the healthcare system. Usually, the preferred method used to estimate the magnitude of the problem is the retrospective cohort study design, with retrospective reviews of the medical records. However this data collection involves a sophisticated sampling plan, and a process of intensive review of sometimes very heavy and complex medical records. Cross-sectional survey is also a valid and feasible methodology to study AEs.
Objectives
The aim of this study is to compare AEs detection using two different methodologies: cross-sectional versus retrospective cohort design.
Setting
Secondary and tertiary hospitals in five countries: Argentina, Colombia, Costa Rica, Mexico and Peru.
Participants
The IBEAS Study is a cross-sectional survey with a sample size of 11 379 patients. The retrospective cohort study was obtained from a 10% random sample proportional to hospital size from the entire IBEAS Study population.
Methods
This study compares the 1-day prevalence of the AEs obtained in the IBEAS Study with the incidence obtained through the retrospective cohort study.
Results
The prevalence of patients with AEs was 10.47% (95% CI 9.90 to 11.03) (1191/11 379), while the cumulative incidence of the retrospective cohort study was 19.76% (95% CI 17.35% to 22.17%) (215/1088). In both studies the highest risk of suffering AEs was seen in Intensive Care Unit (ICU) patients. Comorbid patients and patients with medical devices showed higher risk.
Conclusion
The retrospective cohort design, although requires more resources, allows to detect more AEs than the cross-sectional design.
Keywords: patient safety, adverse events, quality in health care
Strengths and limitation of this study.
The identification of adverse events (AEs) is the first step to improve patient safety. We know cross-sectional studies are easier and less expensive to measure AEs.
This article adds the comparison between different study designs, and finds the most efficient to find AEs in the clinical practice. The study was made in five Latin American countries, so the data are strong for analysis.
We learn with this study that retrospective cohort design allows to detect more AEs compared with cross-sectional. ICU patients and patients with comorbidities have more AEs.
The sample used to evaluate the retrospective cohort was 10% of medical records used in the cross-sectional study, and proportional to hospital size, which could not be representative of all the population attended to in the healthcare system. Tertiary hospitals are more complex and the number of AEs could be overestimated, in comparison with the total number of patients attended to in the country.
Another limitation is the quality of medical records. If the variabilities in the accomplishments between the different countries and healthcare systems were high the comparison between them could be weaker.
Introduction
Valid and timely information about the frequency and impact of healthcare related adverse events (AEs) and about the system’s ability to detect, prevent and manage these AEs is extremely important to understand the failures of healthcare, and to design and evaluate the effectiveness of risk reduction strategies. Increasingly, a large number of research studies have estimated such type of information in various health systems and organisational contexts,1 leading to a growing body of evidence about the burden and nature of AEs caused by healthcare. One of the most important sources of information for such type of data are the patients’ medical records, most frequently through the practice of retrospective reviews following agreed protocols and standard abstract forms. This methodology has consolidated itself as one of the most valid references in the field of patient safety research.2 Nevertheless, despite the advantages of retrospective records reviews in identifying important and observable AEs, there are also some concerns about the capacity to conduct such methodology in facilities with weaker data and research infrastructure, and moreover when certain periodicity is desirable for monitoring the effectiveness of risk reduction strategies.
Every research methodology and data collection system has its strengths and drawbacks.3 Routine information systems have limitations related to compliance and coding bias. Events reporting systems also show preference for the type of events that reporters consider more relevant and have difficulties tracing duplicates, in addition to still facing unresolved legal issues in many contexts, which penalise reporting and limit their effective use. Prospective studies tend to focus on the analysis of higher-risk patients in detriment of other patients. Medical records, electronic or not, are threatened as well by lack of completeness and recording bias, since clinicians tend to record the data that are more meaningful to them from a clinical point of view. In addition, medical retrospective records review involves a sophisticated sampling plan, and a resource intensive process of record retrieval, reviewing and abstracting of sometimes very heavy and complex medical records.
A data collection process that has been less frequently used in the field of patient safety research, despite its potential, consists of running periodic cross-sectional surveys aiming to assess the point prevalence of AEs.4 5, This design has been commonly used to monitor the frequency of healthcare associated infections in many hospitals across Europe and elsewhere,6 where it has proven to be a feasible and valid methodology, capable of being run with no excessive resources on a large scale and across many institutions and organisational cultures. One of the advantages of this design is that instead of requiring a statistically savvy sampling plan, all patients admitted at a given time to the hospital can be surveyed at once, simplifying the sampling process as well as the search and retrieval of records from the archives, since these are usually located near the patients in the wards.7 This design also gives researchers the opportunity to ask the attending clinicians for some clarifications in the records, including some missing data. The unit of observation in this design is typically 1 day of admission, which makes it much shorter and simpler for the reviewers, and gives an estimate of a 1 day prevalence, as opposed to the cumulative incidence of a retrospective record review.8 Because of its greater simplicity, the management of large and multicentred research studies is also simplified.
The IBEAS Study was a multicountry effort aiming to estimate for the first time the frequency of hospital related AEs in a selection of hospitals from Argentina, Colombia, Costa Rica, Mexico and Peru.9 The study was conducted in 2007, in 58 hospitals of the five countries, with the collaboration of Spain, and pan-American organisation and WHO. It used a 1-day cross-sectional design due to the perceived simplicity, lesser demands and greater opportunities for strengthening local capacity and eventual replication of this approach. The researchers involved in designing IBEAS, aware of its innovative approach in the field of AE measurement, were mindful of determining the relationship between the estimates of the 1-day point prevalence design and the more traditional retrospective cohort record review approach. In this same context Michel et al10 evaluated the rates comparing three different methods: cross-sectional, prospective and retrospective. Therefore, we selected a randomised sample of all IBEAS patients to fully examine retrospectively their medical records.
Methods
The AE definitions used in both designs were those published by WHO in the International Classification for Patient Safety.11
A patient safety incident is an event or circumstance that could have resulted, or did result, in unnecessary harm to a patient.
An AE or harmful incident is an incident that results in harm to a patient.
Harm implies impairment of structure or function of the body and/or any deleterious effect arising therefrom, including disease, injury, suffering, disability and death, and may be physical, social or psychological.
In the IBEAS project the AE was defined as:9 ’Any event causing harm to the patient that is perceived to be more related to the healthcare management rather than to the patient’s underlying condition’.
The IBEAS Study had two parts,12 a cross-sectional study and a retrospective cohort study.
The cross-sectional study involved determining how many patients admitted to the participating hospitals experienced harmful incidents attributable to healthcare on a given day (day 0). A prevalent AE is defined as one that originates during hospitalisation and is clinically present on the day of the study, either as an after-effect or undertreatment. This also includes those AEs that were occasioned prior to hospitalisation at any care level and which led to subsequent admission. AEs that had occurred prior to the survey and whose effects had disappeared without prolonging the hospitalisation on that particular day were not included.
The retrospective cohort study was conducted using a sample of patients with the aim of confirming whether the cross-sectional study could replace the conventional retrospective cohort study used to date. Specifically, the study involved reviewing the case notes of a random sample of 10% of patients (1101 patients) hospitalised on day 0, proportional to hospital size, from the entire IBEAS Study population. Case notes were scanned to ascertain whether, at some point during their hospitalisation (or in a previous admission to the hospital), inpatients had experienced a harmful incident, regardless of whether the consequences of the incident were still present on day 0. Patients continued to be monitored until discharge. The sampling strategy and forms are available on request. An incident AE is defined as one that occurs during any patient care process, as it may be detected at another level of care or in another hospital. In practical terms, as we carried out a retrospective study based on clinical hospital records, primary care AEs were not included. Those that led to readmission in the same or another hospital were compensated by the AEs which were detected during this hospitalisation and which had been originated in a previous hospitalisation.
The IBEAS Study was carried out in five countries: Argentina, Colombia, Costa Rica, Mexico and Peru. The number of hospitals included was 58, all of them secondary-level and tertiary-level hospitals. We used a purposive sample of hospitals and the inclusion was voluntary.
The sample size was of 11 379 patients, with a minimum of 2000 patients per country.
In both studies (cross-sectional and retrospective cohort), researchers used two tools to detect harmful incidents, namely a Screening Guide and a Modular Questionnaire13 14 using the medical record review.15 16
First, the screening guide was applied to the patients in the study. This served as an alert and tracking system for possible incidents. All the patients admitted at hospital (except emergency room) were studied. The screening was made by well trained nurses.
If a patient screened positive for one or more of the 19 alert criteria in the screening guide, the case was studied using the case history. An in-depth study of case histories enabled researchers to conclude whether a patient did in fact present with the consequences of a harmful incident (true positive) and if so, to classify the type of event, its severity, any associated factors, and whether or not the incident could have been avoided, and so on. This second confirmatory review was made (in both cross-sectional and retrospective) by medical doctors with at least 5 years of clinical experience. A patient could have more than one AE in the same hospitalisation, and in this case the study collects all of them.
The reviewers' training took place in two stages. First, the trainer workshop addressed the national coordinating teams in Buenos Aires in 2007. Second, the national coordinators, in turn, trained the national investigators. A concordance study was carried out in Bogotá in 2008 using clinical records from each country. The most complex cases were assessed and an agreement was reached.
The preventable AE and the gravity were assessed according to the recommendations in the Modular Questionnaire, and the reviewers were also trained in these criteria.
The cross-sectional and the retrospective cohort studies were made by the same reviewers in each country.
The completed review forms of the retrospective study were entered in electronic files and submitted to a central repository managed exclusively by the principal investigators. Descriptive and multivariate analyses were conducted using SPSS V.14. Logistic regressions were used to estimate the prevalence and incidence of AEs, once taking into account the effect of some covariates, such as patient’s age and comorbidity (intrinsic factors), presence of catheter lines and medical devices (extrinsic factors), type of admission and type of hospital.
Results
In the cross-sectional study 11 379 patients were included (figure 1); 3853 of them (33.9%) fulfilled at least one of the screening criteria. In the second phase of the cross-sectional study 1191 patients had an AE.
Figure 1.
Study patients in the cross-sectional study.
For the retrospective cohort study (figure 2), 1101 patients (10%) were randomly selected from all the 11 379 patients included in the cross-sectional study. The medical records of 13 of these patients (1.2%) could not be retrieved and were excluded from the study.
Figure 2.
Study patients in the retrospective cohort study.
The screening phase of the retrospective review found about 44.5% of the medical records, corresponding to 484 patients, positive for at least one of the 19 triggers included in the forms. At the confirmatory phase, it was determined that 40 of those patients had experienced one or more patient safety incidents without harm or prolonged stay, and 288 patients had experienced at least one AE (harmful patient safety incident). Of these, in 215 patients the AE was considered to be mostly related to the healthcare received rather than to the patient intrinsic vulnerability.
The characteristics of patients in the two types of study are presented in table 1. Patients in the retrospective cohort study were of similar sex composition than the cross-sectional study sample. Though they were slightly older, they did not show significant differences in their intrinsic factors (comorbidity). It seemed there were more patients in surgical wards and with slightly more procedures in the retrospective review sample than the patients in the 1-day cross-sectional study. The composition of participating hospitals and type of admission was comparable in the two types of designs.
Table 1.
Characteristics of the study population
| Cross-sectional | Retrospective cohort | p Value | ||||||
| n | % | 95% CI | n | % | 95% CI | |||
| Sex | Women | 5975 | 52.5 | 51.6 to 53.4 | 547 | 50.3 | 47.3 to 53.2 | n.s. |
| Age | Mean (SD) | 40.2 | 26.9 | 42.1 | 26 | 31.3 to 36.9 | 0.02 | |
| Department | Medical wards | 4045 | 35.5 | 34.7 to 36.4 | 371 | 34.1 | 31.3 to 36.9 | .001 |
| Surgery/gynaecology | 3898 | 34.3 | 33.4 to 35.1 | 435 | 40.0 | 37.1 to 42.9 | ||
| Obstetrics | 1241 | 10.9 | 10.3 to 11.5 | 109 | 10.0 | 8.2 to 11.8 | ||
| Paediatrics | 1701 | 14.9 | 14.3 to 15.6 | 128 | 11.8 | 9.9 to 13.7 | ||
| Intensive care | 494 | 4.3 | 4 to 4.7 | 45 | 4.1 | 3 to 5.3 | ||
| Hospital complexity | Tertiary | 10 520 | 92.5 | 92 to 92.9 | 1011 | 92.9 | 91.4 to 94.4 | n.s. |
| Secondary (with surgery and ICU wards) |
859 | 7.5 | 7.1 to 8 | 77 | 7.1 | 5.6 to 8.6 | ||
| Admission type | Unplanned admission | 8031 | 70.6 | 69.7 to 71.4 | 726 | 66.7 | 63.9 to 69.5 | n.s. |
| Planned admission | 2099 | 18.4 | 17.7 to 19.2 | 190 | 17.4 | 15.2 to 19.7 | ||
| Intrinsic risk factors | Yes | 6128 | 53.9 | 52 to 54.8 | 615 | 56.5 | 53.6 to 5.5 | n.s. |
| No | 5251 | 46.1 | 45.2 to 47.1 | 473 | 43.5 | 40.5 to 46.4 | ||
| Extrinsic risk factors | Yes | 8484 | 74.6 | 73.8 to 75.4 | 844 | 77.6 | 75.1 to 80.1 | 0.03 |
| No | 2895 | 25.4 | 24.6 to 26.2 | 244 | 22.4 | 19.9 to 24.9 | ||
| Patients studied | 11 379 | 1088 | ||||||
n.s., not significant (p>0.05).
Table 2 shows the prevalence of patients with AEs was 10.47% (95% CI 9.90% to 11.03%). As a patient can have more than one AE, the total number of AEs detected was 1349, so the global prevalence of AEs was 11.85% (1349/11379) (95% CI 11.26% to 12.46%).
Table 2.
Differences in result measures in both study designs
| Cross–sectional (prevalence) |
Retrospective cohort (cumulative incidence) |
|
| Patients with adverse events (AEs) | 1191/11379=10.47% (95% CI 9.90% to 11.03%) |
215/1088=19.76% (95% CI 17.35% to 22.17%) |
| Total number of AEs | 1349/11379=11.85% (95% CI 11.26% to 12.46%) |
314/1088=28.86% (95% CI 26.12% to 31.61%) |
Table 2 also shows the cumulative incidence of patients suffering at least one AE related to the care received before or during their hospitalisation was 19.76% (95% CI 17.35% to 22.17%) (215/1088). In total, there were 314 AEs (because a patient could have more than 1 AE) related to healthcare corresponding to a cumulative incidence of total AEs of 28.86% (95% CI 26.12% to 31.60%) (317/1088).
Table 3 shows the results of the cross-sectional study and the retrospective cohort record review per country, showing the rate of positive screening and its positive predictive value (PPV) and the corresponding final estimate in terms of 1-day prevalence and the proportion of patients with at least one AE during their hospitalisation. In the 1-day cross-sectional study, the rate of positive screening review form (SRF) seemed to range more homogeneouslybetween 30% to 39% of all records, with PPVs between 25% and 37%. In the retrospective cohort review, however, the range of positive screening was wider going from about 17% to almost 64% of all records, and also reaching higher PPVs from 24% to over 60%. In all countries, the percentage of patients suffering at least one AE during their hospitalisation was significantly higher than the rate observed in the 1-day study, with values going from 11% of patients to more than 36%.
Table 3.
Adverse event frequency measures and screening form performance
| Cross-sectional study | Retrospective study | |||||
| Positive screening review form | Positive predictive values |
Prevalence of adverse events | Positive screening review form | Positive predictive values | Cumulative incidence of adverse events | |
| Country 1 | 39.0% 926/2373 (95% CI 37.0% to 41.0%) |
33.7% 312/926 (95% CI 30.6% to 36.8%) |
13.1% 312/2373 (95% CI 11.8% to 14.5%) |
61.7% 145/235 (95% CI 6.7% to 8.1%) |
51.7% 75/145 (95% CI 43.2% to 60.2%) |
31.9% 75/235 (95% CI 25.7% to 38.1%) |
| Country 2 | 30.6% 887/2897 (95% CI 28.9% to 32.3%) |
25.3% 224/887 (95% CI 22.3% to 28.2%) |
7.7% 224/2897 (95% CI 6.7% to 8.7%) |
38.9% 112/288 (95% CI 33.1% to 44.7%) |
32.1% 36/112 (95% CI 23.0% to 41.2%) |
12.5% 36/288 (95% CI 8.5% to 16.5%) |
| Country 3 | 35.4% 578/1632 95% CI 33.1% to 37.8%) |
34.3% 198/578 (95% CI 30.3% to 38.2%) |
12.1% 198/1632 (95% CI 6.7% to 8.7%) |
63.7% 107/168 (95% CI 56.1% to 71.3%) |
57.0% 61/107 (95% CI 47.2% to 66.9%) |
36.3% 61/168 (95% CI 28.7% to 43.9%) |
| Country 4 | 34.5% 692/2003 (95% CI 32.4% to 36.7%) |
24.7% 171/692 (95% CI 21.4% to 27.9%) |
8.5% 171/2003 (95% CI 7.3% to 9.8%) |
46.9% 82/175 (95% CI 39.2% to 54.5%) |
24.4% 20/82 (95% CI 14.5% to 34.3%) |
11.4% 20/175 (95% CI 6.4% to 16.4%) |
| Country 5 | 31.1% 770/2474 (95% CI 29.3% to 32.9%) |
37.1% 286/770 (95% CI 33.7% to 40.6%) |
11.6% 286/2474 (95% CI 10.3% to 12.8%) |
17.1% 38/222 (95% CI 11.9% to 22.3%) |
60.5% 23/38 (95% CI 43.7% to 77.4%) |
10.4% 23/222 (95% CI 6.1% to 14.6%) |
| Total | 33.9% 3853/11379 (95% CI 32.9% to 34.7%) |
30.9% 1191/3853 (95% CI 29.4% to 32.4%) |
10.5% 1191/11379 (95% CI 9.9 to 11.0) |
44.5% 484/1088 (95% CI 41.5% to 47.5%) |
44.4% 215/484 (95% CI 39.3% to 48.3%) |
19.8% 215/1088 (95% CI 17.3% to 22.2%) |
In both studies (table 4), the highest risk of suffering AEs was seen in ICU patients. Surgical patients were associated with more risk than patients admitted in the medical wards. Whereas, in the cross-sectional study, obstetrics and paediatric patients also showed higher risk than medical patients. Comorbid patients showed higher risk of suffering AEs in both studies, as well as patients with catheter lines, and other procedures. Similarly, the length of stay before the day of study in the cross-sectional study and the total length of stay in the retrospective cohort were associated witha higher risk of suffering AE. In the retrospective cohort review, emergency hospitalisations did not seem to be associated to the risk of suffering AEs as seemed to be the case in the cross-sectional study. Patient age was not retained as an independent variable or as a confounding factor in the final model in both studies.
Table 4.
Correlates of adverse events in multiple logistic regression analyses
| Cross-sectional | Retrospective cohort | |||||||
| Variables | p Value | OR | 95% CI for OR | p Value | OR | 95% CI for OR | ||
| Department (1) | ||||||||
| Surgery and gynaecology | 0.06 | 1.17 | 0.99 | 1.38 | 0.01 | 1.75 | 1.17 | 2.61 |
| Obstetrics | 0.02 | 1.37 | 1.06 | 1.78 | 0.05 | 0.38 | 0.15 | 0.99 |
| Paediatrics | 0.00 | 1.50 | 1.21 | 1.85 | 0.15 | 0.40 | 0.12 | 1.40 |
| Intensive care | 0.00 | 2.52 | 1.96 | 3.26 | 0.01 | 2.77 | 1.25 | 6.17 |
| Complexity of the hospital (2) tertiary | 0.02 | 1.45 | 1.07 | 1.97 | ||||
| Type of admission (3) urgent | 0.00 | 1.34 | 1.12 | 1.61 | 0.59 | 1.14 | 0.71 | 1.83 |
| Length of stay until the day of study | 0.03 | 1.00 | 1.00 | 1.00 | 0.01 | 1.00 | 1.00 | 1.01 |
| Patient comorbidity (4) any | 0.00 | 1.42 | 1.22 | 1.64 | 0.00 | 2.02 | 1.28 | 3.19 |
| Use of medical devices (5) any | 0.00 | 2.59 | 2.14 | 3.14 | 0.00 | 3.24 | 1.79 | 5.85 |
| Country 1 | ||||||||
| Country 2 | 0.00 | 0.46 | 0.38 | 0.56 | 0.00 | 0.34 | 0.20 | 0.59 |
| Country 3 | 0.37 | 0.91 | 0.73 | 1.12 | 0.16 | 1.44 | 0.86 | 2.41 |
| Country 4 | 0.00 | 0.65 | 0.52 | 0.81 | 0.00 | 0.22 | 0.12 | 0.42 |
| Country 5 | 0.04 | 0.82 | 0.69 | 0.99 | 0.01 | 0.38 | 0.19 | 0.76 |
Reference categories: (1) medical specialities; (2) secondary hospitals of intermediate complexity with at least surgical theatres, and postsurgical resuscitation wards; (3) planned admission; (4) no comorbidities (5) no devices.
The types of AEs identified in both the cross-sectional and the retrospective cohort studies showed similar distribution. The most frequent types of AEs identified in any study were related to the occurrence of healthcare associated infections (more than 35% of all AEs), followed by AEs related to procedures (more than 26%). Medication related AEs represented less than 10% of all AEs in each of the studies (table 5).
Table 5.
Adverse event (AE) types and proportion of total AE
| Type of AE | Prevalence | 95% CI | Cumulative incidence | 95% CI |
| Care provided | 13.27% | 11.46 to 15.08 | 12.16% | 16.24 to 20.32 |
| Medication | 8.23% | 6.76 to 9.69 | 6.57% | 9.87 to 13.17 |
| Healthcare associated infections | 37.14% | 34.56 to 39.72 | 30.68% | 35.99 to 41.30 |
| Related to procedures | 28.69% | 26.27 to 31.10 | 21.86% | 26.75 to 31.65 |
| Diagnostic issues | 6.15% | 6.15 to 7.44 | 2.66% | 5.10 to 7.53 |
| Nosocomial urinary tract infection | 4.08% | 2.98 to 5.17 | 5.09% | 2.50 to 7.69 |
| Nosocomial pneumonia | 9.41% | 7.82 to 11.01 | 6.37% | 3.51 to 9.23 |
| Postsurgical haematoma | 2.89% | 1.96 to 3.82 | 3.50% | 1.31 to 5.69 |
| Phlebitis | 3.41% | 2.40 to 4.41 | 5.73% | 3.00 to 8.46 |
| Neonatal complications | 1.12% | 0.51 to 1.71 | 0.32% | 0.01 to 1.76 |
There were some differences in the impact caused by the AEs identified in the cross-sectional study, versus the retrospective cohort study. The AEs identified through the cross-sectional study seemed to be associated more frequently with hospital readmissions and slightly more with prolonged stay, whereas the frequency of AEs which did not cause prolonged stay or readmission was higher in the retrospective cohort study (table 6).
Table 6.
Impact of adverse events in hospitalisation, n (%)
| Cross-sectional | Retrospective cohort | |||
| Did not have prolonged hospital stay | 228 | 18.91% (95% CI 16.65% to 21.16%) | 87 | 29.9% (95% CI 24.46% to 35.33%) |
| Prolonged hospital stay | 759 | 62.9% (95% CI 60.17% to 65.70%) | 178 | 61.2% (95% CI 51.88% to 63.58%) |
| Extra days same hospitalisation | Mean: 16.1 days SD (29.6) | Mean: 14.9 days SD (19.9) | ||
| Causing admission | 219 | 18.16% (95% CI 15.94% to 20.38%) | 26 | 8.9% (95% CI 5.48% to 12.38%) |
| Extra days new hospitalisation | Mean: 21.4 days SD (69.7) | Mean:19.0 days SE (22.3) | ||
The preventability of AE (See online supplementary appendix) was very similar, with about 65% in the retrospective cohort review and 60% for the cross-sectional study.
bmjopen-2017-016546supp001.pdf (5.7KB, pdf)
Discussion
The choice of the most appropriate epidemiological design in the study of magnitude of AE is not a trivial issue. The question has been analysed in different studies and the generalised consensus is that the choice of method should be based on the aims of the study and the need to combine the minimisation of bias and the validity of AE identification with the reproducibility of value judgements on their iatrogenic nature and/or preventability.17
The retrospective design for the study of AE has been the method used in all national studies.1 Nevertheless it is a method which produces results which may be heavily influenced by the quality of clinical records.
A prospective study offers pedagogical and communicative advantages and facilitates a concomitant analysis of the root causes which provide the conditions for the occurrence of AEs. However not only might it prove too costly, but it would also involve a high workload and excessive complexity.
On the other hand, the cross-sectional design is more time-efficient and resource-efficient and easier to perform. Although it does not allow for a study of the total hospitalisation episode, it has proved capable of sustaining over time a more stable system of observation. We also need to bear in mind that, as a result of a possible survival bias, those AEs which lead to hospital admission will be over-represented, as will those related to nosocomial infection or those which are difficult to identify if the patient is not examined (such as bruising), due to the systemic approach itself of a prevalence study. As in the prospective approach, communication with the ward staff (the patient is hospitalised at this time) makes it easier to judge the causality of the AE and its preventability.
The relationship between prevalence and incidence generally depends on the duration of the event under review and the period of observation.8 In our case we calculated prevalence on a given day and not during the whole period. Consequently this relationship will not be well reflected. In figure 3 we see the possible AEs which may occur and those that are detected on the basis of this approach.
Figure 3.
Study of prevalent AE on a given day. AE, adverse event; HCC, healthcare centre; PC, primary care.
When we compare the results of the cross-sectional study with those of the retrospective cohort study within the context of the IBEAS project, the differences are due exclusively to the design, as the methodology and sample are the same (assuming the representativeness of the incidence subcohort). In (online supplementary appendix), figure 3 and figure 4) which represent the scheme followed in the methodology of this study, we see that the difference between the prevalence and incidence values on a given day are due to those AEs which, having occurred during hospitalisation, are not prevalent on the day of the study (represented by a yellow arrow in figure 3). This also explains why the patients of the retrospective cohort study present more extrinsic risk factors (devices) than in the cross-sectional study.
Figure 4.
Scheme of incident AE study. AE, adverse event; HCC, healthcare centre; PC, primary care.
The SRF has been used in American18 19 and Australian20 21 cohort studies and in different European22 23 countries. It is highly sensitive (84%) in the detection of AEs and we therefore assume that the number of false negatives should be small. We can also detect with the revision of the modular questionnaire.
Appropriateness of the review forms to a point prevalence study was discussed during the training workshop. Modifications to adapt them to the context of Latin America were done bearing in mind vocabulary, and adding common risk factors like malaria or prematurity.
The percentage of patients flagged in the SRF and the predictive value of this phase in the detection of AE are totally compatible with those found in those other AE studies of which we are aware. We can therefore state that the materials are sensitive enough and appropriate for the identification of both prevalence and cumulative incidence of AE. However, in the retrospective cohort study, the PPV of the SRF is higher. This may be due to the fact either that the guide was originally designed for an incidence study and proves more efficient in this type of study or that as the retrospective cohort study was performed after the cross-sectional study, it is possible that the experience of the reviewers raised the performance level of the first questionnaire.
The Spanish version of the modular review form was adapted in Spain for the Identification of Adverse Events (IDEA) Project and modified after the ENEAS (National Study of Adverse Events) Study.4 The researcher must make value judgements through implicit criteria on most occasions. Characterisation of AEs caused by the care rather than the pathological process itself, is done by the reviewer scoring from 1 to 6 the probability that the AE is due to the care. A value of ≥4 is required to confirm this. The same criterion is used to evaluate the AE as preventable. Cross-sectional design allows researchers to consult the medical staff while they are collecting data in order to clarify any uncertainty or doubts associated with the AE. The reliability of the questionnaire in other studies has been assessed as moderate.23
A limitation of this study could be the sample used to evaluate the retrospective cohort. It is 10% of medical records used in the cross-sectional study and proportional to hospital size, which could be not representative of all the population attended to in the healthcare system. Tertiary hospitals have more complexity and the number of AEs could be overestimated, in comparison with the total number of patients attended to in the country. Another limitation is the quality of medical records. If the variability in the accomplishment between the different countries were high the comparison between them could be weaker.
The frequency of AEs in both the cross-sectional and retrospective cohort studies was greater than that found in previous studies, which may be due to the different characteristics of the patients, who had higher average age and more risk factors. The nature of the sample selection and the peculiarities of the different search systems prevent statistical inferences and comparisons either within each country or between the countries which are part of the study.
In any case, higher prevalence means higher cumulative incidence. In some way the interdependence of these frequency measures remains when we use prevalence on a given day. The fact that prevalence is sensitive to the differences in the characteristics of the patients and that it reflects the differences found between countries, would make it a useful tool in the study and follow-up of the frequency of AEs and in comparative studies. Furthermore, as the explanatory model for the occurrence of AE is the same, studying the factors which influence prevalence may provide the same clues when designing strategies for AE control and therefore provide a more efficient tool.
Moreover, the fact that the prevalence design detects proportionally more serious AEs is not a drawback. On the contrary, these are precisely the AEs which need to be prioritised when designing control strategies, and as we commented above, the detected AEs were equally preventable in both designs. This reinforces the idea that preventability and seriousness of the AEs are independent factors.
Due to the fact that the point prevalence design is more efficient in terms of time and resources, its validity is less dependent on the quality of the clinical records and allows simultaneous study through other observation and audit systems, regular prevalence on a given day studies might provide an efficient AE monitoring and control strategy.
Supplementary Material
Footnotes
Contributor: JMAA is the principal publication author, he has coordinated the IBEAS project and he is the team leader. RLR has participated in data collection, data analysis and interpretation of the results, and composition of the manuscript. CAR has been coordinator of the IBEAS project, and he has evaluated the manuscript and contributed to improving the presentation; has has also participated in the data collection. MTG-VdC has participated in data collection, data analysis, evaluation of the results, and she has improved the manuscript; she has coordinated the project with JMAA. FB has participated in the evaluation and improvement of the manuscript and data analysis. IH-A has participated in the design of the study and analysis planning. NLF has participated in the evaluation of the manuscript and she has been the responsible for sending the manuscript to the reviewer and the editorials. CDA has collaborated for manuscript composition and she has contributed to the critical review. ETG has been coordinator of the IBEAS project and he has contributed to the critical review of this paper. PS has collaborated in the data analysis and critical review of this paper. ILJ has contributed in the improvement of the paper composition and she has participated in data analysis. She has also been an IBEAS project coordinator. All the members have contributed towards the data collection.
Competing interests: None declared.
Patient consent: Detail has been removed from these case descriptions to ensure anonymity. The editors and reviewers have seen the detailed information available and are satisfied that the information backs up the case the authors are making.
Ethics approval: The IBEAS Study maintained ethical conduct of research, and was approved by the PAHO Ethics Review Committee and by the National Ethics Review Committees of each participating country.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: Additional data can be available by emailing: jesusmaria.aranaz@salud.madrid.org.
Collaborators: Grupode Trabajo IBEAS: Agra-Varela Y (Ministerio de Sanidad, ServiciosSociales e Igualdad, España), Aibar-Villán L (Hospital San Juan de Dios de Úbeda. España), Casal-Gómez JM (Ministerio deSanidad, Servicios Sociales e Igualdad, España), Fernández-Maíllo M (Ministerio de Sanidad, Servicios Sociales e Igualdad, España), Aranaz-Ostáriz V (Hospital Universitario San Juan de Alicante, España), López Rodríguez-Arias F (Hospital Universitario de Elda, España), Valencia-Martín JL (Hospital Universitario Ramón y Cajal. España), Rincón-Carlavilla A (Hospital Universitario Ramón y Cajal, España), Gonseth-García J (Gerencia Hospital de Especialidades Guayaquil "Dr. Abel Gilbert Pontón", Ecuador), Amarilla AC (Dirección de Calidad de los Servicios deSalud, Ministerio de Salud, Argentina), Restrepo-Parra FR (Dirección general de Calidad, Ministerio de la Protección Social, Colombia), Sarabia-González O (Subsecretaría de Innovación y Calidad, Ministerio de Salud, México), Inga R (Seguro Social de Salud, Perú), Santivañez A (Seguro Social de Salud, Perú), Urroz-Torres O (Dirección Nacional, Caja Costarricense de Seguro Social, Costa Rica), García-Corcuera LV (Ministerio de Salud, Perú).
References
- 1. de Vries EN, Ramrattan MA, Smorenburg SM, et al. The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 2008;17:216–23. 10.1136/qshc.2007.023622 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Lilford RJ, Mohammed MA, Braunholtz D, et al. The measurement of active errors: methodological issues. Qual Saf Health Care 2003;12:8ii–12. 10.1136/qhc.12.suppl_2.ii8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Thomas EJ, Petersen LA. Measuring errors and adverse events in health care. J Gen Intern Med 2003;18:61–7. 10.1046/j.1525-1497.2003.20147.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Requena J, Aranaz JM, Gea MT, et al. Evolución de la prevalencia de eventos adversos relacionados con la asistencia en hospitales de la Comunidad Valenciana. Revista de Calidad Asistencial 2010;25:244–9. 10.1016/j.cali.2010.03.007 [DOI] [PubMed] [Google Scholar]
- 5. Corrales MJ, Limón R, Miralles JJ, et al. Factores asociados a las infecciones evitables relacionadas con la atención sanitaria identificadas en el estudio EPIDEA. Med Prev 2010;16:18–23. [Google Scholar]
- 6. Zarb P, Coignard B, Griskeviciene J, et al. The European Centre for Disease Prevention and Control (ECDC) pilot point prevalence survey of healthcare-associated infections and antimicrobial use. Euro Surveill 2012;17. [DOI] [PubMed] [Google Scholar]
- 7. Sedgwick P. Bias in observational study designs: cross sectional studies. BMJ 2015;350:h1286 10.1136/bmj.h1286 [DOI] [PubMed] [Google Scholar]
- 8. Philippe M, Olsen S, Saillour-Glénisson F, et al. Assessing and tackling patient harm: a methodological guide for data-poor hospitals. Geneva: World Health Organization, 2010. http://www.who.int/patientsafety/research/methodological_guide/PSP_MethGuid.pdf. [Google Scholar]
- 9. Aranaz-Andrés JM, Aibar-Remón C, Limón-Ramírez R, et al. Diseño del estudio IBEAS: prevalencia de efectos adversos en hospitales de Latinoamérica. Revista de Calidad Asistencial 2011;26:194–200. 10.1016/j.cali.2010.12.001 [DOI] [PubMed] [Google Scholar]
- 10. Michel P, Quenon JL, de Sarasqueta AM, et al. Comparison of three methods for estimating rates of adverse events and rates of preventable adverse events in acute care hospitals. BMJ 2004;328:199–203. 10.1136/bmj.328.7433.199 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Conceptual framework for the International Classification for Patient Safety. http://www.who.int/patientsafety/taxonomy/icps_full_report.pdf.
- 12. Aranaz-Andrés JM, Aibar-Remón C, Limón-Ramírez R, et al. Prevalence of adverse events in the hospitals of five Latin American countries: results of the ’Iberoamerican Study of Adverse Events' (IBEAS). BMJ Qual Saf 2011;20:1043–51. 10.1136/bmjqs.2011.051284 [DOI] [PubMed] [Google Scholar]
- 13. O’Neil AC, Petersen LA, Cook EF, et al. Physician reporting compared with medical-record review to identify adverse medical events. Ann Intern Med 1993;119:370–6. 10.7326/0003-4819-119-5-199309010-00004 [DOI] [PubMed] [Google Scholar]
- 14. Michel P, Quenon JL, Sarasqueta AM, et al. L’estimation du risque iatrogène graves dans les établissements de santé en France: les enseignements d’une étude pilote dans the région Aquitaine. Etudes et Résultats 2003;219:1–8. [Google Scholar]
- 15. Michel P, Aranaz JM, Limón R, et al. Siguiendo la pista de los efectos adversos: cómo detectarlos. Revista de Calidad Asistencial 2005;20:204–10. 10.1016/S1134-282X(08)74752-3 [DOI] [Google Scholar]
- 16. Brennan TA, Localio RJ, Laird NL. Reliability and validity of judgments concerning adverse events suffered by hospitalized patients. Med Care 1989;27:1148–58. 10.1097/00005650-198912000-00006 [DOI] [PubMed] [Google Scholar]
- 17. Aranaz-Andrés JM, Aibar-Remón C, Vitaller-Murillo J, et al. Incidence of adverse events related to health care in Spain: results of the Spanish National Study of Adverse Events. J Epidemiol Community Health 2008;62:1022–9. 10.1136/jech.2007.065227 [DOI] [PubMed] [Google Scholar]
- 18. Forster AJ, Asmis TR, Clark HD, et al. Ottawa Hospital Patient Safety Study: incidence and timing of adverse events in patients admitted to a Canadian teaching hospital. CMAJ 2004;170:1235–40. 10.1503/cmaj.1030683 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Thomas EJ, Studdert DM, Burstin HR, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000;38:261–71. 10.1097/00005650-200003000-00003 [DOI] [PubMed] [Google Scholar]
- 20. Wilson RM, Runciman WB, Gibberd RW, et al. The Quality in Australian Health Care Study. Med J Aust 1995;163:458–71. [DOI] [PubMed] [Google Scholar]
- 21. Brown P, McArthur C, Newby L, et al. Cost of medical injury in New Zealand: a retrospective cohort study. J Health Serv Res Policy 2002;7:29–34. 10.1258/135581902320176449 [DOI] [PubMed] [Google Scholar]
- 22. Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001;322:517–9. 10.1136/bmj.322.7285.517 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Neale G, Woloshynowych M, Vincent C. Exploring the causes of adverse events in NHS hospital practice. J R Soc Med 2001;94:322–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2017-016546supp001.pdf (5.7KB, pdf)