A cross-sectional survey using electronic distribution of a questionnaire to subscribers of educational material written by clinicians, for clinicians, to evaluate whether practice change resulted from reading the Clinical Communiqué

Abstract

Objective

To explore whether subscribers reported clinical practice changes as a result of reading the Clinical Communiqué (CC). Secondarily, to compare the characteristics of subscribers who self-reported changes to clinical practice with those who did not, and to explore subscribers’ perceptions of the educational value of the CC.

Design, setting and participants

Online cross-sectional survey between 21 July 2015 and 18 August 2015 by subscribers of the CC (response rate=29.9%, 1008/3373), conducted by a team from Monash University, Australia.

Main outcome measures

Change in clinical practice as a result of reading the CC.

Results

53.0% of respondents reported that their practice had changed after reading the CC. Respondents also found that the CC raised awareness (96.5%) and provided ideas about improving patient safety and care (94.1%) leading them to discuss cases with their colleagues (79.6%) and review their practice (75.7%). Multivariate analysis indicated that working in a residential aged care facility (p<0.05) and having taken part in an inquest (p<0.05) were significantly associated with practice change.

Conclusion

The design and content of the CC has generated a positive impact on the healthcare community. It is presented in a format that appears to be accessible and acceptable to readers and achieves its goals of promoting safer clinical care through greater awareness of the medico-legal context of practice.

Introduction

Throughout the developed world, it has been estimated that about 10% of hospital admissions are associated with an adverse event.¹ The examination of adverse events in healthcare settings is a tool whereby practical information can be generated to stimulate incentives for change at many levels. However, clinical behaviour is notoriously resistant to change.² At an individual level, barriers to (and incentives for) change include awareness, knowledge, attitude, motivation and behavioural routines.³ Systemic organisational factors include change fatigue, resource limitations, restructuring and workplace culture.⁴

Adverse events and patient harm in healthcare settings occur through errors of commission or errors of omission. Errors of commission often manifest as single high-profile catastrophic events, whereas errors of omission may be more pervasive and difficult to identify.⁵ The latter often require careful evaluation to reveal the interplay between remote and unseen factors that may have led to the errors.

In the pursuit of improving patient safety, medico-legal death investigations in healthcare settings can allow for identification of areas suitable for change. Inquests provide a forum within which a coroner conducts a detailed analysis of events that contributed to a patient’s death. Errors are explored and questions of ‘why’, ‘what’, ‘where’ and ‘who’ in relation to practice change can be addressed in the coroner’s comments and recommendations.

Accessibility of this information to healthcare professionals may be limited by the legal format and technical language of coronial findings. The Clinical Communiqué (CC) is an electronic educational publication that encourages practice reflection and change by providing coronial information regarding preventable deaths in acute hospital and community settings. In order to optimise the impact and dissemination of this important information, a relevant clinical context is combined with accessible language and up-to-date expertise (box 1). The CC contains narrative case reports about lessons learnt from coroners’ investigations and its target audience is healthcare professionals, clinicians and managers.⁶

Box 1. History of the Clinical Communiqué (CC).

The CC was launched in September 2014 as a revamped version of the Coronial Communiqué. Established in 2003, the Coronial Communiqué represented the first serial electronic publication of narrative case reports about clinical lessons learnt from patient deaths investigated by the Victorian Coroner’s Office.⁷ Twenty-one editions were released before it went into a hiatus in 2009 due to resource constraints. In its current form, the CC is published quarterly and uses coronial cases from local, interstate and international jurisdictions to explore the challenges that clinicians face every day in providing clinical care. Each issue identifies key themes that are vital to improving patient safety such as communication, supervision, decision making and recognising the deteriorating patient. Many of the themes are reflected in the National Safety and Quality Health Service Standards.⁸ These are presented as case summaries and expert commentaries.

Strengths and limitations of this study.

• This study evaluates the effect that the Clinical Communiqué (CC) has on its readers in terms of practice change.

• There was a substantial number of respondents with over 1000 subscribers participating in the survey.

• The response rate (30%) is higher than that which is usually obtained in online surveys (20%).

• The self-reported change to practice of 53.0% is likely an overestimate; however, there is substantive corroboration with 109 subscribers (13%) agreeing ‘that if CC was not available, change would not have occurred’.

• A control group was not used in our study.

Aim

The primary aim of this study was to explore whether subscribers reported clinical practice changes as a result of reading the CC. It also compared the characteristics of subscribers who self-reported changes to clinical practice with those who did not, and explores subscribers’ perceptions of the educational value of the CC.

Method

Study design and setting

A population-based cross-sectional study was conducted, using an anonymous electronic survey distributed to all registered subscribers of the CC. A team from the Monash University, Australia, conducted the survey from July to September in 2015.

Survey instrument

The survey instrument was refined from previous studies to address the study objectives.^9–11 This involved the refinement of the questionnaire through the changing of phrasing, the addition of new questions and the modification of previous questions by researchers with extensive knowledge of the healthcare system to make clear the premise of each item within the questionnaire. The questionnaire was then piloted with 10 health professionals (five nurses and five doctors) and after reviewing the feedback, the survey was further refined. The final questionnaire was designed and distributed through the web-based application Survey Monkey.

The questionnaire consisted of 33 questions divided into three sections: respondent reading behaviour and evaluation of the CC (10 questions); the impact of the CC, including details of practice change (13 questions); and respondent characteristics (10 questions). A change in practice required the respondent to identify the following elements: location (eg, inpatient ward), discipline involved (eg, medical staff), nature (eg, education, policy, clinical care), edition of the CC that influenced change, impact on subject (eg, patient, staff) and action taken (eg, new initiative). The survey instrument consisted of closed-ended questions, of which six allowed respondents the opportunity to provide further detail if the ‘other’ option was selected from the multiple choices. The closed-ended questions were multiple choice, categorical, dichotomous and Likert-type questions with five-point rating scales.

Only respondents who reported changing practice after reading the CC were able to answer questions about changes in professional practice. No identifying data were collected.

Study population

The survey was sent out via email by study investigator through the MailChimp service to all registered subscribers of the CC at the time the study was conducted, and for whom an email address was available. A modified Dillman protocol¹² was used to guide subscriber participation. Subscribers were contacted directly via email and asked to respond to the survey on 21 July 2015. Two weeks later, a follow-up reminder email was sent to subscribers and a final reminder email was sent after a further 2 weeks. Respondent anonymity was maintained and the researchers blinded by using the web-based survey tool for collection and collation of data. The electronic system automatically identified the non-responders and reminder notices were only sent to subscribers who had not responded to the previous email(s). The Survey Monkey settings were set to refuse multiple responses from the same IP address.

Data analysis

Survey responses were analysed using Statistical Package for the Social Sciences (SPSS) V. 20.0 (SPSS, Chicago, Illinois, USA). Descriptive statistics were used to summarise information about respondents’ reading behaviour and preferences, the impact of the CC, the nature of self-reported changes to practice and characteristics of the respondents.

Responses to questions answered along an ordinal five-point Likert-scale were reported using the median and IQR, and dichotomised: ‘yes’ consisted of 5 (strongly agree) and 4 (agree), whereas ‘no’ consisted of 3 (neutral), 2 (disagree) and 1 (strongly disagree). The ordinal data were collapsed into dichotomous groups as a conservative approach to the analysis using non-parametric tests. Missing data were analysed using pairwise deletion.

Cronbach’s alphas were used to estimate the internal reliability of items relating to respondent opinion about CC content (six items) and influence on professional practice change (four items). Cronbach’s alphas were estimated using the entire sample of participants.

Bivariate analysis was used to compare the characteristics of respondents who self-reported change in practice with those who did not. Characteristics including respondents’ age, gender, professional role, years of experience, frequency of client interaction, practice setting, contact with the Coroners Court and reading behaviour were analysed using the χ² test. Multivariate logistic regression was used to analyse factors associated with readers who reported change in practice compared with those who did not. Factors were included in the regression analysis if p<0.25 in bivariate analysis¹³ or were thought to be important based on expert opinion of senior clinical medical and nursing staff with over 10 years’ experience in medico-legal death investigations. Reference categories were selected based on the category with the largest number of people as this would be the most robust denominator. Collinearities were identified using a pairwise correlations matrix with r>0.40 interpreted as evidence of collinearity.

Non-response bias was analysed by comparing demographical data of survey respondents with that of the ‘register to subscribe’ database of the CC. Since all subscribers were surveyed, differences between the two groups were used to derive non-respondent data. Non-respondent and respondent demographics (gender, age, professional role) were analysed by comparing proportions.

Ethics approval

Institutional ethics approval for this study to proceed was granted from the Victorian Institute of Forensic Medicine Research Advisory and Ethics Committee. Implicit consent for the project was considered when the study participants completed the survey.

Results

Of the 3385 listed subscribers, 3373 had valid email addresses. A total of 1008 individuals completed the survey giving a response rate of 29.9% (1008/3373). All respondents provided valid responses. Cronbach’s alphas demonstrated good internal consistency for opinion about CC content (0.93) and influence on practice change (0.87).

Respondent demographic and occupational characteristics

The demographics of the respondents are shown in table 1. Of the 1008 respondents, most were at or over the age of 45, the majority had worked for over 10 years and there were more females than males. The most commonly identified roles were medical practitioner or nurse. A large proportion worked in clinical roles in Victorian hospitals.

Table 1.

Demographic and occupational characteristics and respondent reading behaviour (n=1008)

Demographic and occupational characteristics	n (%)
Age (years)*
≤34	101 (11.6)
35–44	196 (22.5)
45–54	308 (35.3)
≥55	268 (30.7)
Gender*
Female	643(73.7)
Male	230(26.3)
Professional role*
Allied health professional	25 (2.9)
Medical practitioner	240 (27.5)
Nurse	277 (31.7)
Paramedic	24 (2.7)
Pharmacist	35 (4.0)
Quality and risk manager	99 (11.3)
Other	173 (19.8)
Experience in profession (years)*
≤5	183 (21.0)
6–10	130 (14.9)
11–15	115 (13.2)
16–20	86 (9.9)
≥21	359 (41.1)
Workplace setting*
Government department/agency	41 (4.7)
Hospital—acute	451 (51.7)
Hospital—subacute	47 (5.4)
Primary care	72 (8.2)
Residential aged care service	109 (12.5)
University or other academic	32 (3.7)
Other	121 (13.9)
State or country*
Victoria	549 (62.9)
Other state or territory of Australia	313 (35.9)
Other country	11 (1.3)
Frequency of patient interaction per week (days)*
<1	230 (26.3)
1	58 (6.6)
2 or 3	107 (12.3)
4 or more	478 (54.8)
Level of contact with Coroners Court*
Taken part in an inquest	148 (17.0)
Provided a statement	207 (23.7)
Contacted Court to discuss if death was reportable	338 (38.7)
Contacted Court for other reasons	289 (33.1)
No contact	299 (34.2)
Respondent reading behaviour and preferences
Respondent reading behaviour
Read all four issues†	466 (47.1)
Read all or almost all of each issue‡	747 (79.0)
Respondents regularly read the following sections§,¶
Case summaries	927 (99.3)
Expert commentary	916 (98.0)
Editorial	821 (88.1)
Resources list	525 (56.6)
Use of the Clinical Communiqué‡,¶
I read it as soon as it arrives	802 (84.9)
I encourage my colleagues to read it	681 (72.1)
I refer to it in my job	407 (43.1)
I use it as a teaching aid	590 (62.4)
Number of people with access to the Clinical Communiqué‡,**
Only myself	441 (46.7)
One other person	53 (5.6)
3 to 5 other people	124 (13.1)
6 to 10 other people	85 (9.0)
11 to 20 other people	73 (7.7)
21 to 30 other people	62 (6.6)
More than 31 other people	107 (11.3)

^*One hundred and thirty-five respondents failed to answer this question and were not included in the analysis (n=873).

^†Nineteen respondents failed to answer this question and were not included in the analysis (n=989).

^‡Sixty-three respondents failed to answer this question and were not included in the analysis (n=945).

^§The number of respondents varies for this question (n=934, 935, 932, 928) respectively.

^¶Five-point Likert scale, 5=strongly agree to 1=strongly disagree. Positive responses are counted as the sum of responses that stated ‘strongly agree’ and ‘agree’.

^**The reach of the Clinical Communiqué was calculated from these results. The lower (28 575) and upper ranges (30 958) were determined by multiplying the number of respondents with the extremes of their response. Respondents who answered ‘more than 31 other people’ to this question were given an option to provide the actual number and where available, this response was used in the calculation

NB. Because of rounding not all percentages cumulatively sum to 100

Respondent reading behaviour and preferences

The majority of respondents reported reading all or almost all of each CC issue and approximately half had read all four issues of the CC. Most read the CC as soon as it arrived, and a large proportion used it as a teaching aid. The section most often read was the case summaries (table 1).

Respondent evaluation of the CC

Table 2 shows the respondents’ opinions on the content and efficacy of the CC. A large majority agreed that the content of the CC was useful, reliable, timely, easy to understand and written in plain language. Respondents also found that the CC raised awareness and provided ideas about improving patient safety and care, leading them to discuss cases with their colleagues and review their practice. The case summaries and expert commentary sections were found to be the most useful in aiding the respondents to improve patient care. The length of the CC was regarded as ‘just right’ by most of the respondents, and almost all ‘would recommend the CC to their colleagues’ (table 2).

Table 2.

Respondent evaluation criteria (n=1008)

Respondent evaluation criteria	n	%	Median (25th-75th percentile)
Information given in the Clinical Communiqué is:*,‡
Useful	928	99.1	5 (4–5)
Reliable	907	96.9	5 (4–5)
Timely	728	77.8	4 (4–5)
Easy to understand	912	97.4	5 (4–5)
Written in plain language	917	98.0	5 (4–5)
The following sections of the Clinical Communiqué are useful for improving patient care:*,‡
Case summaries	922	98.5	5 (4–5)
Expert commentary	914	97.6	5 (4–5)
Editorial	799	85.4	4 (4–5)
Resources list	628	67.1	4 (3–4)
Reading the Clinical Communiqué†,‡
Raised awareness about improving patient safety and clinical care	912	96.5	5 (4–5)
Provided ideas for improving safety and clinical care	889	94.1	4 (4–5)
Prompted me to discuss cases with colleagues	752	79.6	4 (4–5)
Prompted me to review my practice	715	75.7	4 (4–4)
The length of the Clinical Communiqué is just right*	766	81.8	–
I would recommend the Clinical Communiqué to my colleagues*,§	927	99.0	5 (5–5)

^*Seventy-two respondents failed to answer this question and were excluded from analysis (n=936).

^†Sixty-three respondents failed to answer this question and were excluded from analysis (n=945).

^‡Five-point Likert scale, 5=strongly agree to 1=strongly disagree. Positive responses are counted as the sum of responses that stated ‘strongly agree’ and ‘agree’.

^§Five-point Likert scale, 5=definitely yes to 1=definitely not. Positive responses are counted as the sum of responses that stated ‘definitely yes’ and ‘probably yes’.

Nature and significance of self-reported change to practice

Four hundred and ninety-six (53.0%, 496/936) respondents reported that their practice had changed after reading the CC. Details regarding the nature of practice change are shown in table 3. Changes were involved in either individual, team-based, or workplace practices. The majority of change was reported to have occurred in staff, at inpatient wards or residential aged care (RAC) facilities, with respect to education and training and clinical practice, involving one or two discipline groups.

Table 3.

Details of self-reported change to practice (n=496)

Characteristics of practice change	n	%
Workplace*
Emergency department	89	21.2
Intensive care unit	42	10.0
Operating theatre	51	12.2
Inpatient ward	146	34.8
Outpatient unit primary care	47	11.2
Residential aged care facility	119	28.4
Other	92	22.0
Who was involved in the practice change?*
One discipline group (eg, doctors only)	157	37.5
Two discipline groups (eg, doctors and nurses)	188	44.9
Three discipline groups (eg, doctors, nurses and pharmacists)	48	11.5
Four or more discipline groups	38	9.1
Practice change was related to:*
Education and training	286	68.3
Policy, procedures and protocols	182	43.4
Clinical practice	283	67.5
Evaluation of care	142	33.9
Documentation of practice	205	48.9
Improving staff morale/attitude	122	29.1
Environment/equipment	65	15.5
Edition of the Clinical Communiqué which influenced practice change*
‘Communiqué cases and the National Health Service Standards’	124	29.6
‘Recognising early warning signs of the deteriorating patient’	341	81.4
‘Communication and decision making at the bedside’	253	60.4
‘Responding as a team to medical emergency’	191	45.6
The practice change had an impact on:*
Patient care	339	80.9
Staff	380	90.7
Environment	215	51.3
Organisation	268	64.0
The action taken in the practice change was to:*
Introduce a new initiative	71	16.9
Alter or modify an existing initiative	382	91.2
Discontinue an existing initiative	24	5.7
Number of initiatives where the Clinical Communiqué was used to change practice*
1	151	36.0
2	175	41.8
3	63	15.0
4	12	2.9
5 or more	19	4.5
Degree to which the Clinical Communiqué influenced practice change*,†
Selecting area of improvement	306	73.0
Defining the scope of the project	173	41.3
Engaging senior management	189	45.1
Engaging point of care staff	276	65.9
Identifying required resources	203	48.5
Setting project timelines	76	18.1
Gathering background research/analysis	227	54.2
The Clinical Communiqué influenced practice change:*,‡
By raising awareness of an issue	409	97.6
By engaging staff	354	84.5
Because of the authority of the publication	310	74.0
By suggesting improvement strategies	359	85.7
By prompting staff to evaluate their existing practice	369	88.1
If the Clinical Communiqué had not been available, the change would have:*,‡
Happened anyway	129	30.8
Taken longer	347	82.8
Happened on a smaller scale	236	56.3
Not occurred	109	26.0

^*Seventy-seven respondents failed to answer this question and were excluded from analysis (n=419).

^†Five-point Likert scale, 5=strongly agree to 1=strongly disagree. Positive responses are counted as the sum of responses that stated ‘strongly agree’ and ‘agree’.

^‡Five-point Likert scale, 5=extremely influential to 1=not at all influential. Positive responses are counted as the sum of responses that stated ‘extremely influential’ and ‘very influential’.

Approximately one quarter of respondents reported that without the CC, change would not have occurred. A larger proportion agreed that the change would have taken longer and occurred on a smaller scale (table 3).

Characteristics and factors associated with respondents who reported change

In bivariate analysis, respondents’ age, professional role and workplace location were found to be significantly associated with practice change (p<0.05) (see online supplementary table S1).

Multivariate analysis indicated that working in an RAC facility and having taken part in an inquest were significantly associated with practice change (p<0.05). Conversely, medical practitioners and pharmacists under the age of 35 years, working in workplaces with minimal patient contact (<1 day per week) who had only read two or less CC issues, were less likely to change their practice (p<0.05). The multivariate logistic regression model had a Pearson’s χ² of 696.7 (degrees of freedom (df)=683, p=0.35) and a c-statistic of 0.72, indicating that it was a good model (c-statistic between 0.70 and 0.80).¹³

Discussion

This population-based cross-sectional study found the CC prompts readers to initiate change in their professional practice to improve patient safety. With a response rate at the higher end of the predicted average rate of 20%–30% for online surveys, and higher than what has been described to meet stringent conditions for response rates when sampling large groups,¹³ this study adds to the existing literature^{9–11 15} on the value of electronic case summaries and commentaries in reaching and influencing the practice of healthcare professionals.

We are not aware of any other similar publications to the CC, apart from our own, that have had a formal evaluation. The impact on changing practice is consistent with our previous studies^9–11 and substantially greater than those reported in a recent Cochrane review¹⁵ which found a small benefit and highlighted significant variability in printed and electronic educational materials’ (PEM) impact on practice.

There are many recognised benefits of PEM. The material is an effective and low cost method of raising awareness. The format and layout can be tailored to appeal to a particular audience to inspire behaviour change. PEM is most effective when combined with other methods, such as using the influence of opinion leaders to disseminate information.¹⁶

The majority of respondents were clinicians who had worked for more than 10 years, suggesting that the CC appeals to junior staff in the early phases of their career path, and to experienced staff in senior roles. The benefits of having a readership that spans all levels of seniority in a healthcare setting are that change can take place at the frontline, in the day-to-day aspects of patient care and also at the policymaking end by staff in positions of clinical and organisational leadership.

The reading behaviour of the respondents showed that there was an overwhelmingly positive and immediate response to the release of each CC issue. Many respondents forwarded the CC to their colleagues, or printed and distributed the issue in communal areas, leading to an estimated reach of more than 28 000 people (box 2). Clearly, not all who read the CC will become subscribers; however, the scale of opportunity to raise awareness and provide ideas about improving patient safety should not be underestimated. It is the very nature of raising awareness that may impart the greatest benefit in influencing practice change. In this study, 97.6% (409/419) of respondents who self-reported practice change selected ‘raising awareness of an issue’ as directly contributing to the change.

Box 2.

As the Clinical Communiqué (CC) is delivered via email, it is difficult to determine exactly how many people are being exposed to the contents of the CC. Therefore, by asking respondents how many people have access to the CC at their workplace, it was calculated that for this sample of 1008 respondents, the CC reached between 28 575 and 30 958 people. Therefore, each person on average shared the CC with at least nine other people.

Limited information is available from quantitative studies to understand the relationship between education and change in clinical practice. Education provides one-sixth of the reasons for changes in clinical practice, and is involved in one-third of the changes.¹⁷ In this study, 53.0% (496/936) of respondents reported a practice change after reading the CC. More than 90% of respondents reported that the CC provided ideas about improving patient safety and care and 75.7% (715/945) of respondents had reviewed their practice. This compares favourably to an earlier study where just under half (41.6%, 290/697) of responding subscribers changed their practice.¹⁰

The CC does not provide rigid guidelines. Instead it provides information that is relayed in the classic human narrative form of storytelling. This is non-hierarchical and the lessons are presented in context. This form of PEM is thought-provoking, and it stimulates dialogue between colleagues. 62.4% (590/945) of respondents reported using the CC as a teaching aid, suggesting they not only read it for personal interest but find the cases and topics highly relevant to their work, or their work setting, team or students, and view the material as an appropriate educational resource.

Reading individual cases of patient deaths engenders self-reflection and self-assessment, and encourages healthcare professionals to ask: can this happen to my patient?

An important finding was that a quarter of all respondents (26.0%, 109/419) attributed a practice change solely to reading the CC. Of those that felt a practice change would have occurred eventually, the majority acknowledged that the change would have been smaller and taken longer.

The survey questions did not explore the extent to which ‘practice change’ was conceived. Instead, this study focused on more tangible outcomes such as projects and initiatives as indicators of practice change. Subtle changes such as a change in communication or simply reflecting on a case during a patient interaction were not asked about, but could occur on a much more frequent basis and may influence practice in ways that are harder to measure.

The multivariate analysis showed that RAC staff were more likely to make practice changes. This was an unexpected finding and may have reflected greater familiarity that some RAC staff have with utilisation of the CC, through its sister publication, the Residential Aged Care Communiqué (now in its 10th year of publication) and its potential to facilitate change. The findings that medical practitioners and pharmacists under the age of 35 years and working with minimal patient contact were less likely to change their practice as a result of reading the CC may suggest that the educational relevance was not as applicable to their working roles and environment. The cases were predominantly set in hospital wards, and the key messages were largely centred around staff tasked with responding to an acutely unwell patient.

A limitation of this study is that subscribers were asked to participate and discuss practice change after only four issues had been released. Clinical guidelines can take up to 3 years to be fully implemented.¹⁶ There are many challenges to the development of clinical guidelines. Failure to address the key areas of funding, clinical involvement, conflicts of interest, intended setting or audience can all hamper the implementation of guidelines that inform practice change to improve patient care.¹⁸

Another limitation of this study is that the scope did not allow the investigators to examine in detail the changes that were reported, or whether the practice changes sustained over time. Therefore, quantifying the risk reduction and lives saved is not possible.

Furthermore, the response rate (29.9%) leaves potential for non-response bias where non-respondents could have characteristics that differ from survey respondents. Analyses for non-response bias (see online supplementary table S2) reveal that the proportions of survey participants were greater for the older age group (>55 years) and for respondents who identified as quality and risk managers. This is likely to contribute to an overestimate of effect of the impact of PEM. If we assume all non-respondents did not change practice, the estimate of impact is still large at 14.7% of all subscribers (496/3373).

In addition, pairwise deletion can be a source of bias as there may be a non-random pattern of missing data.¹⁹ Also, control groups were not used.

Conclusion

Investigations into patient deaths identify preventable errors and enhance knowledge of the many risks to patients that exist in the healthcare setting. The design and content of the CC has generated a positive impact on the healthcare community. It is presented in a format that appears to be accessible and acceptable to readers and achieves its goals of promoting safer clinical care through greater awareness of the medico-legal context of practice.