A systematic review of reporting quality for anaesthetic interventions in randomised controlled trials

Summary

Interventions from randomised controlled trials can only be replicated if they are reported in sufficient detail. The results of trials can only be confidently interpreted if the delivery of the intervention was systematic and the protocol adhered to. We systematically reviewed trials of anaesthetic interventions published in 12 journals from January 2016 to September 2019. We assessed the detail with which interventions were reported, using the Consolidated Standards of Reporting Trials statement for non‐pharmacological treatments. We analysed 162 interventions reported by 78 trials in 18,675 participants. Detail sufficiently precise to replicate the intervention was reported for 111 (69%) interventions. Intervention standardisation was reported for 135 (83%) out of the 162 interventions, and protocol adherence was reported for 20 (12%) interventions. Sixty (77%) out of the 78 trials reported the administrative context in which interventions were delivered and 36 (46%) trials detailed the expertise of the practitioners. We conclude that bespoke reporting tools should be developed for anaesthetic interventions and interventions in other areas such as critical care.

Introduction

Healthcare providers rely on high‐quality randomised controlled trials to assess the effectiveness of interventions. Interventions have to be precisely reported to enable their appropriate implementation and to facilitate evidence‐based decisions with patients. The Consolidated Standards of Reporting Trials (CONSORT) statement, now endorsed by funders and many journals, makes recommendations to improve the quality of reporting in clinical trials [1].

An extension to the CONSORT statement for non‐pharmacological treatments (CONSORT‐NPT) has been developed for complex multi‐component interventions (online Supporting Information Appendix S1) [2, 3]. The CONSORT‐NPT guideline stipulates that “precise details for each intervention should be reported, including how and when they were actually administered” and “a description of the different components of the interventions”, as well as “details of whether and how the interventions were standardised and how adherence of care providers and participants to the protocol was assessed or enhanced”.

The quality of reporting by trials for anaesthetic interventions has been poor, as judged by previous CONSORT statements [4, 5, 6, 7]. An important aspect of complex interventions is how their delivery is standardised and how adherence to protocol is assessed. The lack of consensus definitions and standardised regimens might contribute to poor reporting [8, 9]. The subsequent development of the CONSORT‐NPT guideline should facilitate the replication of interventions and promote their reliable clinical implementation, in keeping with the Medical Research Council’s framework [10].

Different modes of anaesthesia facilitate invasive procedures [8, 11]. The comparison of general anaesthesia vs. regional or local anaesthesia has been ranked as a research priority [12]. Anaesthetic mode is a typical complex intervention for which the CONSORT‐NPT extension is intended [8, 13, 14]. We do not know of any study that has assessed the quality with which mode of anaesthesia has been reported, as judged by CONSORT‐NPT. We aimed to systematically review the quality that randomised controlled trials of mode of anaesthesia reported the intervention.

Methods

We prospectively registered the systematic review [15, 16]. Two researchers independently screened the content pages of 12 English language journals for randomised controlled trials published from 1 January 2016 to 1 September 2019. We chose journals with the highest 2018 Scimago Journal and Country Ranks for: anaesthesia (Anesthesiology; British Journal of Anaesthesia; Regional Anesthesia and Pain Medicine; Anaesthesia; European Journal of Anaesthesiology; Anesthesia and Analgesia); general medicine (New England Journal of Medicine; The Lancet; Journal of the American Medical Association); and general surgery (Annals of Surgery; British Journal of Surgery; Journal of the American Medical Association Surgery) [17, 18, 19].

We included trials that tested anaesthetic interventions during any invasive procedure [20]. The comparisons included general anaesthesia vs. regional anaesthesia and volatile vs. intravenous general anaesthesia. We excluded trials that compared analgesic interventions and cadaveric, laboratory or animal studies. Two researchers independently recorded trial characteristics and we contacted the corresponding author for protocols that had not been published (Clarivate Analytics. EndNote Referencing Software. 2020) [21, 22].

We used 23 items to assess whether included trials met CONSORT standards for reporting multiple‐component interventions [2, 3]. We categorised the detail used to describe interventions as: ‘none’; ‘some’; or ‘precise’ (item 5). We assessed descriptions of the dose, volume, concentration, route and timing of drugs for anaesthesia and sedation. For general anaesthesia these included induction and maintenance, neuromuscular blockade and opioids. For local anaesthetics we also assessed site and, where applicable, sensory and motor blocks. We assessed descriptions of depth of anaesthesia and sedation, airway management and anaesthetic monitoring. We used the training grade to characterise ‘operator expertise’ for anaesthetists (item 15). We defined ‘standardisation’ as a process “to establish a standard consisting of regulations for how something is to be done” (https://www.collinsdictionary.com/about). We recorded how standard anaesthetic protocols were regulated and why and whether personnel adhered to them [3, 23]. We recorded crossover of participants between trial arms. We did not assess the adherence of participants to the intervention (item 5d).

We used the revised Cochrane risk of bias tool to assess: the randomisation process; deviations from the intended interventions; missing outcome data; measurement of the outcome; and selection of the reported result [24]. We did not undertake formal statistic comparisons, in keeping with similar systematic reviews of reporting standards in other research areas [25, 26].

Results

We included 78 trials that reported 162 anaesthetic interventions in 18,675 participants: 79 general anaesthetic; 21 sedation; 59 regional; and 3 local (Fig. 1 and online Supporting Information Appendix S2). The six anaesthetic journals reported 143 (90%) interventions in 69 trials, the three medical journals reported 15 (9%) interventions in seven trials, and the three surgical journals reported 4 (3%) interventions in two trials (online Supporting Information Table S1). We accessed protocols for 19 (24%) trials; 14 were publicly available and five were sent by authors on request. The authors of 59 trials did not reply.

Figure 1.

Study flow diagram.

We categorised the detail with which interventions were described as: none in 19/162 (12%); ‘some’ in 32/162 (20%); and ‘precise’ in 111/162 (69%) (Table 1). Some detail was reported for all allocated groups by 29 trials (37%), while four (5%) trials did not describe any detail for any trial arm.

Table 1.

The detail for 162 anaesthetic interventions published in 78 randomised controlled trials from 1 January 2016 to 1 September 2019 in 12 journals. References are detailed in online Supporting Information Appendix S2.

Description detail	Text
General anaesthetic (n = 79)
None (n = 13)	‘anaesthesia was induced … with the … method chosen at the discretion of the patient's attending anaesthetist’ [S21]
Some (n = 27)	‘induction … through titration … to achieve loss of responsiveness to verbal command […] maintained using volatile anaesthetics … Neuromuscular block … using a non‐depolarising … agent […]’ [S36]
Precise (n = 39)	‘Anesthesia induction was performed with IV fentanyl (5 µg/kg) and propofol (2 mg/kg), and rocuronium bromide (0.6 mg/kg) was used as a neuromuscular blocker…. Additional doses of fentanyl (2 µg/kg) and rocuronium (0.2 mg/kg) were used when necessary.’ [S57]
Sedation (n = 21)
Some (n = 2)	‘a ketamine infusion and a TCI of remifentanil titrated to maintain a pain VAS equal to or less than 30 mm’ [S51]
Precise (n = 19)	‘10‐μg/mL dexmedetomidine … or 0.375‐mg/mL midazolam … or normal saline in an unlabelled 20‐mL syringe … administered intravenously … at … kg (weight of patient) * 0.6 mL/h for a 10‐minute period and then at … kg (weight of patient) * 0.12 mL/h … until the end of surgery.’ [S73]
Regional (n = 59)
None (n = 6)	‘The anaesthetist then performed the CSE based on the ultrasound‐derived markings’ [S16]
Some (n = 2)	‘Needle placement and injection of LA were guided by ultrasonography. A linear‐array high‐frequency transducer probe was used. The tip of the needle was positioned in the interscalene groove posterior to cranial root 5 or the superior trunk of the brachial plexus at the level of the cricoid cartilage using an in‐plane lateral to medial approach through the middle scalene muscle using a 22‐gauge, 2 3/8 inch Chiba needle.’ [S46]
Precise (n = 51)	The ultrasound probe was positioned under the clavicle, medial to the coracoid process, in a parasagittal plane … in‐plane … 20‐gauge 8.89‐cm Tuohy needle was advanced to the posterior side of the axillary artery (6‐o’clock position) until a fascial click … then 30 mL of 1.5% mepivacaine was injected, with … a crescent‐shaped distribution around the artery.’ [S11]
Local anaesthetic (n = 3)
Some (n = 1)	‘a mixture 1:1 of bupivacaine 5 mg/ml and mepivacaine 10 mg/ml, was administered’ [S75]
Precise (n = 2)	‘preperitoneal instillation … total of 30 mL of 0.5% bupivacaine and 8 mg of preservative‐free dexamethasone … into 2 areas (left and right “triangle of pain”)’ [S65]

Eighteen (23%) trials did not report how many hospitals participated, 45 (58%) trials were conducted in one hospital, and 15 (19%) were conducted in multiple hospitals. Forty (51%) trials did not report the type of hospital, 35 (45%) trials were conducted at teaching hospitals and 3 (4%) elsewhere. Invasive procedures were scheduled in 75 (96%) trials, emergency in 2 (3%) trials and either in 1 (1%) trial. Who delivered the anaesthetic interventions was not reported by 42 (54%) trials. Interventions were delivered by consultant anaesthetists in 26 (33%) trials, consultant or trainee anaesthetists in 9 (12%) trials, and by nurses in 1 (1%) trial. The most common types of procedure were orthopaedic in 24 (31%) trials, cardiothoracic in 10 (13%) trials and colorectal in 7 (9%) trials. Countries are listed in online Supporting Information Table S2. Standardisation of interventions and adherence to protocol are illustrated in Table 2. Some participants did not receive the allocated intervention in 37 (47%) trials. No trial reported participant crossover to the unallocated intervention.

Table 2.

Examples of standardisation for 135/162 (83%) interventions and examples of how these were adhered to for 20/135 (15%). References are detailed in online Supporting Information Appendix S2.

Standardising anaesthetic interventions
Kim 2019 [S40]	Supplementary photographs and video
YaDeau 2018 [S76]	Instruction checklist
Luo 2017 [S45]	Ultrasound images detailing site and method
Forster 2018 [S30]	‘Intravenous sedation was standardised and performed by [one] anaesthetist’
Chin 2018 [S16]	‘five study investigators … reviewed each other’s performance before the start of the study; post hoc analysis demonstrated similar success rates’
Adherence to protocol
Maalouf 2016 [S46]	‘A printout scan of the USG block at the level of the cricoid cartilage and verification of needle placement in relation to the plexus was taken for each block.'
Sieber 2019 [S68]	‘The Protocol Deviation Log (Form 50) will be used to document all information on protocol deviations, including deviations in administration of the study treatment’
Auyong 2017 [S6]	‘A video camera was used to record the process of epidural placement, and a blinded investigator reviewed the video at a later time. The time to LOR, number of needle passes, and needle skin punctures attempted were interpreted and documented based on the video recording.'

We categorised the risks of bias as low, medium or high in 30 (38%), 34 (44%) and 14 (18%) trials, respectively (online Supporting Information Table S3).

Discussion

We classified the precision of about two‐thirds of reports sufficient to allow replication of anaesthetic interventions, using the CONSORT‐NPT framework.

The precision with which complex interventions – for instance surgical – are described may permit replication of only about one third [25, 27]. However, classification of report precision with the CONSORT‐NPT guideline is ironically subjective, as it too uses language imprecisely. For instance, the guideline does not define ‘precise’, ‘standardisation’ or ‘adherence’. The two researchers who independently assessed reports in this paper recognised this subjectivity and erred towards generous assessments of descriptive detail. We chose the CONSORT‐NPT framework because it has been recently updated and it encompasses aspects of other tools [3, 28]. A number of other policies have been developed to improve compliance with the reporting of multiple‐component interventions and the standardisation and assessment of adherence to protocol within trials [10, 29, 30].

We limited our systematic review to three years of randomised controlled trials published in 12 journals. We think our findings would be replicated if similar anaesthetic interventions were analysed in trials published recently in other journals, although our findings might be systematically biased by our selection of journals with relatively high impact factors. However, we think that one should not assume that similar rates of compliance with reporting standards apply to interventions that we did not review, such as those employed in peri‐operative medicine, critical care, pain medicine, palliative care and pre‐operative assessment. More extensive systematic reviews will need to develop sensitive search strategies for electronic databases, as reading hundreds of thousands of abstracts would not be feasible [25].

Poor reporting of interventions in trials complicates pooling their results, as heterogeneity in effect might be due to unreported confounding factors interacting with the effect of the intervention [31]. Clinicians will be uncertain how to implement an intervention and whether to expect effects like those reported by the trials. We think that bespoke reporting tools should be developed for anaesthetic interventions, and presumably interventions in other areas such as critical care.

Supporting information

Appendix S1. Sections 4b, 5 and 15 of the 2017 Consolidated Standards of Reporting Trials (CONSORT) checklist of information relating to the delivery of an intervention to be included when reporting a randomised controlled trial assessing non‐pharmacologic treatments.

Appendix S2. References for randomised controlled trials.

Table S1. Reporting of mode of anaesthesia interventions in each included trial.

Table S2. Study baseline characteristics.

Table S3. Risk of bias in each included trial.