MacRae 2004.
| Methods | Trial design: RCT Power calculation: N Funding: Physician's Services Incorporated, Ethicon and Ethicon Endosurgery Ethics approval: Y Lost to follow‐up: 2 |
|
| Participants | Characteristics of participants: active members of the Canadian Association of General Surgeons who had access to the Internet and to e‐mail, and agreed to be randomised, would complete a written exam Exclusion criteria: surgeons with postgraduate training in clinical epidemiology No. randomised: 81 No. analysed: 55 Other characteristics: age: not reported; sex: not reported Setting: Internet‐based Country: Canada |
|
| Interventions | Intervention group: participants in the intervention group received 8 packages (of 2 articles from 1 October 2001 to 1 May 2002) and questions designed to guide critical appraisal. 1‐week listserve discussions centred on methodology of articles, a comprehensive methodologic review of the article prepared by a surgeon with training in clinical epidemiology and a clinical review were also provided. Articles chosen to be of clinical interest to the general surgeons and the methodological article that directly related to the clinical article. This was an internet‐based intervention. Theory‐based intervention: Y (based on adult learning theory and findings from continuing education literature) Control group: received the 8 packages (of 2 articles), clinical articles and had online access to major medical and surgical journals, some of which included articles on critical appraisal Intervention deliverer: sessions moderated by a general surgeon with training in clinical epidemiology at the level of a master of science degree |
|
| Outcomes | Outcomes measured: (locally developed) critical appraisal used previously on a group of general surgery residents (internal consistency Chronbach alpha 0.77) with evidence of face, content and construct validity. Inter‐rater reliability of 2 physicians marking the examination was 0.93. Participants completed rating scales on aspects of study quality and gave free‐text responses to questions about methodology, the validity and applicability of results. Two general surgeons developed an answer key and all answers were marked by a single clinical epidemiologist. Distal follow‐up interval: immediately following intervention. Examinations were mailed to participants in both groups with an e‐reminder 3 weeks later. Asked to submit within 6 weeks. Losses to follow‐up: 26 N randomised: 44 IG, 37 CG N completing follow‐up: 26 IG, 29 CG Reasons for loss to follow‐up: 3 were excluded prior to randomisation, and another 2 were excluded after randomisation (1 in each group) because they had previous clinical epidemiology training Fewer of the experimental group participants (26 surgeons, 58%) than the control group (29 surgeons, 76%) completed the examination. This difference may have been due to trial fatigue, in that we asked the experimental group to complete much more material on a monthly basis. Toward the end of the 8 months, the average number of returned evaluations also decreased, again suggesting the possibility of trial fatigue. (p. 644) |
|
| Notes | ||
| Risk of bias | ||
| Bias | Authors' judgement | Support for judgement |
| Random sequence generation (selection bias) | Low risk | Participants were stratified first by type of practice (academic versus community) then by year of graduation. Within type of practice, participants were blocked into groups of 10 based on year of graduation. This process ensured random assignment within the constraints (1) that exactly half of the academic and exactly half of the non‐academic participants who started the study were assigned to each condition and (2) that, within academic and non‐academic groups, year of graduation would not be confounded with condition. (p.642) |
| Allocation concealment (selection bias) | Unclear risk | Not reported |
| Blinding (performance bias and detection bias) All outcomes | Low risk | The examinations were all marked by a single clinical epidemiologist. All identifying data were blacked out on the examination prior to marking by the research assistant. The control and intervention group examinations were mixed and then marked in random order by a clinical epidemiologist. (p.643) |
| Incomplete outcome data (attrition bias) All outcomes | High risk | Not reported |
| Selective reporting (reporting bias) | Low risk | Expected outcomes reported |
| Other bias | High risk | The major potential bias that may have affected our result was that participants were recruited on a volunteer basis, and are thus more motivated, and perhaps more likely to benefit from the intervention than the general population of surgeons |