Table 3. Effectiveness of CDSS: process outcomes—guideline adherence.

Study	Risk of bias	Practical aspects of CDSS use	Process outccomes	Clinical outcomes	Interpretation
Eccles et al.21	Low	For both groups the median number of active interactions was zero. The number of alerts was approximately zero	No significant difference in drugs prescribed for asthma before and after introduction of CDSS. No significant difference in lung function assessment before and after OR 0.94 (0.67–1.33)	Overall effect of the CDSS on symptom score was non-significant: the parameter estimate from analysis of covariance of scale was −0.62 (95% CI is −2.12 to 0.88).28 No effect on quality of life was measured on the validated Juniper’s Asthma Quality of Life Questionnaire (AQLQ).30 No differences in GP visit rate; OR=0.94 (0.81–1.08)	The design of this British study incorporated two arms, each controlling for the other. The study was a cluster design, with practices as the unit of randomisation. Practices investigating CDSS-driven care for angina provided usual care control data for the asthma CDSS care practices, and vice versa. In addition, the study was very large, with 62 practices across the UK, and so results should have been robust. This trial demonstrated very clearly that CDSS will not be used by clinicians if it is not integrated with their usual workflow. The median usage of the CDSS in this study was zero and there were no differences in consultation rates, process outcomes or clinical outcomes, which were carefully measured.
Martens et al.24,28	Medium	GPs did not have a choice to decide if the CDSS was to be activated	44% of the intervention group were prescribed according to the recommendations compared with 27% of the control group among patients with mildly persistent asthma	No clinical outcomes reported	This Dutch study consisted of 14 general practices in a cluster randomised controlled trial. As in the Eccles study, two arms of the study acted as controls for each other. One arm was given a CDSS to guide on antibiotic, asthma and COPD prescribing, and the other received CDSS for cholesterol prescribing. This design minimises the impact of performance bias and the Hawthorne effect and has therefore contributed to it being rated as only at moderate risk of bias. The study was underpowered (the actual variation was larger than values used to estimate study power), which may have contributed to the non-significant results.
Bell et al.20	High	No difference between groups in the rate at which the CDSS was used (70% of the time during the intervention periods)	Controller medication prescribed more often in urban intervention practices compared with urban control practices (P=0.006) and NSD in suburban practices. Increase in spirometry in intervention sites from 15 to 24% and decrease in control sites from 8 to 1% (P=0.003). Number of asthma plans filed in suburban intervention practices increased compared with suburban controls (P=0.03). NSD in urban practices	No differences in GP visits	Although this US study was graded at high risk of bias, it did have a recognisable cluster design in which steps were taken to try to randomise the baseline differences of poverty and ethnicity in the different urban versus suburban practices. This study demonstrated that CDSS could improve the adherence to guidelines for prescribing, test ordering and use of asthma action plans. No clinical improvements were measured or reported in this trial and a major confounder was the introduction of asthma care-related pay-per-performance incentives during the time period of this trial (though this applied to both groups).
Fiks et al.22	High	Influenza vaccine alerts were active at only 27% of visits	Vaccination rates increased by 3.8% at control practices and by 4.8% at intervention sites	No differences in GP visits	This American study investigated the impact of CDSS for reminding clinicians to give children with asthma an influenza vaccination. The rate of increase in vaccination was not significantly different across the control and intervention groups as the rate increased in both groups. In interpreting this study it should be remembered that there are many influences on the uptake rate of vaccination, including whether a child is acutely unwell or not at the time they attend the clinic, and the health beliefs of the child and their parents.
Kuilboer et al.23	High	The doctor waited for the result of the CDSS analysis in 22% of 10,863 visits. 10,532 comments were produced and 32% of these were read by doctors. The CDSS took on average 31.7 s to analyse the record. The median time spent by the doctor reading comments was 9 s (25th percentile=4 s, 75th percentile=48 s)	Some evidence for a decrease in cromoglycate prescriptions in one of four age brackets, but no other significant changes. More tests were ordered among the CDSS group, but this difference was not always significant	No differences in GP visits except in one of the four age brackets, but risk of multiple testing	This trial provides some evidence of the effectiveness of CDSS in terms of its impact on guideline adherence. There were appreciable increases in the ordering of peak expiratory flow rates and spirometry. In addition, there was some evidence that doctors were more likely to change their prescribing of cromoglycate with the CDSS; however, there were no changes for the other drugs in the guideline (deptropine, antihistamines and oral bronchodilators)—probably because the general practitioners rarely prescribed these drugs anyway. Also measured were changes in the coding of the record: doctors recorded more data in a more structured fashion. It was reported that only a third of the comments were read by doctors. The explanation for this may be that the CDSS provided asthma-related comments irrespective of the reason for the visit.
McCowan et al.25	High	Usually less than 10 min to fill in the template and generate the advice according to a nested study	There was no difference in the proportions of patients in the different categories of maintenance prescribing according to the British asthma guidelines. No difference in PEFRs ordered. No difference in proportion with action plans	Reported no significant differences in asthma symptoms between the intervention and control groups (odds ratio 0.31, 95% CI, 0.03–3.32) In the CDSS intervention group, 12/147 patients had exacerbations and in the control group 57/330 patients had exacerbations; OR=0.43 (95% CI, 0.21–0.85) after adjusting for clustering. Therefore control patients were approximately twice as likely to experience an exacerbation as intervention patients Significantly fewer patients initiated GP consultations in the intervention group; OR 0.59 (0.37–0.95) No difference in emergency department visits: OR=0 (0–9.16) No difference in asthma hospitalisations; OR=0 (0–3.44)	From an initial 46 UK practices who registered to undertake the trial only 12 control practices and 5 intervention practices completed the trial. A significant number from the intervention practices had problems installing and using the software at the trial initiation. The CDSS was apparently partially effective in that there were significantly fewer exacerbations of asthma among intervention patients. However, the majority of outcomes (symptoms, inhaler technique and measurement of peak flow) were not statistically significantly different between control and intervention arms. This is on the basis of those who completed the trial; the data were not analysed by intention-to-treat analysis.
Plaza et al.26	High	Not reported	17.9 of control and 34% of intervention patients conformed to strict treatment guidelines (Wilcoxon P=0.0240). No difference in spirometry rates, X-rays allergy or blood tests	The number of patients with symptoms during the day in the intervention group was significantly less than that in the control group (Wilcoxon P<0.02). There was no difference between the groups in terms of nocturnal symptoms (Wilcoxon P=0.1). Exacerbation rates were not significantly different between the control and the intervention groups. Quality of life reported using the validated Spanish version of the St George’s Respiratory Questionnaire (SGRQ)31 showed significant improvement by more than the minimally important difference of four points in all domains (activity P=0.002, symptoms P=0.003, impact P=0.001) No difference in GP visits (P>0.1) No difference in emergency department visits (P=0.0888) No difference in asthma hospitalisations (P>0.1)	This Spanish study reported randomising groups (clusters) to either the intervention or the control arm. It was a small study with only 10 doctors in each arm. There were two components to the intervention: the CDSS and an asthma education programme for nurses based on the GINA guidelines. This study produced significant improvements in the measures of the St George’s quality of life questionnaire. Daytime symptoms and exacerbations also improved but night-time symptoms did not. This study clearly demonstrated a link between significantly higher prescribing in the intervention arm of long-acting beta-agonists (especially formoterol) and leukotriene antagonists as per the guidelines and improved short-term outcomes (within 6 months). There was no significant difference in the rate of prescribing of inhaled steroids, oral steroids, anticholinergics or cromoglycate. This intervention was applied over a winter to spring period, which may have been a confounding factor in a seasonal condition such as asthma.
Tierney et al.27	High	87–95% of consultations resulted in the generation of a suggestion; doctors complied with only 32–37% of suggestions	5–9% of patients received the suggestion to ‘start inhaled corticosteroids.’ 11–0% of clinicians who received this suggestion adhered to it. Pulmonary function tests: 6% of the 39% in the control group and between 6 and 12% of 40–50% in the three intervention groups who received the suggestion adhered to it	The authors reported that patients with asthma in the pharmacist intervention arm of the trial had significantly (P<0.05) improved scores in the emotion subscale and that this was the only significant result following analysis of covariance of quality of life scores. It seems likely that this result may be significant only as a result of multiple testing No difference in emergency department vists or asthma hospitalisations	This study had four arms: one control and three intervention. The intervention arms consisted of physician CDSS intervention, pharmacist CDSS intervention and both physician and pharmacist intervention. There were no significant differences between the four study groups in adherence to the care suggestions. However, the care suggestions were also generated for the control patients—only that they were on paper, not on the computer. Adherence to care suggestions for the control arm varied from 9 to 71%. Adherence to care suggestions for the physician and pharmacist arm was from 12 to 91%. Overall, there was no clear pattern. It may be surmised that as the adherence to suggestions was very variable and frequently less than 50% this may explain why no significant differences were found in the quality of life and asthma control questionnaires.

Figures in brackets represent 95% confidence intervals.

Abbreviations: CDSS, computer decision support system; CI, confidence interval; COPD, chronic obstructive pulmonary disease; GINA, The Global Initiative for Asthma; GP, general practitioner; NR, not reported; NSD, no significant difference; OR, odds ratio.