Scoring system for methodological adequacy of randomised controlled trials, controlled trials, and controlled before and after studies Sample formation 2=Random allocation 1=Quasi-random allocation 0=Selected, concurrent, historical Baseline differences 2=None or adjusted 1=Differences unadjusted 0=No statement Unit of allocation 2=Practice or clinic 1=Doctor or nurse 0=Patient or family Outcome measures 2=Objective or blind 1=Subjective or not blind 0=No explicit criteria Follow up rate 2=>90% of subjects 1=80-90% of subjects 0=<80% of subjects |
Criteria for methodological adequacy of non-experimental studies Case report Source of case (1=identified; 0=not identified) Description of system (1=information; 0=no information) Validity of measures* (1=statement; 0=no statement) Quality control (1=statement; 0=no statement) Subject compliance rate (1=information; 0=no information) Post-intervention data (1=given; 0=not given) Case series Source of cases (1=identified; 0=not identified) Inclusion/exclusion (1=statement; 0=no statement) Sampling method (1=given; 0=not given) Description of system (1=information; 0=no information) Validity of measures (1=statement; 0=no statement) Lost to follow-up (1=information; 0=no information) Cross sectional study Research question (1=statement; 0=no statement) Source of cases (1=identified; 0=not identified) Inclusion/exclusion (1=statement; 0=no statement) Sample size (1=statement; 0=no statement) Dealing with bias (1=information; 0=no information) Analytic methods (1=description; 0=no description) Cohort study Research question (1=statement; 0=no statement) Source of cases (1=identified; 0=not identified) Inclusion/exclusion (1=statement; 0=no statement) Non-response rate (1=statement; 0=no statement) Starting point (1=definition; 0=no definition) Description of system (1=information; 0=no information) Case-control study Research question (1=statement; 0=no statement) Source of cases (1=identified; 0=not identified) Source of controls (1=identified; 0=not identified) Inclusion/exclusion (1=statement; 0=no statement) Sampling method (1=given; 0=not given) Comparability with controls (1=statement; 0=no statement) Before and after study/time series Research question (1=statement; 0=no statement) Source of cases (1=identified; 0=not identified) Inclusion/exclusion (1=statement; 0=no statement) Sample size (1=statement; 0=no statement) Starting point (1=definition; 0=no definition) Validity of measures (1=statement; 0=no statement) *If there was an explicit statement of validity or measures had face validity, they were given 1. HIf "to evaluate" or "report the impact" was stated but no details were provided, this was given 1. ICondition for patients; clear statement of study background, or origins of evaluation for practitioners. |
Study and design | Research topic(s) | Participants | Effect on performance | Methodology score | Overall score (out of 10) |
Gehlbach et al (1984)w1 Randomised controlled trial | Computerised feedback, generic prescribing | 32 doctors; 3702 scripts | Median generic prescribing increased by 41-48% more for study group | Sample-2; Differences-2; Allocation-1; Outcomes-2; Follow up-0 | 7 |
McDonald et al (1984)w2 Randomised controlled trial | Physician reminder, preventive care | 126 doctors; 4 nurses; 12467 patients | 14-20% increase in response to study actions; study group 2-4 times more likely to apply preventive care | Sample-2; Differences-2; Allocation-2; Outcomes-2; Follow up-0 | 8 |
Roland et al (1985)w3 Randomised controlled trial | Computerised prescribing | 5 doctors; 658 patients | Average of 6 min 37 sec receptionist time saved per 2 hr period, 11 min 38 sec doctor time per 10 scripts; 4.9% fewer queries from pharmacy; 38% fewer telephone requests | Sample-2; Differences-0; Allocation-0; Outcomes-2; Follow up-1 | 5 |
Tierney et al (1986)w4 Randomised controlled trial | Computerised feedback, physician reminder, preventive care | 135 doctors; 6045 patients | 33% greater increase in compliance for faecal occult blood testing, 30% for pneumococcal vaccination, 16% for mammography, 4% for cervical cytology | Sample-2; Differences-0; Allocation-2; Outcomes-2; Follow up-2 | 8 |
McDowell et al (1986)w5 Randomised controlled trial | Patient reminder, flu immunisation | 6 practices (4-6 doctors, 1 nurse); 1420 patients aged ³65 | 13.3% increase with computerised physician reminder, 25.9% with computerised letter, 26.4% with phone reminder; phone most cost effective | Sample-2; Differences-2; Allocation-0; Outcomes-1; Follow up-2 | 7 |
McDowell et al (1989)w6 Randomised controlled trial | Patient reminder, blood pressure screening | 6 practices (4-6 doctors, 1 nurse); 8298 patients aged ³18 | 3.0% increase with phone reminder, 9.6% with computerised physician reminder, 14.6% with computerised letter; physician most cost effective | Sample-2; Differences-2; Allocation-0; Outcomes-1; Follow up-2 | 7 |
Chambers et al (1989)w7 Randomised controlled trial | Physician reminder, mammography screening | 30 doctors; 1262 women aged ³40 | 6.6% increase in screening; numbers fell to pre-intervention levels when reminders stopped | Sample-2; Differences-2; Allocation-0; Outcomes-2; Follow up-unknown | 7.5 |
McPhee et al (1989)w8 Randomised controlled trial | Physician reminder, cancer screening | 62 doctors; 1969 patients | 16-31% increase in tests performed with computer generated reminders; 12-25% with audit and feedback | Sample-2; Differences-2; Allocation-1; Outcomes-2; Follow up-2 | 9 |
McDowell et al (1989)w9 Randomised controlled trial | Patient reminder, cervical screening | 6 practices (4-6 doctors, 1 nurse); 2034 women aged 18-35 | 2.4% increase with computerised physician reminder, 6.3% with phone reminder, 12.2% with computerised letter; physician most cost effective | Sample-2; Differences-2; Allocation-0; Outcomes-1; Follow up-2 | 7 |
Mazzuca et al (1990)w10 Randomised controlled trial | Physician reminder, diabetes management | 114 doctors; 2791 patients; 8132 visits | 5-14% increase in compliance with five diabetes care standards | Sample-2; Differences-2; Allocation-2; Outcomes-2; Follow up-2 | 10 |
McPhee et al (1991)w11 Randomised controlled trial | Physician reminder, cancer screening | 40 doctors; 2420 patients | 10.5-30.7 point increase in physician performance score for stool occult bloods; rectal, pelvic, or breast exams; pap smears; diet or smoking assessment; counselling | Sample-2; Differences-2; Allocation-1; Outcomes-2; Follow up-2 | 9 |
Ornstein et al (1991)w12 Randomised controlled trial | Physician reminder, patient reminder, preventive care | 49 doctors; 7397 patients | Greatest increase with reminders to both; 12-18.6% increase in blood cholesterol tests, faecal occult blood tests, mammograms, tetanus vaccination; decrease in pap smears for all groups | Sample-2; Differences-1; Allocation-2; Outcomes-2; Follow up-2 | 9 |
Rosser et al (1992)w13 Randomised controlled trial | Physician reminder, patient reminder, tetanus vaccination | 6 practices (4-6 doctors, 1 nurse); 8069 patients aged ³20 | 19.6% increase with computerised physician reminder, 20.8% with phone reminder, 27.4% with computerised patient letter; physician most cost effective | Sample-2; Differences-2; Allocation-0; Outcomes-0; Follow up-2 | 6 |
McDonald et al (1992)w14 Randomised controlled trial | Physician reminder, flu immunisation | 126 doctors; 4 nurses; 4555 patients | Average increase of 14.5% in vaccinations over 3 years; study group received vaccinations at almost twice rate of controls | Sample-2; Differences-2; Allocation-2; Outcomes-2; Follow up-0 | 8 |
Rubenstein et al (1995)w15 Randomised controlled trial | Computerised feedback, functional status, quality of life | 73 doctors; 557 patients | Study group physicians diagnosed 9% more patients with depression or anxiety, 12% more with physical, psychological, or social functioning problems; functional status interventions 0.8% higher per patient; 0.8% more clinical problems recorded per patient | Sample-1; Differences-2; Allocation-2; Outcomes-1; Follow up-0 | 6 |
Burack et al (1994)w16 Randomised controlled trial | Physician reminder, patient reminder, mammography screening | 25 doctors (5 sites); 4401 women aged ³40 | 13-29% increase in appointment rates at 6 months; 2-25% at 12 months; no significant increase in mammograms with patient reminder | Sample-2; Differences-2; Allocation-0; Outcomes-2; Follow up-2 | 8 |
Litzelman et al (1993)w17 Randomised controlled trial | Physician reminder, preventive care | 176 doctors; 5407 patients | 3-12% greater compliance for faecal occult blood tests, mammography, pap smears | Sample-2; Differences-2; Allocation-2; Outcomes-2; Follow up-2 | 10 |
Moran et al (1992)w18 Randomised controlled trial | Patient reminder, flu immunisation | 8 doctors; 1 physician assistant; 409 patients | No significant difference in immunisation rates (29.7% v 40% v 38.2%); patients with appointments more likely to be immunised | Sample-2; Differences-2; Allocation-0; Outcomes-2; Follow up-2 | 8 |
Lobach et al (1994)w19 Randomised controlled trial | Computerised protocol, diabetes management | 54 doctors; 2 physician assistants; 2 nurses; 497 patients | 14.4% greater compliance for diabetes care standards; significant difference for physical exam (26.6%), urine protein determination (69.4%), blood cholesterol checks (30.3%); 10 min increase in length if diabetes assessed | Sample-2; Differences-2; Allocation-1; Outcomes-2; Follow up-2 | 9 |
Tierney et al (1987)w20 Randomised controlled trial | Computer display, test results | 111 doctors | 13% reduction in charges per visit; 5.9% fewer tests ordered; 4.7 secs longer to order tests in study group | Sample-1; Differences-2; Allocation-0; Outcomes-2; Follow up-0 | 5 |
McDowell et al (1990)w21 Randomised controlled trial | Patient reminder, flu immunisation | 6 practices; 1175 patients aged ³65 | 29.6% increase in immunisations; rate fell to similar as control group when reminders stopped | Sample-2; Differences-2; Allocation-0; Outcomes-1; Follow up-2 | 7 |
Robson et al (1989)w22 Randomised controlled trial | Computer assisted follow up, preventive care | 5 doctors; 1 nurse; 3206 patients aged 30-64 | 12-28% increase in recording of blood pressure, smoking, blood cholesterol, cervical smears, family history of MI | Sample-2; Differences-2; Allocation-0; Outcomes-2; Follow up-unknown | 7.5 |
Brownbridge et al (1986)w23 Controlled before and after study | Computerised protocol, hypertension management | 3 doctors; 89 consultations | 33% more peripheral pulse exams; 53% more fundal exams; protocol use increased consultation length by 35% | Sample-0; Differences-0; Allocation-2; Outcomes-1; Follow up-not applicable | 4 |
Safran et al (1996)w24 Controlled trial | Computer based record, guidelines, HIV infection | 126 doctors; 349 patients | Median response time with alerts 41 days faster, 385 days faster with reminders | Sample-1; Differences-2; Allocation-2; Outcomes-2; Follow up-2 | 9 |
Warshawsky et al (1994)w25 Before and after study | Computer based record, consultation length, consultation content | 4 physicians; 132 consultations | No significant difference in average length for 3/4 doctors; computer used for 1-11% more of consultation than conventional records, 2-13% less on record review, 5-20% more on documentation; 7-28% increase in non-interaction time | Research question-1; Source of cases-1; Inclusion or exclusion-0; Sample size-0; Starting point-0; Validity of measure-1 | 3 |
Margolis et al (1992)w26 Before and after study | Computer algorithm, paediatrics | 6 doctors | Significant increase in data items collected for otitis media (61%), pharyngitis (55%), URI (58%); significant increase in compliance with management plans recommended for otitis media (41%), pharyngitis (28%); significant decrease in incorrect antibiotic use; no significant increase in length | Research question-1; Source of cases-1; Inclusion or exclusion-0; Sample size-0; Starting point-1; Validity of measure-1 | 4 |
Greatbatch et al (1995)w27 Before and after study | Doctor-patient relationship, communication | 7 doctors; 250 consultations | Increasing familiarity with system did not reduce adverse effect on social interaction; patients attempted to synchronise speech with potential pauses in keyboard use | Research question-0; Source of cases-1; Inclusion or exclusion-0; Sample size-0; Starting point-0; Validity of measure-1 | 2 |
DeVore (1991)w28 Case series | Computerised feedback, geriatric assessment, preventive care | 1 doctor; 22 patients | New medical problem diagnosed for 19/22 patients; new medication prescribed or existing medication altered for 8 patients | Source of cases-1; Inclusion or exclusion-0; Sampling method-0; Exposure-0; Validity of measure-1; Follow up-0 | 2 |
Singh et al (1992)w29 Before and after study | Information management, immunisation, family planning | 5 health centres (2 doctors; 1 pharmacist; 2 community health workers; 1 auxiliary nurse/midwife); only 1 centre studied in this paper | Immunisation rates increased by 34% | Research question-0; Source of cases-0; Inclusion or exclusion-0; Sample size-0; Starting point-0; Validity of measure-1 | 1 |
Tierney et al (1990)w30 Randomised controlled trial | Computer display, test ordering | 121 doctors, 6873 patients (pre-intervention); 121 doctors, 8392 patients (intervention); 74 doctors, 5590 patients (post-intervention) | 8-14% fewer tests ordered during intervention; 11-14% reduction in charges; test ordering was 1.5 seconds longer in study group | Sample-2; Differences-2; Allocation-1; Outcomes-2; Follow up-0 | 7 |
Barton et al (1990)w31 Time series | Physician reminder, patient reminder, flu immunisation | 100 physicians; 845 patients aged ³65 or high risk | No change in immunisations rates for patients ³65 years 1-2, increased by 18% years 2-3; rates for patients <65 increased by 3% years 1-2 and by 14% years 2-3; greatest increase with physician feedback | Research question-0; Source of cases-1; Inclusion or exclusion-1; Sample size-0; Starting point-1; Validity of measure-1 | 4 |
Christensen-Szalanski et al (1982)w32 Time series | Computer algorithm, physician feedback, respiratory illness | 15 nurses; 1 physician assistant; 2 medex; 881 patients | Improved completeness of medical records; 20-75% reduction in tests ordered; 36% reduction in non-provider costs per visit | Research question-0; Source of cases-1; Inclusion or exclusion-0; Sample size-0; Starting point-1; Validity of measure-1 | 3 |
Hutchison (1989)w33 Case-control study | Physician reminder, nurse reminder, flu immunisation | 2 practices; 1211 patients aged ³65 | 15.6% greater increase in overall immunisation rate in study group during intervention; 22.3% more attenders immunised in study group | Research question-0; Source of cases-1; Source of controls 1; Inclusion or exclusion-1; Sampling method-0; Comparability-1 | 4 |
Turner et al (1994)w34 Randomised controlled trial | Physician prompt, preventive care | 44 doctors | Increase of 1-11% in study group for all five measures, none significant; controls increased for flu immunisations (7%), mammography (3%), decreased for stool occult blood tests (5%), pap smear (11%), breast exams (2%) | Sample-2; Differences-1; Allocation-1; Outcomes-2; Follow up-1 | 7 |
Tape et al (1993)w35 Randomised controlled trial | Physician reminder, preventive care | 49 doctors; 1809 patients; 4088 encounters | 2.7% more recommendations for sigmoidoscopy, 3% for tetanus, 7.8% for flu vaccination, 6.6% for pneumococcal vaccination; no difference for faecal occult blood tests, mammography, pap smears, serum thyroxine screening in the elderly | Sample-0; Differences-2; Allocation-2; Outcomes-2; Follow up-unknown | 7.5 |
Harris et al (1990)w36 Time series | Physician prompt, preventive care | 1 medical practice (177 doctors); 450 women aged ³50 | Overall performance increased by 15%; 5% increase from no prompt to nurse prompt, 10% from nurse prompt to physician prompt; 47% increase in flu vaccination, 29% in mammography; non-significant increase of 7-8% in tonometry, breast exam, pneumococcal vaccination | Research question-1; Source of cases-0; Inclusion or exclusion-1; Sample size-0; Starting point-1; Validity of measure-1 | 4 |
Lobach (1996)w37 Randomised controlled trial | Computerised feedback, guidelines, diabetes | 45 doctors | Median compliance was 29.2% greater for physicians in study group; 62% felt feedback increased guideline use; 52% liked getting feedback | Sample-2; Differences-2; Allocation-1; Outcomes-2; Follow up-2 | 9 |
Krall (1995)w38 Time series | Computer based record, physician attitudes, consultation length | 2 practices (30 doctors; 16 physician assistants or nurse practitioners) | After 2 months, doctors spent 2 min 10 sec longer per patient; 1 min 56 sec on >orders and diagnosis=; worked average of 30 days before reaching baseline rates; average of 25-38 hrs per clinician >lost= to decreased productivity learning the system | Research question-0; Source of cases-1; Inclusion or exclusion-0; Sample size-0; Starting point-1; Validity of measure-1 | 3 |
Jones et al (1996)w39 Controlled before and after study | Computer prompting, prescribing, costs | 2 military medical centres; 2 primary care clinics | Reduction in NSAID costs per 100 visits for protocol group, none for computer ordering plus costs group, and increase in computer ordering only group; significant reduction of 13.2% in prescribing expensive NSAIDS for protocol group, 3.6% for computer plus costs group, 1.7% in computer only group | Sample-0; Differences-1; Allocation-2; Outcomes-2; Follow up-unknown | 6 |
Rosser et al (1991)w40 Randomised controlled trial | Patient reminder, preventive care | 6 practices (4-6 doctors, 1 nurse) 8502 patients aged ³15 | 29.5% increase in procedures with computerised letter, 28.8% with phone reminder, 19.2% with computerised physician reminder; physician most cost effective | Sample-2; Differences-2; Allocation-0; Outcomes-2; Follow up-2 | 8 |
Frame et al (1994)w41 Randomised controlled trial | Physician reminder, patient reminder, preventive care | 1 practice (7 physicians, 5 physician assistants); 1665 patients aged ³18 | 11% increase in overall compliance with health maintenance protocol; 2% for mammography or weight, 3% for blood pressure, 6% for blood cholesterol checks, 9% for pap smears, 11% for breast exams,15% for faecal occult blood tests, 21% for tetanus; no change for smoking | Sample-2; Differences-1; Allocation-0; Outcomes-2; Follow up-unknown | 6 |
Chambers et al (1991)w42 Randomised controlled trial | Physician reminder, flu immunisation | 32 doctors; 686 patients | Immunisation rate increased by 21% for those receiving reminders for all patients, 8% for those receiving for half; fell by 10% for those receiving no reminders | Sample-2; Differences-0; Allocation-1; Outcomes-2; Follow up-unknown | 6 |
Hobbs et al (1996)w43 Controlled before and after study | Decision support system, computer prompting, lipid management | 25 practices | No difference in prescribing or gross rate of requests for lipid tests; small increase in patients receiving full lipid profiles, reduction in those receiving partial profiles | Sample-1; Differences-0; Allocation-2; Outcomes-1 ; Follow up-0 | 4 |
Turner et al (1989)w44 Controlled trial | Physician reminder, preventive care | 39 doctors; 253 patients aged ³20 | Significant improvements for mammography (0.7-32%), breast exam (9-16%), stool guaiac test (10-16%), rectal exam (12-18%), pap smears (13-19%); after 1 year completed tests ranged from 33% to 52%, compared with 9-39% before intervention | Sample-0; Differences-0; Allocation-1; Outcomes-2; Follow up-2 | 5 |
Davidson et al (1987)w45 Case series | Decision support system, prescribing, drug interactions | 13 physicians; 103 patients | 152 drug interactions detected for 71 patients; prescription plans changed as a result for 15.5% of patients; physicians gained new information in 46% of cases | Source of cases-1; Inclusion or exclusion-1; Sampling method-0; Exposure-1; Validity of measure-1; Follow up-1 | 5 |
Barnett et al (1983)w46 Randomised controlled trial | Physician reminder, blood pressure management | 1 practice; 115 adult patients | After 24 months, follow up had been attempted for 52% more patients in study group, blood pressure recorded for 18% more | Sample-2; Differences-0; Allocation-0; Outcomes-1; Follow up-1 | 4 |
Burack et al (1996)w47 Randomised controlled trial | Physician reminder, patient reminder, mammography screening | 1 HMO (2 practices); 20 physicians | No significant increase in mammography or visitation rates with patient reminder only; physician reminder increased mammograms by 14% at one site; mammography rate for visitors increased by 2-16% | Sample-2; Differences-2; Allocation-0; Outcomes-2; Follow up-2 | 8 |
Landis et al (1992)w48 Randomised controlled trial | Physician prompt, patient reminder, mammography screening | 1 practice (24 doctors); 122 women aged 50-70 | 20% more patients received a mammography in doctor prompt plus patient reminder group | Sample-2; Differences-1; Allocation-1; Outcomes-2; Follow up-2 | 8 |
Tierney et al (1988)w49 Randomised controlled trial | Computer display, test ordering | 112 doctors; 9496 patients; 23830 visits | 8.8% less charges per visit in study group, from 3.8% for electrocardiogram to 26.8% urine culture; during intervention fell by 10.8% for study group, 3.7% for controls | Sample-2; Differences-0; Allocation-0; Outcomes-2; Follow up-unknown | 5 |
McDonald et al (1980)w50 Randomised controlled trial | Physician reminder, prescribing | 1 practice (26 doctors; 5 nurses) | 16.1% increase in average physician response with reminder alone, 21.1% with reminder plus supporting citations | Sample-1; Differences-2; Allocation-1; Outcomes-1; Follow up-2 | 7 |
Lobach et al (1997)w51 Randomised controlled trial | Computer protocol, guidelines, diabetes care | 54 doctors; 2 physician assistants; 2 nurses | See Lobach et al (1994)w19 | Sample-2; Differences-2; Allocation-1; Outcomes-2; Follow up-2 | 9 |
Herzmark et al (1984)w52 Controlled before and after study | Consultation length, information handling, preventive care | 5 doctors; 374 consultations | Consultation 54 seconds longer; doctors spent twice as much time using computer; discontinued interaction with patient on 24% more occasions | Sample-0; Differences-2; Allocation-1; Outcomes-2; Follow up-2 | 7 |
Pringle et al (1985)w53 Randomised controlled trial | Patient stress and arousal, consultation length, preventive care | 3 doctors; 120 patients | Consultation 48 seconds longer; 16% increase in mention of smears, 55% tetanus, 30% blood pressure; no difference in stress levels; more patients from computer group had higher arousal levels after consultation | Sample-2; Differences-0; Allocation-0; Outcomes-1; Follow up-2 | 5 |
Pringle et al (1986)w54 Randomised controlled trial | Consultation length, doctor activities, patient activities | 3 doctors; 142 patients | Consultation 48 seconds longer; 4.5% increase in doctor speech, 12% less patient speech; 15% increase in physical exams; 12% increase in secondary tasks; 4.5% increase in administration | Sample-2; Differences-0; Allocation-0; Outcomes-2; Follow up-not applicable | 5 |
Pringle et al (1985)w55 Randomised controlled trial | Doctor initiated topics, patient initiated topics | 3 doctors; 120 patients | 29% increase in medical topics raised by doctor | Sample-2; Differences-0; Allocation-0; Outcomes-2; Follow up-not applicable | 5 |
Weingarten et al (1989)w56 Randomised controlled trial | Computer protocol, consultation length, preventive care | 1 doctor; 205 patients | Consultation an average of 90 seconds longer; 3-36% increase in recording of 11 preventive care items over controls | Sample-1; Differences-1; Allocation-0; Outcomes-2; Follow up-1 | 5 |
Brownbridge et al (1985)w57 Case-control study | Standards of care | 6 doctors; 60 patients | No difference in identifying complaints, physical exams, interpreting evidence, reaching medical decisions | Research question-1; Source of cases-1; Source of controls 1; Inclusion or exclusion-0; Sampling method-0; Comparability-1 | 4 |
Garr et al (1993)w58 Before and after study | Physician reminder, patient reminder, preventive care | 44 doctors; 15388 patients | 9.3% increase for cholesterol checks, 9.5% for tetanus immunisations, 3.4% for pap smears; non-significant increase for faecal occult blood tests and mammograms (0.1%) | Research question-0; Source of cases-1; Source of controls-1; Inclusion or exclusion-0; Sampling method-1; Comparability-1 | 4 |
Donald (1986)w59 Before and after study | Prescribing | 1 doctor; 1400 patients | 13% reduction in total cost of prescriptions | Research question-0; Source of cases-1; Inclusion or exclusion-0; Sample size-not applicable; Starting point-1; Validity of measure-1 | 3.5 |
Donald (1989)w60 Before and after study | Prescribing | 1 doctor; 1400 patients | 22%-30% reduction in costs; greater when all prescriptions done on computer | Research question-1; Source of cases-1; Inclusion or exclusion-0; Sample size-not applicable; Starting point-1; Validity of measure-1 | 5 |
Mitchell et al (1997)w61 Controlled before and after study | Chronic disease management, patient outcomes, service use | 12 practices; 1272 patients | 2-20% increase in recording for 11/15 items in the study group, compared with 1-11% for 7/15 items in the control group | Sample-0; Differences-2; Allocation-2; Outcomes-1; Follow up-1 | 6 |
MI=myocardial infarction. URI=upper respiratory infection. HMO=health maintenance organisation.
Table 2 Details of studies of effect of computers in primary care on patient outcomes
Author(s) Study design | Research topic | Participants | Main outcomes | Methodology scores | Overall score (out of 10) |
McAlister et al (1986)w62 Randomised controlled trial | Computerised feedback, hypertension management | 60 doctors; 2231 patients | Moderate hypertensives had 4.8 mm Hg larger decrease in diastolic pressure; 4.1 visits less to GP per year; drop out rate was 4.6% lower | Sample formation-2; Baseline differences-2; Unit of allocation-1; Outcome measures-1; Follow up-1 | 7 |
Fitzmaurice et al (1996)w63 Randomised controlled trial | Decision support system, anticoagulation | 2 practices; 49 patients | 32-66% improvement in INR control; mean recall for study patients extended by 8-12 days; satisfaction in study group was high | Sample formation-1; Baseline differences-0; Unit of allocation-0; Outcome measures-2; Follow up-2 | 5 |
Lewis et al (1996)w64 Randomised controlled trial | Computerised feedback, mental health | 1 practice (6 doctors, 1 trainee); 681 patients | GHQ score improved by 2.5 points at 6 weeks; no difference at 6 months; no significant differences in consultations, prescriptions, referrals to mental health for any group | Sample formation-2; Baseline differences-2; Unit of allocation-0; Outcome measures-1; Follow up-0 | 5 |
Rubenstein et al (1995)w15 Randomised controlled trial | Computerised feedback, functional status, quality of life | 73 doctors; 557 patients | 4.5 point increase in emotional wellbeing scores; limitations on social activities score decreased by 19.6 for patients aged ³70 | Sample formation-1; Baseline differences-2; Unit of allocation-2; Outcome measures-1; Follow up-0 | 6 |
Liaw et al (1996)w65 Randomised controlled trial | Computer health record (CHR), health promotion | 5 practices; 551 patients aged 25-65 | 18-27% more patients in CHR only group attended for health check; 13-22% more in the CHR plus patient health record used the record | Sample formation-2; Baseline differences-0; Unit of allocation-0; Outcome measures-0; Follow up-0 | 2 |
Safran et al (1996)w24 Controlled trial | Computer based record, guidelines, HIV infection | 126 residents; 349 patients | Primary care visits, admission rates, emergency visits, survival were unaffected | Sample formation-1; Baseline differences-2; Unit of allocation-2; Outcome measures-2; Follow up-2 | 9 |
Campbell et al (1996)w66 Before and after study | Appointment system | 1 practice (6-8 doctors) | Median time to make appointment reduced by 4 seconds by phone, 12 seconds in person; patients waiting ³90 seconds to make appointment reduced by 11% by phone, 6% in person; number of doctor movements to check appointment board or computer list reduced by 129 | Research question-0; Source of cases-1; Inclusion or exclusion-0; Sample size-0; Starting point-0; Validity of measure-1 | 2 |
Tierney et al (1990)w30 Randomised controlled trial | Computer display, test ordering | 121 doctors, 6873 patients (pre-intervention); 121 doctors, 8392 patients (intervention); 74 doctors, 5590 patients (post-intervention) | No significant difference in admissions, emergency visits, outpatient visits | Sample formation-2; Baseline differences-2; Unit of allocation-1; Outcome measures-2; Follow up-0 | 7 |
Solomon et al (1995)w67 Randomised controlled trial | Computer based record, patient satisfaction | 2 doctors; 60 patients | No significant differences for familiarity, computer experience, satisfaction with record taking, interaction, physician attentiveness, eye contact | Sample formation-1; Baseline differences-2; Unit of allocation-0; Outcome measures-1; Follow up-2 | 6 |
Strecher et al (1994)w68 Randomised controlled trial | Computer tailoring, smoking cessation | Study 1: 72 patients aged 40-65; Study 2: 296 patients aged 18-75 | 11.8-23.6% greater increase in smoking cessation rates for moderate or light smokers | Sample formation-2; Baseline differences-2; Unit of allocation-0; Outcome measures-0; Follow up-0 | 4 |
Quattlebaum et al (1991)w69 Randomised controlled trial | Patient reminder, appointment dates | 1 practice (27 doctors; 2 behavioural scientists; 1 nurse); 901 appointments | No difference in cancellation rates; 9% greater reduction in no show rates, 8-10% for appointments made before scheduled visits | Sample formation-2; Baseline differences-2; Unit of allocation-0; Outcome measures-2; Follow up-2 | 8 |
Hobbs et al (1996)w43 Controlled before and after study | Decision support system, computer prompting, lipid management | 25 practices | 55% decrease in expected referrals, not significant | Sample formation-1; Baseline differences-0; Unit of allocation-2; Outcome measures-1 ; Follow up-0 | 4 |
Van den Hoogen et al (1990)w70 Case-control study | Computerised feedback, hypertension management | 15 practices | 95 mm Hg target blood pressure achieved for 14% more patients in study group; 31% more hypertensives under permanent surveillance | Research question-1; Source of cases-0; Source of controls-0; Inclusion or exclusion-0; Sampling method-0; Comparability-1 | 2 |
Barnett et al (1983)w46 Randomised controlled trial | Physician reminder, blood pressure management | 1 practice; 115 adult patients | 18% more patients with diastolic pressure <100 mm Hg or on treatment in study group after 12 months, 18% more after 24 months | Sample formation-2; Baseline differences-0; Unit of allocation-0; Outcome measures-1; Follow up-1 | 4 |
Fihn et al (1994)w71 Randomised controlled trial | Computer scheduling of appointment, anticoagulation | 5 practices, 620 patients | Scheduled follow up 0.9 weeks longer in study group, actual follow up still 0.3 weeks longer; no significant differences in anticoagulation control | Sample formation-2; Baseline differences-2; Unit of allocation-0; Outcome measures-2; Follow up-2 | 8 |
Sullivan et al (1992)w72 Before and after study | Patient satisfaction | 1 practice (4 doctors); 900 patients | No difference in overall patient satisfaction, or in individual subscales of general satisfaction, professional care, depth of relationship or perceived time | Research question-0; Source of cases-1; Inclusion or exclusion-0; Sample size-0; Starting point-1; Validity of measure-1 | 3 |
Mitchell et al (1997)w61 Controlled before and after study | Chronic disease management, patient outcomes, service use | 12 practices; 1272 patients | No effects on GP attendance, hospital admission, disease related referrals; no significant difference in satisfaction or health status | Sample formation-0; Baseline differences-2; Unit of allocation-2; Outcome measures-1; Follow up-1 | 6 |
INR=international normalised ratio. GHQ=general health questionnaire.
Table 3 Details of studies of attitudes towards computerisation in primary care
Author(s) Survey method | Participants | Positive aspects mentioned | Negative aspects mentioned |
Liaw (1994)w73 Questionnaire | 315 doctors | 54% planned to computerise records in next 5 years; partially computerised systems more likely to be used by doctors who believed in an empowered doctor-patient relationship | Lack of funds (61%); expensive packages (52%); uncooperative partners (30%) |
Moidu et al (1992)w74 Questionnaire | 43 doctors | Willing to accept computers as part of working environment (98%); main reasons for acquisition were health management (58%) and patient care (53%) | Reservations about use (31%); interference with doctor-patient relationship (26%); dehumanisation of medicine (26%) |
Thakurdas et al (1996)w75 Questionnaire | 268 doctors | Computerisation becoming a necessity (85%); computerisation was a good thing (73%); felt comfortable using computer (50%) | Interference with doctor-patient relationship (43%); privacy (33%); time commitment (28%); cost prohibitive (24%) |
Ridsdale et al (1994)w76 Interview | 1 practice; 30 patients | Efficient tool for access, current medication, or saving time (87%); enjoy discussing what to record (33%); not intrusive if doctor spoke or maintained eye contact (87%); prefer to see screen (97%) | Loss of confidentiality (70%); computers >unfriendly=, eventually take over doctor=s role (17%); unauthorised access (66%) |
Carman et al (1995)w77 Interview | 1 practice; 39 patients | Concerned about who would have access; threat to privacy or confidentiality; should be restricted to non-sensitive information (56%); eager to negotiate with doctor over content | |
Brownbridge et al (1985)w78 Questionnaire | 2 practices; 353 patients | No significant effect on perceptions of doctor attentiveness or rapport, information provision, satisfaction with treatment or post-consultation stress; 87% felt consultation had not been affected | Post-consultation stress linked to patient attitudes about computer use |
Rethans et al (1988)w79 Questionnaire | 1 practice; 263 patients | Contact with doctor was no less personal (92.7%); disagreed that consultation was longer (87.7%); doctor got picture of medical history more quickly (86.6%) | Easier for others to gain information about patients (30.8%); disagreed privacy was more secure (66.5%); does not mean that doctor delivers better care (84.4%) |
Baldwin (1985)w80 Questionnaire | 41 doctors | Patient care improved either a little or a great deal (76%) | More time required than expected (52%); hardware problems (82%); software problems (85%) |
Rowe et al (1995)w81 Questionnaire | 433 1st year family medicine residents | Computers should be a component of training (71%) | Training was a major concern (48%); barriers to this were lack of time (56%), cost (49%), lack of computers (37%), lack of courses (30%); extra training would make use more efficient (56%) and increase skills and comfort with using computer (27%) |
Krall (1995)w38 Questionnaire | 2 practices (9 physicians; 12 internists; 9 paediatricians; 16 physician assistants or nurse practitioners) | Only 15% had reservations about computers pre-intervention; after 4 months, system was worth time and effort to use it (85%); system was easy to use (79%) | After 4 months 15% agreed they would return to the old system if given a choice |
Bolton et al (1995)w82 Questionnaire | 398 RACGP members | 49.2% computerised; 41.6% believed patients= were positive; highest ranked priorities for choosing system were market presence (7.8), upgrade of existing system (6.8), networking (6.7) | Computer records less secure; highest ranked barriers to use were no advantage to practice management (8.9), fear (6.5), poor training (6.4), poor support (6.3), cost (6.2) |
Valenta et al (1996)w83 Group / questionnaire | 46 doctors categorised as 4 types | Useful for obtaining data and access to literature; did not believe computers were hard to learn, depersonalised consultation or made medical care too standardised | Concern over confidentiality and security; consultation use was depersonalising, medical care too standardised; negative view of computers for research, literature retrieval, data sharing |
Ornstein et al (1994)w84 Interview | 1 practice; 16 patients | No worries over confidentiality if security measures taken (69%); doctor-patient relationship unchanged (88%); better access; time saved, better legibility and durability, less paperwork, improved prescribing | 3 patients saw other patient=s record, had missing information, or saw doctor getting frustrated at malfunction; concern over confidentiality (31%); system downtime, data back up, training |
Cruickshank (1984)w85 Questionnaire | 645 patients (229 outpatients, 416 primary care) | Patients with experience were more positive; time saving (81% of outpatients, 70% of primary care patients); check against mistakes (84%, 80%); save money (43%, 40%); lead to better treatment or diagnosis (42%, 33%) | See even less of doctor (48% of outpatients, 35% of primary care patients); loss of personal touch (56%, 52%); doctor would spend more time checking computer than patient (39%, 30%) |
Potter (1981)w86 Questionnaire | 2 practices; 462 patients | Majority had positive attitude towards computerisation | Anxiety over doctor using computer (19%); items they would not like recorded on computer (14.5%); unwilling to speak frankly (26.5%); change doctor if computer used (6.7%) |
Pringle et al (1984)w87 Questionnaire | 2 practice; 304 patients | In general patients had positive attitude towards computerisation | Did not favour computer use (17%); loss of confidentiality (31%); consultations would become less personal (15%); negative views of the cost of computerisation (13%) |
Als (1997)w88 Interview | 5 practices; 5 doctors; 12 patients | The more patients knew about computers the more positive their attitudes | Doctors unaware of using computer to guide conversation or provide >time out=; saw possible intrusion on doctor-patient relationship; only 5/12 patients aware of computerised records; 11/12 thought information was accessible to others; concern over confidentiality; some had lack of confidence in doctor |
Bright (1991)w89 Questionnaire | 1 practice; 144 patients | Did not mind or were glad computer was there (97% in near group, 96% in far group); patients preferred having computer near to themselves and doctor during consultations | Computer interfered with consultation (1.5% in near group, 5.6% in far group) |
Herzmark et al (1984)w52 Interview | 5 doctors; 374 consultations | Doctors found computer use stressful; felt communication with patient was impaired | |
Sullivan et al (1992)w72 Questionnaire | 1 practice (4 doctors); 900 patients | Before computer was introduced, 77% patients thought it was a good idea; increased to 85% six weeks after introduction; increased to 88% 6 months after |
RACGP=Royal Australian College of General Practitioners