Douglas Carnall
Associate Editor
British Medical Journal
BMA House
Tavistock Square
London
WC1H 9JR
22nd August 2000
Dear Dr Carnall
Paper 001870 — A descriptive feast but an evaluative famine — a systematic review of published papers on primary care computing 1980-1997
Please find enclosed on paper copy and one electronic copy of our paper for publication. We have revised it as suggested and have responded to the points made in your letter of 26 July as follows.
| 1a. | Search criteria: making the number of eligible studies more explicit |
| Page 1, para 3. We have changed the title of this paragraph from Subjects to Methods to make it explicit to the reader that the paper contains a methods section. The numbers of articles which were identified, were eligible and which were included in the review have been outlined more explicitly in the Results section. | |
| 1b. | Why was CINAHIL ommitted from the initial search |
| CINAHL was excluded from our search strategy based on previous experience, which showed that the database contained relatively few articles relating to primary care and even fewer to the field of medical informatics. The majority of articles are also referenced in Medline. In case of any oversight, we conducted a subsequent search and found only five potentially relevant papers. One of these was indexed after the date of our search, the other four were identified through the other databases. | |
| We also note the reviewer's comment on the issue of nursing articles relating to the discussion section of the paper (page 10, para 3). The references which he kindly included are either descriptive summaries of systems used in nursing or are studies of research carried out in a hospital or nursing home setting. The point which we make in the discussion relates to the number of articles identified as part of this review, which look at use of computers by nurses in a primary care setting. We have stated this more explicitly in the paper (page 10, para 3). | |
| 2. | Consideration of the QUORUM guidelines |
| Medical informatics research is still a relatively new field and this is reflected in the literature. Conversely, the use of informatics in primary care is widespread. Computerisation exists at various levels in practice and evaluation is undertaken using several different approaches. In light of this, we felt that it was innappropriate to restrict our review to randomised controlled trials. The QUORUM guidelines apply to reports of randomised controlled trials. Less than two thirds of the papers included in our review used an experimental study design and as such it was not applicable or feasible to synthesize the studies in the manner suggested by the guidelines. Nevertheless, we did employ a systematic methodology for searching the literature and we did have pre-defined inclusion and exclusion criteria. These were not listed in the text of the paper due to the word count restriction. Instead, we referenced our previous review which does list these criteria. We also independently and rigorously evaluated the papers and we did present descriptive data for each study and combine the results in a manner which we regarded as appropriate to the research question. We regard this work as a systematic review, but will adhere to the committee's decision as to whether the word 'systematic' should be removed. | |
| 3. | Papers dealing with consultation length |
| Six of the nine papers looking at the consultation reported on consultation length. Only five of those studies found that consultation length increased. Time was measured objectively in each case; by time-stamped video recording in three studies and by direct timed observation by the researchers in two. The remaining study found no significant difference in consultation length for 3 of the 4 doctors using the computer. The text has been altered to include this (page 6, para 1). There were no studies which found that computer use shortened the consultation. | |
| All significant study outcomes were recorded for each article. This was done after their suitability for inclusion had been determined and after they had been independently rated by each of the authors. We did not frame questions regarding study outcomes before conducting the literature search, so this cannot be reflected in our paper. | |
| 4. | Diversity of computer applications |
| The purpose of this review was to summarise the literature surrounding the use of computers in terms of their impact on the outcome of primary care consultations. We were not interested in looking at specific outcomes, but rather at those related to any aspect of the consultation itself, to practitioner performance or to patients. Similarly, we were not interested in looking at specific systems or applications. We believe that it is more appropriate to collate the evidence in this review by area of practice (e.g. prescribing or preventive care) or by patient outcome (e.g. disease management or satisfaction). We believe that this will be of interest to practitioners and is more relevant in an area such as this, where there is little comprehensive information. | |
| Had the review included studies evaluating the impact of computer use on a specific outcome, such as immunisations, it would be have been more appropriate to organise the paper by application type, as one type of system may have been more or less beneficial than others. However, this review includes studies covering a broad range of topic areas and we believe that organising it by application would be of less value in an evolving field where systems are constantly changing and being replaced. | |
| 5. | Additional comments from the reviewer |
| 5.1 The reviewer found table II unhelpful | |
| We have included a possible alternative to this table although it is rather dense since it outlines scoring systems for six research designs. | |
| 5.2 Are the overall scores listed in Table III a combination of the methods | |
| Studies were scored only once using the method which was relevant to the study design. This has been made more explicit in the text (page 4, para 2). | |
| 5.3 Mention excluded articles in the Methods section rather than Results | |
| We regard inclusion criteria, databases and search terms used as methods, but view numbers of articles identifed, excluded and included as the results of a search. On this basis, we have retained the figures in the Results section. | |
| 5.4 Horizontal lines in Table IV | |
| The lines are not significant but were an oversight on our part. They have been removed from the table. | |
| 5.5 Results section misleading on patients wishing to change doctor | |
| This has been altered in the text to reflect that it was an issue for some patients in one study (page 9, para 2). |
We hope that we have dealt adequately with your suggestions. Please do not hesitate to contact me if you need further clarification on any of these points.
Yours sincerely
Elizabeth Mitchell
Research Fellow