Report of the BMJ’s full Information in Practice editorial committee

Members of the committee were:

Douglas Carnall
Nick Booth
John Williams

Decision: revise to author.

The committee took the view that the paper extends a valuable example of a methodology for reviewing websites, but that there are several points to be addressed in a revision before it can be published.

1. The description of how the search was done needs to be more detailed. How were the 21 sites reviewed arrived at? Were they just the most frequently accessed sites? How confident are you that DirectHit measures this accurately?

2. What is the status of the ACPAR guidelines? Evidence based? Widely accepted? Needs to be clarified for an international audience. One of the main criticisms of the Impiccatore paper was that they did not use evidence-based criteria.

3. How generalisable would the message be for a different subject area? How does the process of converting the guidelines into a set of evaluation criteria for websites take place?

4. The committee contemplated asking the authors to replicate their methodology in a different subject area, but rejected this as too onerous a requirement. An allusion in the discussion to the potential of the method would nevertheless be interesting and worthwhile.

5. The organisation of the paper is confusing: It would be preferable to ensure that the headings of quality, accountability, site characteristics are repeated in the same order through the paper (in abstract, methods)