Table 2. Q-set statements and factor array.
| No. | Statements | Factors | ||
|---|---|---|---|---|
| A | B | C | ||
| 1 | The evidence from rapid reviews is good enough to inform low-risk, emergent policy or decision-making needs when the alternative is the use of no evidence | 1 | 1 | 0 |
| 2 | When time allows, a comprehensive systematic review of all available evidence should always be conducted | −1 | 2 | −3 |
| 3 | Deviating from accepted systematic review methods may introduce bias and impact the validity of the resulting rapid review, which may be an unacceptable risk for some for knowledge users | 0 | 3 | 1 |
| 4 | Further research comparing the methods and results of rapid reviews and systematic reviews is required before I decide how I feel about rapid reviews | −1 | 2 | 0 |
| 5 | Rapid reviews are too focused in scope and/or context to be generalizable to a variety of knowledge users | −2 | −2 | −1 |
| 6 | Rapid reviews mean different things to different people | 1 | 2 | 1 |
| 7 | Rapid reviews should only precede a more comprehensive and rigorous systematic review | −3 | −3 | −3 |
| 8 | The opportunity cost of a comprehensive SR or HTA is too high and it is more advantageous to conduct rapid reviews when timeliness is a factor | 0 | −2 | 1 |
| 9 | Rapid reviews do not replace SRs or HTAs | 0 | 2 | 0 |
| 10 | All evidence synthesis products, including rapid reviews, SRs, or HTAs, can be conducted very well or very poorly | 3 | 2 | 0 |
| 11 | Rapid reviews are comparable to SRs except they are done in a more timely fashion | −1 | 1 | 1 |
| 12 | Rapid reviews are ‘quick and dirty’ systematic reviews | −2 | 1 | −2 |
| 13 | Rapid reviews need to be tailored to the specific needs of the knowledge user | 0 | 2 | 3 |
| 14 | Rapid reviews meet the needs of knowledge users | 1 | 0 | 2 |
| 15 | There is a paucity of evidence on rapid reviews, so I cannot support or oppose their use in decision-making | −1 | 0 | −1 |
| 16 | There is so much overlap across the various evidence synthesis methods that I cannot generalize my opinion to favor one over the other without the context of the decision at hand | 0 | 0 | −1 |
| 17 | There is a risk involved in tailoring accepted SR methods to produce rapid reviews that we do not yet understand | −1 | 0 | 2 |
| 18 | Using rapid reviews to inform decisions is better than using no evidence at all | 2 | 0 | 0 |
| 19 | It is always appropriate to conduct a rapid review | −2 | −3 | −1 |
| 20 | Rapid reviews and all other evidence synthesis products hold the same value as long as they retain the core value of being transparent in conduct, include the highest quality evidence available and present results with a qualification on the strength of evidence | 2 | −1 | 0 |
| 21 | Appropriateness of a rapid review varies with the type of decision being made, and any financial, legal or other important contextual facets tied to the decision | 2 | 1 | 1 |
| 22 | My confidence in a rapid review is impacted by which methods are tailored to speed up the review process | 1 | 0 | 0 |
| 23 | My confidence in a rapid review is directly tied to results being presented and contextualized by the strength and applicability of the evidence | 0 | 0 | −1 |
| 24 | It is important to have minimum standards for the methodological conduct of rapid reviews | 0 | 1 | 2 |
| 25 | It is important to have minimum standards for the reporting of rapid reviews (e.g., a PRISMA-RR) | 2 | 0 | 0 |
| 26 | Standardization of rapid review methods may conflict with the needs of knowledge users | 0 | −2 | 1 |
| 27 | The value of rapid reviews in the context of emergent decision-making needs outweighs the disadvantages or risk of bias and potentially ‘imperfect’ evidence | 1 | −1 | 2 |
| 28 | Knowledge users don’t always need all of the evidence, they just need the best evidence to support their decision, and what is ‘best evidence’ is specific to the knowledge user | 1 | −2 | 3 |
| 29 | Knowledge users do not fully understand the implications of streamlining evidence synthesis methods to produce a more timely evidence product | 1 | 0 | 0 |
| 30 | Reporting of the results of rapid reviews must be tailored to the knowledge user(s) who commissioned the review | 0 | 0 | 0 |
| 31 | Rapid reviews that omit an assessment of the quality of included studies are useless to knowledge users | 0 | −1 | −1 |
| 32 | Rapid reviews can be timely and valid, even when methodological concessions are made | 1 | 1 | 1 |
| 33 | Transparency of process is more important than the actual methods used to produce rapid reviews, as transparency allows the end user to make their own assessment on validity and appropriateness | 0 | −2 | 2 |
| 34 | It is appropriate to endeavor to define a single, unique methodology for rapid reviews | −1 | −1 | −2 |
| 35 | Rapid reviews are not a unique methodology, they are simply a variation of a systematic review that can fall anywhere on the continuum of evidence synthesis methods | 1 | −1 | 0 |
| 36 | The results from a systematic review may not differ from those of a rapid review, but more research is needed to support this theory and quantify why results may be the same or different | 1 | 1 | 0 |
| 37 | I put more confidence in evidence produced in a systematic review than of a rapid review | −1 | 1 | −1 |
| 38 | The more time spent conducting the review of the evidence, the more valid the results of the review will be | −3 | −1 | −2 |
| 39 | Achieving a precise estimate of effect (from a SR) may not inform the decision-at-hand any better than a general estimate of effect (produced by a rapid review) | 2 | −1 | 1 |
| 40 | Rapid reviews should only be conducted when the alternate option is the use of no evidence to inform a decision | −2 | 1 | −1 |
| 41 | A well-conducted rapid review may produce better evidence than a poorly conducted systematic review | 3 | −1 | 2 |
| 42 | Any review of evidence that takes longer than three months to produce is not a rapid review | −1 | −2 | −1 |
| 43 | Any review of evidence that takes longer than one month to produce is not a rapid review | −2 | −1 | −2 |
| 44 | A rapid review must be justified with a valid rationale for both speeding up the process and tailoring rigourous methods for evidence synthesis | −1 | 0 | 0 |
| 45 | A good quality review of evidence is determined by the methods used, not by the speed at which it is completed | 2 | 1 | 0 |
| 46 | It is difficult to tell a rapid review from a systematic review unless very specific nomenclature is used in the title or description of methods | −1 | 0 | 1 |
| 47 | A rapid review cannot be a systematic review | −2 | 3 | −2 |
| 48 | ‘Rapid review’ is too broad a phrase—doing a review in a more timely way can only be relative to how long it takes the same team to produce a full systematic review | 0 | −1 | −1 |
| 49 | Producers are more concerned with the methodology and validity of rapid reviews than knowledge users | 0 | 0 | 1 |
| 50 | It is difficult to judge the validity of a rapid review as the reporting is often truncated and protocols are not published | 0 | 0 | −2 |
Note:
Variance = 2.08, Standard Deviation = 1.4.