Skip to main content
. 2014 Nov 13;2014(11):CD009519. doi: 10.1002/14651858.CD009519.pub2

2. Risk of bias and applicability items and criteria for their assessment.

Item Description
Domain 1: Patient selection
A. Risk of bias
Patient sampling The study design will be listed here 
Was a consecutive or random sample of participants enrolled? 'Yes' if a consecutive or random sample of participants were enrolled
 'No' if a consecutive or random sample of participants were not enrolled
 'Unclear' if the study does not describe the method of participants enrolment
Was a case‐control design avoided? 'Yes' if the study has not used a case‐control design
 'No' if the study has used a case‐control design
 'Unclear' if the study does not report enough information to ascertain whether a case‐control design was used
Did the study avoid inappropriate exclusions? 'Yes' if the characteristics of the participants are well described and probably typical of a secondary healthcare setting
 'No' if the sample is unrepresentative of people with potentially resectable lung cancer in general
 'Unclear' if the source or characteristics of participants is not adequately described
Could the selection of participants have introduced bias? A judgement of low, high, or unclear risk of bias will be made based on a balanced assessment of the responses to the above signalling questions
B. Concerns about applicability
Patient characteristics and setting The information detailed under 'Patient characteristics and setting' in the 'Characteristics of included studies' table will be listed here
Are there concerns that the included participants and setting do not match the review question? A judgement of low, high, or unclear concerns about applicability will be made based on a balanced assessment of the response to the third signalling question above and on how closely the sample matches the target population of interest
Domain 2: Index test
Index test The information detailed under 'Index test' in the 'Characteristics of included studies' table will be listed here
Were the index test results interpreted without knowledge of the results of the reference standard? 'Yes' if the report stated that the person undertaking the index test did not know the results of the reference tests or if the 2 tests were carried out in different places
 'No' if the report stated that the same person performed both tests or that the results of the index tests were known to the person undertaking the reference tests
 'Unclear' if insufficient information provided
Did the study provide a clear definition of what was considered to be a positive result? 'Yes' if the definition of a positive result is clearly stated (e.g., SUV)
 'No' if no definition of what was considered a positive result is stated or the definition of a positive result varied between the participants
 'Unclear' if not enough information is given to permit judgement
If a threshold was used, was it prespecified? 'Yes' if prespecified
 'No' if the authors selected the optimal cut‐off value based on the results of the study
 'Unclear' if there is a range of cut‐off values and there is doubt which cut‐off has been used, or if there is no mention at all of a cut‐off value
Could the conduct or interpretation of the index test have introduced bias? A judgement of low, high, or unclear risk of bias will be made based on a balanced assessment of the responses to the above signalling questions
Are there concerns that the index test, its conduct, or interpretation differ from the review question? A judgement of low, high, or unclear concerns about applicability will be made based on a balanced assessment of the information detailed under 'Index test' with particular reference to the definition of test positivity/malignancy
Domain 3: Reference standard
Target condition and reference standard(s) The information detailed under 'Reference standard(s)' in the 'Characteristics of included studies' will be listed here
Is the reference standards likely to correctly classify the target condition? 'Yes' if reference standard is sampling of mediastinal nodes with pathological diagnosis
 'No' if there is no sampling of mediastinal nodes with pathological diagnosis
'Unclear' if insufficient information is provided
Were the reference standard results interpreted without knowledge of the results of the index tests? 'Yes' if the report stated that the person undertaking the index test did not know the results of the reference tests, or if the 2 tests were carried out in different places
 'No' if the report stated that the same person performed both tests, or that the results of the index tests were known to the person undertaking the reference tests
 'Unclear' if insufficient information provided
Could the reference standard, its conduct, or its interpretation have introduced bias? A judgement of low, high, or unclear risk of bias will be made based on a balanced assessment of the responses to the above signalling questions
Are there concerns that the target condition as defined by the reference standard does not match the question? The answer to this question will always be 'low' because the target condition that the reference standard defines will always be the target condition of the review, i.e., pathologically confined non‐small cell lung cancer. Otherwise, the study will not be included
Domain 3: Flow and timing
Flow and timing The information detailed under 'Flow and timing' in the 'Characteristics of included studies' will be listed here
Was there an appropriate interval between index test and reference standard? 'Yes' if the time period between PET‐CT and the reference standard is ≤ 8 weeks
 'No' if the time period between PET‐CT and the reference standard is > 8 weeks
'Unclear' if insufficient information is provided
Did all participants receive the same reference standard? 'Yes' if the same reference test was used regardless of the index test results
 'No' if different reference tests are used depending on the results of the index test
 'Unclear' if insufficient information is provided
 If any participants received a different reference test, what were the reasons stated for this, and how many participants were involved?
Were all participants included in the analysis? 'Yes' if there are no participants excluded from the analysis, or if exclusions are adequately described
 'No' if there are participants excluded from the analysis and there is no explanation given
 'Unclear' if not enough information is given to assess whether any participants were excluded from the analysis
 Report how many participants were excluded from the analysis for reasons other than uninterpretable results
Report how many results were uninterpretable (of the total)
Could the patient flow have introduced bias? A judgement of low, high, or unclear risk of bias will be made based on a balanced assessment of the responses to the above signalling questions and of the information listed under 'Flow and timing'
Was the study free of commercial funding? 'Yes' if the funding source is clearly stated and is not commercial
 'No' if the funding source is clearly stated and is commercial
 'Unclear' if not enough information is given to assess whether the funding source is commercial

PET‐CT = positron emission tomography and computed tomography.
 SUV = standardised uptake value.