Skip to main content
. 2020 May 22;117(21):365–372. doi: 10.3238/arztebl.2020.0365

eTable 2a. Instrument for evaluating the quality of studies included in the systematic review.

Major risk of bias domains* Risk Criteria Hints/ notes
1. Recruitment procedure & follow-up (in cohort studies):

For cohort studies

HINT: We are looking for selection bias:
– Was the cohort representative of a defined population? #
– Was everybody included who should have been included? #
–If response on day care center level is slightly <50% but does not indicate selection bias, it will be listed as a demerit in extraction table.
Preliminary ruling:
–If the cohort recruitment is based on a convenient/self-reported sampling OR if response is <10% or if response was not reported/calculable, the study will be excluded from analysis.
low
  • Cohort recruitment was acceptable.#

  • Baseline response on both educator and day care center level is acceptable (50% or more) OR is <50% and >30%, but substantial differential selection could be excluded (e. g. by a non-responder analysis).

  • Loss to follow-up is below 20% in total and not different between the two groups (up to 10% difference).*

high
  • Cohort recruitment was not acceptable.#

  • Response not reported/ not calculable.

  • Total loss to follow-up is larger than acceptable (20% or more)* OR drop out differs between the groups by more than 10%* OR the reasons for drop out considerably differ between exposed and non-exposed groups.*

For case-control studies

HINT: We are looking for selection bias:
– Were the cases and control subjects representative of the same defined population (“study base”; geographically and/or temporally)? #
– Was there an established reliable system for selecting all the cases? #
– The same exclusion criteria are used for both cases and controls. #
– Comparison is made between participants and non-participants to establish their similarities or differences. #
– If response on day care center level is slightly <50% but does not indicate selection bias, it will be listed as a demerit in extraction table.
Preliminary ruling:
– If the recruitment is based on a convenient/self-reported sampling OR if response is <10% or if response was not reported/calculable, the study will be excluded from analysis.
low
  • Case selection and recruitment was acceptable.#

  • Control subjects’ selection and recruitment was acceptable.#

  • Non-response was less than 50% for cases and/or control subjects OR it was >50% and <70%, but substantial differential selection of cases and control subjects could be excluded (e.g. by a non-responder analysis)*

high
  • Case selection and recruitment was not acceptable.#

  • Control subjects’ selection and recruitment was not acceptable.#

  • Non-response was >70% for cases or control subjects OR it was >50% and<70%, but substantial differential selection of cases and control subjects could not be excluded.*

  • Response not reported/ not calculable

For cross-sectional studies

HINT: We are looking for selection bias:
– Was the study population representative of a defined population? #
– Was everybody included who should have been included? #
– If response on day care center level is slightly <50% but does not indicate selection bias, it will be listed as a demerit in extraction table.
Preliminary ruling:
– If the recruitment is based on a convenient/ self-reported sampling OR if response is <10% or if response was not reported/calculable, the study will be excluded from analysis.
low
  • Recruitment of the study population was acceptable.#

  • Non-response was less than 50% OR it was >50% and <70%, but substantial differential selection of the study population could be excluded (e.g. by a non-responder analysis).*

high
  • Recruitment of the study population was not acceptable.#

  • Non-response was >70% OR it was >50% and <70%, but substantial differential selection of the study population could not be excluded.*

  • Response not reported/ not calculable.

2. Exposure definition and ‧measurement low
  • Exposure definition included at least basic job characteristics (e.g., job tasks, length of employment).

  • Exposure was accurately measured to minimize bias.

  • Adequate comparison group of non-exposed workers (e.g. office workers) included.

high
  • Exposure does not cover basic job characteristics.

  • Exposure was not accurately measured.#

  • Different methods were used to measure exposure in different groups/cases and control subjects (in case-control studies).

  • No adequate comparison group of non-exposed workers included (only for outcome 1b)

unclear
  • Not reported.

3.1a Outcome “infection rate”. Source and validation low
  • Outcome was accurately/ objectively measured to minimize bias (positive serology, medical diagnosis).#

  • Measurement methods were similar in the different groups.#

high
  • Outcome was not accurately or subjectively measured (self-reported).#

  • Measurement methods were different in the groups.#

unclear
  • Not reported.

3.1b Outcome “risk of infection”. Source and validation low
  • Outcome was accurately/ objectively measured to minimize bias (e.g. positive serology, medical diagnosis).#

  • Measurement methods were similar in the different groups.#

high
  • Outcome was not accurately or subjectively measured (e.g. self-reported).#

  • Measurement methods were different in the groups.#

unclear
  • Not reported.

3.2 Outcome “Immunization coverage of the nursery-school teachers”. Source and validation
Only applicable to vaccine-preventable diseases.
low
  • Outcome was accurately/ objectively measured to minimize bias (e.g. checked by certificates of vaccination, use of validated instruments).#

  • Measurement methods were similar in the different groups.#

high
  • Outcome was not accurately or subjectively measured (e.g. self-reported).#

  • Measurement methods were different in the groups.#

unclear
  • Not reported.

3.3 Outcome “Immunity status of the nursery-school teachers”. Source and validation low
  • Outcome was accurately/ objectively measured to minimize bias (e.g. positive serology).#

  • Measurement methods were similar in the different groups.#

high
  • Outcome was not accurately or subjectively measured.#

  • Measurement methods were different in the groups.#

unclear
  • Not reported.

4. Confounding and effect modification
HINT: If the immunity status of the children in care is not being considered, it will be listed as a demerit in extraction table.
low
  • If risk estimators were calculated, major confounding factors (at least age, sex, SES and for vaccine-preventable diseases, immunization status of child care workers) & effect modifiers were considered.

  • If only prevalence or incidence was assessed, at least sex, age and immunization coverage (at least mean values for the study population) are described.

high
  • Major confounding factors or effect modifiers were not considered.

unclear
  • Not reported.

5. Analysis method: methods to reduce research specific bias
HINT: If the prevalence of serology is very high, we will not accept Prevalence Odds Ratios as adequate.
low
  • Authors used adequate statistical models to reduce bias (e.g., standardization, matching, adjustment in multivariate model, stratification, propensity scoring). For prevalences, matching/stratification may not be required as long as a good description of the age structure and immunization status of the population is given.

high
  • Authors did not use adequate statistical models to reduce bias.

unclear
6. Chronology low
  • Incident diseases were included.#

  • Temporal relation may be established (exposure precedes the outcome).#

  • Negative serology known at baseline (career entry, baseline of study) AND was accurately/ objectively measured.

  • For outcomes 2 and 3, cross-sectional studies are appropriate.

high
  • Prevalent diseases were included OR prevalent diseases of baseline were not excluded (in cohort studies).#

  • Temporal relation cannot be established.

  • Serology is unknown at baseline.

  • Cross-sectional studies without basic information about temporal course (not applicable to outcomes 2 or 3)

unclear
  • Not reported.

7. Funding low
  • Grant/ non-profit-organizations*

  • Study was clearly not affected by sponsors.*

high
  • Sponsoring organization participated in data analysis.

  • Study was probably affected by sponsors.

unclear
  • Industry, combined industry+grant*, unclear if study was affected by sponsors.

  • Not reported.

8. Conflict of interest low
  • Reported not having conflict of interest or clear from report/ communication that study was not affected by author(s) affiliation.*

high
  • Conflict of interest exists (at least one author).*

unclear
  • Not reported.

*according to Ijaz et al. (2013), with modifications; # SIGN/CASP