Skip to main content
. 2020 Feb 5;2(1):100032. doi: 10.1016/j.ocarto.2020.100032
Risk of bias item Criteria for answers (please circle one option) Additional notes and examples
External Validity
1. Was the study's target population representative of most TKR populations on relevant demographic and clinical variables, e.g. age, sex, pain severity, osteoarthritis grade?*
  • Yes (LOW RISK): The study's target population was a close representation of most TKR populations.

  • No (HIGH RISK): The study's target population was clearly NOT representative of most TKR populations.

The target population refers to the group of patients to which the results of the study will be generalised. Examples:
  • The study was a survey of patients in a hospital department and the sample was drawn from a list that included all individuals operated on over a two-year period. The answer is: Yes (LOW RISK).

  • The study was conducted in one province only, and it is not clear if this was representative of the national population. The answer is: No (HIGH RISK).

  • The study was undertaken asking responses from people considering revision surgery and it is clear this was not representative of most TKR populations. The answer is: No (HIGH RISK).

2. Was the sampling frame a true or close representation of the TKR population?*
  • Yes (LOW RISK): The sampling frame was a true or close representation of the TKR population.

  • No (HIGH RISK): The sampling frame was NOT a true or close representation of the TKR population.

The sampling frame is a list of the sampling units in the target population and the study sample is drawn from this list. Examples:
  • The sampling frame was a database of every individual who received a TKR within a hospital. The answer is: Yes (LOW RISK).

  • The study asked responses from anonymous people in an online chat group. The answer is: Yes (LOW RISK).

3. Was some form of consecutive or random selection used to select the sample?
  • Yes (LOW RISK): Some form of consecutive or random selection was used to select the sample (e.g. simple random sampling, stratified random sampling, cluster sampling, systematic sampling).

  • No (HIGH RISK): Some form of consecutive or random selection was NOT used to select the sample.

In a survey, only part of the sampling frame is sampled. In these instances, consecutive or random selection of the sample helps minimise study bias.
Examples:
  • Every person in a consecutive sample was surveyed. The answer is: Yes (LOW RISK).

  • The sample was selected using simple random sampling. The answer is: Yes (LOW RISK).

  • A clinician asked a non-consecutive sample of his/her patients. The answer is: No (HIGH RISK).

4. Was the likelihood of non-response bias minimal?*
  • Yes (LOW RISK): The response rate for the study was ≥75%, OR, an analysis was performed that showed no significant difference in relevant demographic and clinical characteristics between responders and nonresponders

  • No (HIGH RISK): The response rate was <75%, and if any analysis comparing responders and non-responders was done, it showed a significant difference in relevant demographic and clinical characteristics between responders and non-responders.

Examples:
  • The response rate was 68%; however, the researchers did an analysis and found no significant difference between responders and non-responders in terms of age, sex and clinician status. The answer is: Yes (LOW RISK).

  • The response rate was 65% and the researchers did NOT carry out an analysis to compare relevant characteristics between responders and non-responders. The answer is: No (HIGH RISK).

  • The response rate was 69% and the researchers did an analysis and found a significant difference in age, sex and clinical status between responders and non-responders. The answer is: No (HIGH RISK).

Internal Validity
5. Were data collected* directly from the participants (as opposed to a proxy)?
  • Yes (LOW RISK): All data were collected directly from the participants.

  • No (HIGH RISK): In some instances, data were collected from a proxy.

A proxy is a representative of the subject. Examples:
  • All eligible participants were surveyed directly.

  • The answer is: Yes (LOW RISK).

  • A clinician, or series of clinicians, estimated how satisfied their patients were.

  • The answer is: No (HIGH RISK).

6. Was an acceptable participant definition TKR used in the study?*
  • Yes (LOW RISK): An acceptable participant definition was used.

  • No (HIGH RISK): An acceptable case participant definition was NOT used.

In a study, the following participant definition was used: “All participants must have had a TKR, which is a surgical procedure to replace the weight-bearing surfaces of the knee joint to relieve pain and disability.” The answer is: Yes (LOW RISK).
  • In a study, the following participant definition was used: “Participants needed to have received some form of knee surgery.” The answer is: No (HIGH RISK).

7. Was the study instrument that measured satisfaction shown to have reliability and validity (if necessary)?
  • Yes (LOW RISK): The study instrument had been shown to have reliability and validity, e.g. test-retest, piloting, validation in a previous study, etc.

  • No (HIGH RISK): The study instrument had NOT been shown to have reliability or validity

  • The authors used a questionnaire, which had previously been validated. They also tested the inter-rater reliability of the questionnaire. The answer is: Yes (LOW RISK).

  • The authors developed their own questionnaire and did not test this for validity or reliability. The answer is: No (HIGH RISK).

8. Was the same mode of data collection used for all participants?
  • Yes (LOW RISK): The same mode of data collection was used for all subjects.

  • No (HIGH RISK): The same mode of data collection was NOT used for all subjects.

The mode of data collection is the method used for collecting information from the subjects. The most common modes are face-to-face interviews, telephone interviews and self-administered questionnaires. Examples:
  • All eligible subjects had a face-to-face interview. The answer is: Yes (LOW RISK).

  • Some subjects were interviewed over the telephone and some filled in postal questionnaires. The answer is: No (HIGH RISK).

9. Was the length of the measurement period of satisfaction with TKR appropriate?*
  • Yes (LOW RISK): The shortest measurement period of satisfaction with TKR was appropriate.

  • No (HIGH RISK): The shortest measurement period was not appropriate

The measurement period is the length of time post-surgery. The shorter the measurement period, the greater the likelihood of the participant's satisfaction being about the operative and rehabilitation process rather than about the medium-term or longer-term residual pain and functional capacity. Examples:
  • Participants were asked about satisfaction with TKR when they were 12-months post-surgery. The answer is: Yes (LOW RISK).

  • Participants were asked about satisfaction when they were 2-months post-surgery. The answer is: No (HIGH RISK).

  • Note: A follow-up of >6 months was part of the inclusion criteria for this review. Therefore all studies with <6 months were excluded from further consideration in the full text screening.

10. Were the statistical measures of satisfaction appropriate?*
  • Yes (LOW RISK): The paper presented adequate description of how the summary statistics were calculated, the statistics were appropriate and would be possible to be reproduced in a replication study.

  • No (HIGH RISK): The paper did present adequate description of the statistics or one or more of these were inappropriate.

  • The individual items in the satisfaction questionnaire were scored and summarised using the method the questionnaire developers validated, and a group mean score was reported with 95% confidence intervals). The answer is: Yes (LOW RISK).

  • It is not clear how the measure of satisfaction was scored and/or summarised. The answer is: No (HIGH RISK).

11. Summary item on the overall risk of study bias
  • LOW RISK OF BIAS: Further research is very unlikely to change our confidence in the estimate.

  • MODERATE RISK OF BIAS: Further research is likely to have an important impact on our confidence in the estimate and may change the estimate.

  • HIGH RISK OF BIAS: Further research is very likely to have an important impact on our confidence in the estimate and is likely to change the estimate.

Items with an asterisk will exclude a paper from further consideration (items 1, 2, 4, 5, 6, 9 and 10).