Mode of administration bias

Self-report questionnaires are widely used as proxy measures of clinical outcomes. The results of the questionnaires are typically tabulated into a single score and used to describe a selected construct or dimension of health (e.g. disability, function, pain).^1,2 The recent increase in the popularity of self-report questionnaires has led to a concomitant increase in the validation of each instrument and the subsequent assessment of how bias can influence outcomes.³

According to Sackett and colleagues,⁴ bias is considered as any systematic deviation of an observation from the true clinical state. Unfortunately, bias associated with self-report questionnaires is quite common and can potentially influence the outcome of the targeted dimension of health.⁵ Minor variations in the structure of the questions of the self-report questionnaire (e.g. question wording and order) can lead to significant discrepancies in findings.^6,7 Furthermore, variations in mode of administration (how, when, and in what manner the self-report questionnaire is provided) can also be a dramatic source of study bias.^7,8

Different modes of administration include: (1) traditional paper and pencil self-administration ‘interview’ methods, handled through postal services, clinical administration, either in person of in absence of the clinician; (2) computer-assisted (electronic) self-administered ‘interview’ methods by use of a computerized interface; or (3) face-to-face verbal interviews between the patient and the clinician.⁹ Mode of administration bias occurs during data collection and involves intentional or unintentional alteration of information collected from the patient.⁸

When examining within-session effects of manual therapy interventions, mode of administration of outcomes measures is often either a written self-report assessment (such as a global rating of change score or a visual analog scale for pain) or a verbal report using a standardized outcomes measure. Studies that examine within-session or immediate effects routinely capture patient changes (either verbally or by written self-report) directly after a single intervention. How these measures are captured can lead to dramatically variable results.¹⁰ The goal of this editorial is to outline the potential forms of bias associated with mode of administration and to discuss methods to reduce the risk of inflated findings.

Forms of mode of administration bias

Mode of administration bias can be sub-divided into two broad categories: (1) unintentional or intentional patient bias; and (2) unintentional or intentional clinician-centric bias. Patients have been shown to bias their own thoughts and considerations regarding their outcomes, especially in the selected forms of mode of administration. For example, it has been reported that a patient’s willingness to admit complaints is lessened in a face-to-face interview.¹¹ In addition, patients often attempt to please the clinician or researcher or withhold negative information to best model the intervention provided,¹² thereby generating overly optimistic responses relative to the responses given on the self-completed section of the survey. Clinician-centric biases may result when a clinician (or researcher) unconsciously underestimates possible complaints and unfavorable answers of the patient, or when research administrators or personnel tend to influence (consciously or unconsciously) answers to interview or written self-reports from patients.

Mode of administration: patient biases

Forms of patient biases include: attention bias (Hawthorne effect), reporting bias, regression dilution bias, and extreme response bias. Each of the biases is described below and in Table 1 for further understanding.

Table 1. Forms of mode of administration bias and explanations.

Forms of biases	Definition and explanation
Patient biases
Attention bias	When subjects change their behavior or demonstrate an improvement in their outcome because they know they are being observed
Reporting bias	An intentional situation in which the patient selects that they have improved after an intervention because they feel it is appropriate to do so and because they do not want to disappoint their clinician researcher. Includes obsequiousness bias and social responsibility bias
Regression dilution bias	The phenomenon associated with an extreme value on the initial measurement followed by less extreme subsequent assessment (a regression to the mean)
Extreme response bias	Occurs when the practitioner is present during taking of outcomes measures and when the patient selects the extremes of the choices to emphasize the importance of their particular situation
Clinician-centric biases
Observer expectation bias	Occurs when observers erroneously record data to match expected and desired outcomes
Interviewer bias	Involves the tendency of the interviewer to obtain answers that support preconceived notions; typically through biasing face-to-face interviews. Includes therapeutic personality bias in which the clinician functions in a method that drives a specific outcome of the clinically tested intervention

Attention bias

Attention bias, or the Hawthorn effect, occurs when research subjects or patients change their behavior when they know they are being observed.¹³ In many cases, the research subject behaves in an artificial manner that they think is appropriate for the study dynamics. Patients in a clinical setting where the research study is performed may also modify their own behavior when they know it is a component of observation.

Reporting bias

A similar concept of attention bias is reporting bias. Reporting bias⁸ occurs when research subjects collaborate with researchers to give answers in the direction they perceive are of interest to the study. Often, the consent form and the explanation of the study can drive the research subject to provide the expected outcome examined in the study. Two forms of reporting bias include social responsiveness bias and obsequiousness bias. Social responsiveness bias occurs when subjects provide what they feel are socially desirable responses (i.e. more positive comments after an intervention) when examined secondary to concerns of confidentiality.^3,14 The bias is enhanced when the research subject knows that the clinician will be privy to the results of their outcomes. In contrast, obsequiousness bias occurs when questions are answered in a manner that is sycophantic to the research.⁸

Regression dilution bias

Regression dilution bias is the phenomenon associated with an extreme value on the initial measurement followed by less extreme subsequent assessment (a regression to the mean).^15,16 With regression dilution bias, the initial measure is typically inflated. The phenomenon generally occurs in longitudinal studies but can greatly influence the outcomes in immediate effects studies when only one measurement point is captured. The finding is not to be confused with the Proteus Phenomenon,¹⁷ which is a form of bias that occurs over a number of studies. With the Proteus Phenomenon, the first investigation of a particular trial design will typically have inflated effect sizes, versus follow up studies that lack the compelling findings of the initial study.

Extreme response bias

Extreme response bias⁸ occurs when the practitioner is present during taking outcomes measures and when the patient selects the extremes of the choices to emphasize the importance of their particular situation. The phenomenon is more likely present during research studies than clinical practice, specifically if the impact of the research study is high and the research subjects feel that they play a large role in the outcomes of the study.

Mode of administration: clinician-centric biases

Forms of clinician-centric biases include observer expectation bias and interview bias. Each of the biases is described below and in Table 1 for further understanding.

Observer expectation bias

Observer expectation bias occurs when observers erroneously record data to match expected and desired outcomes.^18,19 This finding is most prevalent during clinical scoring mechanisms (range of motion, strength testing, etc.) and less prevalent with self-report measures.

Interviewer bias

As with observer expectation bias, interview bias involves the tendency of the interviewer to subconsciously obtain answers that support preconceived notions. Interviewer bias differs from observer expectation bias in that the bias is most prevalent during face-to-face interviews when patients seek clarification of scoring values. This is most evident when certain words are emphasized during the interview of cases but not of controls (or vice versa) or when clarifications of the interventions are provided that are not part of the protocol.¹⁶ Germane to interviewer bias is therapeutic personality bias. Therapeutic personality bias involves elements of interviewer and expectancy bias and is an umbrella term for the unblended clinician’s influence on the patient’s perception of benefit.¹⁸ When patients and clinicians interact in a clinical manner the influence of this form of bias is most significant.²⁰

How to control for mode of administration bias

Patient report of outcomes is a necessity for research, thus it is paramount that researchers and clinicians utilize effective methods for controlling bias in a study. The most effective methods include: (1) blinding (or masking) of allocation; (2) standardization of outcome measure dispensation; (3) removal of clinicians from the outcomes collection process; (4) use of written report only; and (5) fair diligence in informing the research subject of study purpose and objectives.

Blinding data collectors to treatment allocation will reduce the likelihood of therapeutic personality bias, expectancy bias, social responsibility bias, and obsequiousness bias. It removes the relationship aspect between the assessor and patient and lessons the unconscious influence of the clinician toward the desired study effects. For research studies, data collectors should not only be blinded, but may also benefit from being non-clinicians. This reduces the likelihood of expectancy bias and obsequiousness bias.

Research organizers should tell patients that the clinicians involved in the interventions will not have access to results. This further removes the risk of social responsibility bias, therapeutic personality bias, and obsequiousness bias and reduces the fear of reporting bias associated with confidentiality. In turn, clinicians should be blinded from results until the end of the study.

Outcomes measures should be written, self-report only. This controls the effects of extreme response bias and regression dilution bias but also deters the possibility of interviewer bias and therapeutic personality bias. When completing the written, self-report forms, the patient should be sequestered from others to reduce the effects of attention bias as well as interviewer bias.²¹ The mode of administration should be standardized; including timing and outcome explanation.

Lastly, an independent explanation of expected study benefits and purpose may assist in lessening the influences of reporting bias. Patients often attempt to help researchers meet their study goals and too much information regarding the desired outcomes of the study will lean the study results in that direction. Using an independent party to explain the study purpose should reduce unintentional persuasion toward a specific finding.

Main points

Mode of administration involves the how, when, and in what manner a self-report questionnaire is dispensed.

1. Mode of administration bias can be sub-divided into two broad categories: unintentional or intentional patient bias; and unintentional or intentional clinician-centric bias.

2. The most effective methods to control for mode of administration bias include: blinding; standardization of outcome measure dispensation; removal of clinicians from the outcomes collection process; use of written report only; and fair diligence in informing the research subject of study purpose and objectives.