Table 2.
Comparison of differences in key question characteristics for agree-disagree (AD) and item-specific (IS) questions
| Question Characteristics | Operationalization within and across studies | ||
|---|---|---|---|
| Category | Description | AD | IS |
| Manner of questioning | Whether the sentence to be evaluated is structured as declarative (a statement) or interrogative (a question) | Indirect, structured as a statement | Direct, structured as a question |
| Response dimensions | Continuum that a question asks respondents to consider when constructing their answer | Offered response dimension (intensity of agreement) and underlying response dimension (intensity, quantity, or frequency) do not match | Offered and underlying response dimensions match and measure intensity, quantity, or frequency |
| Threshold words | Intensifiers (e.g., “very”), quantifiers (e.g., “most”), or frequency markers (e.g., “rarely”) that establish a threshold for comparison | Often, but not always, included in the statement | Not applicable |
| Polarity | Whether response dimension is bipolar with both poles or ends of the response dimension presented or unipolar with only one pole presented | Usually bipolar (“strongly agree … strongly disagree”) | Usually unipolar (“not at all satisfied … extremely satisfied”), but may be bipolar (“extremely satisfied … extremely dissatisfied”) |
| Number of response categories | Number of categories or points offered on the response continuum | Category number in AD-IS experiments is almost always held constant between AD-IS comparisons within a study; across studies, category number varies from 4 to 11, with 5 categories being the most common implementation | |
| Labeling of response categories | Labeling of all or only some of the categories using various combinations of words and numbers | Category labeling in AD-IS experiments is almost always held constant between AD-IS comparisons within a study; across studies, labeling varies, with categories fully labeled with words and no numbers being the most common implementation | |
| Direction of response categories | Whether the categories increase in value (e.g., “strongly disagree … strongly agree”, “not at all” … “extremely”) or decrease in value (e.g., “strongly agree … strongly disagree”, “extremely” … “not at all”) | Varies, but categories often decrease in value (e.g., “strongly agree … strongly disagree”) | Varies, but categories often increase in value particularly for unipolar quantity (e.g., “none … a great deal”) and frequency (e.g., “never … always”) dimensions |
| Middle category | For bipolar questions, whether the response categories include a conceptual middle where the dimension transitions from positive to negative; category may be neutral category or mixed | Commonly used bipolar questions often include a middle category (e.g., “neither agree nor disagree”) | If unipolar, no conceptual middle category |
| Battery | Whether questions appear alone or as part of a battery of topically-related items with a common response format | Use same response categories for all items included in a battery | Response categories for items in the battery will likely vary depending on the underlying response dimension |
| Valence of the construct and target objects | Whether the construct and target objects asked about in the questions are inherently positive, negative, neutral, or ambiguous | Valence of the construct and objects in AD-IS experiments is almost always held constant between AD-IS comparisons within a study; across studies, valence varies | |
| Alignment with the construct | Whether the construct and the response categories are positively aligned (e.g., higher-valued response categories indicate higher levels of the construct) or negatively aligned (e.g., higher-valued response categories indicate lower levels of the construct) | Alignment of a construct for an AD-IS experimental pair sometimes varies; across studies alignment varies | |