Table 2.
Article sample results.
| Sample article | Example |
|---|---|
| Persuasive rhetoric of measurement | |
| Explicitly persuasive forms of rhetoric | |
| Reported a “substantial link” between the independent and dependent variables where estimated effects were normatively small (i.e., r = 0.17 and d = 0.34). | Article 1 |
| Explicit reference to the importance of “objective measures,” without elaboration of what constitutes objective in reference to the measure used. | Articles 6, 18, 33 |
| Stated the measure used in the study “has undergone rigorous evaluation and been found to perform well relative to similar measures,” without reporting explicit psychometric evidence to justify. | Article 19 |
| Described instrument used in study as the “gold standard” for the assessment of the phenomenon without elaboration of why this marker of excellence was provided. | Article 36 |
| Common or rote expressions and tropes | |
| Vague gestures to previous research, validity, and reliability | |
| “Previous research has shown that…measures are more sensitive to [focal phenomenon].” | Article 2 |
| “Previous research finds the [measure]has adequate test–retest reliability.” | Article 19 |
| “Previous research has demonstrated the validity of [the measure].” | Articles 28, 37 |
| Reported “reliability and validity” as a general property. | Articles 12, 13, 22, 27, 31 |
| Metaphors and other literary styles | |
| Metaphors | |
| Measure was described “tap[ping]* children’s ability to suppress a dominant response and undertake a subdominant response.” | Article 5 |
| “The results revealed* a significant three-way interaction between age group, condition, and perceived partner closeness.” | Article 26 |
| References to “emerge” or “emerging” in relation to measured phenomena. | Articles 5, 12, 20, 21, 22, 27, 29 |
| References to “detect” or “detection” in relation to measured phenomena. | Articles 1, 2, 6, 8, 17, 19, 21, 22, 25, 26, 28, 30, 32, 33, 35, 38 |
| Use of “metaphorical story-telling” (Carlston, 1987). | Articles 16, 20 |
| Use of passive voice | |
| “The Structured Clinical Interview for the DSM–IV, nonpatient edition … was administered* to assess for Axis I DSM–IV disorders.” | Article 15 |
| “Reward valuation ability* was assessed…” | Article 18 |
| Misascribing actions or capacity | |
| e.g., “the measure* assessed” or “items access” as opposed to “We [the researchers] assessed … with the measure/items,”“this study* conceptualized…” instead of “We conceptualized…” | |
| A growing literature has explored…” instead of “A growing number of researchers have explored…” | Articles 3, 4, and 12 |
| Confusing expressions, ambiguous, or unjustifiable claims | |
| Construct validity | |
| “Such improvements in ADHD knowledge, use of behavioral strategies, and adaptive thinking skills, as measured by our study-specific measures, speak to their potential role as clinical change mechanisms, lending support to the construct validity of our design*” | Article 3 |
| “[Cited authors] have provided evidence for the construct and criterion-related validity of this measure.” | Article 31 |
| Constructs | |
| “As implicated in [cited study] meta-analysis, alliance is a living, * evolving, and dynamic construct that can be perceived and reported differently throughout the course of therapy.” | Article 1 |
| Describe the construct of “functioning” as representing* “a rather multifaceted construct, whose complexity may not have been captured by [the measure].” | Article 16 |
| Described the relationship between the focal construct and other constructs as follows: “anxiety, depression, and posttraumatic stress disorder (PTSD) are constructs that display* significant overlap with alexithymia.” | Article 18 |
| Generativity is a distinct construct driven by* the underlying desire to contribute to the community and future generations through one’s own legacy.” | Article 34 |
| Missing Information | |
| “It is beyond the scope of this article to report on all of the behavioral outcomes that were assessed in the current study but, in addition to measures of subjective response…” | Article 19 |
| Hedging | |
| “Various measurement approaches have been utilized in the field … Each of these measurement approaches has associated advantages as well as disadvantages and may capture distinct aspects of daily life.” | Article 27 |
| Other | |
| Conflating ordinary and technical meanings of terms (e.g., reliable [as in dependable] measurement tools and measurements demonstrating high psychometric reliability). | Articles 1, 3, 5, 8, 17 and 30 |
| Conflating aggregate statistical findings with individual-level causal claims (e.g., “Previous research has demonstrated the validity of this manipulation, showing, for example, that social exclusion makes individuals more aggressive … and reduces prosocial behavior,” and “Participants in the frustration condition further reported lower levels of satisfaction of the need for self-esteem”). | Article 28 |
| Confusing statements | |
| “[Cited article] reported that the [measure] can be applied in a four dimensional or unidimensional structure to collect data with good reliability and validity.” | Article 13 |
| “…the experimental design could detect the presence/absence of the [measure] effect moderately well, but likely does not reliably detect small changes in the [measure] effect across conditions. To reliably detect a 15 ms change in the [measure] effect at roughly 80% power, for example, we estimate would require 100 participants per group.” | Article 21 |
*Emphasis added.