Table 2.
Inter-rater reliability results for all coded themes
| Theme | Percent agreement | Krippendorff’s alpha(95% CI) |
|---|---|---|
| A. Overall themes: Crowdsourced final response vs research team member (subsample n=350) | ||
|
| ||
| Tweet is about dabbing marijuana concentrates | 91% | 0.79 (0.66 – 0.91) |
| Currently dabbing, recently dabbed, or wanting/planning to dab a | 85% | 0.62 (0.43 – 0.77) |
| Describes first experience with dabbing | 100% | 1.00 (1.00 – 1.00) |
| Mentions that dabbing aids in relaxation, sleep, or solving problems | 96% | 0.69 (0.40 – 0.93) |
| Heavy or excessive dabbing | 94% | 0.77 (0.57 – 0.93) |
| Describes intense or extreme effects from dabbing | 95% | 0.82 (0.65 – 0.96) |
| Mentions other forms of marijuana, alcohol, or other substances | 94% | 0.74 (0.52 – 0.91) |
| Advertisement or promotion of dabbing products | 98% | 0.75 (0.35 – 1.00) |
|
| ||
| B Specific extreme effects from dabbing: All tweets with “intense or extreme effects” theme further coded by two research team members | ||
|
| ||
| Physiological effects | 92% | 0.83 (0.71 – 0.93) |
| Passing out/losing consciousness | 99% | 0.98 (0.94 – 1.00) |
| Loss of body control or inability to move | 97% | 0.78 (0.50 – 1.00) |
| Respiratory effects | 100% | 1.00 (1.00 – 1.00) |
| Perspiring | 100% | 1.00 (1.00 – 1.00) |
| Crying/tearing up | 100% | 1.00 (1.00 – 1.00) |
| Nausea/vomiting | 99% | 0.94 (0.81 – 1.00) |
| Cognitive or psychological effects | 93% | 0.73 (0.51 – 0.92) |
| Memory loss/forgetfulness | 97% | 0.89 (0.68 – 1.00) |
| Confusion/distorted reality | 91% | 0.79 (0.57 – 1.00) |
|
| ||
| C. In-depth examination of respiratory effects: All tweets with specific respiratory-related keywords of interest coded by two research team members | ||
|
| ||
| Tweet is about respiratory effects from dabbing | 94% | 0.82 (0.67 – 0.94) |
| Tweet mentioned coughing after dabbing | 98% | 0.96 (0.90 – 1.00) |
| Describes excessive coughing | 91% | 0.83 (0.74 – 0.90) |
| Dabbing is hard on the lungs | 99% | 0.96 (0.87 – 1.00) |
| Difficulties with breathing after dabbing | 99% | 0.94 (0.77 – 1.00) |
| Tweeter explicitly dislikes respiratory effects from dabbing | 98% | 0.55 (0.00 – 1.00) |
|
| ||
| D. In-depth examination of “passing out”: All tweets with specific passing out-related keywords of interest coded by two research team members | ||
|
| ||
| Tweet is about passing out/losing consciousness from dabbing | 97% | 0.81 (0.57 – 1.00) |
| Tweeter uses dabs in order to pass out, or plans/desires to pass out | 93% | 0.86 (0.76 – 0.96) |
| Tweeter actually experienced passing out after dabbing | 93% | 0.83 (0.71 – 0.95) |
| Someone else (besides tweeter) experienced passing out after dabbing | 96% | 0.85 (0.70 – 0.96) |
| Tweeter explicitly dislikes the effects of passing out after dabbing | 99% | 0.62 (0.00 – 1.00) |
a
Percent agreement and Krippendorff’s alpha for the theme concerning currently/wanting/planning to dab was relatively low in comparison to the other themes (77% and Krippendorff’s alpha 0.49). However, when we restricted use of this code to only tweets with no other theme present, reliability was better (reported in table above). Thus, this theme is reported only in the absence of the other themes.