Table 3.
Factors affecting levels of agreement on rating and domain allocations: statistically significant variables in the logistic regressions.
| Vignette | Potential sources/indicators of agreement/disagreement |
|
|---|---|---|
| Rating level | Domain allocation | |
| V1 | • Little variation/High agreement; no significant variables | • Little variation/High agreement; no significant variables |
| V2 | • Ratings given to other vignettes • Experience of inspections • Seniority: rated Outstanding by 72% of junior-level inspectors, compared with 36% of more senior colleagues • Content knowledge: rated Outstanding by 54% of End of life care sub-team members, compared with 35% of members of other sub-teams | • Profession: Fewer CQC staff (65%) and patients/experts by experience (65%) allocated Caring, compared with doctors (89%) and nurses/allied health professionals (73%) |
| V3 | • Ratings given to other vignettes • Experience of inspections • Profession: rated Inadequate by 97% of Experts by Experience, compared with 60% of other inspectors | • Little variation/High agreement; no significant variables |
| V4 | • Ratings given to other vignettes | • No significant variables |
| V5 | • Experience of inspections • Seniority: rated Inadequate by 73% of junior level inspectors and Experts by Experience, compared with 50% of other inspectors • Profession: rated Requires Improvement by 60% of CQC inspectors, compared with 39% of other inspectors | • No significant variables |
| V6 | • Profession: rated Outstanding by 30% of doctors, compared with 3% of other inspectors | • Perceived accuracy of ratings • Ratings given across all vignettes |
| V7 | • Ratings given to other vignettes • Profession: rated Outstanding by 74% of nurses, compared with 57% of other inspectors | • No significant variables |
| V8 | • No significant variables | • Profession: More CQC staff (28%) allocated Effectiveness, compared with nurses/allied health professionals (9%), doctors (11%) and patients/experts by experience (15%) |
| V9 | • Little variation/High agreement; no significant variables | • Profession: Safety was the most popular domain allocation among doctors (48%); Effectiveness the most popular among patients/experts by experience (65%), CQC staff (58%) and nurses/allied health professionals (49%) |
| V10 | • Ratings given to other vignettes | • Experience of inspections: Allocated Responsiveness by 28% of inexperienced inspectors, rising to 63% for the most experienced inspectors |