Table III.

Data collection and analysis methods used in the evaluation of CQC’s new model of hospital inspection

Method	Implementation and analysis	Participants	Topics covered
Interview	Face to face; semi-structured Conducted individually by two researchers Recorded and transcribed Thematic analysis using Dedoose	18 key stakeholders within CQC and other national-level organisations involved in the regulation of hospitals or service improvement	The rationale for the design of the model: how it is meant to work, how it differs from previous models, the problems that it seeks to solve, any concerns, what success would look like, the expected impacts, how the impacts will be sustained over time, how the model will work alongside other regulatory processes and organisations
	Telephone; semi-structured Conducted individually by all researchers Recorded and transcribed Thematic analysis using Dedoose	35 inspection team members from 17 teams, spanning the range of roles and professional backgrounds	The inspection process: the usefulness of pre-inspection preparations, whether the composition of the team was right, how KLOEs were determined, the usefulness of KLOEs, how findings and ratings were arrived at, how the unannounced and announced inspections compared, how the process might be improved, interest in participating in future inspections
		25 hospital staff from 13 inspected organisations – senior managers or other staff responsible liaising with CQC about the inspection, plus some operational staff in inspected service areas	The ability of the inspection to identify important performance issues and promote performance improvement: how services prepared prior to inspection, how well the inspection process worked, the accuracy of the inspection report and ratings, the impact on services and service improvement
Observation	Non-participant observation Conducted by pairs of researchers Semi-structured free text reflective summary sheet completed. Used to inform interviews and surveys; an aide-memoire for triangulation with other data	Preparation (1 day) and inspection (lasting 2-4 days) of six organisations, spanning a range of sizes, CQC risk categories and governance types. Shadowing individual inspection team members within selected sub-teams and observing on-site and off-site team meetings Subsequent single-day observations by individual researchers of inspections of three organisations to check for process changes	Inspection team expertise, use of intelligence/surveillance data, preparation and planning, logistics, inspection team (roles and responsibilities, dynamics, leadership, functioning), inspection process, use of evidence to form judgements, feedback process, provider and stakeholder engagement and response
Survey	Online; mix of Likert scales, tick boxes and free text boxes Personalised e-mail invitations with up to two reminders Univariate and bivariate statistical analyses using SPSS. Thematic analysis of free text data using Excel	369 team members from inspections of 19 organisations. Response rate 66%	Motivations for joining the inspection team, the usefulness of various tools and processes designed to support the inspection, confidence in having the necessary skills to gathering information using the various mechanisms available, the accuracy of ratings, intentions to participate in future CQC inspections
		698 managers and senior clinicians from 18 inspected organisations. Response rate 40%	Preparations made for the inspection visit, the ability of various inspection activities to provide inspectors with accurate information, the knowledge and skills of the inspectors, how well the CQC identifies good practices and concerns, actions likely to be taken as a consequence of the inspection, the impact of those actions