Table 2.

Barriers to developing and implementing readmission models

Domain	Barriers	Evidence*	Example†	Strategies to overcome barriers
Validation	Poor generalisability	Substantial	“[We are] questioning the generalization of the model to our population.” (33)	More advanced modelling strategies than regression Better access to data to improve prediction External rather than internal validation and calibration Account for different clinical subpopulations
	Low discriminative value	Limited	“[The biggest barrier is] having a good model to start with.” (48)
Features	SDH not included	Extensive	“We need to look well beyond SES, etc. to social support and healthcare beliefs and behaviors.” (2)	Consider SDH and HAF when developing models Place SDH and HAF in structured data fields Record better measures of SDH and HAF Improve natural language processing and extraction More research on the factors leading to readmission Assess the possibility of using shorter timeframes
	HAF not included	Substantial	“We need to look beyond academic status and for-profit status … to understand processes.” (2)
Timeframe	Timeframe not optimised	Limited	“30 days is probably too long to provide an accurate prediction.” (48)
Data access	Barriers to data access	Extensive	“No matter how complex and good the model is, it is only as good as the data it has.” (20)	Federal incentives for data sharing Federal sanctions for data blocking Aggregate data sets from multiple sources Include sources with multiple data types Better documentation practices and training Human resources for data entry and validation Better tools or interfaces for data collection Implement better structured data capture processes Reduce unnecessary documentation burdens on healthcare providers, so the quality can improve
	Inadequate interoperability	Substantial	“We need access to databases, especially linked primary and secondary care ones.” (9)
	Insufficient data	Substantial	“We don’t have the necessary data and we don’t know what the necessary data even are.” (27)
	Poor quality data	Substantial	“When we use routinely collected data in EHRs, the quality is less reliable.” (38)
	Lacks current information	Limited	“If and when the factors change, we don’t know what they are - the case managers do.” (34)
Resources	Lacks personnel or expertise	Substantial	“[We lack] staffing resources for adequate capture of data and analytics of accumulated data.” (43)	More support from the hospital administration Better financial support for model development Better packages in statistical software to help developers move beyond logistic regression
	Financial barriers	Substantial	“[There is] reluctance to make the necessary investments to access the EHR’s back end.” (1)
Vision	Competing priorities	Limited	“It [the model] is not a priority… they [the administration] have competing priorities. (14)	Engage key stakeholders and senior decision-makers Establish governance over model implementation Select a clinical or administrative champion Implement financial and other incentives for hospitals
	Lack of leadership	Very limited	“[There is] no operational leadership, so the model hasn’t been implemented.” (21)
Clinical relevance	Poor perceived relevance	Extensive	“Risk score doesn’t necessarily flag the patients in whom we can most usefully intervene.” (9)	Models must determine how providers can intervene More research on hospital processes and needs Invest more human resources to prevent readmission Feedback for providers if readmission averted (or not) Validate and calibrate model in the new clinical setting Ensure users receive the most up-to-date predictions
	Unclear usefulness	Substantial	“[Models must] fit into a workflow where an intervention can be made.” (16)
	Poor perceived accuracy	Substantial	“I’ve found that it [the model] is not accurate at the individual patient level.” (6)
Workflow integration	Poor workflow integration	Extensive	“Getting it inserted into the EHR in a way that requires little provider effort is tough.” (48)	Consider who will receive the results, when, and how Tie integration to physical risk-reduction interventions Train, orient, and support potential users of the model Plan to iteratively improve on integration over time
	Alert fatigue	Very limited	“Clinicians get alert fatigue and stop paying attention to the results.” (33)
Maintenance	Antiquated model or interface	Limited	“Our commercial partner no longer supports the front end they developed [for our model].” (9)	Integrate within system with long-term IT support Designate responsibility of updating model annually

*Extensive evidence (≥8 mentions); substantial evidence (4–7 mentions); limited evidence (2–3 mentions), very limited evidence (1 mention).

†Numbers in parentheses refer to participant study ID.

EHR, electronic health record; HAF, healthcare-associated factors; IT, information technology; SDH, social determinants of health; SES, socioeconomic status.