Table 3.

The reach, effectiveness, adoption, implementation, and maintenance evaluation measures of InCharge Health implementation.

Domains and measures		Data sources
Reach	Sociodemographic characteristics of patients at each site Proportion and representativeness of patients screened for the study (numerator) among all patients who receive hydroxyurea treatment (denominator) at each site Proportion and representativeness of patients eligible for the study (numerator) among all patients who receive hydroxyurea treatment (denominator) at each site Proportion and representativeness of patients participating/enrolled in the study (numerator) among all patients who receive hydroxyurea treatment and were eligible (denominator) at each site	Clinic data collection forms Clinic population demographics and treatment data, study database Screening log Qualitative interviews
Effectiveness	Primary outcome >20% improvement in the PDCa for hydroxyurea among those receiving the intervention	Prescription drug claims (PDC for hydroxyurea refills)
	Secondary outcomes Change in Quality of life, self-efficacy, perceived health literacy Change in percentage of patients with EDb visits, hospitalizations since the last study visit Change in biomarkers of hydroxyurea effect (MCVc, ANCd, ARCe, indirect bilirubin, HbFf, Hbg, LDHh)	Patient surveys (ASCQ-Mei, PROMISj, Perceived health literacy, self-efficacy) Medical chart abstraction Qualitative interviews
Adoption	Proportion and description of clinics in each site agreeing to support InCharge Health Proportion and description of providers in each clinic agreeing to support InCharge Health (ie, proportion enrolled on the study)	Clinic administrative data and data collection forms Qualitative interviews
Implementation	Consistency with which sites are able to implement the app as planned Qualitative assessment of any adaptations or enhancement to recruitment strategies needed to meet enrollment by the clinic, by site Assess adaptation of training needed to improve InCharge Health implementation at each site Engagement with the app: percentage, number, and representativeness of patients who used InCharge Health during the study period (low, medium-low, medium, or high use; in the entire practice) Proportion, number, and characteristics of patients who complete the study among those who initiate the use of the app but then later discontinue at each site Percentage and characteristics of patients who reported satisfaction with the InCharge Health app (MARSk scale) Clinic/provider assessment of perceptions of InCharge Health app for further scale-up or sustainability—ease of use, preferred features, and so on	App usage statistics Patient surveys Qualitative interviews
Maintenance/sustainability	Extent to which program leaders express a desire or intent to continue providing the app with patients at the conclusion of the research Percentage of patients who continue to use the app beyond the study period and their representativeness	Pharmacy claims data (PDC for hydroxyurea refills) App use statistics Clinic data collection forms Qualitative interviews

^aPDC: proportion of days covered.

^bED: emergency department.

^cMCV: mean corpuscular volume.

^dANC: absolute neutrophil count.

^eARC: absolute reticulocyte count.

^fHbF: fetal hemoglobin.

^gHb: hemoglobin.

^hLDH: lactate dehydrogenase.

ⁱASCQ-Me: adult sickle cell quality of life measurement information system.

^jPROMIS: Patient-Reported Outcomes Measurement Information System.

^kMARS: mobile app rating scale.