Table 2.

Summary of implementation using the Behavior Interventions using Technology framework.

Domains		Metrices	Results	Sources
Acceptability
	User level	TEIa mean score and percentage above the acceptability cutoff (>27)	Mean 30.7, 86% moderate-to-high acceptability	Patient survey
	Organization level	Provider attitudes toward the app (1 “Strongly disagree” to 5 “Strongly agree”)	Helpful to provide CBTb: mean=4.6 My patients would benefit: mean 4.6 Improves the quality of care: mean 4.5 Better use of resources: mean 4.4 Fills in an important need: mean 4.3	Provider survey
Adoption
	User level	Percentage of participants who downloaded the app	n=68/73 (93%)	Back-end data
	User level	Percentage of participants who used WMMc after first log-in	n=68 (100%)	Back-end data
	User level	Percentage of participants who completed ≥1 module	n=54 (74%)	Back-end data
Appropriateness
	User level	Score on app perceptions (0 “Did not like it” to 5 “Liked it very much”)	Appearance: mean 3.6 Navigation: mean 3.9 Theme: mean 3.7 Content: mean 3.3	Patient survey
Feasibility
	Organization level	Percentage of clinics agreeing to continue making referrals	All clinics agreed (100%)	Administrative tracking data
	Organization level	Percentage of clinics making referrals	7/8 (88%)	Administrative tracking data
	Organization level	Final sample out of planned sample	n=143/120 (119%)	Administrative tracking data
	User level	Number of technical issues reported or complaints	n=0 (0%)	Administrative tracking data
	User level	Participants comments	Not enough space to download the app, n=2 (3%)	Patient survey
	Organization level	Providers comments	It was easy to refer patients; WMM is something useful that can be integrated in the practice	Provider survey
Fidelity
	User level	Number of days tracking symptoms	Mean 30.5 (SD 29.4); median=19; range 2-56	Back-end data
	User level	Number of participants completing the treatment	n=29 (40%)	Back-end data
Implementation costs
	Organization level	App development costs	As planned	Budgets
	Organization level	Making WMM publicly available	Exceed budget by 7%	Budgets
Penetration
	Organization level	New patients using the app during the referral period and clinic they were referred from	Total N=56 Clinic 1=7 Clinic 2=0 Clinic 3=3 Clinic 4=6 Clinic 5=26 Clinic 6=3 Clinic 7=6 Clinic 8=5	Administrative tracking data; back-end data
Sustainability
	Organization level	Referrals made	100% of the clinics agreed; 88% kept referring	Administrative tracking data
	Organization level	“I will encourage my patients to use the app after the study is over”	92% agreed or strongly agreed with the item	Provider survey

^aTEI: Treatment Evaluation Inventory.

^bCBT: cognitive behavioral therapy.

^cWMM: WebMAP Mobile.