. 2019 Jul 26;56:0046958019861257. doi: 10.1177/0046958019861257

Table 2.

Indicator Description.

Author	Endpoints	Indicator type	Indicator denomination/description	Indicator definition
Quantitative studies
Garcia-Cardenas et al.²³	Acceptability^° (“The perception among implementation stakeholders (patients and GP) that the MRF service is agreeable, palatable, or satisfactory”)	PBI	“Acceptability rate of patient-targeted care plans”	Numerator: No. of “recommendation[s] given by the pharmacist to the patient to prevent and/or solve a negative outcome associated with a medication” that were accepted	Denominator: The total number of recommendations given by the pharmacist to the patients
		PBI	“Acceptability rate of GPs-targeted care plans”	Numerator: No. of “recommendation[s] given by the pharmacist to the GP to prevent and/or solve a negative outcome associated with a medication during multidisciplinary collaboration” that were accepted	Denominator: The total number of recommendations given by the pharmacist to the GPs
	Feasibility^° (“The extent to which the MRF service can be successfully used or carried out within the pharmacy”)	CBI	“Patient recruitment rate”	No. of patients recruited
		PBI	“Retention-participation rate”	Numerator: No. of patients who received the service.	Denominator: No. of patients recruited
		ROBI	“Service-offering rate”	Numerator: “Service offering by the pharmacy”	Denominator: “service request by the patient ratio”
	Fidelity^° (“The degree to which the MRF service is implemented and provided as it was described”)	CBI	“Dose: The amount, frequency and duration of MRF service provision”	“Number of patient’s visits to the pharmacy for service provision and time per patient spent on service provision”
	Implementation costs^° (“Cost impact of the MRF implementation effort”)	CBI	“Implementation costs”	“Direct measures of implementation costs, including the cost of the service provider and resources needed for service provision”
	Penetration^° (“Level of integration of the MRF service within the pharmacy and its subsystems”)	SROBI	“Penetration rate”	Numerator: “Number of eligible patients who use the service”	Denominator: “[N]umber of potential patients eligible for the service”
	Service implementation efficiency^§ (“The degree to which the service provider improves his/her skills and abilities to provide it”)	CBI	“Service implementation efficiency”	“Change in the time spent on the service provision through the implementation program”
Garner et al.¹	Fidelity ^°	CBI	“Session exposure”	“average number of A-CRA sessions that the organization delivered to its respective adolescent client”
	Fidelity ^°	ROBI	“Procedure exposure”	“average number of unique A-CRA procedures that the organization delivered to its respective adolescent client”
	Penetration ^°	CBI	“Client penetration”	“unduplicated number of adolescent clients that the organization delivered at least one A-CRA procedure over the course of their SAMHSA/CSAT project”
	Penetration ^°	CBI	“Staff penetration”	“total number of ‘staff A-CRA certification days’”
Guenther et al.²⁷	Penetration ^*	PBI	“KMC service availability: Percentage of facilities with in-patient maternity services with operational KMC”	Numerator: “Number of health facilities in which KMC is operational^*”	Denominator: “Number of health facilities with inpatient maternity services”
	Adoption ^*	PBI	“Weighed at birth: Percentage of newborns weighed at birth”	Numerator: “Number of newborns weighed at birth”	Denominator: “Number of live births”
		PBI	“Identification of newborns ≤2000 g: Percentage of live births identified as ≤2000 g”	Numerator: “Number of newborns identified as ≤2000 g”	Denominator: “Number of live births”
		SROBI	“KMC coverage‡: Percentage of newborns initiated on facility–based KMC”	Numerator: “Number of newborns initiated on facility-based KMC”	Denominator: “Expected number of live births OR expected number of LBW babies”
		PBI	“KMC monitoring: Percentage of KMC newborns who are monitored by health facility staff according to protocol”	Numerator: “Number of newborns admitted to KMC who are monitored by health facility staff according to protocol (includes at minimum: assessing feeding, STS duration, weight, temperature, breathing, heart rate, urine/stools)”	Denominator: “Number of newborns initiated on facility-based KMC”
		PBI	“Status at discharge from KMC facility: Percentage of newborns discharged from KMC facility who: met facility criteria for weight gain/health status; left against medical advice; referred out; or died before discharge”	Numerator: “Number of newborns discharged from facility–based KMC who: 1) met facility criteria for weigh gain, health status, feeding, thermal regulation, family competency, etc.; 2) left against medical advice; 3) referred out for higher level care; 4) died before discharge”	Denominator: “Number of newborns discharged from facility-based KMC”
	Fidelity ^*	PBI	“KMC follow-up: Percentage of newborns discharged from facility-based KMC that received follow-up per protocol”	Numerator: “Number of newborns discharged from facility-based KMC that received follow-up per protocol”	Denominator: “Number of newborns discharged alive who received facility-based KMC”
Heidkamp et al.⁹	Penetration ^*	ROBI	“HSA-to-population ratio”	Numerator: “HSAs working at time of assessment (Data source: MOH district records)”	Denominator: “Total population under 5 years (Data source: NSO Census 2008 - 2013 Projection)”
		PBI	“Proportion of HSAs trained in iCCM”	Numerator: “HSAs trained in iCCM”	Denominator: “HSAs working at time of assessment (MOH district records)”
		PBI	“Proportion of hard-to-reach areas with iCCM-trained HS”	Numerator: “HSAs trained in iCCM who work in HTRA (Data not available)”	Denominator: “Total no. of HTRA (Data not available)”
		RABI	“Proportion of iCCM HSAs who have seen a sick child in the past 7 days”	Numerator: “HSAs who have seen a sick child in the past 7 days”	Denominator: “Surveyed HSAs working in iCCM at the time of the assessment”
		PBI	“Proportion of iCCM HSAs who are living in their catchment area”	Numerator: “HSAs who live in their catchment area”	Denominator: “Surveyed HSAs working in iCCM at the time of the assessment”
	Fidelity ^*	RABI	“Proportion of iCCM HSAs with supply of key iCCM drugs in last 3 months”	Numerator: “HSAs with no stockouts of more than 7 days of cotrimoxizole, lumefantrine-artemether, ORS, and/or zinc in the last 3 months (HSA must have at least one dose of unexpired drug at the time of survey)”	Denominator: “Surveyed HSAs working in iCCM at the time of the assessment”
	Fidelity ^*	RABI	“Proportion of iCCM HSAs with supply of life-saving CCM [sic] drugs in the last 3 months”	Numerator: “HSAs with no stockouts of any duration of three life-saving drugs (cotrimoxizole, lumefantrine-artemether, and ORS) in the last 3 months (HSA must have at least one dose of unexpired drug at the time of survey)”	Denominator: “Surveyed HSAs working in iCCM at the time of the assessment”
	Penetration ^*	ROBI	“Utilization”	“Mean no. of sick children seen per iCCM-trained HSA in the last 1 month”
	Adoption ^*	RABI	“Proportion of iCCM HSAs supervised at village clinic in the last 3 months”	Numerator: “HSAs supervised at village clinic in CCM [sic] in the last 3 months”	Denominator: “Surveyed HSAs working in iCCM at the time of the assessment”
	Adoption ^*	RABI	“Proportion of iCCM HSAs supervised in the last 3 months with reinforcement of clinical practice”	Numerator: “HSAs supervised at village clinic with observation of case management or practicing case scenarios or mentored in health facility in the last 3 months”	Denominator: “Surveyed HSAs working in iCCM at the time of the assessment”
Karim et al.²⁵	Penetration ^* *(“Program intensity”)*	RABI	“period prevalence of household visits by HEWs”	“the percentage of women in a kebele who were visited by a HEW during six months preceding the survey”
		RABI	“period prevalence of household visits by CHPs”	“the percentage of women in a kebele who were visited by a CHP during the last six months”
		PBI	“proportion of households with a FHC”	Not mentioned
		PBI	“proportion of model families”	“percentage of respondents who reported that their household was a model family household or they were working towards it”
McCullagh et al.²⁸	Acceptability ^§ Unclear	ROBI	“iCPR tool use”	Numerator: “opened tool”	Denominator: “iCPR encounters”
		Unclear	“iCPR tool completion”	–
		Unclear	“iCPR smartset completion”	–
Saldana et al.²⁹	Feasibility ^*	ROBI	“Duration score”	“the amount of time that a county/agency takes in a stage is calculated by dates of entry through date of final activity completed”
Saldana et al.²⁹	Fidelity ^*	PBI	“Proportion score”	“the proportion of activities completed within a stage”
Stiles et al.³⁰	Penetration ^{^}	RABI	“Penetration rate” “within a specified time frame”	Numerator: “the number of eligible persons who used the service(s)”	Denominator: “the total number of eligible persons”
		RABI	“Annual penetration”	Numerator: “Total persons having any service contact during the year”	Denominator: “Total persons eligible at some point during the year”
		RABI	“Average monthly penetration”	Numerator: “Total user months over a 1-year period”	Denominator: “Total eligible months over a 1-year period”
Weir and McCarthy³¹	Adoption ^*	PBI	Not mentioned	“Proportion of orders entered by each clinical role (M.D., R.N., R.N. verbal, pharmacist, clerk)”
		PBI	“Proportion of orders entered by R.N.s”	“^# of R.N., verbal, and telephone orders/total # of orders”
	Fidelity ^*	PBI	“Mean time for unsigned orders”	“^# of orders left unsigned after 6 hours/total # of orders”
		CBI	Not mentioned	“^# of orders incorrectly entered as text orders”
		PBI	“Proportion of antibiotic orders expiring w/out a DC order”	“^# of antibiotic orders expiring w/out a DC order/total # of antibiotic orders”
	Penetration ^*	PBI	“Origin of order (direct entry, order set, or personal order set)”	“^# of each type/total ^# of orders”
	Fidelity ^*	PBI	“Proportion of orders with entry error”	“^# of orders DC’ed by each service and reentered (e.g., lab, pharmacy)/ total ^# of orders per service”
	Fidelity ^*	CBI	Not mentioned	“^# of orders incorrectly entered as text orders”
	Unclear	CBI	Not mentioned	“^# of automatically generated co-orders (e.g., corollary, expired, transfer, etc.)”
	Feasibility ^*	CBI	Not mentioned	“Time for each service between orders and event (e.g., lab collection, imaging procedure done)”
		CBI	Not mentioned	“^# of times unplanned computer down per week”
		CBI	Not mentioned	“^# of DC’ed orders by clerk with ‘duplicate’ as reason”
	Fidelity ^*	CBI	Not mentioned	“^# of missing print lab and pharmacy labels”
	Fidelity ^*	CBI	Not mentioned	“^# of alerts unresolved after 12 hours (indicating possible delivery to wrong clinician)”
	Feasibility ^*	CBI	Not mentioned	“Time from order of stat meds to actual delivery”
	Feasibility ^*	CBI	Not mentioned	“^# of incident reports for errors”
	Fidelity ^*	CBI	Not mentioned	“^# of duplicate ‘sticks’ for the same lab order”
		CBI	Not mentioned	“^# of hours that alerts for out-of-range lab values remain unsolved”
		CBI	Not mentioned	“^# of hours that out-of-range lab values remain out of range, e.g., potassium, haematocrit”
		ROBI	Not mentioned	“Average time for X-ray readings to be available”
		ROBI	Not mentioned	“Average time for orders to be acknowledged by R.N.”
		CBI	Not mentioned	“^# of hours between the two signers for chemo-therapy orders”
		CBI	Not mentioned	“^# of minutes postop orders are activated after transfer to floor”
		CBI	Not mentioned	“^# of minutes between activation of admission orders and official bed assignment”
Proctor et al.⁵	Penetration	PBI	Not provided	“the number of providers who deliver a given service or treatment, divided by the total number of providers trained in or expected to deliver the service”

Note. GP = general practitioners; MRF = pharmacist-led medication review with follow-up; PBI = proportion-based indicator; CBI = continuous variable indicator; ROBI = ratio-based indicator; SROBI = standardized ratio–based indicator; RABI = rate-based indictors; A-CRA = Adolescent Community Reinforcement Approach; SAMHSA = Substance Abuse and Mental Health Services Administration; CSAT = Center for Substance Abuse Treatment; KMC = Kangaroo Mother Care; LBW = low birth weight; STS = skin-to-skin; HSA = Health Surveillance Assistant; MOH = Ministry of Health; NSO = National Statistical Office; iCCM = Integrated Community Case Management; HTRA = hard-to-reach areas; ORS = oral rehydration solution; HEW = Health Extension Workers; CHPs = Community Health Promoters; FHC = Family Health Card; iCPR = integrated clinical prediction rule; M.D.= physician; R.N. = registered nurse; DC = discontinue.

^°

Indicators categorized by outcomes consistent with Proctor et al.⁵ by the authors of the identified publications.

^§

Indicators categorized by described outcomes of the authors of the identified publications.

Indicators categorized by outcomes described by the authors of this article consistent with Proctor et al.⁵

^{^}

Indicators categorized by Proctor et al.⁵ according to their defined outcomes.