Table 3.

Overall strength of evidence by outcome

Outcome	Relevant studies	Evidence statement	Strength of evidence	Comments
Safety	=Kellermann et al 12 =Little et al 13 =Middleton et al 17 =Poote et al 18 =Price et al 22 Madan26	No evidence of a difference in risk between health professionals and symptom checkers	Weaker	Rating changed from stronger based on study numbers and design to weaker because of low numbers of adverse events reported
Clinical effectiveness	−Little et al 13 ?Madan26	Insufficient evidence to draw any firm conclusions	Very limited
Costs/cost effectiveness	+Babylon Health23 ±Cowie et al 25 +Madan26	Insufficient evidence to draw any firm conclusions	Inconsistent
Diagnostic accuracy	?Berry et al 15 −Berry et al 38 − Berry et al 16 − Price et al 22 ?Semigran et al 6 −Semigran et al 21 =Sole et al 19	Symptom checkers appear inferior to health professionals in terms of diagnostic accuracy	Stronger	Mainly for specific conditions or preprepared vignettes
Disposition accuracy	=Babylon Health23 −Berry et al 16 =Middleton et al 17 ?Nijland et al 20 − Poote et al 18 ±Semigran et al 6 ±NHS England 7	Inconsistent findings on accuracy of disposition	Inconsistent	Performance variable between different systems
Service use/diversion	?Kellermann et al 12 ±Little et al 13 ±Poote et al 18 ?Carter et al 24 ?Cowie et al 25 +Madan26 ±NHS England 7 +Babylon Health23 −Luger et al 33	Inconsistent findings on effects on service use	Inconsistent
Compliance	?Nijland et al 20 ?NHS England7	No comparative data on compliance	Very limited
Patient/carer satisfaction	?Nagykaldi et al 35 ?Nijland et al 20 ?Price et al 22 +Yardley et al 14 ?Carter et al 24 ?Cowie et al 25 ?Madan26 ?NHS England7 ?Lanseng and Andreassen30	Most studies report high rates of patient satisfaction with symptom checkers and e-consultation systems generally	Weaker	Few studies with comparator data

Controlled studies in bold; =means no significant difference in outcomes; +means better outcome with symptom checker; ±varying results within study; ? results difficult to interpret in comparative terms.