Table 3.

The assessment criteria (and supporting references) used in Step 2 to assess the solution options

Criteria	Sub-criteria
Market strength
(Gezici et al. 2019;	∙ Recommendations
Adewumi et al. 2016)	(Le Blanc and Korn 1994; Kornecki and Zalewski 2005)
	∙ Current user base (Le Blanc and Korn 1994)
	∙ Supplier state (Alnafjan et al. 2013; Church and Matthews 1995;
	Du Plessis 1993; Le Blanc and Korn 1994)
	∙ *Supplier relationship (Taherdoost and Brard 2019)
	∙ Fashion, buzz (Suomi 1992)
Strategy & Culture
(Le Blanc and Korn 1994)	∙ Innovation adoption
	∙ Planning vs organic culture (Prather 1993; Lester and Wilkie 2004)
Productivity (Du Plessis 1993)
	∙ Collaborative support (Biffl et al. 2009; Rivas et al. 2008;
	Post and Kagan 2000; Gezici et al. 2019)
	∙ Support for work practices (Bashroush et al. 2017;
	Håkansson and Bjarnason 2020; Biffl et al. 2009; Rivas et al. 2008;
	Le Blanc and Korn 1994; Lester and Wilkie 2004)
	∙ Freedom of choice (Lending and Chervany 2002; Biffl et al. 2009;
	Lester and Wilkie 2004; Phillips et al. 1998)
	∙ *Ease of use & effectiveness (Bashroush et al. 2017;
	Håkansson and Bjarnason 2020; Petersen et al. 2018;
	Biffl et al. 2009; Church and Matthews 1995; Rivas et al. 2008;
	Pelechano et al. 2006; Lester and Wilkie 2004;
	Kornecki and Zalewski 2005; Post and Kagan 2000;
	Delahaye and Du Bousquet 2015; Phillips et al. 1998)
	∙ Learning curve (Håkansson and Bjarnason 2020;
	Torchiano and Morisio 2004; Church and Matthews 1995;
	Rivas et al., 2008; Le Blanc and Korn 1994; Phillips et al. 1998)
	∙ Tolerance (Du Plessis 1993)
	∙ Subjective user satisfaction (Håkansson and Bjarnason 2020;
	Church and Matthews 1995; Le Blanc and Korn 1994;
	Pelechano et al. 2006)
Quality
	∙ Performance (Petersen et al. 2018; Le Blanc and Korn 1994;
	Prather 1993; Lester and Wilkie 2004; Kornecki and Zalewski 2005;
	Miller and Yeoh 2006; Gezici et al. 2019)
	∙ Standards adherence (Bashroush et al. 2017;
	Torchiano and Morisio 2004; Biffl et al. 2009; Brown 1994;
	Du Plessis 1993; Pelechano et al. 2006; Kornecki and Zalewski 2005)
	∙ Robustness (Du Plessis 1993; Kornecki and Zalewski 2005;
	Phillips et al. 1998)
	∙ Security (Petersen et al. 2018; Alnafjan et al. 2013; Du Plessis 1993;
	Krawatzeck et al. 2015; Post and Kagan 2000; Miller and Yeoh 2006;
	Gezici et al. 2019)
	∙ Future proof (Du Plessis 1993)
	∙ Documentation (Håkansson and Bjarnason 2020;
	Delahaye and Du Bousquet 2015; Du Plessis 1993;
	Le Blanc and Korn 1994; Pelechano et al. 2006;
	Lester and Wilkie 2004; Kornecki and Zalewski 2005)
	∙ Feedback (Du Plessis 1993; Miller and Yeoh 2006; Gezici et al. 2019)
Integration
(Brown 1994; Du Plessis 1993;	∙ Effort, initial and continuous (Miller and Yeoh 2006)
Miller and Yeoh 2006)	∙ Ecosystem adherence (Håkansson and Bjarnason 2020; Brown 1994;
	Delahaye and Du Bousquet 2015; Le Blanc and Korn 1994;
	Kornecki and Zalewski 2005)
	∙ Product life cycle (Du Plessis 1993; Kornecki and Zalewski 2005)
	∙ Look’n’feel (Brown 1994; Du Plessis 1993; Rivas et al. 2008;
	Le Blanc and Korn 1994; Lester and Wilkie 2004)
	∙ Automation fit (Miller and Yeoh 2006; Gezici et al. 2019)

Criteria marked with * were added or modified after the focus group