Table 2.

Challenges to automation identified by meeting participants and invited speakers

Broader challenges

• Social acceptance of automation technology

• Development of flexible systems for different disciplines

• Acquiring resources for development

• Fostering collaboration in a competitive environment

• Keeping up with rapidly evolving technologies and approaches, such as open data

• Making automation approaches compatible with stakeholder transparency needs, that is, the “black box” nature of many technologies such as machine learning

Technological challenges

• Designing an application programming interface that meets the needs of multiple scientific domains and goals for different systematic reviews

• Integrating an application programming interface into both new and existing software tools

• Creating cross-compatibility of tools

• Addressing issues of intellectual property

• Meeting review-specific/data-specific challenges

• Extracting data from full texts

• Developing approaches for algorithm and tool validation