TRAAC framework to improve regulatory acceptance and wider usability of tools and methods for safe innovation and sustainability of manufactured nanomaterials

Abstract

There has been an increasing use of advanced materials, particularly manufactured nanomaterials, in industrial applications and consumer products in the last two decades. It has instigated concerns about the sustainability, in particular, risks and uncertainties regarding the interactions of the manufactured nanomaterials with humans and the environment. Consequently, significant resources in Europe and beyond have been invested into the development of tools and methods to support risk mitigation and risk management, and thus facilitate the research and innovation process of manufactured nanomaterials. The level of risk analysis is increasing, including assessment of socio-economic impacts, and sustainability aspects, moving from a conventional risk-based approach to a wider safety-and-sustainability-by-design perspective. Despite these efforts on tools and methods development, the level of awareness and use of most of such tools and methods by stakeholders is still limited. Issues of regulatory compliance and acceptance, reliability and trust, user-friendliness and compatibility with the users' needs are some of the factors which have been traditionally known to hinder their widespread use. Therefore, a framework is presented to quantify the readiness of different tools and methods towards their wider regulatory acceptance and downstream use by different stakeholders. The framework diagnoses barriers which hinder regulatory acceptance and wider usability of a tool/method based on their Transparency, Reliability, Accessibility, Applicability and Completeness (TRAAC framework). Each TRAAC pillar consists of criteria which help in evaluating the overall quality of the tools and methods for their (i) compatibility with regulatory frameworks and (ii) usefulness and usability for end-users, through a calculated TRAAC score based on the assessment. Fourteen tools and methods were assessed using the TRAAC framework as proof-of-concept and for user variability testing. The results provide insights into any gaps, opportunities, and challenges in the context of each of the 5 pillars of the TRAAC framework. The framework could be, in principle, adapted and extended to the evaluation of other type of tools & methods, even beyond the case of nanomaterials.

Graphical abstract

Highlights

• •TRAAC framework evaluates the regulatory readiness of tools and methods.
• •Its scores show strengths, gaps, opportunities and challenges for tools and methods.
• •Its scores guide tool users and developers to well-documented tools and methods and their further development needs.

1. Introduction

In the last two decades, a considerable rise of interest in manufactured nanomaterials (MNMs) has occurred (OECD, 2016; EC, 2012). As a Key Enabling Technology, the use of MNMs allows for considerable innovation in a variety of sectors such as healthcare, energy, aerospace, cosmetics, and electronics (EC, 2018). Their potential to improve the performance and quality of some products and processes have led to the rise of a global market which is projected to reach 30 billion euros by 2030 (Tewari, 2021).

In the EU, MNMs are addressed under several regulations such as the Registration, Evaluation, Authorization, and Restriction of Chemicals framework (REACH) (Regulation (EC) 1907/2006), and the Classification, Labelling, and Packaging regulation (CLP) (Regulation (EC) 1272/2008). Certain regulations address MNMs for specific product types like e.g. biocidal products (Regulation EC 528/2012), cosmetic products (Regulation (EC) 1223/2009), medical devices (Regulation (EU) 2017/745) and novel foods (Regulation (EU) 2015/2283).

Within the REACH regulation, for which TRAAC framework is formulated, MNMs have until recently been treated similarly to all/bulk chemical substances. REACH was selected as a starting point for elaborating the framework as it is considered one of the overarching EU regulations for chemicals in Europe. It applies to the manufacture, placing on the market or use of all chemical substances. However, scientific evidence showed a need for a nano-specific approach for materials identification and risk assessment (Dekkers et al., 2016; EFSA Scientific Committee, 2009; EFSA Scientific Committee, 2018; Gottardo et al., 2017; SCENIHR, 2009). To meet the new needs, the REACH regulation (Regulation (EC) 1907/2006) was amended with Regulation (EU) 2018/1881 to address nanoforms of substances (i.e. a form of a natural or manufactured substance containing particles, in an unbound state or as an aggregate or as an agglomerate and where, for 50% or more of the particles in the number size distribution, one or more external dimensions is in the size range 1 nm–100 nm, including also by derogation fullerenes, graphene flakes and single wall carbon nanotubes with one or more external dimensions below 1 nm (REACH regulation EC 1272/2008 Annex VI)), which came into effect on January 1, 2020. Since then, manufacturers and importers must register nanoforms and their nano-specific human and environmental risk assessments.

So far, compliance with the new regulatory requirements, set out by the EC, has proven to be challenging for industries. According to the European Chemical's Agency (ECHA), the lack of validated tools and methods (e.g. procedures, guidance) for the characterization, testing and risk assessment of MNMs plays a role in the low compliance rate to the legal obligations (ECHA, 2020). In addition, lack of knowledge about the tools, user capacity, training and usage difficulty impede the successful implementation of the new regulatory requirements among industrial users (Kirkegaard et al., 2020). Despite this, large and diverse type of tools and methods dealing with the assessment of risks, have been developed and published in the last decade by national and international research organizations, companies, government agencies and other actors. Several of such tools also assess the sustainability and socio-economic impacts of MNMs which are essential elements in the EC-JRC's Safe and Sustainable by Design approach for MNMs within EC's Chemicals Strategy for Sustainability and the Zero Pollution Act (EC, 2019b; EC, 2022a). In the context of this study, ‘tool’ can be interpreted as an instrument (e.g. computational model, database) to obtain a specific result and a ‘method’ as a description of the process to achieve the result (e.g. risk assessment framework, technical and procedural guidance, standard).

There currently is a demand for validated regulatory-oriented tools and methods (T&Ms) to carry out the assessments of MNMs for their potential risks and socio-economic impacts. The mass market for existing MNMs, their increasing complexity and need for regulatory oversight, will further increase the demand for reliable T&Ms. for MNMs and for their integration into standard practices of chemical risk management, as it happens for the T&Ms. for bulk chemicals. Thus, we focus on T&Ms. dealing with the risk assessment, risk management, socio-economic impact assessment and sustainability of MNM, including T&M combining all these different aspects. This is in line with most recent policy proposals, such as the Safe and Sustainable by Design framework and the Safe (r) Innovation Approach from OECD, that are looking for T&M helping to predict and model risks of novel and complex materials, such as MNM, since the early phases of the development (in a by -design perspective) and all along the research and development process and life cycle of the material (integrated and holistic assessment approach). (EC, 2022; OECD, 2020; Soeteman-Hernandez et al., 2019a, Soeteman-Hernandez et al., 2019b).

‘Regulatory orientation’ of the research data and the results provided by the T&Ms. have constantly been highlighted in the scientific literature as a key component (and sometimes main deterrent) to the wider use of these T&Ms. (van Teunenbroek et al., 2017; Porcari et al., 2019; Sørensen et al., 2019; Hasselbring et al., 2020). Regulatory needs entail well defined, standardized, reliable, reproducible and exchangeable results, and implies that the T&Ms. must be aligned with- and support the regulatory frameworks in place and should be widely used and accepted by different stakeholders (regulators, end-users, and the scientific community) (van Teunenbroek et al., 2017; Porcari et al., 2019). However, as demonstrated by Halamoda Kenzaoui et al. (2019), new state-of-art methods, instruments, approaches or tools have not yet sufficiently proven their reliability and relevance for the purpose of assessing risks and socio-economic impacts of MNMs and nanotechnology-enabled products. Recently, however, an OECD initiative assessed a large set of tools for their regulatory preparedness based on analysis of the tools' accessibility, sensitivity and performance in regards to nano-specific environmental, consumer and occupational exposure assessment (OECD, 2021a, No. 345, 346, 347 and 348).

Besides regulatory orientation or acceptance, the T&Ms. should also demonstrate their alignment with specific needs of their end-users/stakeholders, such as public research and/or manufacturing companies, regulators, policy makers, risk assessing service providers/consultancy and supra-national organizations dealing with the issues related to the safety of MNMs. As shown by a recent study (Porcari et al., 2020), stakeholder needs include extended functionality, (easier) accessibility, (easy) usability, affordability, and more reliable and efficient management and exploitation of safety data and databases. In addition, the role of the T&Ms. with respect to different factors or aspects of the safe innovation of MNMs (e.g. risk assessment, risk management, risk-benefit analysis, socio-economic impact assessment) are also of prime interest to the stakeholders. The lack of both proper guidance and training is a recurrent issue which hinders the wide usability of the T&Ms. (Kirkegaard et al., 2020).

In the present study, a framework is proposed to assess different T&Ms. which are applicable to MNMs, inspired by decision-making strategies in chemical and environmental policy settings. The assessment of T&Ms. is done with respect to their compatibility with regulatory orientation or requirements (i.e. regulatory readiness) and stakeholders needs. The framework is designed to help the aforementioned T&Ms. end-users navigate through the wide number of existing (and future) T&Ms., to identify and select the most suitable ones to address their needs.

The framework is composed of five core pillars: Transparency, Reliability, Accessibility, Applicability, and Completeness (i.e. TRAAC). The TRAAC framework is intended to help (i) regulators and other authorities by pointing out the most appropriate and relevant T&Ms. (i.e. regulatory orientated) for the safe innovation of MNMs; (ii) aforementioned end-users of T&Ms. by selecting the most appropriate options for their specific needs; (iii) industries by increasing their trust and confidence around the T&Ms. and thus accompanying them with the regulatory compliance process; (iv) T&Ms. developers by compiling end-users' needs into an easy-to-use framework, which may help them to improve their T&Ms.

The present paper describes the development of the TRAAC framework and results of from testing it on a selection of existing qualitative to quantitative T&Ms. The framework can be further adapted and used for other T&Ms. This could, for example, be for Safety and Sustainability by Design (SSbD) tools intended to facilitate the implementation of the EU's Chemicals Strategy for Sustainability which foresees the reduction of the human health and the environment impact of these chemicals and materials. The TRAAC approach was used for final documentation and selection of T&Ms. in the updated caLIBRAte x Gov4Nano platform and tools-supported nano-risk governance framework (http://www.nanoriskgov-portal.org/Public/Catalogue).

This study builds upon the work conducted in former EU H2020 projects such as caLIBRAte (Grant Agreement, GA 686239) in which a web-based Nano Risk Governance Portal (NRGP) and stage-gate innovation governance framework was launched, NANoREG (and NanoReg2) in which a Safe Innovation Approach (SIA) for MNMs was developed and Gov4Nano (GA 814401), NANORIGO (GA 814530) and RiskGone (GA 814425) projects in which a new NRGP was established, directing users to different nano-risk governance tools and methods and external nano-risk governance platforms supported by T&Ms.

2. TRAAC framework

The TRAAC framework was developed in several steps, as further elaborated below, which consisted of:

• (i)the identification of key elements of the framework, by a literature search of general issues and “themes” with regards to regulatory acceptance and wider usability of T&Ms.;
• (ii)the development of the pillars and criteria of the framework structure;
• (iii)the review and improvement of the framework structure, based on feedbacks collected in a multi-stakeholder workshop;
• (iv)the demonstration of the use and robustness of the framework, by inter-criteria correlation analysis and user variability analysis
• (v)the application of the TRAAC framework and the evaluation of results on 14 selected T&Ms.

The development of TRAAC is inspired by existing decision-making frameworks in the fields of chemical and environmental risks, where policy makers need to choose among different and conflicting information on (chemical, environment) management alternatives, based on many parameters/variables, in particular MCDA- Multi Criteria Decision Analysis (Linkov et al., 2021; Musuamba et al., 2014; OECD, 2021b). Out of several essential elements relevant to decision making, the TRAAC framework (i) defines clear and transparent decision criteria/rules, (ii) identifies decision uncertainties, data gaps, and ways to organize and prioritize information, and (iii) performs comparative assessment among alternative T&Ms. These aspects correspond to some of the most relevant stakeholder needs regarding the choice of T&Ms., emerged during the literature search and the multi-stakeholder workshop. The TRAAC approach is, however, conceptually a simpler construct based on different criteria of additive characteristics (not conflicting) for each topic in question.

2.1. Literature search

A literature search with regards to risk and sustainability aspects of MNMs (i.e. risk assessment, risk governance, functionality, economic impact and societal impact) and available T&Ms. helped to identify the key elements needed to design the TRAAC framework. In addition, attention was also given to the REACH regulation EC 1272/2008 and its recent amendments, as well as to the reports of previous EU funded projects, such as caLIBRAte and NanoReg2, which analyzed topics of risk assessment, material/product functionality and nano-risk governance. Additionally, relevant material published by governmental agencies (e.g. EPA, 2017; ECHA, 2011; EUON, 2020) and international organizations (e.g. OECD, 2005; IOMC, 2016) about REACH requirements for environment and human health assessments of MNMs was considered. The views and perspectives of stakeholder needs on the risk governance of MNMs and nano-related products (as identified by Kirkegaard et al., 2020 and Porcari et al., 2020), based on selected EU funded projects, initiatives and literature resources, were further used to support the applicability of the framework.

2.2. Workshop

An interactive workshop was organized involving 36 stakeholders from different EU countries. The workshop was organized as a dedicated session within a joint EU H2020 NMBP-13 project meeting in 2021. The collaborators from the three independent European projects (Gov4Nano, RiskGONE and NANORIGO), who were already participating in the joint meeting, also attended the workshop. Coming from different backgrounds (researcher/consultants: 46%, academicians: 24%, industry/business: 23%, regulators: 3% and government agencies: 4%), the involved participants/stakeholders are regarded experts in the field of impact of MNMs on human and environmental health, and have participated in many major European and National projects and authority activities dealing with these topics.

The objectives of the workshop were to demonstrate the TRAAC framework, to get it recognized and supported by a wider audience and to incorporate their feedback to further refine the framework. Built upon the literature search, a draft framework was presented and, for each of the five pillars, the participants were asked to vote and rate pre-selected criteria (detailed in following sections) using an online survey platform in the order of their significance to the given pillar (which contributed to the weightage assignment to TRAAC criteria) and propose additional criteria. The participants could see the number of received votes for each criterion in real-time which fostered lively discussion among them.

2.3. Structure

As shown in Fig. 1, the framework follows a multi-faceted approach aiming to evaluate the regulatory readiness of T&Ms. and their alignment with stakeholder needs. It rests on five pillars:

• (i)Transparency: Ownership, clear communication about development, methods, strengths and limitations (e.g. boundary of use);
• (ii)Reliability: Quality, correctness, and consistency of output;
• (iii)Accessibility: Usability, findability, and user experience evaluation;
• (iv)Applicability: Applicability domain and adequacy to address user needs;
• (v)Completeness: Comprehensiveness regarding EU regulatory frameworks (REACH) and requirements for MNMs.

Fig. 1.

Overview of the TRAAC framework: five pillars of the framework, i.e. Transparency, Reliability, Accessibility, Applicability and Completeness, and their respective constituting criteria; five modules within the Completeness pillar are also shown with the parameters/criteria for the completeness evaluation within each module.

Each of these five pillars consists of several criteria, characterized by scores (on a scale of 0.1, 0.5, 1) and weights (on a scale of 1 to 5), to assess potential requirements for T&Ms. regulatory readiness and their users' needs.

As described in the subsequent sections, the criteria in turn are comprised of three scoring options which evaluate the degree of their fulfilment: 0.1- unfulfilled, 0.5- partially fulfilled and 1-completely fulfilled. There are certain criteria for which there are two scoring options, 0.1- unfulfilled and 1-completely fulfilled, as partial fulfilment for such criteria is not possible (more details in subsequent sections). The numerical weight assigned to a criterion signifies its importance in defining the feature and scope of the pillar, with 5 being the most important and 1 being the least important. The criteria and their respective weights in each pillar are shown in Table 1, Table 2, Table 3, Table 4. The weights of these criteria are assigned a priori and were derived based on their significance found during the literature search and workshop. The compilation of scores and weights is based on a combination of literature search, and experts' judgment (based on the authors expertise, review during the workshop, testing with end-users).

Table 1.

Criteria in the transparency pillar.

Criteria ref	Criteria name	Description	Assigned weight (out of 5)	Scoring options
T.1	Knowledge of developers, their affiliations and communication with end-users	The T&M (or its guidance) provides information about its developers, contributors, and funding when applicable. The end-users can contact the developers for feedback, questions, or to request additional guidance.	3	Score 1: The contact details of the developers of the T&M, contributors, and involved institutions are known from the documentation. The end-users can thus contact or communicate with them directly to provide feedback, obtain additional information, and receive guidance. Funding (when applicable) is also publicly disclosed.
				Score 0.5: Limited information about the developers, contributors, institutions, or funding is disclosed in the documentation. This only allows the end-users to submit remarks and comments to the T&M developers through a web form.
				Score 0.1: Developers, contributors, institutions, and funding are not disclosed at all or are unknown to the user. The end-users, thus, cannot communicate or contact them.
T.2	Publicly available detailed description	The aim of the T&M, scope, source, processing of the data, applicability, and endpoints are addressed by developers. It concerns the information on “what” the T&M can be used for. The information on “how” to use the T&M is accounted by A.1.	5	Score 1: Aim of the T&M, scope, source, and processing (when applicable) of the data, methodology, applicability, endpoints, and decision criteria are described. Contribution of the T&M to the assessment process is stated. Comprehension, use, and/or replication are possible.
				Score 0.5: The description of the T&M includes some of the considerations listed above. Comprehension, use, and/or replication is possible but difficult.
				Score 0.1: The description of the T&M includes few or none of the considerations listed above. Comprehension, use, and/or replication is not possible.
T.3	Knowledge of strengths and limitations	The strengths and limitations of the T&M are disclosed by the developers (e.g. in a user guidance or separate research paper).	4	Score 1: The knowledge of strength(s) and limitation(s) of the T&M is provided by the developers and is known to the users.
				Score 0.5: The knowledge of either strength(s) or limitation(s) (or partial knowledge of both strength(s) and limitation(s)) of the T&M is provided by the developer and is known/identifiable to the user.
				Score 0.1: No such knowledge is provided by the developer, and it cannot be known/identified by the user.

Table 2.

Criteria in the reliability pillar.

Criteria Ref	Criteria Name	Description	Assigned weight (out of 5)	Scoring options
R.1	Testing and/or validation	The T&M has been tested by comparing its output with preferably MNMs measurement studies or, in case data was lacking, with similar tools/methods. The comparison shows that the T&M output is reasonable (e.g. a strong Pearson correlation coefficient between estimated and measured values).	5	Score 1: The T&M has been tested/validated against MNMs specific data.
				Score 0.5: The T&M has been tested against outcomes of tested/validated models.
				Score 0.1: The T&M has not been tested or no such information is available to the user.
R.2	Peer-review	The development of the T&M is published in a scientific journal (peer-reviewed) to ensure that the scientific soundness of the underpinning method or design of the T&M is checked by an independent reviewer. Unlike other criteria which have three scoring options (0.1, 0.5 and 1), there are only two scoring options here because the T&M can either be peer-reviewed or not. There is no partial peer-review.	2	Score 1: The scientific publications of the developers about the T&M are peer-reviewed. This also includes reports/documents issued by governmental organizations (e.g. OECD, ECHA), regulatory authorities (e.g. RIVM, BAuA) or public standards organization (e.g. CEN) which are often based on peer-reviewed scientific publications.
				Score 0.1: The scientific publications of the developers about the T&M are not peer-reviewed or there is no scientific publication.
R.3	Quality of the data	Measurement data used for the development of the T&M has been collected with use of validated measurement methods (published by renowned institutes). Sufficient description of the context (i.e. contextual adequacy) is provided to either know the values for relevant parameters or to make informed and justified expert judgment on several parameters.	5	Score 1: The T&M developers address data quality in a guidance document or a scientific article. The data used during the development is well documented, measurements are performed with validated measurement methods (or closely resembling methods), and contextual adequacy (see description) is addressed.
				Score 0.5: The data quality and measurement methods are not addressed by the T&M developers or are unknown. Score 0.1: The T&M is developed without the use of measurement data or no such information is available to the user.
R.4	User variability and clear decision making	The required information by the T&M is clear, and multiple users of the T&M, with the same set of required information to execute the T&M, reach the same output with an unambiguous interpretation.	3	Score 1: A study was conducted, and user variability was concluded to be acceptable by the authors.
				Score 0.5: A study was conducted, and user variability was concluded by the authors to be high.
				Score 0.1: No study was conducted to investigate the user variability, or no such information is available to the user.
R.5	Uncertainty analysis	The uncertainty of the T&M output has been analyzed (e.g. for a quantitative T&M) or addressed (e.g. for a qualitative T&M). Like R.2, the uncertainty of T&M can either be dealt with or not, hence two scoring options.	4	Score 1: The T&M includes/addresses uncertainty with its output.
				Score 0.1: The T&M does not include/address uncertainty with its output, or no such information is available to the user.
R.6	Collaborative effort	The T&M was developed with different stakeholders (e.g. research community, industry, regulatory institutions). There can either be a collaborative effort or not and thus there are only two scoring options like R.2 and R.5.	2	Score 1: The T&M was developed and funded with partners and collaborators from the research community, the industry, and the regulatory institutions. It can also be the product of an EU research project.
				Score 0.1: The T&M was not developed or funded with partners and collaborators from different communities, or this information is not disclosed.

Table 3.

Criteria in the accessibility pillar.

Criteria Ref	Criteria Name	Description	Assigned weight (out of 5)	Scoring options
A.1	Quality guidance material	Up-to-date guidance for use of the T&M latest version is provided which can include resources like T&M user instructions (i.e. how to use), study-cases, access information, audio visual resources, FAQs etc.	4	Score 1: The user is provided with up-to-date guidance material for basic and/or advanced features of the latest version of T&M. The guidance material is complete and clear to understand. The guidance material includes case studies, and information for accessing the T&M. Additional resources (e.g. guidance, training) are listed on the platform or within the T&M itself. The user guide is task oriented and features illustrations and examples for each step. Audio visual guides (e.g. Webinars, tutorials) or FAQs can also be considered as guidance material.
				Score 0.5: The user is provided with some guidance material which addresses basic features of a T&M and lacks clarity. Additional material such as other resources or audio-visual guides are not included or are not currently available.
				Score 0.1: No guidance is provided to the users.
A.2	Compatibility with various levels of expertise	Basic and more advanced features of the T&M are understandable and usable without an elevated level of expertise in material, safety, or sustainability relevant knowledge of MNM.	3	Score 1: The basic and advanced features of T&M are easily understandable and usable without an elevated level of expertise. The T&M results and decision-making criteria are clear and can be interpreted by a wide range of end-users.
				Score 0.5: The basic and advanced features of T&M are understandable and usable with a moderate level of expertise. The T&M results and decision-making criteria are clear and can be interpreted by most end-users.
				Score 0.1: The basic and advanced features of T&M are understandable and usable with an elevated level of expertise. The T&M results and decision-making criteria cannot be interpreted without substantial expertise.
A.3	Availability in more than one language	The T&M (or its guidance material) is available in English and in other languages. As previously observed for the Reliability pillar, this criterion also has only two scoring options because T&M can either be present in more than one language or not.	1	Score 1: The T&M (or its guidance material) is available in English and additional languages.
				Score 0.1: The T&M (or its guidance material) is only available in English.
A.4	Inclusion of a license	The T&M includes a license determining conditions for (re)usage, reproduction, modification, and distribution of the T&M.	1	Score 1: The T&M includes a license addressing (i) authorship, (ii) conditions/terms for usage, (iii) reproduction and modification, (iv) distribution, and (v) liability.
				Score 0.5: The T&M includes a license. Only some of the aspects listed above are addressed.
				Score 0.1: The T&M is not accompanied by any license, or its development is not advanced enough to include a license.
A.5	Citations/mention in scientific, regulatory, or industry publications	The T&M is cited or mentioned in publications from the industry, the scientific community, or governmental agencies.	2	Score 1: The T&M is cited/mentioned in at least 3 scientific, regulatory, or industry publications (excluding publications from the developers or associated project).
				Score 0.5: The T&M is cited/mentioned in at least 1 scientific, regulatory, or industry publication (excluding publications from the developers or associated project).
				Score 0.1: The T&M is not cited/mentioned in publications other than the ones by its developers or associated project.
A.6	Findability and accessibility	The T&M is accessible online or through download. The T&M can either be easy or difficult to find and access. Thus, there are only two scoring options.	4	Score 1: The T&M is easily accessible to end-users on the web, either on a web platform or through download.
				Score 0.1: The T&M is either inaccessible (for instance, due to no user interface), difficult to access or is only available upon request.
A.7	User-friendliness and inter-operability	The T&M is easy to operate and navigate. In some cases, the T&M can be used in combination with other T&M or databases (interoperability).	5	Score 1: The T&M is user friendly: the GUI (Graphical User Interface) or the step-by-step methodology is clear and simple. Information is easily findable. In its user interface, inputs controls and navigation components are easy to understand and interact with. In some cases, and when applicable, the T&M is inter-operable and can be used in combination with other T&M.
				Score 0.5: Some elements from above are missing but the T&M is still easy to navigate and inter-operable.
				Score 0.1: The T&M is not user-friendly or is still in an early development stage to consider user-friendliness and it is not inter-operable.

Table 4.

Criteria in the Applicability pillar.

Criteria Ref	Criteria name	Description	Assigned weight (out of 5)	Scoring options
P.1	Applicable life cycle stage(s)	The T&M is applicable to all life cycle stages of a product: (i) R&D/Synthesis, (ii) Manufacturing, (iii) Use, (iv) End of life.	1	Score 1: The T&M is applicable to all the following life cycle stages.
				Score 0.5: The T&M is applicable to two or more life cycle stages.
				Score 0.1: The T&M is specifically applicable to one life cycle stage.
P.2	Applicable exposure routes and/or population types	The T&M is applicable to all exposure routes (i.e. inhalation, dermal and ingestion) and/or exposed populations (i.e. workers, consumers, environment, and general population).	4	Score 1: The T&M is applicable to all exposure routes and/or populations.
				Score 0.5: The T&M is applicable to 2 or more exposure routes and/or 2 or more populations.
				Score 0.1: The T&M is specific in its application and allows for one applicable exposure route and/or one applicable population.
P.3	Nano-specificity	The T&M considers relevant physico-chemical properties and endpoints for MNMs specific behaviour, and thus can be applied to the MNM relevant analysis. As previously seen for several other criteria, this criterion also has only two scoring options because the T&M can either be nano-specific or not.	5	Score 1: The T&M is developed with the purpose of being applicable to MNMs' specific behaviour and physiochemical properties.
				Score 0.1: MNMs are out of the scope or beyond scope of the T&M or its applicability to MNMs is yet unknown.
P.4	Quantitative output	The output of the T&M is quantified.	3	Score 1: The T&M has a quantitative output (e.g. exposure concentration in cm−3, ED50 in mg/kg).
				Score 0.5: The T&M has a semi-quantitative output (e.g. relative scores).
				Score 0.1: The T&M has a qualitative output (e.g. control bands).
P.5	Handling data gaps	The T&M can be applied despite limited data/information being available. This criterion has also two scoring options because the T&M can either handle the data gaps or not.	4	Score 1: The T&M can be executed despite limited information availability. When insufficient information is available to the user, some required information can be left blank, and a result can still be obtained. It considers worst-case estimates or uses typical default inputs (a priori assumptions) based on expert judgment.
				Score 0.1: The T&M requires the user to fill in all the input fields to carry on to the next steps and generate a result.
P.6	Time and cost efficiency	The T&M is considered time and cost efficient for users depending on the innovation stage to which it can be applied. For the early innovation stages (ideation/scope/business case/R&D), which focus on ideas screening and building up the concept, the T&M should not need more than a few minutes for the assessment. For the later stages (test & validation/launch/monitoring), which require more in-depth assessment, the T&M should not need more than a few hours for the assessment. The T&M can either be time and cost efficient or not, and thus there are two respective scoring options.	3	Score 1: The T&M is time and cost efficient depending on the innovation stage.
				Score 0.1: The T&M is not time or cost efficient. The T&M requires substantial training and/or time, even for early innovation stage.
P.7	Continuously supported by developers	The latest version the T&M can be applied without any user hesitation because it stays up to date with new requirements, new data, and studies. It is updated (e.g. successive versions) when needed. The continuous support from T&M developers can either be there or not, and thus there are two respective scoring options.	5	Score 1: The T&M is continuously supported by developers: it stays relevant and takes into consideration new studies, data, or regulatory requirements. It is updated when needed. Continuous support of the T&M also includes updates for the T&M interface/user experience.
				Score 0.1: The T&M has never been updated after its launch (e.g. only one version since its launch several years ago).
P.8	Advice for improving the output	The T&M can be applied to provide advice to the user on how to improve the assessment output to an accepted level (e.g. for exposure assessment, a suitable advice would be on how to lower the estimated exposure). The two respective scoring options for this criterion represent the ability of the T&M to either generate the advice or not.	2	Score 1: The T&M is able generate advice to the user on the measures to take to improve the assessment output to an accepted level.
				Score 0.1: The T&M simply provides an output with no such advice.

2.3.1. Transparency

Transparency towards the scientific community, regulators, and end-users is a key factor for the stakeholders' acceptance of T&Ms. As underlined by Hristozov et al., 2012, EC, 2019a, and EU, 2019, transparency is a prominent part of the risk governance process. In the context of the T&Ms., it implies clear information about the authors/developers, the methods and data (and their scientific evidence) used for their development, and the communicated uncertainties with regards to the tools' outputs.

A multi-stakeholder study in 2017 which considered the opinions of 97 stakeholders (through online survey and subsequent face-to-face workshop) over their perceptions regarding the use and potential impact on society and the environment of MNMs and nano-related products has shown that 38% of policymakers and 24% of the industry have only a basic to intermediate understanding of MNMs and their potential impact on the society and the environment (Porcari et al., 2019). A transparent approach to risk assessment from the tool developers helps these stakeholders to better understand the risk assessment and safe innovation process for MNMs, increases their trust and confidence in the tools and technology, and enables potential downstream users to interpret and communicate risks clearly (Isigonis et al., 2019; EC, 2012).

The TRAAC framework takes transparency into account by answering three underlying questions (EC, 2019a; Gomez-Diaz and Recio, 2019; OECD, 2005), which lead to the development of four transparency related criteria in Table 1.

• −Who created the T&M? This allows for accountability, facilitates collaboration, and enables regulators and industries to directly contact the T&M developers.
• −Why was the T&M created? A clear scientific rationale and publicly defined purpose for T&M is necessary for end-users to assess the appropriateness of the T&M for their intended purpose, e.g. risk assessment, socio-economic impact assessment.
• −How does the T&M work and what are the restrictions of use? A detailed description of the tool helps end-users understand how the tool works, how relevant it is for them, and how reliable the T&M approach is, by addressing uncertainties, strengths, and limitations.

2.3.2. Reliability

Reliability is often considered as the pre-requisite for regulatory acceptance and stakeholders. Results from T&M need to be reliable, accurate (e.g. in line with reality or a realistic worst-case estimate), and the uncertainty of the estimates needs to be dealt with. The uncertainty is generally reflected by model uncertainty and can be expressed as a certain percentile (e.g. the 90th percentile) in which case the user knows that the T&M estimates are intended to be conservative. Additionally, between- and within-user variance provides additional insights in the uncertainty and can be used to calculate different percentiles. The reliability pillar of the TRAAC framework focuses on aforementioned aspects by assessing the correctness and consistency of the tools' outputs (see Table 2). Three underlying questions are addressed (Aerts, 2017; EPA, 2017; Gomez-Diaz and Recio, 2019; Hristozov et al., 2016; Isigonis et al., 2019; JRC, 2018; Morris et al., 2010; OECD, 2005; Sørensen et al., 2019) when considering the reliability of the T&M, which lead to the development of six reliability related criteria in Table 2.

• -Has the T&M been verified and received support within the scientific community? For scientific knowledge to be approved and accepted by regulators, their quality must be high, and their findings should be approved and supported throughout, for instance, peer reviewing approaches involving the scientific community of reference (EC, 2019).
• -Has the scientific quality of T&M been assessed/measured? Documented applications, validating the T&M results against results from other tools or (preferably) measurements, and evaluation of the quality of the data used are important reliability evaluation criteria for stakeholders (EPA, 2017; Hristozov et al., 2016; Isigonis et al., 2019; OECD, 2005; Porcari et al., 2019).
• -Is the uncertainty of the T&M assessed or dealt with? Information with regards to the uncertainty of a T&M outcome or result is considered crucial to assess the reliability.

2.3.3. Accessibility

The Accessibility pillar of the TRAAC framework evaluates the T&Ms. in terms of user experience (see Table 3). It assesses the usability and findability of the T&Ms. by users particularly in accordance with the stakeholder needs as described by Porcari et al., 2020, FAIR principles (Jeliazkova et al., 2021; Wilkinson, 2016) and other relevant studies (Aerts, 2017; EC, 2019a; Gomez-Diaz and Recio, 2019; Hristozov et al., 2016; Isigonis et al., 2019; JRC, 2018; OECD, 2005; Soeteman-Hernandez et al., 2019a; Soeteman-Hernandez et al., 2019b; Trump et al., 2018).

A large fraction of the organizations developing, producing and using MNMs are Small and Medium Size Enterprises (SMEs) (Teunenbroek et al., 2017). Accessibility of relevant T&M can influence the choice of T&M, as most T&Ms. require some training and a certain degree of expertise. Consequently, the more complex a T&M is to use, the more resource demanding and difficult is its use. As SMEs often do not have in-house expertise about potential environmental and human health risks of MNMs and work with limited budgets, accessibility of T&Ms. in terms of their use is important to consider (Kirkegaard et al., 2020; Porcari et al., 2019). Three underlying questions help to determine the seven criteria for this pillar:

• -how easily findable is the T&M?;
• -how easily accessible (in terms of use) is the T&M?;
• -what type of additional resources (e.g. guidance, training) do the developers provide to users?

Ideally, a T&M should be easy to use, understandable, and compatible with various levels of expertise (Isigonis et al., 2019; Sørensen et al., 2019). Among other criteria, as mentioned in Table 3, quality guidance material, a user-friendly interface, and built-in case studies can help accelerate the learning period. When assessing accessibility, other aspects are also to be looked at, for example, the availability of licenses for redesigning or reusing the existing T&M in a new project/work, citations for scientific credibility and recognition by a wider stakeholder community, and availability of the T&M (or its guidance material) in different languages for a wider audience for whom the default language of the T&M can be a deterrent feature.

2.3.4. Applicability

The applicability pillar addresses two aspects of the T&Ms.: (i) their applicability domain and (ii) the extent to which they fulfill some specific stakeholders' needs as identified in previous works (Beaudrie and Kandlikar, 2011; Franken et al., 2020a, Franken et al., 2020b; Isigonis et al., 2019; Porcari et al., 2020). The criteria evaluating the T&Ms. applicability are shown in Table 4 and they are determined based on three underlying questions in line with Hristozov et al., 2016; Isigonis et al., 2019; JRC, 2018; Morris et al., 2010; OECD, 2005; Porcari et al., 2019; Sørensen et al., 2019; Teunenbroek et al., 2017; Trump et al., 2018:

• -Is the T&M nano-specific? Unlike bulk chemicals, MNMs require a T&M to consider nano-specific parameters, such as additional physico-chemical characteristics.
• -What are the specific usage-relevant characteristics of the T&M? Due to the data gaps for MNMs, the capacity of T&M to work with a small number of input data is desirable for potential end-users. Quantitative types of T&Ms. and risk management options also ease the risk assessment and safe innovation process by providing end-users with more specific information for making decisions depending on the innovation stage. Moreover, continuous support and development of the T&M is useful for them to stay relevant and up to date. A ‘one-size-fits-all’ approach of the T&M, measured in terms of their comprehensiveness for different populations (worker, consumer, general population, environment), exposure routes and life cycle stages, is certainly a factor that also increases their applicability. T&M can be highly effective and useful in a single process, life cycle stage, population, or exposure route for which it was developed. However, its inability to be applicable to other processes and/or life cycle stages, populations or exposure routes is considered to limit its general applicability from a user-perspective.
• -Does it cost a lot of resources (money and time) to work with the T&M? Efficiency is measured by evaluating the time and costs implied using T&Ms. Sometimes, it can be a trade-off between saving time and spending money (in the form of a user fee). For example, a T&M can ask for a user fee for the easy management of risk assessment files for REACH registration which saves time in building REACH dossiers compared to free versions.

2.3.5. Completeness

The last pillar of the TRAAC framework evaluates the comprehensiveness of the T&M in relation to EU's REACH regulatory standards and requirements for MNMs, as mentioned in modified REACH Annexes I, III and VI-XII in EC Regulation (EU) 2018/1881. In addition, the criteria/parameters in the completeness pillar are also based on the regulatory guidance documents, including EUON's summary of the new REACH requirements, ECHA Guidance R.14 (ECHA, 2016a) and Appendix R14–4 (ECHA, 2016b), an EU H2020 caLIBRAte project report on quality criteria for data (Nymark et al., 2017), and ECHA's Socio-Economic Analysis guidance report.

There cannot be a universal set of completeness criteria which are applicable to all types of T&Ms. because the input parameters in T&Ms. depend on the type of the assessment it does (e.g. assessment of risk, exposure, hazard, socio-economic impact). Thus, unlike the previous four pillars, the Completeness pillar follows a modular approach. Within its current scope, there are 5 modules in this pillar (as shown in Fig. 1 and Table 5) with each module representing a specific aspect which is relevant for the safe innovation/safety-by-design of MNMs: (i) Human risk assessment (ii) Environmental risk assessment (iii) Human exposure assessment (iv) Socio-economic impact assessment (v) Environmental impact assessment. Each of the required parameters in a module (e.g. C.1.1 to C.1.27 criteria for human risk assessment, C.1.1 to C.1.14 and C.2.1 to C.2.6 criteria for environmental risk assessment) serves as the completeness evaluation criterion with all criteria having the same assigned weight (i.e. 1). There are only two scoring options for all criteria, i.e. Yes (1) and No (0.1). If ‘Yes’ is selected for a particular parameter and for a particular T&M, it implies that particular T&M addresses that particular parameter. The option ‘No’ implies otherwise. Since each module has its own set of parameters/criteria, it is important to know that a T&M should be evaluated based on the module to which it is applicable and other inapplicable modules should be omitted from the evaluation. If a T&M is applicable to more than one module, separate Completeness scores for each module should be evaluated to check how much a particular T&M is “complete” with respect to a specific module.

Table 5.

Criteria in the Applicability pillar and its 5 modules (Module 1: Human risk assessment; Module 2: Environmental risk assessment; Module 3: Human exposure assessment; Module 4: Socio-economic impact assessment; Module 5: Environmental impact assessment) with each module shown for which a given criterion is valid (validity is shown with an ‘X').

Criteria Ref	Criteria name	Description	Assigned weight (out of 5)	Scoring options	Criterion valid for module
					1	2	3	4	5
C.1.1	Composition	Composition of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.2	Aspect ratio	Aspect ratio of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.3	Crystallinity	Crystallinity of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.4	Size range	Size range of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.5	Size distribution	Size distribution of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.6	Purity	Purity of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.7	Shape morphology	Shape morphology of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.8	Surface chemistry	Surface chemistry of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.9	Agglomeration/ Aggregation	Agglomeration/ Aggregation of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.10	Surface area	Surface area of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.11	Surface charge	Surface charge of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.12	Solubility/ Dispersion	Solubility/ Dispersion of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.13	Octanol water partition coefficient	Octanol water partition coefficient of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.14	Density	Density of MNM addressed by the T&M?	1	Score 1: Yes	X	X	X
				Score 0.1: No
C.1.15	Type of product	Details on the product addressed by the T&M in which MNM is present?	1	Score 1: Yes	X		X
				Score 0.1: No
C.1.16	Physical form	Physical form of MNM addressed by the T&M?	1	Score 1: Yes	X		X
				Score 0.1: No
C.1.17	Dustiness/ viscosity	Dustiness of MNM or viscosity of the MNM liquid dispersion addressed by the T&M?	1	Score 1: Yes	X		X
				Score 0.1: No
C.1.18	Boiling/melting point	Boiling/melting point of the product addressed by the T&M in which MNM is present?	1	Score 1: Yes	X		X
				Score 0.1: No
C.1.19	Route of exposure	Route of exposure of MNM addressed by the T&M?	1	Score 1: Yes	X		X
				Score 0.1: No
C.1.20	Frequency of exposure	Frequency of exposure of MNM addressed by the T&M?	1	Score 1: Yes	X		X
				Score 0.1: No
C.1.21	Duration of exposure	Duration of exposure of MNM addressed by the T&M?	1	Score 1: Yes	X		X
				Score 0.1: No
C.1.22	Level of exposure	Level of exposure of MNM addressed by the T&M?	1	Score 1: Yes	X		X
				Score 0.1: No
C.1.23	Toxicokinetics	Toxicokinetics of MNM addressed by the T&M?	1	Score 1: Yes	X
				Score 0.1: No
C.1.24	Mutagenicity	Mutagenicity of MNM addressed by the T&M?	1	Score 1: Yes	X
				Score 0.1: No
C.1.25	Acute toxicity	Acute toxicity of MNM addressed by the T&M?	1	Score 1: Yes	X
				Score 0.1: No
C.1.26	Short/long term repeated dose toxicity	Short/long term repeated dose toxicity of MNM addressed by the T&M?	1	Score 1: Yes	X
				Score 0.1: No
C.1.27	Reproductive toxicity	Reproductive toxicity of MNM addressed by the T&M?	1	Score 1: Yes	X
				Score 0.1: No
C.2.1	Biodegradation/ bioaccumulation	Biodegradation/bioaccumulation of MNM addressed by the T&M?	1	Score 1: Yes		X
				Score 0.1: No
C.2.2	Hydrolysis	Hydrolysis of MNM addressed by the T&M?	1	Score 1: Yes		X
				Score 0.1: No
C.2.3	Absorption/ desorption	Absorption/desorption of MNM addressed by the T&M?	1	Score 1: Yes		X
				Score 0.1: No
C.2.4	Short term toxicity on invertebrates/ fishes	Short term toxicity on invertebrates/fishes of MNM addressed by the T&M?	1	Score 1: Yes		X
				Score 0.1: No
C.2.5	Effects on terrestrial organisms	Effects on terrestrial organisms of MNM addressed by the T&M?	1	Score 1: Yes		X
				Score 0.1: No
C.2.6	Growth inhibition in aquatic plants	Growth inhibition in aquatic plants of MNM addressed by the T&M?	1	Score 1: Yes		X
				Score 0.1: No
C.3.1	Innovation	Innovation of MNM use addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.2	Marketability	Marketability of MNM use addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.3	Increased efficiency compared to bulk alternative	Increased efficiency due to MNM use, compared to bulk alternative, addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.4	Profitability	Profitability of MNM use addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.5	Time to market launch	Impact of MNM use on time to market launch addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.6	Availability and suitability of alternative substances	Availability and suitability of alternative substances addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.7	Impact on trade, competitiveness and investment	Impact of MNM use on trade, competitiveness and investment addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.8	Capacity to improve quality of life	Capacity of MNM use to improve quality of life addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.9	Impact on the next generation	Impact of MNM use on the next generation addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.10	Occupational health risk	Occupational health risk due to MNM use addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.11	Consumer health risk	Consumer health risk due to MNM use addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.12	Employment opportunities	Impact of MNM use on employment opportunities addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.3.13	Public health and safety	Impact of MNM use on public health and safety addressed by the T&M?	1	Score 1: Yes				X
				Score 0.1: No
C.4.1	Energy consumption	Impact of MNM use on energy consumption addressed by the T&M?	1	Score 1: Yes					X
				Score 0.1: No
C.4.2	Material consumption	Impact of MNM use on material consumption addressed by the T&M?	1	Score 1: Yes					X
				Score 0.1: No
C.4.3	Green-house gasses (GHG) emissions	Impact of MNM use on green-house gasses emissions addressed by the T&M?	1	Score 1: Yes					X
				Score 0.1: No
C.4.4	Emission of hazardous substances in air, soil, or water from manufacturing, use, or recycling	Impact of MNM use on emission of hazardous substances in air, soil, or water from manufacturing, use, or recycling addressed by the T&M?	1	Score 1: Yes					X
				Score 0.1: No
C.4.5	Waste from the manufacturing, use, or recycling	Impact of MNM use on waste from the manufacturing, use, or recycling addressed by the T&M?	1	Score 1: Yes					X
				Score 0.1: No
C.4.6	Product lifetime	Impact of MNM use on product lifetime addressed by the T&M?	1	Score 1: Yes					X
				Score 0.1: No
C.4.7	Environmental risk	Impact of MNM use on environmental risk addressed by the T&M?	1	Score 1: Yes					X
				Score 0.1: No

Human exposure assessment is part of the human risk assessment in which exposure and hazard levels are combined to assess the overall risk. However, these two aspects have been kept separate within the TRAAC framework. This can be clearly seen in Fig. 1 in which the modules 1 and 3 share parameters C.1.1 to C.1.22 but are still separate modules. The reason behind this separation is that there are several T&M which are developed to focus only on the human exposure assessment and exclude the hazard assessment. Scoring them for their completeness based on risk assessment parameters/criteria will unjustifiably lower their Completeness score. Thus, it was decided for human exposure assessing T&M to have their own Completeness module with the parameters/criteria dedicated to human exposure assessment.

2.4. Scoring system and assumptions

To allow for a comprehensive and objective assessment (as much as possible), the framework has been designed to yield a score per pillar as well as an overall score combining the individual scores per pillar. Fig. 2 shows the scoring system considered in the TRAAC framework schematically.

Fig. 2.

Scoring system in TRAAC framework.

Within each pillar, as mentioned earlier, two or three scoring options (1, 0.5 and 0.1) are available when attributing a fulfilment score to a criterion (Table 1, Table 2, Table 3, Table 4). For instance, the criterion T.1: “Knowledge of developers, their affiliations and communication with end-users” in Table 1 has an assigned weight of 3 and offers three scoring options to the user: score 1, score 0.5 and score 0.1 (see Table 1 for details on these scores).

If the second option is valid for a given T&M, the score for the given criterion is equal to 1.5 (=0.5 × 3). A similar procedure is followed for each criterion to obtain individual scores. A weighted mean of the individual scores is taken, as shown in eq. 1, to obtain a final score of the T&M per pillar.

$$\mathit{Score}\mathit{per}\mathit{pillar}=\frac{\sum _{i=}^n{W}_i{S}_i}{\sum _{i=}^n{W}_i}\left(1\right)$$

In eq. 1, W_i is the assigned a priori weight to the criterion (based on expert judgment and stakeholder views), S_i is the scoring option valid for the criterion and n is the number of criteria in a pillar. Such score-based decision-making tends the framework to be inherently based on two assumptions:

• (i)Direct value/utility functions are used with simple relations (simple value function, could have 0.1 utility at the lowest potential value and 1 at the highest potential value). Thus, it is assumed that the decision is made based on the rational to prefer more value to less and willingness to make the trade-offs implied via their value functions.
• (ii)All value functions are combined and assessed simultaneously, instead of using a hierarchical/filtering/rule-based approach, assessing only specific functions. It is, thus, assumed that all our pillars are equally important, and none can be excluded by the judgment.

2.5. TRAAC robustness evaluation

The robustness of the TRAAC framework was evaluated based on: inter-criteria correlation analysis and user variability analysis.

2.5.1. Inter-criteria correlation analysis

For the inter-criteria correlation analysis, each criterion from all 5 pillars was picked one-by-one and was checked for its correlation with the rest of the criteria in a correlation matrix. It also allowed to determine whether there is any inherent bias within the framework towards any specific aspect/criterion. Since the scores could be varied only on 3 levels, i.e. 0.1, 0.5 and 1 (or sometimes at 2 levels for some criteria- 0.1 and 1), we could not perform any regression analysis between two criteria to determine the correlation coefficient between them. Instead, the correlation was assessed on a qualitative level in which it was checked whether changing the score of a criterion from 0.1 to 0.5 and then from 0.5 to 1 led to any change in the score of the other criterion.

Based on this methodology, we did not observe correlations between any given pair of criteria in all 5 pillars, except for the reliability criterion R.2, i.e. Peer-review, which was observed to have an effect on the criteria T.1, T.2 and T.3, and thus the correlation. The peer-review of a T&M (i.e. score for R.2 is changed from 0.1 to 1) ensures that the developers and affiliations are known and hence can be communicated with (i.e. score of T.1 changes to 1). A peer-review publication on the T&M also ensures that its detailed description is publicly available (i.e. score of T.2 changes to 0.5 or 1) and its strengths and limitations are properly scrutinized (i.e. score of T.3 changes to 0.5 or 1). However, there is no such correlation between R.2 and the three Transparency criteria (T.1, T.2 and T.3) when the score for R.2 is changed back to 0, i.e. absence of any peer-review of the T&M (i.e. score for R.2 = 0.1) does not necessarily imply that T.1, T.2 or T.3 = 0.1 because the knowledge of the T&M developers and their affiliations or a detailed description of the T&M or the knowledge of the T&M strengths and limitations can be made available through other media too (e.g. guidance material, interactive sessions). One does not have to rely on the peer-review publications for these aspects.

2.5.2. Between-users variability analysis

Another factor to consider when assessing or validating a tool is the user variability (i.e. when multiple users assess the same T&M, do they come to the same conclusions and how much do the conclusions agree/differ with each other). To analyse such user variability, 12 experts in the field of risk assessment of MNMs (2 of them from respective regulatory agencies) were invited to use the TRAAC framework to assess three T&M, namely NanoSafer, LICARA nanoSCAN, and the Precautionary Matrix for synthetic nanomaterials (see Table 6). They independently filled in their assessments in pre-defined excel forms. The assessment results (details are provided in Supplementary information) were subsequently entered in a dataframe using R statistical software. A mixed effect model was then used with the different TRAAC pillars as random components to assess the variability in the scoring between the 12 assessors. The user agreement was assessed by calculating the Intraclass correlation coefficient (ICC).

Table 6.

List of selected T&M to demonstrate TRAAC framework user-variability.

T&M		Nano-specific	Type	Application domain	Route of exposure	Applicable population
1	NanoSafer	Yes	Tool	Risk assessment and management	Inhalation only	Worker
2	LICARA NanoScan	Yes	Tool	Socio-economic impact assessment, Environmental impact assessment, Risk-benefit analysis	Inhalation only	Environment, Worker, Consumer, General population
3	Precautionary Matrix for synthetic nanomaterials	Yes	Tool	Exposure assessment	All routes	Environment, Worker, Consumer

The ICC was calculated to be equal to 0.55 for the overall assessment (including all three T&M). In terms of individual T&M, ICC is equal to 0.58, 0.49 and 0.72 for the LICARA nanoSCAN, NanoSafer and Swiss Precautionary matrix respectively. According to the ICC classification and interpretation by Cicchetti (1994), the obtained ICC values signify fair to good agreement among the user assessments.

The spread of the TRAAC score from all user assessments per T&M is shown in Fig. 3 (together with its quartile-based coefficient of variation, CV = IQR/Median) for all five pillars. The outliers shown in the Fig. 3 are statistically calculated (i.e. TRAAC scores lesser than Q1 − 1.5 × IQR or greater than Q3 + 1.5 × IQR) and do not signify any error in the user assessment. In addition, the aforementioned ICC calculation includes these outliers and thus the ICC values are bound to be higher if outliers are to be excluded from the analysis. Irrespective of the T&M, Fig. 3 shows that the variability in the TRAAC scores is consistently highest in case of the Reliability pillar for which the greatest CVs, equal to 40%, 57% and 57%, are observed respectively for LICARA nanoSCAN, NanoSafer and Swiss Precautionary matrix.

Fig. 3.

Variability of the TRAAC scores per pillar for the 3 assessed T&M, i.e. LICARA nanoSCAN (LIC), NanoSafer Control Banding (NS) and Precautionary Matrix for synthetic nanomaterials (PM) obtained from 12 independent assessments and their comparison with authors' estimates (shown as ‘own estimate’); details on the calculation of ‘Own estimate’ for each T&M are provided in Supplementary information.

2.5.3. Usability for other T&M

To demonstrate the usability of the TRAAC framework, several types of T&Ms., as shown in Table 7, were evaluated.

Table 7.

List of T&M evaluated using TRAAC framework; N/A: not applicable.

Selected T&M		Nano-specific	Type	Application domain	Route of exposure	Applicable population
1	Stoffenmanager Nano	Yes	Tool	Risk assessment and prioritization	Inhalation only	Worker
2	MARINA Framework	Yes	Method	Risk assessment	All routes	Environment, Worker, Consumer, General population
3	LRI Ambit 2	Partly	Tool and method	Risk assessment	All routes	Environment, Worker, Consumer, General population
4	Advanced REACH Tool (ART)	No	Tool	Exposure assessment	Inhalation only	Workers
5	NewHoRRIzon Societal Readiness Thinking (SRT)	No	Method	Social-economic impact assessment, Environmental impact assessment	N/A	N/A
6	ECETOC TRA	No	Tool	Exposure assessment	Inhalation, Dermal	Environment, Worker, Consumer
7	RiskofDerm (ROD)	No	Tool	Exposure assessment	Dermal only	Worker
8	Future Nano Needs Bayesian Belief Network (FNN-BBN)	Yes	Tool	Exposure assessment	Inhalation only	Worker
9	SimpleBox4Nano	Yes	Tool	Exposure assessment	N/A	Environment
10	ConsExpo nano	Yes	Tool	Exposure assessment	Inhalation only	Consumer
11	NanoDUFLOW Model	Yes	Tool	Exposure assessment	N/A	Environment

The 11T&Ms. in Table 7 were selected from a comprehensive inventory of 160 state-of-the-art and emerging T&M candidates for the Nano Risk Governance Portal (OECD, 2021a; Shandilya and Franken, 2020). Evaluating all 160 T&M using the TRAAC framework was not practical due to the limited resources and thus certain selection boundaries had to be established to exclude the following types of T&M. Nevertheless, it does not imply that they cannot be evaluated using TRAAC framework.

• -The T&M which are not publicly available, have a broken access link, or are in a language other than English; there were such 10 T&Ms.
• -The T&M classified as “regulatory documents” or “standards documents” and published by governmental agencies or governments for which regulatory acceptance is already established; there were such 46 T&Ms.
• -The guidance T&M, databases, system of T&Ms. and web platforms compiling T&M and data, which are either meant as strictly informative for the process (providing guidance), uses several T&M to provide an assessment or simply serve as a library of T&M to guide a user to an appropriate T&M; there were such 28 T&Ms.
• -The T&M requiring a fee for access or considered a service, and thus requiring the intervention of a third party to carry out an assessment; there were such 4 T&Ms.

Consequently, the remaining 72 T&Ms. were considered eligible for an evaluation. Diversity of the T&Ms. in terms of their nano-specificity, type, application domain, applicable population and route of exposure was further considered as the further basis of the T&Ms.' selection, as shown in Table 7. As a result, the list of 72 T&Ms. was refined down to the 14 T&Ms. The diversity of these 14 T&Ms. in terms of the aforementioned aspects ensure that they represent a general sub-set of the Nano Risk Governance Portal inventory. Together with NanoSafer, LICARA nanoSCAN, and the Precautionary Matrix for synthetic nanomaterials, the TRAAC assessment results for all 14 assessed T&Ms. are shown in Fig. 4.

Fig. 4.

TRAAC scores per pillar for different T&M; from left to right: LICARA nanoSCAN (LIC), ECETOC-TRA, NanoSafer Control Banding (NS), Precautionary Matrix for synthetic nanomaterials (PM), Advanced REACH Tool (ART), LRI Ambit-2, Stoffenmanager Nano (STMn), Future Nano Needs Bayesian Belief Network (FNN-BBN), RiskofDerm (ROD), SimpleBox4Nano (SB4N), The NewHoRRIzonSocial Readiness Thinking Tool (SRT), ConsExpo nano, NanoDUFLOW (NDF) and the MARINA Framework (MARINA). For each T&M, score per pillar is shown in the pillar dedicated colour band.

2.5.3.1. Transparency

The transparency scores obtained for the tested T&Ms. range from 0.37 to 1 (Fig. 4). Overall, the T&M perform relatively well for this pillar as the average score is equal to 0.81 which is highest compared with the other 4 pillars. There are particularly 5 T&M which score perfect transparency score of 1, i.e. ART, FNN-BBN, RiskofDerm, SimpleBox4Nano and NDF. It signifies that the tested T&Ms. generally have good transparency towards their end-users. The details on the calculated scores for each T&M are provided in the Supplementary information. During the analysis, it was observed that while most T&Ms. provide contact details for end-users and detailed descriptions of the T&Ms., the developers and partners associated with a T&M are not always clearly disclosed. This is more noticeable for industry developed T&M, such as LRI AMBIT 2 and ECETOC TRA, which disclose organization affiliations rather than the names of the developers in most cases. The T&M score was generally low for criteria T.3 (regarding the disclosure of the strengths and limitations).

2.5.3.2. Reliability

With regards to reliability, scores range from 0.27 to 0.93, with an average of 0.6 (see Fig. 4). The top scoring T&Ms. in terms of reliability are ART (0.93), RiskofDerm (0.93), Stoffenmanager nano (0.7) and SimpleBox4Nano (0.7). The socio-economic impact assessment T&M (i.e. SRT), as well as MARINA framework, generally did not perform well in this pillar, with results below average. For the MARINA framework, its lowest score (0.27) is largely due to its early development phase and conceptual nature, which makes it difficult to validate using data, or quantify uncertainty. As for LICARA nanoSCAN and SRT, their lack of testing as well as SRT's inability to consider uncertainty affect its performance. While the T&Ms. score generally good (or above average) for most of the criteria, for criteria R.4 and R.5 which refer to the user variability and uncertainty, their performance is rather poor because of the absence of studies on the user variability (i.e. R.4) or a lack of description of the output uncertainty (R.5). The details on the calculated scores for each T&M are shown in the Supplementary information.

2.5.3.3. Accessibility

The scores in the accessibility pillar range from 0.32 to 0.96, with an average score of 0.69. As shown in Fig. 4, the top scoring T&M are ECETOC TRA (0.96), PM (0.93) and LRI Ambit 2 (0.88) which have not been developed within the research community but involved either the industry (i.e. LRI Ambit 2, ECETOC TRA) or a governmental agency (i.e. PM). While most T&M are easily findable (A.6) and often cited (A.5), inequalities in the quality of the guidance material have been noticed (A.1). User friendliness (A.7) is also an additional area for improvement. The three criteria where the T&M particularly lag are A.2, A.3 and A.4. Almost all T&M need some level of expertise in a relevant domain (e.g. material science, exposure science, toxicology) to be able to use them (criterion A.2). With regards to the other two criteria, A.3 (multiple languages) and A.4 (availability of a license), most of the T&Ms. are not available in multiple languages and do not have a license of use agreement. The T&Ms. with high relatively complexity of use (e.g. SimpleBox4Nano) or are conceptual frameworks or methods (e.g. MARINA Framework) or which are only available and detailed within academic research papers (e.g. NanoDUFLOW model), typically rank lower in this pillar. Operationalizing and using these T&Ms. require a greater effort on the part of end-users, making them less accessible as a result. The details on the calculated scores for each T&M are shown in the Supplementary information.

2.5.3.4. Applicability

The scores range between 0.4 and 0.88 in the Applicability pillar, with an average score of 0.67 (Fig. 4). The top scoring T&Ms. are LICARA nanoSCAN, ECETOC TRA, PM and SimpleBox4Nano, with two of them are from industrial sector and one developed by a regulatory agency. The top ten ranking T&M of the pillar are all nano-specific, with the exception of ECETOC TRA. This is expected as 9 of the 14 T&M are nano specific and nano-specificity (i.e. P.3) is the heaviest weighted criterion (i.e. 5). ECETOC TRA performs well for every criterion due to having a very broad applicability domain, except for P.3 (nano-specificity). The majority of the T&Ms. generally score lower for the criterion on applicable exposure routes and/or population (P.2) because they are specific to either one exposure route (e.g. inhalation, dermal) or one population type (e.g. worker, environment, consumer). Similarly, providing some advice on relevant measures to improve the assessment output to accepted level is out of scope for a lot of T&Ms. due to which they score poorly for the criterion P.8 (advice on improvement of the result). The details on the calculated scores for each T&M are shown in the Supplementary information.

2.5.3.5. Completeness

The scores of the T&Ms. for the completeness pillar range from 0.3 to 0.96, with an average score of 0.56 (Fig. 4), which is lowest among all 5 pillars. The completeness of a T&M is evaluated using criteria of the particular module to which it belongs, i.e. NS, STMn belongs to Module 1; LRI Ambit 2 belongs to both Module 1 and 2; ECETOC TRA, ART, FNN-BBN, ROD and ConsExpo nano belong to Module 3; LICARA nanoSCAN and SRT belong to both Modules 4 and 5. The cases for which a T&M belongs to more than one Completeness module, only one module was selected and is shown in Fig. 4. Two exposure assessment T&Ms., i.e. SimpleBox4Nano, NanoDUFLOW Model, were not graded for this pillar, as their focus is on environmental fate for which several parameters like photoreaction, heteroaggregation, sulfidation etc., are highly relevant which can only be considered within an environmental fate dedicated module. The Completeness pillar is also not well-suited for conceptual T&M, like MARINA framework, due to which it was also not graded. The T&Ms. which scores >0.7 are LICARA nanoSCAN, NS and LRI Ambit 2. The rest of the other T&M, irrespective of the modules they belong to, score ≤ 0.5 (approx.) which illustrates that only a few of these T&Ms. have the potential to fulfill the new REACH regulatory requirements or high compatibility with the ECHA requirements for the risk or socio-economic assessment, wherever applicable. The details on the calculated scores for each T&M are shown in the Supplementary information.

3. Discussion

3.1. Designing TRAAC

The TRAAC framework was designed as a structured approach to identify the potential readiness of T&Ms. for the assessment of MNMs with respect to regulatory acceptance and stakeholder needs. As a proof of concept, the framework was tested with a wide range of T&Ms., providing insight into the usability of the framework, the inter-criteria correlation and the between-users variability. In addition, this exercise provided explorative insights into the strengths and weaknesses of the investigated T&Ms. (most of them are recognized by different sources as state-of-the-art T&Ms. for their specific purposes, see OECD, 2021a, Shandilya and Franken, 2020).

The TRAAC framework addresses the aspect of regulatory acceptance of T&Ms. through its 5 pillars based on three elements: (i) regulatory compliance (compliance with the law, which is embodied by the Completeness pillar), (ii) working principles (e.g. transparency, multi-stakeholder approaches), and (iii) values upheld by the regulatory institutions and stakeholders (e.g. veracity of data/scientific method, output correctness, open-access, and re-usability of knowledge).

3.2. T&M scoring

While scoring the T&Ms., it was observed that the T&Ms. with the highest score for a particular pillar achieved, in general, a good balance between different pillar criteria. However, the results showed that most of the nano-specific T&Ms. score consistently low for the Reliability pillar particularly because of the lack of their validation studies and uncertainty analysis compared to the T&Ms. which are used under REACH for bulk chemicals and have been checked on reliability (e.g. ECETOC TRA, Stoffenmanager and ART) (Kupczewska-dobecka et al.,2011; Delmaar et al., 2013; Hesse et al., 2015; Lamb et al., 2015; Riedmann et al., 2015; Jung et al., 2016; Marquart et al., 2017; van Tongeren et al., 2017; Franken et al., 2020a, Franken et al., 2020b). This aspect will likely change though with the advent of the recently accomplished OECD initiative on the assessment of T&M accessibility, sensitivity and performance regarding nano-specific environmental, consumer and occupational exposure assessment (OECD, 2021a), indicating a possible shift from development of new T&M, towards validation of existing T&M for their nano-specificity.

In the context of the Completeness pillar, low scores of most of the T&Ms. were observed in each module. The performance further lowered down in the case of the ‘human exposure assessment’ module. The reason for such a low performance can be attributed to the fact that these T&Ms. are limited to particular exposure routes (e.g. inhalation, dermal, oral) and, therefore, their input and output information are relevant to only those exposure routes. The criteria within the Completeness pillar, on the other hand, are constituted in such a way that they are relevant to any exposure route in accordance with the regulatory requirements. The criteria, thus, inadvertently lower down the individual Completeness scores of the T&Ms. Consequently, for these T&Ms. to score higher, they must improve their applicability to multiple exposure routes (criterion P.2 in Applicability pillar).

The TRAAC framework is intended to be easy-to-use when grading a variety of T&Ms. However, a certain degree of knowledge on the T&Ms. is needed to apply it. Some literature search is also required to assess some of the criteria (e.g. investigate for available validation studies, available guidance documents and publications with regards to the T&M), which was found to be occasionally time consuming and the information was scattered over various sources. This was consistently remarked by almost all TRAAC users during the user-variability study. Therefore, the intended target users of this framework are risk managers, regulators, industrial users of existing T&Ms. who possess certain knowledge of the T&Ms. and are interested in assessing their regulatory readiness, and T&M developers too who can specifically investigate where their own T&M might improve upon in future updates. Based on their needs, the target users can select the T&Ms. which score high (e.g. higher than the average scores obtained for different T&Ms) either overall for all five pillars or for a particular pillar which they deem is more important over other pillars.

From the experiences of previous EU projects (e.g. NANoREG2, caLIBRAte), it has been observed that the qualitative T&Ms. which require less data and are considered quicker to operate are preferred by the stakeholders, particularly SMEs, over more complicated (and often quantitative) ones because the latter can require data inputs which are often unknown. The general lack or low availability of the data/information on human and environmental impact of rapidly innovated MNMs basically accentuates this constraint. Although the qualitative T&Ms. serve their purpose of risk screening in especially the early stages of innovation value chain, and can help users set “early-warnings”/“red-flags” based on possible health risks, or lack of benefits, they provide outputs with higher uncertainty than the quantitative ones which are more to be used along the later stages of innovation value chain when more data/information is known. Shandilya et al., 2020 and Soeteman-Hernandez et al., 2019b recommend the use of certain T&M type based on the stage within the innovation value chain: the higher the stage level within the innovation process, more quantitative the T&M is to be used and relied upon.

Within the Reliability and Accessibility pillars, there are two respective criteria named as Uncertainty analysis (criterion R.5) and Quantitative output (criterion P.4) which give higher scores to the quantitative T&Ms. (e.g. FNN-BBN) than to the qualitative ones (e.g. PM) which, in the perspective of legislative frameworks, is logical considering quantitative input is required in the product registration/authorization dossiers. On the other hand, the Accessibility and Applicability pillars consider respective criteria named as User-friendliness (criterion A.7) and Time and cost efficiency (criterion P.6) for which qualitative T&Ms. score higher over quantitative ones. Thus, at the end, any undue advantage over the T&Ms. type (qualitative or quantitative) balances out and the TRAAC framework allows for an unbiased assessment.

3.3. Between-users variability

The between-users variability assessment showed fair to good agreement between the 12 assessors for the three T&Ms. which were investigated. User variability assessment is a useful method to provide insight on the expertise needed for the use of such T&Ms. The current assessment was performed with external assessors who had no previous knowledge with regards to the TRAAC framework and received no training on its use. Only the criteria descriptions were provided to them, as given in Table 1, Table 2, Table 3, Table 4, Table 5, prior to the assessment and they were asked to provide their assessment results in pre-defined excel forms. The assessors reported a general duration of 2 h to 3 h per T&M to finish their assessments. While the assessors are recognized as experts in the field of risk assessment and innovation of MNMs, their expertise for all three assessed T&Ms. was unknown and their knowledge between the three T&Ms. might vary. Usually, only the knowledge of a model and its input parameters is investigated to measure the user-variability regarding the model. In such case, model users have to translate test scenario descriptors to model input parameters based on their interpretation of the input parameters (Schinkel et al., 2014). However, in the present case, the user variability accounts for not only the assessors' interpretation of input parameters of the three T&Ms., but also for the understanding of the TRAAC criteria. For the latter, although the criteria have been described as objectively as possible (to minimize the possibility of multiple interpretations of one criterion), there will always be some subjectivity associated with the criteria description. Thus, these two sources of variability equally impact the shown user agreements in this study.

Between-users variability was further demonstrated by looking at individual assessments within the Reliability pillar which showed highest variability among all five pillars. Agreement was high for criteria “R.2 Peer-review” and “R.6 Collaborative effort” as they were easy to assess, even for a non-expert, merely on the basis of available documentation. However, for the criteria like “R.1 Testing and/or validation”, “R.3 Quality of the data” and “R.5 Uncertainty analysis”, the assessment scores varied greatly. Such criteria require in-depth knowledge of the T&M being assessed. Therefore, this exercise showed that the TRAAC framework is to be used by experts on risks and risk management of MNMs, with expert knowledge of the T&M being assessed. In accordance with Schinkel et al. (2014) which demonstrated a positive training effect, training material and courses are currently being developed for future TRAAC framework users to lower the variability associated with their comprehension of the TRAAC criteria.

3.4. Strengths, limitations and future perspectives

This study investigated how available T&Ms. can be assessed for their regulatory and wider use readiness in a structured manner and as objectively as possible. This is especially important in the field of nanomaterial science as the field progresses quickly and numerous T&Ms. are continuously developed and introduced while regulators and industries have difficulty in selecting the most appropriate T&M(s) for their use. Most of the criteria proposed within the TRAAC framework are based on a literature search of (i) existing regulatory frameworks, (ii) stakeholder needs in regards to MNMs governance and (iii) T&M for the assessment of MNMs for their human and environmental risk as well as socio-economic impacts. This ensures that the needs brought forward by the TRAAC framework for the further development of the T&M for improving their potential for regulatory acceptance and downstream use are supported and grounded in literature as well as supported by the current vision of the stakeholders' needs. This helps in imparting regulatory preparedness, as underlined within the Safe Innovation Approach (Soeteman-Hernandez et al., 2019b) and constitutes the main strength of the TRAAC framework.

The entire framework, its pillars, criteria and its weights are based on expertise and results of some of the largest EU projects and initiatives which, in the last years, dealt with the development of T&Ms. for MNMs, thus providing a unique synthesis of available knowledge on these T&Ms.

Moreover, the results of the investigated T&Ms. showed that the overall results provided by the TRAAC framework seem to correlate well with the results obtained in previous reviews of the existing T&Ms. which considered T&Ms. assessment with regards to stakeholder needs and available regulations (Franken et al., 2020a, Franken et al., 2020b; Sørensen et al., 2019; Isigonis et al., 2019; Trump et al., 2018). These review studies also considered some of the top performing T&M in the TRAAC framework, e.g. LICARA NanoScan, NanoSafer CB and Stoffenmanager nano, as promising options for supporting risk governance for MNMs.

The TRAAC framework also has the potential to be useful for T&M developers themselves, as it allows for them to identify the strengths and weaknesses of their own respective T&Ms., and identify future opportunities for T&M improvement.

With regards to the reliability assessment, the framework currently assesses it through several criteria, but occasionally it is sufficient to simply indicate that certain information with regards to a T&M is known. For example, a T&M receives a score of 1 when the uncertainty around the T&M results is known and reported. However, the current framework does not state what level of uncertainty is regulatory accepted. Similarly, it is stated that a T&M needs to be validated preferably with data, but the framework does not set conditions to when a T&M can be considered validated. The framework, thus, can be refined and updated in the future in this particular aspect as specific criterial agreements are made. In addition, the framework can further include a thorough assessment of the scientific quality of the T&Ms., which will require the results of the performance testing or validation study and establishing proper metrics to measure the performance of a T&M (OECD, 2021a). With regards to the Completeness pillar, it was developed using REACH as an example. Further developments in this pillar can address other regulatory frameworks as well, like the CLP 1272/2008 or the cosmetics regulation (EC 1223/2009) to make it more relevant to different regulations in place and broaden its applicability domain. This may result in the modification of the existing modules/criteria and/or new modules, e.g. life cycle assessment, classification and labeling etc.

When compared to other existing approaches which pertain to the T&Ms. assessment in the field of chemical and environmental risk, the distinctive feature of the TRAAC framework is the comprehensiveness and objectivity of its scope, as represented in its five pillars. Previous pertinent studies have either discussed such aspects in a qualitative manner (Isigonis et al., 2019; Hristozov et al., 2016; Musuamba et al., 2014; Fadeel et al., 2018) or proposed criteria similar to only accessibility and transparency pillars of TRAAC, particularly the ones adapting the FAIR data principles to general research software (Hasselbring et al., 2020; Gruenpeter et al., 2020; Lamprecht et al., 2020). The TRAAC framework adds value to the existing state-of-art by providing a more integrated approach to the evaluation of T&Ms. in the field of chemical and environmental risk, considering not only reliability, but also other aspects such as regulatory requirements, accessibility, transparency, and stakeholder's needs. In addition, the framework proposes a novel approach to quantify and combine the different evaluation criteria, using scores and weights inspired by MCDA approach, guiding users in understanding and prioritizing the different criteria and thus performing a comparative evaluation among alternative T&Ms.

Currently, the TRAAC framework has been designed to evaluate T&Ms. In the future, the use of the TRAAC framework is planned to be further extended to consider data and knowledge readiness level for efficient risk governance. This effort on data is currently being undertaken within the framework of EU NMBP-13 NANORIGO and Gov4Nano projects. This will include the data stored as a resource and managed according to formal data resource management concepts, principles, and techniques. In addition, the TRAAC framework was developed with respect to EU legislation (REACH), and the framework would need a revised “completeness” pillar for the framework to be used for different legislations outside the EU. The transparency, reliability, accessibility and applicability pillars are not dependent on legislation and can be easily used outside the EU. Several reasons have motivated our choice to focus specifically at this moment on the REACH regulation for the completeness pillar of TRAAC

• -the complexity of chemical regulations and potential hindrances to the drafting of a completeness pillar accounting for other non-EU regulations (e.g. time, language barriers, user-friendliness of the framework, different levels of maturity for the chemicals legislations regarding MNMs, and different definitions of MNMs upon which these regulations are based).
• -the unique positioning of REACH at the global level: indeed, REACH is and remains the most advanced and comprehensive chemicals regulation and chemicals knowledge base in the world. At this present time, how extensively MNMs are addressed by global chemicals regulations can vary greatly. In view of this, we developed the completeness pillar using REACH as a comprehensive foundation. Please note, however, that there is a possibility for the completeness pillar to be easily adapted to other regulations than REACH (e.g. UK-REACH, K-REACH from Korea) now and in the future, should the need for it arise.

4. Outlook

By developing a multi-faceted and multi-criteria approach, founded upon an extensive literature search, experts' judgment and the current regulatory requirements for MNMs, the TRAAC framework proved to be a useful tool to evaluate a large variety of state-of-the-art T&Ms., which are relevant for the safe innovation of MNMs. The concept and the structure of the TRAAC framework were found useful to evaluate regulatory compliance and end-users acceptance of T&Ms., and thus help to increase the knowledge and confidence of the stakeholders in the T&M. This also aligns with the successful implementation of EC-JRC's SSbD approach for MNM within Chemicals Strategy for Sustainability and the Zero Pollution Act (EC, 2019b; EC, 2022a). The TRAAC framework provides essential input into the quality of the T&M to assess MNM safety and sustainability at each step of the EC-JRC's SSbD approach: the JRC SSbD framework is based on 5 sequential steps to be followed before market launch, those are composed of intrinsic hazard assessment, Occupational Health, Consumer and Environmental Health, Environmental Sustainability Assessment and Social and Economic Sustainability Assessment. However, the JRC SSbD framework is initially considered for chemicals and may not capture the particulars of the MNMs. The TRAAC framework helps to address most of the above steps through the identification of the most suitable method per step, and providing the user with dedicated tools for MNMs. Hence the TRAAC framework not only becomes a key element in the identification of the most relevant methods to feed the particular steps but anticipates issues regarding the MNMs which have not yet been dealt within the current case studies testing the EC JRC framework.

Funding

This work was supported by the EU NMBP-13 H2020 Gov4Nano project to develop the TRAAC framework (grant number 814401).

CRediT authorship contribution statement

Neeraj Shandilya: Conceptualization, Methodology, Formal analysis, Writing – review & editing, Visualization. Marie-Sophie Barreau: Writing – original draft, Investigation, Resources. Blanca Suarez-Merino: Investigation, Writing – review & editing. Andrea Porcari: Methodology, Formal analysis, Writing – review & editing, Visualization. Daniela Pimponi: Writing – review & editing. Keld Alstrup Jensen: Conceptualization, Formal analysis, Writing – review & editing, Funding acquisition. Wouter Fransman: Writing – review & editing, Supervision. Remy Franken: Methodology, Formal analysis, Writing – review & editing, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Editor: Bernd Nowack

Appendix A. Supplementary data

Tools and Methods scores within each TRAAC pillar

Experts' assessments of tools to anlyse between-users variability of the framework

Data availability

No data was used for the research described in the article.