Abstract
Ultrasonication is a widely used and standardized method to redisperse nanopowders in liquids and to homogenize nanoparticle dispersions. One goal of sonication is to disrupt agglomerates without changing the intrinsic physicochemical properties of the primary particles. The outcome of sonication, however, is most of the time uncertain, and quantitative models have been beyond reach. The magnitude of this problem is considerable owing to fact that the efficiency of sonication is not only dependent on the parameters of the actual device, but also on the physicochemical properties such as of the particle dispersion itself. As a consequence, sonication suffers from poor reproducibility. To tackle this problem, we propose to involve machine learning. By focusing on four nanoparticle types in aqueous dispersions, we combine supervised machine learning and dynamic light scattering to analyze the aggregate size after sonication, and demonstrate the potential to improve considerably the design and reproducibility of sonication experiments.
Ultrasonication is a widely used and standardized method to redisperse nanopowders in liquids and to homogenize nanoparticle dispersions. Here, we use Machine Learning to predict the outcome of ultrsonication experiments on oxide nanoparticles.
Engineered nanoparticles (ENPs) have been used in commercial products, such as foodstuff, cosmetics and personal care for over two decades.1–3 Typical materials are zinc oxide, titania, ceria, and silica.4,5 Consumers may find titania ENPs in toothpaste,6 and zinc oxide in sunscreen.7 Such particles are usually produced and sold in large quantities, and thus stored in a dry powdered form (nanopowder). When stored as powders, particles can form agglomerates, which are single particles clumping together, owing to attractive interparticle interactions.8 Given that the usefulness of ENPs is due to their specific and tunable physicochemical properties that are mediated by the vast surface-to-volume ratio, powders are to be liquified and redispersed before supplying them to manufacturing processes. To redisperse powders, ultrasonication is the most often-used method.9,10 Indeed, sonication, acting via fluid cavitation, is preferred over ball milling or high shear mixing due to its greater efficiency.11,12 Furthermore, ultrasonication is applied not only in industrial labs but in academic research labs as well, preparing ENPs in toxicological and environmental studies.11,13,14 If overdone, ultrasonication, however, may considerably alter the physicochemical properties of the particles.11,12,15,16 This effect has been observed especially for mass-produced metal oxide nanopowders, such as silica, titania or alumina, that are synthesized in the high temperature vapor phase.12,15,16 For example, when it comes to hazard assessment addressing environmental or biological systems, any physicochemical change of the primary particles induced by sonication is undesired.11 In such cases, the reduction of the primary particle size may strongly influence the pathway of cellular internalization,17 cytotoxicity,18 and biodistribution,19 and thus, the overall fate in environmental milieu.20 At the same time, compared to the primary particles, agglomerates may render particle dosimetry characterization more difficult,21–23 and may also exhibit different hazards.24 Dominantly, two approaches have been followed in the sonication of ENPs: on the one hand, a generic standard operating procedure (SOP) is carried out,25–28 which enforces uniformity in the experimental settings but directly comes with the disadvantage of non-adjustable experimental parameters and the eventuality of having unwanted changes in the particles, such as size reduction, shift in zeta-potential, or even the appearance of sonication-induced agglomerates.11,29 On the other hand, custom-made protocols dedicated to given materials and given experimental conditions may be developed.6,14,30–33 This is, in essence, the more attractive option, offering customization as well as the prospect of standardization. There is a caveat though: the parameter space is vast, and classical methods of optimization become severely limited by time and cost.14,30–34
To get around this bottleneck eventually, here we approached the problem via machine learning (ML). ML is data-driven and known for the ability to surpass human performance in various situations by being able to capture non-intuitive nonlinear and multivariate relationships. Using supervised ML, we built an algorithmic model that is able to capture characteristic features of the complex relationships between (a) the outcome of redispersion and deagglomeration, and (b) the combinations of sonication parameters—such as particle concentration, dispersion volume, sonicator type, duration of sonication, and sonication power—and selected physicochemical properties of the particles—such as size, zeta-potential, isoelectric point, surface coating and material type. We use a gradient boosted decision tree algorithm and exclusively focus on four ENP types, where we expect similar behavior to ultrasonication. We characterize the outcome via the intensity-weighted Z-average hydrodynamic size (diameter) and polydispersity index (PDI) of the particles via dynamic light scattering (DLS). According to the fundamental principles of DLS, these two are the most reproducible parameters one can determine from DLS experiments.35
Supervised ML aims at establishing a quantitative relationship between features (inputs) and labels (outputs). The outputs of sonication were the Z-average hydrodynamic radius and the PDI, and the inputs were the sonication parameters and particle properties. Our data-driven ML approach was performed on two consecutive studies (Fig. 1). In the first part of the study, we addressed well-defined—but not strictly monodisperse—model ENPs, which were synthesized, processed, and characterized in our own laboratory. For this, we synthesized aqueous dispersions of non-crystalline silicon dioxide nanoparticles (SiO2 ENPs), commonly known as colloidal silica. The SiO2 ENPs were synthesized with nominal diameters of roughly 40 nm, 70 nm, and 100 nm, respectively, using a co-condensation reaction adapted from Stöber et al.36 Depending on the desired particle size, different relative amounts of ethanol, ammonia and water (MilliQ) were mixed and heated to 40 °C (ESI, Table SI1†). After stirring the mixture for 1 h at constant temperature for equilibration, tetraethyl orthosilicate was added, and the mixture was stirred for another 8 h. After the mixture cooled down to room temperature, the SiO2 ENPs were purified by centrifugation (Thermo Scientific, 5000g, 5, 10, 15 and 20 min, five cycles in total). To mimic the extensive degree of agglomeration in powders, the ENPs were agglomerated via the change of ionic strength,37 by adding magnesium chloride hexahydrate (50 g L−1) to the aqueous dispersion. Agglomerated samples were purified again by dialysis and were resuspended in water before undergoing different sonication processes.
Fig. 1. Workflow of our studies. In the first part of the study, experimental data are generated by characterizing agglomerated dispersions of model SiO2 ENPs, before and after ultrasonication, using different, systematically combined sonication parameters. These data are used for supervising a ML algorithm, where the sonication parameters and particle sizes are fed into the algorithm as features (inputs), and the DLS results (Z-average and PDI) as predicting labels (outputs). By mapping and approximating the functional relationships between labels and features, the goal of the ML model is to predict the outcome of ultrasonication in terms of DLS analysis. In the second part of the study, meta-analysis is performed on data mined from peer-reviewed publications addressing the ultrasonication of particle systems. We focused on oxides—namely ZnO, SiO2, CeO2 and TiO2—given their evident presence in consumer products.38,39.
The overall number of parameter combinations was obtained via a generalized fractional factorial experimental design aimed at extracting an optimal amount of information from the smallest possible number of experiments, while still being able to understand the relationship between the different parameters and parameter values with the experiment outcome.40 In this design of experiments we considered that the primary particle size, particle concentration, dispersion volume, duration and power of sonication were found to have an effect on the degree of deagglomeration of particle agglomerates.11,12,41–44 We designed the sonication experiments using the pyDOE2-library,45 and used a reduction factor of seven, which reduced the number of experiments from 108 (full factorial design) to 14, with every experiment carried out in triplicate. Each triplicated design contained one of either two or three levels of combinations of particle concentration, water volume, sonicator type (probe vs. bath), duration of sonication, and effective energy and energy density of sonication (Table 1).
The experimental design space of ultrasonication. These sonication parameter combinations were used for all the three particle sizes. These parameters are features in the ML model. In the second study (meta-analysis), four additional features (particle properties) were included: isoelectric point, zeta-potential, material type, and surface coating.
| Sample vol. (mL) | Particle conc. (mg mL−1) | Type | Energy (J) | Duration (min) | Energy density (J mL−1) | |
|---|---|---|---|---|---|---|
| 1 | 1 | 1 | Probe | 179 | 1 | 179 |
| 2 | 1 | 1 | Probe | 17 256 | 20 | 17 256 |
| 3 | 1 | 1 | Bath | 2025 | 45 | 2025 |
| 4 | 1 | 5 | Probe | 3576 | 20 | 3576 |
| 5 | 1 | 5 | Probe | 38 826 | 45 | 38 826 |
| 6 | 1 | 10 | Probe | 8046 | 45 | 8046 |
| 7 | 5 | 1 | Probe | 3576 | 20 | 715 |
| 8 | 5 | 1 | Probe | 38 826 | 45 | 7765 |
| 9 | 5 | 5 | Probe | 8046 | 45 | 1609 |
| 10 | 5 | 10 | Bath | 194 | 1 | 39 |
| 11 | 10 | 1 | Probe | 8046 | 45 | 805 |
| 12 | 10 | 5 | Bath | 194 | 1 | 19 |
| 13 | 10 | 5 | Bath | 45 | 1 | 5 |
| 14 | 10 | 10 | Bath | 100 | 20 | 388 |
The ultrasonication devices were bath and horn-probe sonicators (Elmasonic P 60 H, ELMA and Branson SFX550 Sonifier equipped with a standard 13 mm diameter disruptor horn, Branson Ultrasonics Corp.). The sonication power and the corresponding energy release were calibrated by calorimetric measurements described elsewhere,28 and the details of calibration are presented in the ESI (Tables SI3 and 4†). After sonication, the particles were characterized by DLS (Malvern Panalytical, Zetasizer Nano-ZS) at room temperature (25 °C). For DLS analysis, the given sample volumes (500, 100, and 50 μL) were diluted with 1 mL water (MilliQ) depending on the particle concentration (1, 5, and, 10 g L−1). Dilution was necessary to minimize any potential bias owing to the negative impact of multiple light scattering and collective diffusion affecting Brownian dynamics.46 Each diluted triplicate was then measured three times, and the auto-correlation functions were analyzed by the methods of cumulants35,47 to determine the so-called Z-average and PDI.35,47 Typical examples of the DLS field auto-correlation functions, and representative TEM micrographs of the agglomerates are shown in the ESI (Fig. SI1, 2 and 4†). The Z-average and PDI are both functions of the intensity-weighted particle size distribution, which is affected by the parameters of sonication. Loosely speaking, the smaller the Z-average, the higher the impact of sonication on the agglomerates. The overall goal of sonication is to redisperse (that is, disintegrate) agglomerates without altering the primary particles’ properties.
For this straightforward goal, it is essential to closely approximate the unknown but existing functional relationship between the inputs (parameters of sonication) and the corresponding outputs (Z-average and PDI). This task is called multivariate regression analysis, and it is used for predicting and forecasting. The functional relationship is nevertheless complex, and reliable and transferable quantitative regression models are not available yet, to the best of our knowledge. This is the point where we invoke supervised machine learning. The model we implemented is based on a gradient boosted decision tree (GBDT) algorithm by using the XGBoost-library.48 The benefit of using GBDT is that it offers good efficiency and flexibility while being relatively fast and relatively easy to implement as well as interpret.48–51 Therefore, GBDTs are optimal for limited datasets due to their robustness in comparison with e.g., a deep-learner.52 Additionally, they enable a straightforward ranking of the feature importance. This offers insights into decision-making of the model, and ranks the importance of the underlying physical processes during ultrasonication. The structure of decision trees is composed of nodes and branches, where branches make ‘one-way’ connections between the nodes. Besides the root node—the very first node—there are two elementary types of nodes: the first type is non-leaf nodes, which are internal crossroad junctions of the decision route in the tree, representing either an attribute (e.g., “ultrasonication via bath sonicator”, “ultrasonication via horn sonicator”) or a question (e.g., is the particle size under 50 nm?”, “is the released energy density over 15 J mL−1?”). The second type is leaf nodes at the end of the decision-making process, which offer the prediction of the label. To improve the predictive accuracy of decision trees, a regression tree algorithm can be deployed, where trees are trained in consecutive learning cycles. In these cycles, the final prediction of a tree is tested on a measured data point. If the tree fails to predict the target, another tree is built on this error until a tree is trained where the prediction and the measured value overlap. With this, the final tree model can map, generalize and compare rules from an ensemble of specific and individual observations. A set of observations forms a dataset, which is described by two main attributes: features (inputs: parameters of sonication and particle properties) and labels (outputs: Z-average and PDI).
Supervising an ML has three main phases: training, validation, and testing. During training, the ML model is given known features and corresponding labels to find a relationship between these two. In a certain number of training rounds, the predictive accuracy of the model is improved by altering the tree structure and the decision rules. To achieve this improvement, the accuracy of the model's predictive power has to be tested after every training step, which is called validation. Briefly, a part of the training data is withheld during training and used to test the success of the trained model. If the prediction for the validation data is incorrect, the model starts another learning round with an adjusted tree. In the testing phase, the ability to generalize and approximate these relationships is tested by quantifying the agreement between ML-prediction and the so-far unseen data. To train, validate, and test the model, feature values must be formatted. First, categorical feature values (for example, parameter corresponding to sonicator type: horn vs. bath) were transformed into numerical values by One-Hot encoding, using the Scikit-learn library in Python.53 Encoding creates a ‘feature vector’ for each category of the parameter and fills it with either 1 or 0 to encode the presence or absence of the feature.51,54,55 Second, feature values were power-transformed to approximate a standard normal distribution.51,56,57 To define training and test data, we used random sampling and stratified splitting. Stratification was based on the average values of the features (Z-average or PDI), and as a result of stratification, the training and test sets were balanced, in the sense that both were representative of the population of the observations we had at hand. Stratification forces the model to learn on the full range of label values, and thus, it promotes higher prediction quality. The training set and test set were non-intersecting, that is, they had no common element. This was important to prevent the phenomenon of the so-called data leakage, which is the simultaneous occurrence of data with identical features–label combo in the training as well as in the test set.51 Therefore, the experimental triplicates of identical labels were never split, were kept in a given set, and any triplicate went either into the training or the test set. We used 70% of the data in the training sets and 30% in the test sets. To train our ML model, the algorithm was presented to features and the corresponding labels of the training set. During training, the model was repetitively tested on a small set, which is referred to as the validation set. In our case, 20% of the training set was allocated into the validation set. The validation set was withheld from the actual training rounds, but it was presented to test the prediction quality after single training steps. This was necessary to tune the hyperparameters of the model.58 To optimize the values of the hyperparameters (ESI,† machine learning terms), we used the tree of the Parzen estimator algorithm (maximum 200 trials) implemented in the Optuna library.59 At the end of supervision, the quality of learning, that is, the corresponding prediction accuracy was evaluated on the test set.51 The quality of prediction was quantified by the R2 score, which is the coefficient of determination. R2 is in fact equal to the square of the Pearson (linear) correlation coefficient, and quantifies the agreement between the experimentally measured values and ML-predicted values. By definition R2 = explained variation/total variation, and it may take values between 0 (no agreement) and 1 (perfect agreement). The structure of our ML approach is summarized in Fig. 2. The comparison between experimental values and values predicted by our ML model addressing the colloidal silica particles is shown in Fig. 3.
Fig. 2. The main units of our supervised machine learning. (a) Definition of data subsets we used for training, validating and testing the ML model. Data splitting is done randomly, but in a stratified manner. The goal of stratification is to obtain balanced subsets wherein the data population is represented equally. (b) Depiction of the algorithm of decision trees, where decisions are taken at nodes and propagating to the next nodes via branches. The validation set is used for hyperparameter optimization, which is achieved by adjusting the number of nodes and branches of the trees. Once the training has finished, the best model is ‘blind-tested’ with data unseen during training. (c) If the model passes this evaluation step, it may be useful for interpreting the relationship between inputs (features) and outputs (labels) by ranking the importance of the features in the ML model prediction.
Fig. 3. Parity plots of log10Z-average and log10 PDI values of the SiO2 ENPs synthesized, processed, and characterized in our lab. Data in gray color indicate ten independent training and testing rounds, while the data in turquoise/red color highlight the most successful training (88 data points) and testing (38 data points). The two models for Z-average and PDI show a R2 of 0.76 and 0.75. These values correspond to a linear correlation coefficient better than 0.87. The dashed black lines indicate perfect predictions (R2 = 1). A distribution of the R2 score for the test set of 100 newly randomly seeded and trained models can be found in Fig. SI7.†.
To test the ability to extrapolate by our ML model, we predicted labels whose features were not from the interval of the training set. For this, we synthesized and characterized a new batch of particles (approx. 80 nm SiO2 ENP) and constructed a new experimental design (Table 2) with new parameter levels. Apart from one instance, the agreement between experimental and predicted triplicates is very good with a 6% relative error on average, but the model struggles with predicting accurately larger Z-average values. This, in part, is due to the fact that agglomerates are very heterogeneous in size, and the degree of heterogeneity scales with size. Therefore, the larger the mean aggregate size, the broader the size distribution, and thus, the expectable noise is larger.60 Second, the uncertainty of bath sonication is larger than probe sonication, and the noise in the corresponding data points is larger. This indicates some challenges in the reproducibility of bath sonication, likely due to variations in the experimental conditions, such as water and room temperature, relative humidity, bath volume and the vertical and horizontal position of the sonicated vessel.11,61
The performance of our ML model on the sonication experiment of 80 nm diameter SiO2 particles. The ML model was neither trained nor validated nor tested on these particles beforehand.
| Parameters of sonication | Result | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Sample vol. (mL) | Particle conc. (mg mL−1) | Type | Amplitude (%) | Duration (min) | Energy density (J mL−1) | Predicted Z-ave (nm) | Measured Z-ave (nm) | Predicted PDI | Measured PDI | |
| 1 | 2 | 2 | Bath | 50 | 5 | 258 | 337 ± 67 | 324 ± 61 | 0.38 ± 0.12 | 0.30 ± 0.23 |
| 2 | 7.5 | 2 | Bath | 50 | 30 | 298 | 239 ± 22 | 796 ± 26 | 0.21 ± 0.10 | 0.50 ± 0.13 |
| 3 | 7.5 | 2 | Probe | 20 | 5 | 322 | 124 ± 17 | 141 ± 2 | 0.08 ± 0.04 | 0.10 ± 0.01 |
| 4 | 7.5 | 7.5 | Bath | 50 | 5 | 69 | 325 ± 40 | 331 ± 33 | 0.44 ± 0.25 | 0.31 ± 0.18 |
| 5 | 2 | 7.5 | Probe | 20 | 30 | 3878 | 150 ± 39 | 112 ± 5 | 0.12 ± 0.11 | 0.05 ± 0.01 |
After successfully constructing the ML model and predicting the outcome of ultrasonicating the silica particles, in the second part of this study we apply our ML approach to the meta-analysis of published and peer-reviewed work reported on the sonication and DLS characterization of oxide particles, such as ZnO, CeO2 and TiO2. Following the guidelines of Field and Gillett,62 we compiled a set of 203 data points collected from 12 peer-reviewed articles.12,30,31,43,63–70 Articles relevant to the project were searched online, by using the combinations of the keywords of “ultrasonication”, “nanoparticles”, and “oxide”. Compared to the laboratory study, the number of features (Table 1) could be increased by adding particle properties like zeta-potential in water, surface hydrophobicity/hydrophilicity, and isoelectric point. Owing to larger dataset, the ML model performed better (Fig. 4), while the structure of supervising the machine learning algorithm was very similar to the lab-based study.
Fig. 4. Parity plots of log10Z-average and log10 PDI values of oxide ENPs synthesized, processed and characterized elsewhere (meta-analysis). Data in gray color indicate the progress of ten independent training and testing rounds, and data in turquoise/red color show the best models. We had a total of 383 data points for training and 289 data points for testing, and compared to the lab-based ML analysis, we achieved better performance (R2 = 0.82 and 0.84 for the Z-average and PDI, which correspond to a linear correlation coefficient higher than 0.9). The dashed black lines indicate perfect predictions (R2 = 1). A distribution of the R2 score for the test set of 100 newly randomly seeded and trained models can be found in Fig. SI7.†.
Nevertheless, similar to the lab-based study, the accuracy at large Z-average values and with bath sonicated samples is also decreased. As the final evaluation of the performance of our ML model, we tested two commercially available ENPs synthesized on a large-scale (aeroxide TiO2 P 25 and aerosil SiO2 200, both by Evonik Operations GmbH) following the experimental design detailed in Table 2. The values predicted by the ML model and the values measured by DLS are listed in Table 3.
The performance of our ML model on the sonication of commercialized large-scale produced ENPs. The ML model was nor trained nor validated nor tested on these parameter combinations beforehand.
| Aeroxide (TiO2) | Aerosil (SiO2) | |||||||
|---|---|---|---|---|---|---|---|---|
| Predicted size (nm) | Measured size (nm) | Predicted PDI | Measured PDI | Predicted size (nm) | Measured size (nm) | Predicted PDI | Measured PDI | |
| 1 | 289 ± 14 | 302 ± 15 | 0.40 ± 0.09 | 0.42 ± 0.08 | 501 ± 41 | 476 ± 51 | 0.71 ± 0.34 | 0.64 ± 0.24 |
| 2 | 261 ± 20 | 922 ± 63 | 0.23 ± 0.17 | 0.61 ± 0.17 | 171 ± 53 | 309 ± 39 | 0.24 ± 0.08 | 0.57 ± 0.15 |
| 3 | 371 ± 29 | 352 ± 13 | 0.34 ± 0.10 | 0.38 ± 0.07 | 181 ± 17 | 173 ± 8 | 0.09 ± 0.08 | 0.13 ± 0.02 |
| 4 | 275 ± 36 | 285 ± 51 | 0.38 ± 0.09 | 0.40 ± 0.09 | 339 ± 87 | 306 ± 8 | 0.66 ± 0.22 | 0.57 ± 0.19 |
| 5 | 143 ± 12 | 144 ± 26 | 0.22 ± 0.09 | 0.19 ± 0.07 | 114 ± 17 | 123 ± 2 | 0.29 ± 0.10 | 0.34 ± 0.02 |
Next, we were interested in identifying the relative importance of the individual ultrasonication parameters and particle properties we used as an input on the outcome of the predictions. With this, we were also able to see what parameters are the most decisive—in the eyes of our model—in the process of particle ultrasonication, giving us insights into the underlying physical process of ultrasonication. To obtain these insights, we performed a so-called feature importance analysis (FIA). Feature importance analysis supports strongly the interpretation of ML-prediction.71 FIA, in essence, assigns a score to each feature used in the ML model, based on their relative importance in predicting the label values. The higher the score, the greater the influence of the feature. Hence, if one wants to optimize ultrasonication experiments in the future, it will be time-saving to start tweaking the most influential parameter and then to follow the importance hierarchy. Our FIA is based on the so-called Shapely values.72 Shapley values were invented in the field of cooperative games, and loosely speaking, they are intended to establish a basis of merit-based payoff, by quantifying the marginal contributions of players of a team in a given game and the associated reward.73 In a cooperative game, the success of each player depends not only on what they do, but also on how the players cooperate together. Accordingly, the most useful team-player gets the highest share of the reward. Calculating Shapley values is computational very intensive, and thus, we compute with the method introduced by Lundberg and Lee (SHapley Additive exPlanations, SHAP).74 According to the SHAP values, in our ML model trained on the silica ENPs synthesized and processed in our laboratory, the sonicator type has the highest influence on the deagglomeration success and the obtained PDI (Fig. 5a and b). This most likely reflects the fact that the available range of sonication energy is dependent on the type of sonication. The comparison of SHAP values between the models for PDI and Z-average shows only a marginal difference. In the meta-analysis, it is interesting to see that in the ML model, the isoelectric point, the zeta-potential, and surface coating only show a low influence for the ultrasonication process, their cumulative share is less than 15% (Fig. 5c and d). Apart from the energy-related quantities, the material type and particle size are however important features of the model. Their influence may be interpreted by using colloidal science: the amplitude of van der Waals forces—binding the particle agglomerates—is particle size and particle material dependent,75,76 and thus the model picks up their important role in the binding energy of the particle agglomerates and their influence on cluster deagglomeration.
Fig. 5. Feature importance analysis by normalized Shapley values. The more important the feature in our ML model, the higher the share in the pie chart. Parameters relating to the sonication process are presented in a cold (blue/green) color palette, and parameters relating to particle physicochemical properties are shown in hot (red/yellow) color palette. (a) Analysis of Z-average for model SiO2 NPs synthesized, processed and characterized in our lab. (b) Analysis of PDI for model SiO2 NPs. (c) Analysis of Z-average for oxide NPs synthesized, processed and characterized elsewhere. (d) Analysis of PDI for oxide NPs.
Last but not least, we acknowledge that our study has limits, which point out potential subjects for future studies. First, while DLS is one of the most frequently used method in the characterization of particle dispersions, it is an ensemble technique that is very sensitive to outliers, which may lead to bias, and thus, requires carefully prepared and reproducible samples. Therefore, while DLS has its own merits, other in situ, and perhaps more robust characterization techniques—such as particle tracking analysis, Taylor dispersion analysis, small-angle scattering and diffraction methods—may serve the purpose equally well, if not better. Our choice of experimental characterization technique was due to the widespread use and accessibility of DLS—and therefore the largest number of published data points. Second, our ML model was developed on so-called horn and bath ultrasonicators, but cup horn sonication—also a frequently used device type—is not addressed in this study due to the lack of published data. Third, sonication may benefit from the use of dispersing agents, but we do not address their presence and role here. Fourth, in this study, we concentrated on a given class of ENPs with somewhat similar physicochemical properties, but other highly relevant materials, such as iron oxides, aluminum oxides, quantum dots, carbon nanotubes, or even particle mixtures were not addressed. Additionally, the model might show lower predictability for data points out of the range the model was trained on, e.g. micro sized particles or different particle morphologies like sheets or wires. Fifth, while our analyses are sound, we are able to offer only a hierarchy of importance and degree of association of the features to interpret the prediction of our ML model. Therefore, a detailed mechanistic understanding of the model and a casual inference is missing. To describe in quantitative detail the ML model in terms of cause and effect (by, for example, closed-form analytic algebraic expressions) is beyond our current capacity. Explainable and fully transparent ML is an active field of debate,77–81 which, however, concerns not only us, but any ML models where information content available (data) is limited.
As a final note for outlook, we co-published a web-based application with a graphical user interface (https://sonipredict.herokuapp.com/) where we offer quantitative guidance for designing sonication processes. While the application in its current form is based on the ML approach presented here, with creating a larger data bank, we hope to extend the model to cover increasing number of parameters and experimental scenarios of greater complexity, such as different dispersants and material types and sizes. The ML model can be greatly improved by incorporating new data, as any ML model learns best on data of high quality and of high volume, and we also call to the community to support us and send their own results on ultrasonicated nanoparticle dispersions. To collect more data, we also co-published a new database (https://tineglaubitz-sonidb-app-8z3bkw.streamlitapp.com/) for researchers to send in their data points. With a collective effort we aim at improving ML analyses, and promoting reproducibility in this impactful field.
Abbreviations
- DLS
Dynamic light scattering
- ENP
Engineered nanoparticle
- GBDT
Gradient boosted decision tree
- ML
Machine learning
- PDI
Polydispersity index
Author contributions
CG conceived the idea, synthesized the particles, developed the experimental design, performed and analyzed the experiments, built and trained the ML model. SB supervised CG in project management and data analysis. CG and SB wrote the manuscript through contributions of AF, BRR, and ML. AF, BRR, and ML joint-supervised CG during the experiments.
Conflicts of interest
There are no conflicts to declare.
Supplementary Material
Acknowledgments
The authors are grateful for the support of Dr Patricia Taladriz-Blanco, Laetitia Haeni, Liliane Ackermann Hirschi, Dr Miroslava Nedyalkova and Dr Christoph Geers. The authors are grateful for the financial support of the Adolphe Merkle Foundation, the University of Fribourg, and the Swiss National Science Foundation through the National Centre of Competence in Research Bio-Inspired Materials and the grant SNF no. 200020_184635.
Electronic supplementary information (ESI) available. See DOI: https://doi.org/10.1039/d2nr03240f
References
- Swain B. Park J. R. Park K.-S. Lee C. G. Synthesis of cosmetic grade TiO2-SiO2 core-shell powder from mechanically milled TiO2 nanopowder for commercial mass production. Mater. Sci. Eng., C. 2019;95:95–103. doi: 10.1016/j.msec.2018.10.005. [DOI] [PubMed] [Google Scholar]
- Polat S., Ağçam E., Dündar B. and Akyildiz A., Nanoparticles in Food Packaging: Opportunities and Challenges, in Health and Safety Aspects of Food Processing Technologies, Springer International Publishing, 2019, pp. 577–611 [Google Scholar]
- Raj S. Sumod U. Jose S. Sabitha M. Nanotechnology in cosmetics: Opportunities and challenges. J. Pharm. BioAllied Sci. 2012;4(3):186–193. doi: 10.4103/0975-7406.99016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kessler R. Engineered Nanoparticles in Consumer Products: Understanding a New Ingredient. Environ. Health Perspect. 2011;119(3):A120–A125. doi: 10.1289/ehp.119-a120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dahle J. Arai Y. Environmental Geochemistry of Cerium: Applications and Toxicology of Cerium Oxide Nanoparticles. Int. J. Environ. Res. Public Health. 2015;12(2):1253–1278. doi: 10.3390/ijerph120201253. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Al-Salman F. Ali Redha A. Al-Shaikh H. Hazeem L. Taha S. Toxicity evaluation of TiO2 nanoparticles embedded in toothpaste products. GSC Biol. Pharm. Sci. 2020;12(1):102–115. doi: 10.30574/gscbps.2020.12.1.0205. [DOI] [Google Scholar]
- Gulson B. McCall M. Korsch M. Gomez L. Casey P. Oytam Y. Taylor A. McCulloch M. Trotter J. Kinsley L. Greenoak G. Small Amounts of Zinc from Zinc Oxide Particles in Sunscreens Applied Outdoors Are Absorbed through Human Skin. Toxicol. Sci. 2010;118(1):140–149. doi: 10.1093/toxsci/kfq243. [DOI] [PubMed] [Google Scholar]
- Shrestha S. Wang B. Dutta P. Nanoparticle processing: Understanding and controlling aggregation. Adv. Colloid Interface Sci. 2020;279:102162. doi: 10.1016/j.cis.2020.102162. [DOI] [PubMed] [Google Scholar]
- Feng H., Barbosa-Canovas G. and Weiss J., Ultrasound Technologies for Food and Bioprocessing, Springer, 2011 [Google Scholar]
- Chemat F. Zill-e-Huma Khan M. K. Applications of ultrasound in food technology: Processing, preservation and extraction. Ultrason. Sonochem. 2011;18(4):813–835. doi: 10.1016/j.ultsonch.2010.11.023. [DOI] [PubMed] [Google Scholar]
- Taurozzi J. S. Hackley V. A. Wiesner M. R. Ultrasonic dispersion of nanoparticles for environmental, health and safety assessment–issues and recommendations. Nanotoxicology. 2011;5(4):711–729. doi: 10.3109/17435390.2010.528846. [DOI] [PubMed] [Google Scholar]
- Mandzy N. Grulke E. Druffel T. Breakage of TiO2 agglomerates in electrostatically stabilized aqueous dispersions. Powder Technol. 2005;160(2):121–126. doi: 10.1016/j.powtec.2005.08.020. [DOI] [Google Scholar]
- Laux P. Riebeling C. Booth A. M. Brain J. D. Brunner J. Cerrillo C. Creutzenberg O. Estrela-Lopis I. Gebel T. Johanson G. Jungnickel H. Kock H. Tentschert J. Tlili A. Schäffer A. Sips A. J. A. M. Yokel R. A. Luch A. Challenges in characterizing the environmental fate and effects of carbon nanotubes and inorganic nanomaterials in aquatic systems. Environ. Sci.: Nano. 2018;5(1):48–63. doi: 10.1039/C7EN00594F. [DOI] [Google Scholar]
- Cohen J. M. Beltran-Huarac J. Pyrgiotakis G. Demokritou P. Effective delivery of sonication energy to fast settling and agglomerating nanomaterial suspensions for cellular studies: Implications for stability, particle kinetics, dosimetry and toxicity. NanoImpact. 2018;10:81–86. doi: 10.1016/j.impact.2017.12.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Aoki M. Ring T. A. Haggerty J. S. Analysis and Modeling of the Ultrasonic Dispersion Technique. Adv. Ceram. Mater. 1987;2(3A):209–212. doi: 10.1111/j.1551-2916.1987.tb00082.x. [DOI] [Google Scholar]
- Vasylkiv O. Sakka Y. Synthesis and Colloidal Processing of Zirconia Nanopowder. J. Am. Ceram. Soc. 2001;84(11):2489–2494. doi: 10.1111/j.1151-2916.2001.tb01041.x. [DOI] [Google Scholar]
- Rejman J. Oberle V. Zuhorn I. S. Hoekstra D. Size-dependent internalization of particles via the pathways of clathrin- and caveolae-mediated endocytosis. Biochem. J. 2004;377(1):159–169. doi: 10.1042/bj20031253. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pan Y. Neuss S. Leifert A. Fischler M. Wen F. Simon U. Schmid G. Brandau W. Jahnen-Dechent W. Size-Dependent Cytotoxicity of Gold Nanoparticles. Small. 2007;3(11):1941–1949. doi: 10.1002/smll.200700378. [DOI] [PubMed] [Google Scholar]
- Li S.-D. Huang L. Pharmacokinetics and Biodistribution of Nanoparticles. Mol. Pharm. 2008;5(4):496–504. doi: 10.1021/mp800049w. [DOI] [PubMed] [Google Scholar]
- Darlington T. K. Neigh A. M. Spencer M. T. Nguyen O. T. Oldenburg S. J. Nanoparticle characteristics affecting environmental fate and transport through soil. Environ. Toxicol. Chem. 2009;28(6):1191. doi: 10.1897/08-341.1. [DOI] [PubMed] [Google Scholar]
- Hirsch V. Kinnear C. Rodriguez-Lorenzo L. Monnier C. A. Rothen-Rutishauser B. Balog S. Petri-Fink A. In vitro dosimetry of agglomerates. Nanoscale. 2014;6(13):7325–7331. doi: 10.1039/C4NR00460D. [DOI] [PubMed] [Google Scholar]
- Cohen J. M. Teeguarden J. G. Demokritou P. An integrated approach for the in vitro dosimetry of engineered nanomaterials. Part. Fibre Toxicol. 2014;11(1):20. doi: 10.1186/1743-8977-11-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rodriguez-Lorenzo L. Rothen-Rutishauser B. Petri-Fink A. Balog S. Nanoparticle Polydispersity Can Strongly Affect In Vitro Dose. Part. Part. Syst. Charact. 2015;32(3):321–333. doi: 10.1002/ppsc.201400079. [DOI] [Google Scholar]
- Murugadoss S. Brassinne F. Sebaihi N. Petry J. Cokic S. M. Van Landuyt K. L. Godderis L. Mast J. Lison D. Hoet P. H. van den Brule S. Agglomeration of titanium dioxide nanoparticles increases toxicological responses in vitro and in vivo. Part. Fibre Toxicol. 2020;17(1):10. doi: 10.1186/s12989-020-00341-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Preparing suspensions of nanoscale metal oxides for biological testing, Version 2.3, NanoCare, 2007 [Google Scholar]
- PROSPEcT, Protocol for Nanoparticle Dispersion, 2010 [Google Scholar]
- Preparing suspensions of nanomaterials in serumcontaining medium, Version 2.0, NanoGEM, 2011 [Google Scholar]
- Gilliland D.; , Standardised dispersion protocols for high priority materials groups, NanoDefine Technical Report D2.3: Wageningen, 2016
- Pradhan S. Hedberg J. Blomberg E. Wold S. Wallinder I. O. Effect of sonication on particle dispersion, administered dose and metal release of non-functionalized, non-inert metal nanoparticles. J. Nanopart. Res. 2016;18(9):285. doi: 10.1007/s11051-016-3597-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kaur I. Ellis L.-J. Romer I. Tantra R. Carriere M. Allard S. Mayne-L’Hermite M. Minelli C. Unger W. Potthoff A. Rades S. Valsami-Jones E. Dispersion of Nanomaterials in Aqueous Media: Towards Protocol Optimization. J. Visualized Exp. 2017;130:56074. doi: 10.3791/56074. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cohen J. M. and Demokrito P., Dosimetry for In Vitro Nanotoxicology: Too Complicated to Consider, Too Important to Ignore, in Nanoparticles in the Lung, CRC Press, 2015, pp. 267–293 [Google Scholar]
- Grall R. and Chevillard S., SOP for Dispersion, NANoREG, 2017 [Google Scholar]
- Hellack B. and Nickel C., Dispersion Protocol for Nanoparticle Suspension by Cup Horn Sonication, Version 1.1, nanOxiMet, 2016 [Google Scholar]
- Jensen K. A., The NANOGENOTOX standard operational procedure for preparing batch dispersions for in vitro and in vivo toxicological studies, Version 1.2, NANOGENOTOX, 2018 [Google Scholar]
- ISO 22412:2017, Particle size analysis—Dynamic light scattering (DLS), International Organization for Standardization 2017
- Stöber W. Fink A. Bohn E. Controlled growth of monodisperse silica spheres in the micron size range. J. Colloid Interface Sci. 1968;26(1):62–69. doi: 10.1016/0021-9797(68)90272-5. [DOI] [Google Scholar]
- Pfeiffer C. Rehbock C. Hühn D. Carrillo-Carrion C. de Aberasturi D. J. Merk V. Barcikowski S. Parak W. J. Interaction of colloidal nanoparticles with their local environment: the (ionic) nanoenvironment around nanoparticles is different from bulk and determines the physico-chemical properties of the nanoparticles. J. R. Soc., Interface. 2014;11(96):20130931. doi: 10.1098/rsif.2013.0931. [DOI] [PMC free article] [PubMed] [Google Scholar]
- The appropriateness of existing methodologies to assess the potential risks associated with engineered and adventitious products of nanotechnologies; SCENIHR/002/05 SCIENTIFIC COMMITTEE ON EMERGING AND NEWLY IDENTIFIED HEALTH RISKS (SCENIHR): 2006
- Raj S. Sumod U. Jose S. Sabitha M. Nanotechnology in cosmetics: Opportunities and challenges. J. Pharm. BioAllied Sci. 2012;4(3):186. doi: 10.4103/0975-7406.99016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Surowiec I. Vikström L. Hector G. Johansson E. Vikström C. Trygg J. Generalized Subset Designs in Analytical Chemistry. Anal. Chem. 2017;89(12):6491–6497. doi: 10.1021/acs.analchem.7b00506. [DOI] [PubMed] [Google Scholar]
- Mahbubul I. M. Saidur R. Amalina M. A. Elcioglu E. B. Okutucu-Ozyurt T. Effective ultrasonication process for better colloidal dispersion of nanofluid. Ultrason. Sonochem. 2015;26:361–369. doi: 10.1016/j.ultsonch.2015.01.005. [DOI] [PubMed] [Google Scholar]
- Marín R. R. R. Babick F. Lindner G.-G. Wiemann M. Stintz M. Effects of Sample Preparation on Particle Size Distributions of Different Types of Silica in Suspensions. Nanomaterials. 2008;8(7):454–472. doi: 10.3390/nano8070454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Meißner T. Oelschlägel K. Potthoff A. Dispersion of nanomaterials used in toxicological studies: a comparison of sonication approaches demonstrated on TiO2 P25. J. Nanopart. Res. 2014;16:2228. doi: 10.1007/s11051-013-2228-7. [DOI] [Google Scholar]
- Tantra R. Sikora A. Hartmann N. B. Sintes J. R. Robinson K. N. Comparison of the effects of different protocols on the particle size distribution of TiO2 dispersions. Particuology. 2015;19:35–44. doi: 10.1016/j.partic.2014.03.017. [DOI] [Google Scholar]
- Sjögren R. and Svensson D., https://github.com/clicumu/pyDOE2; , 2018
- Finsy R. Particle sizing by quasi-elastic light scattering. Adv. Colloid Interface Sci. 1994;52:79–143. doi: 10.1016/0001-8686(94)80041-3. [DOI] [Google Scholar]
- Koppel D. E. Analysis of Macromolecular Polydispersity in Intensity Correlation Spectroscopy: The Method of Cumulants. J. Chem. Phys. 1972;57(11):4814–4820. doi: 10.1063/1.1678153. [DOI] [Google Scholar]
- Chen T. and Guestrin C., XGBoost: A Scalable Tree Boosting System, in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining [Internet], ACM, New York, NY, USA, 2016, pp. 785–794 [Google Scholar]
- Friedman J. H. Greedy function approximation: A gradient boosting machine. Ann. Stat. 2001;29(5):1189–1232. [Google Scholar]
- Kim B., Khanna R. and Koyejo O., Examples are not enough, learn to criticize! criticism for interpretability, in Proceedings of the 30th International Conference on Neural Information Processing Systems, Curran Associates Inc., Barcelona, Spain, 2016, pp. 2288–2296 [Google Scholar]
- Jablonka K. M. Ongari D. Moosavi S. M. Smit B. Big-Data Science in Porous Materials: Materials Genomics and Machine Learning. Chem. Rev. 2020;120(16):8066–8129. doi: 10.1021/acs.chemrev.0c00004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jiang J. Wang R. Wang M. Gao K. Nguyen D. D. Wei G.-W. Boosting Tree-Assisted Multitask Deep Learning for Small Scientific Datasets. J. Chem. Inf. Model. 2020;60(3):1235–1244. doi: 10.1021/acs.jcim.9b01184. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Buitinck L., Louppe G., Blondel M., Pedregosa F., Mueller A., Grisel O., Niculae V., Prettenhofer P., Gramfort A., Grobler J., Layton R., Vanderplas J., Joly A., Holt B. and Varoquaux G., API design for machine learning software: experiences from the scikit-learn project 2013, p. arXiv:1309.0238. https://ui.adsabs.harvard.edu/abs/2013arXiv1309.0238B; (accessed September 01, 2013)
- Virtanen P. Gommers R. Oliphant T. E. Haberland M. Reddy T. Cournapeau D. Burovski E. Peterson P. Weckesser W. Bright J. van der Walt S. J. Brett M. Wilson J. Millman K. J. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods. 2020;17:261–272. doi: 10.1038/s41592-019-0686-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harris D. and Harris S., in Digital design and computer architecture, Morgan Kaufmann, San Francisco, California, 2013 [Google Scholar]
- Yeo I.-K. Johnson R. A. A New Family of Power Transformations to Improve Normality or Symmetry. Biometrika. 2000;87(4):954–959. doi: 10.1093/biomet/87.4.954. [DOI] [Google Scholar]
- Virtanen P. Gommers R. Oliphant T. E. Haberland M. Reddy T. Cournapeau D. Burovski E. Peterson P. Weckesser W. Bright J. van der Walt S. J. Brett M. Wilson J. Millman K. J. Mayorov N. Nelson A. R. J. Jones E. Kern R. Larson E. Carey C. J. Polat İ. Feng Y. Moore E. W. VanderPlas J. Laxalde D. Perktold J. Cimrman R. Henriksen I. Quintero E. A. Harris C. R. Archibald A. M. Ribeiro A. H. Pedregosa F. van Mulbregt P. Vijaykumar A. Bardelli A. P. Rothberg A. Hilboll A. Kloeckner A. Scopatz A. Lee A. Rokem A. Woods C. N. Fulton C. Masson C. Häggström C. Fitzgerald C. Nicholson D. A. Hagen D. R. Pasechnik D. V. Olivetti E. Martin E. Wieser E. Silva F. Lenders F. Wilhelm F. Young G. Price G. A. Ingold G.-L. Allen G. E. Lee G. R. Audren H. Probst I. Dietrich J. P. Silterra J. Webber J. T. Slavič J. Nothman J. Buchner J. Kulick J. Schönberger J. L. de Miranda Cardoso J. V. Reimer J. Harrington J. Rodríguez J. L. C. Nunez-Iglesias J. Kuczynski J. Tritz K. Thoma M. Newville M. Kümmerer M. Bolingbroke M. Tartre M. Pak M. Smith N. J. Nowaczyk N. Shebanov N. Pavlyk O. Brodtkorb P. A. Lee P. McGibbon R. T. Feldbauer R. Lewis S. Tygier S. Sievert S. Vigna S. Peterson S. More S. Pudlik T. Oshima T. Pingel T. J. Robitaille T. P. Spura T. Jones T. R. Cera T. Leslie T. Zito T. Krauss T. Upadhyay U. Halchenko Y. O. Vázquez-Baeza Y. SciPy C. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods. 2020;17(3):261–272. doi: 10.1038/s41592-019-0686-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bergstra J. Komer B. Eliasmith C. Yamins D. Cox D. D. Hyperopt: a Python library for model selection and hyperparameter optimization. Comput. Sci. Discovery. 2015;8:014008. doi: 10.1088/1749-4699/8/1/014008. [DOI] [Google Scholar]
- Akiba T., Sano S., Yanase T., Ohta T. and Koyama M., Optuna: A Next-generation Hyperparameter Optimization Framework, in Proceedings of the 25rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019
- Friedlander S. K., Smoke, Dust, and Haze: Fundamentals of Aerosol Dynamics, Oxford University Press, 2nd edn, 2000 [Google Scholar]
- Nascentes C. C. Korn M. Sousa C. S. Arruda M. A. Z. Use of Ultrasonic Baths for Analytical Applications: A New Approach for Optimisation Conditions. J. Braz. Chem. Soc. 2001;2(1):57–63. doi: 10.1590/S0103-50532001000100008. [DOI] [Google Scholar]
- Field A. P. Gillett R. How to do a meta-analysis. Br. J. Math. Stat. Psychol. 2010;63(3):665–694. doi: 10.1348/000711010X502733. [DOI] [PubMed] [Google Scholar]
- Bacon J., Rielly C. and Özcan-Taşkin N. G., in Breakup of Silica Nanoparticle Clusters Using Ultrasonication, Presented at the 16th European Conference on Mixing (Mixing 16), Toulouse, France, Toulouse, France, 9–12 September 2018 [Google Scholar]
- Beltran-Huarac J. Zhang Z. Pyrgiotakis G. DeLoid G. Vaze N. Hussain S. M. Demokritou P. Development of reference metal and metal oxide engineered nanomaterials for nanotoxicology research using high throughput and precision flame spray synthesis approaches. NanoImpact. 2018;10:26–37. doi: 10.1016/j.impact.2017.11.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bihari P. Vippola M. Schultes S. Praetner M. Khandoga A. G. Reichel C. A. Coester C. Tuomi T. Rehberg M. Krombach F. Optimized dispersion of nanoparticles for biological in vitro and in vivo studies. Part. Fibre Toxicol. 2008;5:1–14. doi: 10.1186/1743-8977-5-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cohen J. Deloid G. Pyrgiotakis G. Demokritou P. Interactions of engineered nanomaterials in physiological media and implications for in vitro dosimetry. Nanotoxicology. 2013;4:417–431. doi: 10.3109/17435390.2012.666576. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jensen K. A., Kembouche Y., Christiansen E., Jacobsen N. R., Wallin H., Guiot C., Spalla O. and Witschger O., Towards a method for detectingthe potential genotoxicity of nanomaterials: Final protocol for producing suitable manufactured nanomaterial exposure media, NANOGENOTOX, 2010 [Google Scholar]
- Mahbubul I. M. Elcioglub E. B. Saidur R. Amalin M. A. Optimization of ultrasonication period for better dispersion and stability of TiO2−water nanofluid. Ultrason. Sonochem. 2017;37:360–367. doi: 10.1016/j.ultsonch.2017.01.024. [DOI] [PubMed] [Google Scholar]
- Marín R. Babick F. Stintz M. Ultrasonic dispersion of nanostructured materials with probe sonication−practical aspects of sample preparation. Powder Technol. 2017;318:451–458. doi: 10.1016/j.powtec.2017.05.049. [DOI] [Google Scholar]
- Taurozzi J. S. Hackley V. A. Wiesner M. R. A standardised approach for the dispersion of titanium dioxide nanoparticles in biological medi. Nanotoxicology. 2011;7(4):389–401. doi: 10.3109/17435390.2012.665506. [DOI] [PubMed] [Google Scholar]
- Roscher R. Bohn B. Duarte M. F. Garcke J. Explainable Machine Learning for Scientific Insights and Discoveries. IEEE Access. 2020;8:42200–42216. [Google Scholar]
- Lipovetsky S. Conklin M. Analysis of regression in game theory approach. Appl. Stoch. Models Bus. Ind. 2001;17(4):319–330. doi: 10.1002/asmb.446. [DOI] [Google Scholar]
- Shapley L. S., A Value for n-Person Games, in Contributions to the Theory of Games (AM-28), ed. H. W. Kuhn and A. W. Tucker, Princeton University Press, 1953, vol. II, pp. 307–318 [Google Scholar]
- Lundberg S. M. and Lee S.-I., A unified approach to interpreting model predictions, in Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., Long Beach, California, USA, 2017, pp. 4768–4777 [Google Scholar]
- Chen J. Anandarajah A. van der Waals Attraction between Spherical Particles. J. Colloid Interface Sci. 1996;180(2):519–523. doi: 10.1006/jcis.1996.0332. [DOI] [Google Scholar]
- Hamaker H. C. The London—van der Waals attraction between spherical particles. Physica. 1937;4(10):1058–1072. doi: 10.1016/S0031-8914(37)80203-7. [DOI] [Google Scholar]
- Breiman L. Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) Stat. Sci. 2001;16(3):199–231. [Google Scholar]; , 33
- Cox D. R. [Statistical Modeling: The Two Cultures]: Comment. Stat. Sci. 2001;16(3):216–218. [Google Scholar]
- Bareinboim E. Pearl J. Causal inference and the data-fusion problem. Proc. Natl. Acad. Sci. U. S. A. 2016;113(27):7345–7352. doi: 10.1073/pnas.1510507113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lipton Z. C. The mythos of model interpretability. Commun. ACM. 2018;61(10):36–43. doi: 10.1145/3233231. [DOI] [Google Scholar]
- Rudin C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 2019;1(5):206–215. doi: 10.1038/s42256-019-0048-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.





