Artificial neural networks for the inverse design of nanoparticles with preferential nano-bio behaviors

Abstract

Safe and efficient use of ultrasmall nanoparticles (NPs) in biomedicine requires numerous independent conditions to be met, including colloidal stability, selectivity for proteins and membranes, binding specificity, and low affinity for plasma proteins. The ability of a NP to satisfy one or more of these requirements depends on its physicochemical characteristics, such as size, shape, and surface chemistry. Multiscale and pattern recognition techniques are here integrated to guide the design of NPs with preferential nano-bio behaviors. Data systematically collected from simulations (or experiments, if available) are first used to train one or more artificial neural networks, each optimized for a specific kind of nano–bio interaction; the trained networks are then interconnected in suitable arrays to obtain the NP core morphology and layer composition that best satisfy all the nano–bio interactions underlying more complex behaviors. This reverse engineering approach is illustrated in the case of NP-membrane interactions, using binding modes and affinities and early stage membrane penetrations as training data. Adaptations for designing NPs with preferential nano-protein interactions and for optimizing solution conditions in the test tube are discussed.

I. INTRODUCTION

The ability to rationally design nanoparticles (NPs) that interact with biological structures, such as proteins, membranes, cells, and tissues (referred to as nano–bio interactions), in specific ways would have a significant impact in biomedicine,^1–5 including drug transport and delivery, modulation of enzyme activity, imaging, and photothermal and photodynamic therapy of cancer. Determining the optimal nanocore size, shape, and surface chemistry for specific purposes is challenging due to the extreme complexity of the biological environment. Unlike other areas of nanotechnology, such as electronics, photonics, or catalysis, the successful use of exogenous nanostructures in a living organism requires numerous independent conditions to be satisfied simultaneously to produce the desired outcome and avoid a host of potentially harmful side effects. For example, a nanoparticle (NP) must avoid self-aggregation in biological solutions (e.g., blood serum or interstitial fluids), have low affinity for plasma proteins (e.g., albumin or fibrinogen), escape the surveillance of the immune system (e.g., macrophages), preserve the functionalized surface against the degrading effects of the environment during circulation, target proteins or membranes selectively, and bind with specificity to target sites. Failure to control these processes may lead to suboptimal therapeutic efficacy or undesirable systemic effects, e.g., acute or chronic toxicity,¹ including inflammation,^6,7 production of reactive oxygen species (ROS),⁸ or interference with gene expression or DNA repair mechanisms.⁹ The molecular bases of these processes are not well understood, so progress in designing NPs for diagnostics and therapeutics has been slow and by trial and error. A more rational and systematic approach is desirable, especially in emerging areas of treatment, such as theranostics and precision medicine.

The aim of the method proposed here is to help guide the design of nanostructures with preferential nano-bio behaviors. This is a realistic goal in the ultrasmall size regime (i.e., NPs smaller than ∼5 nm in at least one dimension): unlike their larger counterparts (typically, ∼30 nm to 60 nm across), ultrasmall NPs are protein mimics,¹⁰ with sizes comparable to that of an average globular protein (∼50 kDa), and interact with biological structures as proteins do,¹¹ with distinct binding modes, affinities, and kinetics. These NPs could, in principle, be engineered to interact with proteins in vitro in specific ways. For example, ultrasmall NPs have recently been designed to target specific sites of enzymes to modulate their activities,⁴ mimicking the effect of a drug-like compound, or change their protein binding kinetics from a reaction-limited to a diffusion-limited mechanism,¹² an important step in the design of biopharmaceuticals.

The method introduced here combines multiscale modeling with a pattern-recognition technique based on artificial neural networks. The multiscaling approach^13,14 describes the essential physics of the system and enables the rapid generation of training data in the absence of (or complementary to) experimental data. The use of neural nets for pattern recognition is well established,¹⁵ versatile enough to be adapted to different scenarios and architectures, and can incorporate increasing levels of complexity into the NP design through network arrays (cf. Sec. III). Unlike other machine learning methods more common for sound and image recognition,¹⁶ these features make neural nets particularly well suited for biophysical problems, in general, and biomedical applications, in particular. Neural networks have been used recently to design multi-layer nanophotonic devices¹⁷ and graphene-based metamaterials¹⁸ with specific electromagnetic responses. These applications rely on single-output deep nets and make use of different networks for training and design. By contrast, the method proposed here uses a single, shallow multi-output net to predict physically distinct properties. Networks optimized for unrelated properties can then be interconnected to find the design parameters that best satisfy multiple conditions imposed on the NP behavior. Therefore, the inherently complex problem posed by the biological environment is reduced to several smaller, independent problems that are simpler to handle and understand; the overall complexity is left to be dealt with by the network array in the most optimal way possible.

To illustrate, the method is applied to ultrasmall NPs interacting with membranes. Potential adaptations for reverse-engineering the thermodynamic conditions of the aqueous solution for in vitro experiments and for designing NPs with preferential nano-protein interactions are discussed based on simulations of NPs in the presence of plasma proteins.

II. METHODS

The method developed for the systematic study of nano–bio interactions, which will be used throughout this paper, was reported previously.¹⁴ It combines a self-adaptive multiscaling algorithm for efficient simulations of proteins in crowded conditions¹³ with a practical prescription for modeling nanostructures of arbitrary designs. This combined approach is summarized in Sec. II A (technical details of the algorithm and a discussion of scope and performance are given in Refs. 13 and 14). The physical basis of the network architecture and the equations that relate the output to input data are presented in Secs. II B and II C, respectively; training and performance of the network are discussed in Sec. II D, whereas the equations used in the inverse design algorithm are presented in Sec. II E.

A. Nanoparticles design and multiscale simulations

A nanoparticle consists of a metallic (crystalline) core covered by a layer of organic molecules attached to the core¹⁴ by covalent or coordinate bonds¹⁹ (e.g., gold-thiol in the case of AuNPs). The core morphology is defined by a flattened elliptical torus (cf. Fig. 1, left panel) geometrically characterized by a set of parameters (d), which can be changed continuously to represent different shapes and sizes.¹⁴ The outer boundary of the coating layer is defined by a surface parallel to that of the core and contains the functional groups that determine the layer chemical properties. The thickness of the layer and the size and surface density of the functional groups, all of which are known to affect the NP behavior, are characterized by a second set of parameters (l), which can also be changed continuously to represent different layer compositions (Fig. 1, left). This modeling protocol, which enables arbitrary nanostructures to be designed through a few (intuitive) parameters, allows for systematic high-throughput screening of NPs and is ideally suited for reverse engineering, in general.

FIG. 1.

Definitions of input/output units in the network depicted in Fig. 2 (cf. details in Fig. 3), as used in practice. Input variables: The NP core d is determined by seven parameters that characterize the geometry of a flattened elliptical torus and enable continuous transitions between shapes and sizes (A, B, a, b, and τ given in nm). The coating layer l is determined by seven additional parameters corresponding to the physical features known to control the nano-bio interactions, including width and surface charge distribution (δ and σ are given in nm, s in nm⁻², and q as a multiple of the unit charge e); the prolate ellipsoidal nanocore shown in the figure is obtained from parameters midway between a nanosphere and a nanorod. The membrane m is defined in analogy to l; in the application in Sec. III, however, none of these quantities are optimized, so m is used only to select a membrane, likewise, for the solution conditions c, which are fixed in all the applications and not shown (cf. Sec. II B). In general, the inverse design approach can optimize a lipid type as well, e.g., in the design of synthetic membranes or the solution conditions in vitro. Output variables: The output units are given by the following arrays: ε (binding affinity of a NP for a membrane), with dimensionless elements 0 or 1 depending on the bin in which the binding energy falls (e.g., [0, 0, 0, 1] for high binders or [1, 0, 0, 0] for weak binders); ρ (binding mode, or angles at which the NP binds a membrane), with dimensionless elements in the [0, 1] range (e.g., all elements are equal to 1/18 for non-specific binding modes, i.e., all angles with the same probability, which is most common for nanospheres, or all the elements equal to 0 except one equal to 1 for a highly specific mode of association at this angle); and μ (membrane penetration, or depth at which the NP first penetrates the membrane; see the discussion on the physical and statistical meaning of this quantity in Ref. 14; see also Sec. IV), with dimensionless elements 0 or 1 depending on the bin in which the deepest NP falls, as measured perpendicular to the membrane (z direction) and with respect to some reference, here the outer boundary of the layer, as calculated from molecular dynamics simulations (e.g., [1, 0, 0, 0] for a NP entering deep into the membrane, pass the glycerol group, or [0, 0, 0, 1] for NP that barely perturbs the membrane structure, a common situation for larger nanospheres). The bins are defined within the space spanned by the glycerol group, the phosphate group (both common to all the phospholipids), and the lipid head groups (unique to each lipid).

In the multiscaling method,¹³ a molecule in an aqueous solution interacts with another molecule at a level of resolution that depends on their relative spatial configurations. In practice, each molecule p (a protein, nanoparticle, lipid, or any other species but water, which is treated as a continuum) in a system of $\mathcal{M}$ molecules is assigned a set of $\mathcal{M}$ − 1 scaling parameters λ, which determine the spatial resolutions of p with respect to each of the other molecules and depend on the spatial configuration of the system. In a simulation, the configuration changes continually, so the value of each λ moves back and forth between a minimum and a maximum to adapt to the new state in a self-guided, continuous process. A fully atomic representation is only needed when two molecules come in close contact, e.g., when a NP binds to a membrane or protein. A significant speedup in the computation of intermolecular forces can thus be achieved while still retaining the relevant physics of the interactions. Such computational efficiency is critical for the investigation of ultrasmall nano–bio interactions in real biological media because of the need to consider many NPs and macromolecules in crowded conditions. This is a challenge that has precluded systematic studies and data collection for training purposes. The multiscaling algorithm becomes relatively more efficient as the molecular crowding increases, so the method is particularly apt for nano-bio applications. The structural coarsening is incorporated into the force field as a smooth, continuous function of the atomic coordinates only and has continuous first derivatives everywhere in the configurational space, so the energy is conserved across scales. Moreover, because the a priori probability distribution is homogeneous and is kept fixed throughout a Monte Carlo simulation, detailed balance is preserved.

B. Neural network architecture

The method described in Sec. II A was recently used to study the behavior of ultrasmall nanostructures in the presence of several phospholipid membranes.¹⁴ The data collected therein will be used in Sec. II D to train the network, whereas the insight gained from the analysis of the interactions suggests the optimal network architecture. Three basic properties relevant for biomedical applications were analyzed:¹⁴ the NP-membrane binding modes and affinities, and the NP penetration in the early stage of association, i.e., before the NP and the membrane undergo structural adaptation (cf. discussion in Sec. IV). These properties were shown to depend critically on the nanocore shape and size, as well as on the layer thickness and charge distribution.¹⁴ It was argued that this sensitivity could be exploited to design NPs that interact with membranes in specific ways (e.g., to bind preferentially to one membrane over another, or to disrupt the structural integrity of some membranes while stabilizing others). As shown in this paper, this goal can be accomplished, in practice, with the multi-output neural net schematically depicted in Fig. 2 and described in detail in this section.

FIG. 2.

Schematic representation of the neural network proposed for the inverse design of nanostructures with specific nano-bio interactions (cf. Fig. 3 for details). The input layer is fully connected to a single hidden layer divided into three nodes, each containing n_i neurons, which are only partially connected to the output nodes. The two lateral paths incorporate physical information into the weights (see text). The input units p and e characterize the particle (nano-) and the environment (-bio); each one is divided into two nodes: d and l determine the NP core morphology and the coating layer (cf. Fig. 1); m and c select the type of membrane and the solution conditions. The network may be improved by including additional hidden layers (deep nets) or convolution along the deeper paths.

For a given core morphology d and coating layer l (together comprising the “nano” side of the nano–bio system) in the presence of a membrane m under solution conditions c (both defining the “bio” component), the three quantities of interest are the binding energies ε, the binding modes ρ, and the membrane penetrations μ.¹⁴ To limit redundancy, the network is only partially connected, whereas the lateral (cross) paths from ρ to the hidden layers of nodes ε and μ are intended to incorporate physical information into the weights. This is important for the inverse-design protocol proposed here, which uses the same optimized network to obtain input from the output. The introduction of these two lateral paths is motivated by the observation¹⁴ that the binding modes (ρ) ultimately determine the ability of a NP to disrupt/stabilize the membrane structure (μ) and are related to the strength of the NP-membrane interactions (ε). Therefore, enforcing a correlation with ρ (or, in general, introducing as much physicochemical information as possible into the network design) is expected to be advantageous when predicting properties. For example, a nanoplate that binds parallel to a membrane—information embedded in the binding mode characterized by ρ—is known a priori to be unable to reach deep into the membrane—information in μ; the lateral paths aim to enforce these physical correlations.

The relatively simple architecture proposed reduces dimension while increasing predictive power in the presence of limited training data (cf. Sec. II D), a common occurrence in experimental areas, especially at the nanoscale, where systematic data collection is not always feasible. As shown in the results (Sec. III) and in a series of cross-validation tests (Sec. II D), the network in Fig. 2 is remarkably robust and efficient for the intended application. Other nano–bio systems, however, e.g., NPs interacting with proteins, may require different architectures (cf. discussion in Sec. IV); the ability to mix multiple, independently optimized networks in a hybrid array is a salient feature of the grid-based approach proposed here (cf. Sec. III).

1. Input nodes

In Fig. 2, each input unit $\mathit{x}=\left[x_1,x_2,\dots ,x_{n_{\mathit{x}}}\right]$ contains n_x neurons and is fully connected to the second (hidden) layer. These nodes are defined by the column arrays (details in Fig. 1) d = [A, B, a, b, τ, υ, h], l = [δ, σ₊, σ₋, q₊, q₋, s₊, s₋], $\mathit{m}=\left[{\delta }^{\prime },{\sigma }_{+}^{\prime },{\sigma }_{-}^{\prime },q_{+}^{\prime },q_{-}^{\prime },s_{+}^{\prime },s_{-}^{\prime }\right]$, and c = [c₊, c₋, pH] (transpose signs omitted). In this paper, m and c are used only for selecting the (bio) environment of the NPs and are kept fixed during the inverse design optimization; thus, the input space is reduced, in practice, to (at most) the 14 variables in d and l. A full description of all the variables is nonetheless given here for completeness. The components d_i’s determine the morphology of the nanocore, represented by the parameters of the torus¹⁴ (cf. Sec. II A and Fig. 1): A, B, a, and b determine the shape and size of the main tube; τ, υ, and h = ±1 are the helix pitch, number of turns, and handedness, respectively (for non-helical NPs, these three parameters are set to zero); the core material plays a minor role in nano–bio interactions because NPs are typically coated, except bare magnetite, and is thus not part of the input data; l_i’s describe the coating layer (Fig. 1): δ is the thickness, and σ_±, q_±, and s_± are the radii, charges, and surface density of the chemical groups; m_i’s describe the basic features of a membrane in analogy to the features of the layer (cf. discussion in Sec. IV and Fig. 1 for the four types of phospholipid bilayers considered in this study): δ′ is the thickness of the hydrophilic head group, estimated here as the average distance between the phosphorus atom and the membrane outer boundary;¹⁴ all the other quantities are defined in analogy to l for the NP; information on the solution conditions are contained in c_i’s and specified by the concentrations c_± of cations and anions and by the pH of the solution (this characterization is a simplification, especially for highly charged solutes like NPs or other polyelectrolytes, since different solution species are known to play different roles in mediating interactions, either directly or indirectly, near charged interfaces^20,21); in addition, P and T may need to be included.²²

2. Output nodes

Each output node $\mathit{a}=\left[a_1,a_2,\dots ,a_{n_{\mathit{a}}}\right]$ contains n_a neurons and can be defined¹⁴ as the column arrays (details in Fig. 1, right panel) ε = [ε₁, ε₂, …, ε_N], $\mathbf{\rho }=\left[{\rho }_1,{\rho }_2,\dots ,{\rho }_{N^{\prime }}\right]$, and $\mathbf{\mu }=\left[{\mu }_1,{\mu }_2,\dots ,{\mu }_{N^{″}}\right]$. Each of these arrays is normalized to one: ε_n is 1 if the binding energy E falls in the interval (n − 1)ξ < E ≤ nξ and 0 otherwise (particles with E > Nξ are grouped in the last bin as strong membrane binders); ρ_n is the (average) value of the probability distribution ρ(ϕ) of the binding angle ϕ in the interval (n − 1)ν < ϕ ≤ nν; and μ_n is 1 if the maximum penetration z of the NP into the membrane falls in the interval (n − 1)ζ < z ≤ nζ and 0 otherwise (particles with z > N″ζ are grouped in the last bin as strong membrane-structure disruptors; z is measured with respect to a common reference, here the outer boundary of the membrane¹⁴). Error estimates¹⁴ and account of current limitations of the force field to model interactions between highly charged interfaces^14,20,21,23 suggest that N = 4 with ξ = 20 kcal/mol is a reasonable compromise for both zwitterionic and charged lipids; calculations also suggest¹⁴ that N′ = 18 with ν = 5° and N″ = 4 with ζ = 1 Å provide a good characterization of the binding modes and membrane penetrations. Changes to these parameters would only change the dimensions of the input units (features space) and have no bearing on the proof of principle or the conceptual basis of the method.

C. Feedforward and backpropagation equations

The network in Fig. 3 is shallow, except along the deeper lateral paths, and uses sigmoidal activations (f) in all the nodes except the output layer, where softmax (s) is used for normalization purposes. This mixture is common practice and simplifies implementation. In compact notation, the output data are obtained from the input data through the matrix equations $\mathbf{\epsilon }=s\left\{\hat{\mathbf{W}}f\left(\hat{\mathbf{Z}}\mathbf{x}+\hat{\mathbf{U}}\mathrm{\rho }\right)\right\}$, $\mathbf{\mu }=s\left\{{\hat{\mathbf{W}}}^{\prime }f\left({\hat{\mathbf{Z}}}^{\prime }\mathbf{x}+{\hat{\mathbf{U}}}^{\prime }\mathrm{\rho }\right)\right\}$, and $\mathbf{\rho }=s\left\{{\hat{\mathbf{W}}}^{″}f\left({\hat{\mathbf{Z}}}^{″}\mathbf{x}\right)\right\}$ (ρ computed first), where the column array x contains information from the input layer, and the definitions of the matrices $\hat{\mathbf{Z}}$, $\hat{\mathbf{W}}$, and $\hat{\mathbf{U}}$, and their single- and double-primed versions, follow immediately from Fig. 3 (all transpose signs omitted). Explicit functional forms of these equations and those used in the backpropagation algorithm are presented below, following standard derivation.¹⁵

FIG. 3.

Detailed representation of the network in Fig. 2, showing explicitly all the quantities used to derive the equations for the feedforward [Eqs. (1)–(5)], backpropagation [Eqs. (6)–(17)], and inverse-design [Eqs. (18)–(20)] implementations.

Each unit, indicated by a circle in Fig. 3, in layer i is composed of n_i neurons; the weight matrices connecting the different units are given by ${\hat{\mathbf{u}}}_j$, ${\hat{\mathbf{v}}}_j$, and ${\hat{\mathbf{w}}}_j$ and the biases by the vectors b_j (and by their primed versions). Because the input layer is fully connected to the hidden layer, it can be grouped into different units as deemed more practical. Here, the four input units in Fig. 2, which represent the nano–bio system in all its conceptual parts, are divided into two components: the particle (nano) p = (d, l) and the biological environment (bio) e = (m, c). The feedforward equations are then given explicitly by

$$\begin{array}{ccc}\hfill \mathbf{\epsilon }& =\hfill & s\left({\hat{\mathbf{w}}}_{3}f\left({\hat{\mathbf{v}}}_2\mathbf{p}+{\hat{\mathbf{w}}}_2\mathbf{e}+{\hat{\mathbf{u}}}_2\mathbf{\rho }+{\mathit{b}}_2\right)+{\mathit{b}}_3\right)\hfill \\ \hfill & =\hfill & s\left\{\hat{\mathbf{W}}f\left(\hat{\mathbf{Z}}\mathbf{x}+\hat{\mathbf{U}}\mathbf{\rho }\right)\right\},\hfill \end{array}$$ (1)

$$\begin{array}{ccc}\hfill \mathbf{\mu }& =\hfill & s\left({{\hat{\mathbf{w}}}^{\prime }}_{3}f\left({{\hat{\mathbf{v}}}^{\prime }}_2\mathbf{p}+{{\hat{\mathbf{w}}}^{\prime }}_2\mathbf{e}+{{\hat{\mathbf{u}}}^{\prime }}_2\mathbf{\rho }+{\mathit{b}}_2^{\prime }\right)+{\mathit{b}}_3^{\prime }\right)\hfill \\ \hfill & =\hfill & s\left\{{\hat{\mathbf{W}}}^{\prime }f\left({\hat{\mathbf{Z}}}^{\prime }\mathbf{x}+{\hat{\mathbf{U}}}^{\prime }\mathbf{\rho }\right)\right\},\hfill \end{array}$$ (2)

$$\begin{array}{ccc}\hfill \mathbf{\rho }& =\hfill & s\left({{\hat{\mathbf{w}}}^{″}}_{2}f\left({{\hat{\mathbf{v}}}^{″}}_2\mathbf{p}+{{\hat{\mathbf{w}}}^{″}}_2\mathbf{e}+{\mathit{b}}_2^{″}\right)+{\mathit{b}}_3^{″}\right)\hfill \\ \hfill & =\hfill & s\left\{{\hat{\mathbf{W}}}^{″}f\left({\hat{\mathbf{Z}}}^{″}\mathbf{x}\right)\right\},\hfill \end{array}$$ (3)

which define the vectors x ≡ (p, e) and the matrices $\hat{\mathbf{U}}$, $\hat{\mathbf{Z}}$, and $\hat{\mathbf{W}}$ (the latter two containing implicitly the biases b₂ and b₃, respectively), as well as the vectors z₂ and z₃ in Fig. 3; f and s are the sigmoidal and softmax activation functions, respectively, given by

$$1/f\left(z\right)=1+\text{exp}\left(-z\right),$$ (4)

$$s\left(z_i\right)=\text{exp}\left(z_i\right)/{\sum }_{j=1}^n\text{exp}\left(z_j\right),$$ (5)

where z is one of the components of z₂ in the hidden layer and z_i is one of the components of z₃ in an output unit containing n neurons.

To derive the equations for the backpropagation algorithm, let u_j,pq, v_j,pq, and w_j,pq be the elements (p, q) of the weight matrices ${\hat{\mathbf{u}}}_j$, ${\hat{\mathbf{v}}}_j$, and ${\hat{\mathbf{w}}}_j$ and b_j,p and z_j,p be the elements p of the bias vectors b_j and z_j. For the component I of the output unit ρ, the explicit equations needed for the calculation of the gradient of the cost are given by

$$\partial {\rho }_I/\partial w_{3,pq}^{″}=s_p^{\prime }\left(z_{3,I}^{″}\right)f\left(z_{2,q}^{″}\right),$$ (6)

$$\partial {\rho }_I/\partial v_{2,pq}^{″}=p_q{f}^{\prime }\left(z_{2,p}^{″}\right){\sum }_{k=1}^{n_3^{″}}{s}_I^{\prime }\left(z_{3,k}^{″}\right)w_{3,kp}^{″},$$ (7)

with $\partial {\rho }_I/\partial w_{2,pq}^{″}$ as in Eq. (7) after replacing p_q (component q of input p) by e_q (component q of input e), where f′(z) ≡ df/dz = f(z)[1 − f(z)] and $s_a^{\prime }\left(z_b\right)\equiv \left(\partial s/\partial z_a\right){|}_{z_b}=s\left(z_b\right)\left[{\delta }_{ab}-s\left(z_b\right)\right]$, and δ is the Kronecker delta. Likewise, for the component I of the output unit ε, the backpropagation equations are given explicitly by

$$\partial {\epsilon }_I/\partial w_{3,pq}=s_p^{\prime }\left(z_{3,I}\right)f\left(z_{2,q}\right),$$ (8)

$$\partial {\epsilon }_I/\partial v_{2,pq}=p_q{f}^{\prime }\left(z_{2,p}\right){\sum }_{k=1}^{n_3}{s}_I^{\prime }\left(z_{3,k}\right)w_{3,kp},$$ (9)

$$\begin{array}{ccc}\hfill \partial {\epsilon }_I/\partial w_{3,pq}^{″}& =\hfill & f\left(z_{2,q}^{″}\right){\sum }_{k=1}^{n_3}{\sum }_{l=1}^{n_2}{\sum }_{m=1}^{n_3^{″}}{s}_I^{\prime }\left(z_{3,k}\right)\hfill \\ \hfill & \hfill & \times w_{3,kl}{f}^{\prime }\left(z_{2,l}\right)u_{2,lm}{s}_p^{\prime }\left(z_{3,m}^{″}\right),\hfill \end{array}$$ (10)

$$\begin{array}{ccc}\hfill \partial {\epsilon }_I/\partial v_{2,pq}^{″}& =\hfill & p_q{f}^{\prime }\left(z_{2,p}^{″}\right){\sum }_{k=1}^{n_3}{\sum }_{l=1}^{n_2}{\sum }_{m=1}^{n_3^{″}}{\sum }_{n=1}^{n_3^{″}}\hfill \\ \hfill & \hfill & \times s_I^{\prime }\left(z_{3,k}\right)w_{3,kl}{f}^{\prime }\left(z_{2,l}\right)u_{2,lm}{s}_m^{\prime }\left(z_{3,n}^{″}\right)w_{3,np}^{″},\hfill \end{array}$$ (11)

with $\partial {\epsilon }_I/\partial w_{2,pq}$ and $\partial {\epsilon }_I/\partial u_{2,pq}$ as in Eq. (9) after changing p_q for e_q and by $s\left(z_{3,q}^{″}\right)$, respectively, and $\partial {\epsilon }_I/\partial v_{2,pq}^{″}$ as in Eq. (11) with e_q instead of p_q. [The innermost sums over m in Eq. (10) and n in Eq. (11) stem from the softmax function in ρ, which mixes all the neurons of a given output unit.]

For the component I of μ, the corresponding equations follow immediately from Eqs. (8)–(11) using the symmetry of the network architecture. The corresponding derivatives with respect to the bias components b_j,p follow from Eqs. (6)–(11), i.e.,

$$\partial {\rho }_I/\partial b_{3,p}^{″}=\left(\partial {\rho }_I/\partial w_{3,pq}^{″}\right)/f\left(z_{2,q}^{″}\right),$$ (12)

$$\partial {\rho }_I/\partial b_{2,p}^{″}=\left(\partial {\rho }_I/\partial v_{2,pq}^{″}\right)/p_q,$$ (13)

$$\partial {\epsilon }_I/\partial b_{3,p}=\left(\partial {\epsilon }_I/\partial w_{3,pq}\right)/f\left(z_{2,q}\right),$$ (14)

$$\partial {\epsilon }_I/\partial b_{2,p}=\left(\partial {\epsilon }_I/\partial v_{2,pq}\right)/p_q,$$ (15)

$$\partial {\epsilon }_I/\partial b_{3,p}^{″}=\left(\partial {\epsilon }_I/\partial w_{3,pq}^{″}\right)/f\left(z_{2,q}^{″}\right),$$ (16)

$$\partial {\epsilon }_I/\partial b_{2,p}^{″}=\left(\partial {\epsilon }_I/\partial v_{2,pq}^{″}\right)/p_q,$$ (17)

and the corresponding derivatives of μ_I follow from the symmetry of the network.

D. Network training and cross-validation tests

Ideally, to be of practical use to experimentalists, the network should be trained with experimental data. However, no systematic collection of data is presently available for ultrasmall NPs interacting with membranes, so results from computer simulations are used here as a substitute. This theoretical approach is enough to assess the network’s predictive performance and does not impinge on the validity of the method as a whole (cf. further discussion on Sec. IV). A total of 128 NP/membrane combinations were previously studied¹⁴ using the approach described in Sec. II A (cf. data in the SI of Ref. 14). The structures span a spectrum of shapes and sizes (nano-spheres, nano-plates, nano-rings, nano-rods, nano-tubes, and nano-helices, with linear dimensions covering a broad range within the ultrasmall domain), core materials (gold and magnetite), and layer compositions (L-glutathione and p-mercaptobenzoic acid); four phospholipid bilayers were considered: charged POPA and POPG, and zwitterionic POPC and POPE (cf. the supplementary material for details).

Training of the network follows standard procedures: optimization is carried out by stochastic gradient descent, using an adaptive mini-batch ranging in size from 10 (early stages of training) to 30 (late-stage) data points; these and other hyperparameters (e.g., the learning rate) were determined after a series of preliminary tests. The total cost J is the sum of the partial cross-entropy losses, J = J_ε + J_ρ + J_μ, with $J_{\mathit{a}}={\mathcal{N}}^{-1}{\sum }_{i=1}^{\mathcal{N}}{j}_{\mathit{a},i}$, where $\mathcal{N}$ is the number of training data i and $j_{\mathit{a},i}=-{\sum }_{k=1}^{n_{\mathit{a}}}\left\{a_{ik}^0\text{\hspace{0.17em}ln\hspace{0.17em}}{a}_{ik}+\left(1-a_{ik}^0\right)\text{ln}\left(1-a_{ik}\right)\right\}$, where $a_{ik}^0$ and a_ik are the components k of the target and actual values of the output node a for data i; L2 regularization is used. The optimal number of neurons in the hidden layer was selected after systematic hold-out cross-validation and underfitting/overfitting assessment (cf. supplementary material), resulting in n₁ = n₃ = 10 and n₂ = 20 in Fig. 2. The performance of the optimized network is remarkably good, with an average of about 80% of output data predicted correctly (cf. Fig. S1 and discussion in the supplementary material). This success rate is a validation of the proposed architecture. However, given the limited size of the training set and the seeming complexity of the problem, it may also indicate that the dependence of the output on the NP design parameters may be simpler than assumed and suggests that an analytical (empirical) model that relates the output and input data, e.g., a linear combination or quadratic form of the input variables with optimized coefficients, or a simpler machine learning approach for multiclass classification might be conceived.

E. Inverse design equations

In this paper, “inverse design” specifically means finding input data that are most compatible with the desired output data. Once the network is trained [i.e., once the weights and biases in Eqs. (1)–(3) are fully optimized, as described in Sec. II D, using the available training data], the input variables x ≡ (p, e) can be obtained for a given set of output variables a ≡ (ε, ρ, μ) using gradient descent while keeping the parameters of the network fixed; technically, the input nodes in Fig. 2 now become the output nodes and vise versa. (Simulated annealing Monte Carlo is an alternate optimization approach not discussed here.) In this optimization process, the NP morphology and layer composition, as well as the properties of the bio-environment, are modeled as continuous variables. Many combinations of input data x may be compatible with a given output a, so repeated runs may be needed to find experimentally feasible designs (see below and results in Sec. III). In practice, each inverse-design run can take from a few seconds to a few minutes of central processing unit (CPU) time, depending on the demands imposed on the NP behavior (cf. Sec. III) or, more generally, on the complexity of the network array (cf. Fig. 4).

FIG. 4.

Three network arrays used for the inverse design of NPs (thick black lines: specifications; thick gray lines: parameters to be optimized; and thin lines: input or output not part of the specifications; nodes c are the same for all and not shown). In these examples, all the networks are identical and optimized for a single type of nano-bio interactions, but hybrid arrays with nets optimized for different nano-bio interactions can incorporate increasing levels of complexity in the NP behavior. The same method can be used to optimize the solution conditions (input node c) or to design a synthetic membrane.^12,22,34

Using sigmoidal activations across the entire net, it is straightforward to show that the derivatives with respect to the component i of the input unit p are given explicitly by (cf. Fig. 3)

$$\partial {\rho }_I/\partial p_i=f^{\prime }\left(z_{3,I}^{″}\right){\sum }_{k=1}^{n_2^{″}}{w}_{3,Ik}^{″}{f}^{\prime }\left(z_{2,k}^{″}\right)v_{2,ki}^{″},$$ (18)

$$\begin{array}{ccc}\hfill \partial {\epsilon }_I/\partial p_i& =\hfill & f^{\prime }\left(z_{3,I}\right){\sum }_{k=1}^{n_2}{w}_{3,Ik}{f}^{\prime }\left(z_{2,k}\right)v_{2,ki}+f^{\prime }\left(z_{3,I}\right){\sum }_{k=1}^{n_2}{\sum }_{l=1}^{n_3^{″}}{\sum }_{m=1}^{n_2^{″}}\hfill \\ \hfill & \hfill & \times w_{3,Ik}{f}^{\prime }\left(z_{2,k}\right)u_{2,kl}{f}^{\prime }\left(z_{3,l}^{″}\right)w_{3,lm}^{″}{f}^{\prime }\left(z_{2,m}^{″}\right)v_{2,mi}^{″},\hfill \end{array}$$ (19)

$$\begin{array}{ccc}\hfill \partial {\mu }_I/\partial p_i& =\hfill & f^{\prime }\left(z_{3,I}^{\prime }\right){\sum }_{k=1}^{n_2^{\prime }}{w}_{3,Ik}^{\prime }{f}^{\prime }\left(z_{2,k}^{\prime }\right)v_{2,ki}^{\prime }\hfill \\ \hfill & \hfill & +f^{\prime }\left(z_{3,I}^{\prime }\right){\sum }_{k=1}^{n_2^{\prime }}{\sum }_{l=1}^{n_3^{″}}{\sum }_{m=1}^{n_2^{″}}{w}_{3,Ik}^{\prime }{f}^{\prime }\left(z_{2,k}^{\prime }\right)u_{2,kl}^{\prime }\hfill \\ \hfill & \hfill & \times f^{\prime }\left(z_{3,l}^{″}\right)w_{3,lm}^{″}{f}^{\prime }\left(z_{2,m}^{″}\right)v_{2,mi}^{″},\hfill \end{array}$$ (20)

whereas the derivatives with respect to the component i of the input unit e follow from Eqs. (18)–(20) using the symmetry of the network. The first term (single sum) on the rhs in each of Eqs. (19) and (20) stem from the shallow pathways, while the second terms (triple sums) follow the deeper paths via ρ.

The partial loss used for optimization is a conventional quadratic form, namely, $h_{\mathit{a}}={\left(2\mathcal{N}\right)}^{-1}{\sum }_{i=1}^{\mathcal{N}}{\sum }_{k=1}^{n_{\mathit{a}}}{\left(a_{i,k}^0-a_k\right)}^2$, where $a_{i,k}^0$ is the components k of the value imposed on the output node a by condition i and a_k is its current value; the sum in k runs only over the neurons on which the condition i applies. Because the input nodes have physical meaning, the optimization is performed within a subspace x′ ⊂ x determined by a set of external potentials V_k(x_k) ≡ V_k of the form V_k = 0 if x_m,k < x_k < x_M,k, V_k = k(x_k − x_m,k)² if x_k < x_m,k, and V_k = k(x_k − x_M,k)² if x_k > x_M,k, where k, x_m,k, and x_M,k are user-defined parameters. In practice, the main goal of these restraints is to restrict the search to the ultrasmall regime, i.e., the size domain within which the network was trained; this can be accomplished, in practice, by restraining all linear dimensions within ∼5 nm, as done in this paper. However, these external potentials play a more prominent role when designing specific surface chemistries or morphologies (e.g., to restrict the design to a nanosphere or a nanotube or to favor a prolate over an oblate spheroid nanocore), as illustrated in some of the examples in Sec. III. Thus, the total cost J is the sum of h_a over the relevant output nodes and external potentials, i.e., J = ${\sum }_{\mathit{a}}{h}_{\mathit{a}}+{\sum }_{\mathit{x}}{\sum }_{k=1}^{n_{\mathit{x}}}{V}_k$, where, in practice, the sum on x is akin to a regularization term. The update of x_k in the gradient descent is thus x_k → x_k − η∂J/∂x_k; for ε and μ, ∂h_a/∂x_k has contributions from two neural paths, namely, the shallow one and the deeper lateral connections.

III. RESULTS

In this section, the method is applied to three (arbitrary) systems that display increasing levels of complexity in their behavior. Figure 4 shows three scenarios that illustrate how arrays of networks can be interconnected to satisfy multiple conditions imposed on a NP. In these examples, all the networks are identical and optimized, as described in Sec. II D. However, the modular paradigm proposed here can incorporate different network architectures optimized for different kinds of nano–bio interactions (cf. discussion in Sec. IV), including nets trained with computational or experimental data and collected by different laboratories or techniques.

In any given application, there is generally no guarantee that a suitable combination of parameters can be found to be compatible with all the requested conditions, which is only an aspirational goal for the NP behavior. This is still valuable information to avoid futile design efforts and adjust the expectations. Thus, the value of the cost J (cf. Sec. II E) is only a measure of how well a predicted structure conforms to the (ideal) set of prescriptions; unlike the cost used for training (cf. Sec. II D), the value of J in the inverse design optimization should be regarded with caution as the only measure of fitness, as discussed in the examples below. The simplest case is depicted in Fig. 4(a), which instructs a single net to find the shape and size of a non-helical ultrasmall AuNP covered with L-glutathione that binds POPA at a specified angle. These conditions on the binding modes are generally more difficult to satisfy. The only input node to be optimized is d = [A, B, a, b, 0, 0, 0] since all the others are fixed; the only output node relevant for the calculation of the cost is ρ = [0, …, 1, …, 0], with 1 in the angle of interest, here 45°. The lowest cost (J ≈ 0.1) is found for a nanorod [Fig. 5(a), left panel], which binds with the expected tilt, although some non-specific modes with more planar configurations are also predicted [Fig. 5(a), right panel, red line]; this structure happens to have low affinity for the membrane and relatively high penetrating power. The second-best design (J ≈ 0.3) is a nanoring [Fig. 5(b), left] although it can adopt planar and normal configurations as well [Fig. 5(a), right, black]; this structure also has a low binding affinity and modest membrane penetration. Other designs [Fig. 5(b), middle and right] are more appropriate for the other membranes although they all show low degrees of binding specificity [Fig. 5(a), right, blue, and green]. These results make good physical sense in light of the results discussed in Ref. 14 and are consistent with the information incorporated into the network during training: it can be seen from the distributions ρ vs ϕ in Ref. 14 that sharply tilted binding modes usually require a wedge (high curvature in at least one dimension) on the NP surface and multiple stabilizing interactions with the membrane, conditions satisfied by all the predicted structures.

FIG. 5.

Reverse-engineered ultrasmall NPs from the arrays in Fig. 4: (a) GSH-covered gold nanorod with optimized design for binding POPA at an angle of ϕ = 45° using the single net of Fig. 4(a) [functional groups are shown in green; binding modes distribution ρ(ϕ) indicated by the red line]. (b) Left: the second-best design obtained from the same network is a fully coated circular nanoring (black line). Middle: for POPG, the best design is a flattened ellipsoidal nanoring (blue line; GSH layer not shown). Right: for POPC, the best design is a narrow nanoplate (green line; GSH layer not shown). For POPE, no designs were found with binding modes centered near the desired angle. (c) Magnetite NP of a triaxial ellipsoidal shape found with the two-net array of Fig. 4(b). (d) Left: Circular nanorod covered with a mixed layer found with the three-net array of Fig. 4(c). The functional groups are shown in red (–CO₂⁻) and blue (–NH₃⁺), both assumed to be charged at pH ∼7; the layer thickness and coverage density are consistent with diethylene glycol. Right: corresponding binding-mode distribution.

Figure 4(b) shows an array of two nets for the design of a non-helical Fe₃O₄–AuNP that has a stronger preference for POPA (m₁) over POPC (m₂) and modest penetration for both. The input node to be optimized is d = [A, B, a, b, 0, 0, 0], whereas the relevant output nodes are ε₁ = [0, 0, 0, 1] and ε₂ = [1, 0, 0, 0] (or any other combinations that reflect the required condition on the relative affinity), and μ₁ = [1, 0, 0, 0] and μ₂ = [1, 0, 0, 0]. Figure 5(c) shows a typical design with the lowest cost (J ≈ 0.05). The shape can be described as a triaxial ellipsoid; other optimal designs are found to be variations of this morphology and are all predicted to bind with the major axis parallel to the membrane, hence their limited disruptive capability. Here, too, the results are consistent with the training set: large curvatures tend to adopt a planar configuration, which prevents deep penetration into the membranes.

A more complicated set of conditions is shown in Fig. 4(c), in which an array of three nets is used to find the core size of a gold nanorod and the coverage density²⁴ of a mixed layer composed of HS-PEG-NH₂ and HS-PEG-COOH short chains (low PEG M_w) so that, at neutral pH, the NP binds perpendicularly to POPA (m₁), has a high affinity for POPC (m₃), and penetrates deeper into POPA than into POPE (m₂). In this case, 2d = [L − d, 0, d, d, 0, 0, 0] and l = [δ, 2.0, 2.2, +1, −1, s₊, s₋], where L is the rod length and d is the diameter of its (circular) cross-section; σ_± (in Å) are estimated from the volumes of the –NH₃⁺ and –CO₂⁻ groups, which are the same used for GSH. The required outputs are, e.g., ρ₁ = [1, 0, …, 0], μ₁ = [0, 0, 0, 1] and μ₂ = [1, 0, 0, 0] (or any other suitable combination of the relative penetrations), and ε₃ = [0, 0, 0, 1]. The optimal design [Fig. 4(d), left panel] only partially meets all the conditions (J ≈ 0.7) as the NP binds to POPC with lower affinity than expected (ε₃ ≈ [0.5, 0.25, 0.25, 0]); the other conditions are satisfied, though, including the relative membrane penetration and the binding angle [Fig. 5(d), right], which has a sharp distribution centered at 25°, close to the ideal value (for the other two membranes, the distribution has a peak around 90°; not shown). At the predicted coverage density (s₊ ∼3.2 nm⁻² and s₋ ∼5 nm⁻², which are within the expected values²⁵ for PEG-AuNPs) and layer thickness (∼5 Å), the spacers are most consistent with PEG2.

IV. DISCUSSION AND CONCLUSIONS

The use of synthetic nanostructures for diagnosis and therapy requires multiple independent conditions to be satisfied at once, both for efficacy and safety.^26,27 Besides well-known concerns (e.g., low affinity for plasma proteins, limited self-aggregation, preservation of the functionalized surfaces, and efficient accumulation in target cells), ultrasmall NPs are protein mimics that may be designed to display drug-like behavior;^4,12 thus, selectivity for a target protein or membrane and specificity for binding sites must also be considered.

Automation in the design and synthesis of NPs with desired nano-bio behaviors would help overcome the intuition-driven, trial-and-error approach common in the field. The method introduced in this paper, which integrates multiscale and pattern recognition techniques, aims to accomplish this goal. The proposed network architecture reduces redundancy and incorporates physical information through its connectivity, a combination that enhances efficiency and predictive power. Simplicity in the architecture is desirable because of the potential complexity of arrays of networks trained for different types of nano–bio interactions. Unlike other systems for which there are no apparent laws that connect input and output (e.g., voice, digits, or image recognition), the underlying law in a physical system should facilitate training and prediction if the network architecture takes advantage of it. The fact that a few hundred data points may be sufficient to predict the nano-membrane behavior indicates that the proposed network may be recognizing the operating law quite effectively. This suggests that an analytical equation, e.g., a linear or quadratic form in the design parameters with coefficients optimized with a simpler ML technique, may go a long way toward predicting nano–bio interactions. Such an empirical law would be of great aid to bioengineers and experimentalists, in general.

Arguably, to be of immediate practical use, the network should be trained mostly with experimental data. Systematic collection of data for NP/membrane systems is not presently available, though, so results from simulations were used here as a surrogate. This theoretical approach was enough to assess the robustness of the method and its potential to produce physically realistic designs. None of these conclusions are expected to change if experimental data were used instead, although this assertion still needs to be corroborated. To collect an experimental training set similar in size and scope to that used here, synthetic layers, artificial cell membranes, or micelles can be used for systematic studies in vitro.²⁸ The multiscale method would still be needed to supplement this dataset, e.g., to explore regions in the NP morphology not accessible experimentally. Enveloped viruses are a practical real-life alternative, both for raining and testing, given their simplicity when compared to plasma membranes in living cells, which are composed of a mixture of lipids arranged in a mosaic-like structure containing bound and embedded proteins.^29,30 Once trained, however, the method can be used to design ultrasmall NPs that do interact with real cells: unlike their larger counterparts, which interact with extended areas of “unclean” membrane surfaces, ultrasmall NPs can bind directly to pristine regions of the membrane, so lipid specificity is still an important consideration.¹⁴

The reverse-engineering approach proposed here is in line with recent developments in other areas of NP design in material science.^17,18 The use of a pattern recognition technique makes sense if there is a pattern to be recognized in the first place. This is indeed the case for ultrasmall NPs interacting with membranes. There seems to exist a difference in the general behavior of NPs (e.g., as measured by ρ) in the presence of charged or neutral membranes (cf. supplementary material in Ref. 14). In addition, binding modes, affinities, and membrane penetration depend on features that are present on both the NP and the membrane before binding,^14,31 including match/mismatch of atomic interactions and surface complementarity between the NP and the crevices formed on the structure of the membrane. If direct electrostatic interactions are favorable (match), larger contact areas between the NP and the membrane would be preferred, leading, e.g., to planar binding modes; if less favorable (mismatch), the contact areas would be minimized, leading, e.g., to tilted configurations. Even if direct electrostatics favored extensive contacts, the unfavorable electrostatic self-energy would prevent desolvation of polar groups, explaining the difference in behavior between, say, two nanoplates of different diameters or a plate and a ring of the same diameter. These competing effects depend on the local curvature of the NP core and the composition of the coating layer—features captured in d and l—and on the local topography of the membrane and regularity of the lipid arrangement, which depend on the charge, size, and packing of the groups in the lipid hydrophilic heads—features captured in m. The composition of the bulk and interfacial liquids modulate these effects through non-trivial mechanisms, here represented simply by c. The same approach used to reverse engineer d and l can be used to optimize either m or c, e.g., to design artificial membranes or calibrate solution conditions for in vitro nano-bio experiments.

Preliminary data suggest that nano-protein interactions may be amenable to the same treatment than nano-membrane interactions, as long as the size of the protein is comparable to (or larger than) that of the NP, a condition satisfied in the ultrasmall size domain. A set of simulations that parallel those conducted for NPs in the presence of membranes¹⁴ were performed here for NPs in the presence of three plasma proteins: albumin, globulin, and fibrinogen (cf. the supplementary material for details of the simulations setup). These are among the largest and most abundant proteins a NP would encounter upon entering the bloodstream and are known to be part of the soft and hard protein coronas of large NPs.³² An analysis of the simulations suggests a pattern of interactions similar to those in membranes: the small crevices that open transiently on the membrane surface, which are essential to accommodate and stabilize locally a NP, are replaced by larger (nanometer-scale) concave areas that are part of the stable protein structure (cf. Fig. 6). The importance of surface complementarity is more readily apparent in NP-protein than in protein–protein associations, possibly due to the regularity in the distribution of charges on the NP surface, making it more likely for the NP to find favorable contacts. Match/mismatch of interactions and competition between direct electrostatics and self-energies still play similar roles than they do in membranes and ultimately determine the final modes of binding.

FIG. 6.

Complexes of ultrasmall NPs with two plasma proteins obtained from multiscale canonical Monte Carlo simulations^13,14 using the CHARMM force field³⁵ (version c42; simulations setup in the supplementary material). (a) Albumin bound to nanospheres of two nanocore sizes (left: bare magnetite; right: GSH-AuNP with functional groups colored gray). (b) Fibrinogen bound to magnetite NPs (upper panel: spheres and rings; middle: tubes and rods; lower: plates and helices). Simulations in the presence of globulins show similar trends (to be reported).

The above discussion suggests that synthesizing NPs with a mismatch of interactions or surface complementarity may hinder or even prevent NP-protein association; by contrast, matching both could enhance selectivity or specificity. These are conditions that may need to be imposed on a NP design through the reverse engineering protocol described in this paper, albeit with different network architectures. Between these extremes, fine control of affinity may also be necessary: simulations have shown²² that the morphology and cohesiveness of NP/albumin aggregates can be modulated through the NP design and solution conditions. This is of clinical significance because the dissolution of porous, hybrid aggregates can be controlled, and the bioactive coating thus protected from degradation or detachment,³³ increasing efficiency while mitigating toxicity. For example, some metallic NPs are known to cause oxidative stress, and evidence suggests that the ROS generated by gold NPs^8,9 may result in protein carbonylation or lipid peroxidation. Although not yet fully understood, these undesirable effects may stem from exposure of the nanocores to the biological medium, despite their chemical inertness and assumed non-cytotoxic nature. Designing NPs that can withstand the rigors of the environment by promoting the formation of small, labile aggregates that shield and protect the NP layer may be another element to consider in the inverse design.

SUPPLEMENTARY MATERIAL

See the supplementary material for the results from the network validation tests, training, and size optimization, and description of the NP/protein simulation setup.

DATA AVAILABILITY

The data that support the findings of this study are available within the article and its supplementary material, and from Ref. 14 and its supplementary material therein. Computational codes (prototypes) and input/output data files used for training are available from the author upon reasonable request.