Predictive modelling of drop ejection from damped, dampened wings by machine learning

Abstract

The high frequency, low amplitude wing motion that mosquitoes employ to dry their wings inspires the study of drop release from millimetric, forced cantilevers. Our mimicking system, a 10-mm polytetrafluoroethylene cantilever driven through ±1 mm base amplitude at 85 Hz, displaces drops via three principal ejection modes: normal-to-cantilever ejection, sliding and pinch-off. The selection of system variables such as cantilever stiffness, drop location, drop size and wetting properties modulates the appearance of a particular ejection mode. However, the large number of system features complicate the prediction of modal occurrence, and the transition between complete and partial liquid removal. In this study, we build two predictive models based on ensemble learning that predict the ejection mode, a classification problem, and minimum inertial force required to eject a drop from the cantilever, a regression problem. For ejection mode prediction, we achieve an accuracy of 85% using a bagging classifier. For inertial force prediction, the lowest root mean squared error achieved is 0.037 using an ensemble learning regression model. Results also show that ejection time and cantilever wetting properties are the dominant features for predicting both ejection mode and the minimum inertial force required to eject a drop.

1. Introduction

Translational and rotational aerodynamics explain wing motion precision and sensitivity to change in morphology [1–11]. Flying insects face many different environmental challenges, contending with rain, fog, dew and other airborne particles which can deposit additional mass onto the wing surface [12–15]. The accumulated drops across the insect body can be many times its mass, inhibiting or preventing flight [13,16]. To overcome this threat, mosquitoes apply a modified wingbeat during flight startup, the flutter stroke [13]. The flutter stroke persists for approximately 4 ms at nearly double the in-flight wingbeat frequency, during which the wingtip deflection is around 10% of a normal stroke, imparting high accelerations in excess of 2400 gravities to particles adhered to wings. This simple yet effective technique inspires further investigation of systems where the dynamics of the substrate and fluid are coupled, pushing the paradigm of liquid drop motion control to a greater level of complexity.

Drop motion control efforts have primarily aimed to displace drops linearly [17], merge multiple drops [18,19] or establish internal flow [20]. In the simplest motions, however, complicating nonlinear drop deformation arises which is not fully understood for all substrate perturbations. Previous studies have investigated drop motion by moving contact lines [21–27] and pinch-off [28,29] using ultrasound- [30,31] and vibration-induced motion [32–35]. The transition between pinned motion and contact line movement occurs when the deviation of contact angle exceeds the contact angle hysteresis for both horizontally [36] and vertically [33] vibrated sessile drops. If contact angle hysteresis is sufficiently large, drops which are otherwise stationary on a vertical surface can be migrated against gravity with surface vibration [37]. Vertical perturbations of horizontal substrates induce axisymmetric, linear drop oscillations and pinned contact lines at low amplitudes [33]. At high vibration amplitudes contact lines exhibit stick–slip motion and fluid oscillations are nonlinear, with nodes developing on the surface. The critical Bond number by which sliding, or contact line depinning, occurs under gravity can change with the history of inertial force [38]. At ultrasonic vibration frequencies, the drop can experience internal flow that deforms the free surface and unpins the contact line [30], or shatters the drop altogether [39,40].

The aforementioned studies consider drop motion by one-dimensional substrate displacement. The two-dimensional drop–cantilever system studied here was first introduced to investigate the impact dynamics of drops on hydrophobic laminar leaves [41]. Environmental contaminants and seasonal changes to leaves which lower the water-repellency of foliage increase the torque exerted to the leaf impacted by a raindrop. Inspired by insects, we study the drop–cantilever system from the opposing perspective—drop removal by the cantilever [42]. Drop ejection from a beam occurs via three different modes [42]: normal-to-substrate ejection, sliding, and pinch-off. Photo sequences of drop deformation and time to complete release are shown in figure 1a–c. Normal-to-substrate ejection (electronic supplementary material, movie S1) occurs when beam accelerations from small deflections produce receding contact lines. Inertial force must overcome the drop adhesion without drops failing cohesively. Sliding (electronic supplementary material, movie S2) occurs for relatively larger deflections allowing tangential inertial force to overcome contact angle hysteresis. In pinch-off (electronic supplementary material, movie S3), inertial force generates contact line motion but the drop ultimately fails in cohesion, leaving a small child droplet attached to the substrate. Three ejection modes are highly correlated with the selection of cantilever wetting properties, vibration characteristics, and drop location and size. Theoretical prediction of ejection modes [42] with a given set of input variables is not a closed problem and complicated by drops’ time-dependent deformation, or sloshing, which is neither periodic nor axisymmetric [43–46]. Closing the problem would require solving coupled Navier–Stokes and elastica equations in time and space. At best, we can use semi-empirical relations [42] to predict the transition between complete and partial liquid removal for a fixed set of system properties. The number of system features including drop and cantilever properties, and drop location exacerbates the need for a data-driven approach. Data-driven predictive modelling of ejection modes is achievable by only a partial characterization of the ejection mode parameter space [29,47,48].

Figure 1.

Photo sequences of drop ejection via (a) normal, (b) sliding and (c) pinch-off. (Online version in colour.)

Machine learning techniques have recently received significant attention in fluid mechanics because of their ability to handle copious amounts of experimental and numerical data [49] in sub-disciplines such as shape optimization [50,51] and particle image velocimetry [52–54]. The relation between satellite droplet formation and three-dimensional printing variables has been predicted using artificial neural networks for drop-on-demand bioprinting [55]. Wavelength analysis by the maximum entropy method can be used to predict the drop size distribution in primary breakup of liquid jets [56]. Data-driven approaches in splash demonstration significantly improve the visual monitoring of the splash detail [57]. Ensemble learning, which combines different machine learning techniques, has been successfully deployed to characterize interfacial kinetics [58] and droplet formation in ink-jet-based bioprinting [59].

In this study, we employ ensemble learning to tackle the classification problem of predicting drop ejection modes from a forced cantilever beam with base amplitude ±1 mm and frequency of 85 Hz across a range of surface wettability, drop position, and drop size. Another set of base learners is used to perform regression analysis to predict the related inertial forces. We thus characterize drop ejection across a greater range of system variable values than we have explored experimentally. Experimental and theoretical considerations for our drop–cantilever system can be found in a previous work [42]. We begin with a description of our experimental methods in §2a. The selection of base learners to perform ensemble classification and regression is described in §2b. The predictive results from our algorithm for the classification of ejection mode are presented in §3a. We present the prediction results of minimum inertial force required to eject a drop in §3b and provide concluding remarks in §4.

2. Drop ejection experiments

(a). Experimental methods

More detailed experimental methods are provided in Alam et al. [42]; we thus provide a brief description here. Polytetrafluoroethylene cantilevers with elastic modulus E = 480 MPa are cut to shape with a Universal Laser Systems ILS12.150D laser cutter. Finished, mounted cantilevers measure 10 × 3.6 × 0.05 mm. We affix cantilevers to the post of a K2007E01 electrodynamic shaker with cyanoacrylate adhesive. LabVIEW interfaces with a Keysight 33210A signal generator to drive the shaker at a constant frequency of 85 Hz via a NI9263 ± 10 V 4-Channel C series module and a bus-powered compactDAQ cDAQ9171 USB chassis. A schematic of the drop release experimental set-up is shown in figure 2. To increase cantilever hydrophobicity, we use NeverWet^® hydrophobic spray and mechanically remove the coating to generate a wide range of wetting properties. Fluid properties are augmented by mixing glycerin and water, 1:1 by volume. Drops are placed in different locations on the cantilever surface using a 10 ml syringe. We film the cantilever–drop motion with Photron UX-100 and AX-200 high-speed cameras fitted with Nikon 105 mm lenses at 8000 fps. Amscope LED-50W and 30W gooseneck lights provide localized illumination for filming. We extract the drop size, drop location and cantilever position using Tracker, an open source image analysis software.

Figure 2.

Schematic of experimental set-up. (Online version in colour.)

(b). Experimental results and theoretical considerations

Drop release experiments are performed using a horizontal 10-mm cantilever and a singular drop of water or a 1:1 water–glycerin solution by volume resting on the upward-facing surface. To produce a particular ejection mode, we vary drop location x₀, drop size R, liquid viscosity μ and contact angle θ_e while noting ejection type and measuring ejection time t. We plot the distribution of the three principal ejection modes by pairwise system variables in figure 3. Ejection modes appear as clusters in the physical variable space. The diagonal in figure 3 shows univariate distribution of the data for the variable in the respective column. The surface tension σ of water (72.9 dyn cm⁻¹) and 1:1 water–glycerin solution by volume (67.5 dyn cm⁻¹) are within 7% and density within 2% of one another. Thus, we do not include surface tension or density as variables. The viscosity of the glycerin solution, however, is μ = 6.13 ± 0.05 cP (N = 3), or approximately 7 times that of water. Data corresponding to μ in figure 3 appear in rows and stacks because only two discrete viscosity values are put on test. These plots, particularly the rightmost histogram, likewise demonstrate sliding is more rarely witnessed for pure water.

Figure 3.

Pair plot of ejection modes. Plots along the diagonal are histograms. (Online version in colour.)

Drops experience inertial force imposed by cantilever deflection, the quantification of which is challenging because of nonlinearity [60,61]. The addition of periodic base motion makes the problem more complicated by the addition of a non-homogeneous term. Recently, this problem was solved by combining modified Hamilton’s principle [62], Euler–Bernoulli kinematic assumptions and elastica theory [61]. With the assumed mode technique, cantilever motion is described by a nonlinear equation of motion [42], similar to a forced mass–spring–damper system:

$$M\ddot{\delta }+C\dot{\delta }+K_{\text{L}}\delta +K_{\text{NL}}{\delta }^3=-M_0{\ddot{\delta }}_0(t),$$ 2.1

where δ(x, t) represents the amplitude of the assumed mode and δ₀ is the maximum vertical displacement of the cantilever base. Constants M and C are the equivalent mass and damping coefficients, respectively, while K_L and K_NL are the linear and nonlinear stiffness terms, respectively. M₀ is a mass term combining the base (rigid) and deformable motions of the cantilever–drop system. Solving equation (2.1) with boundary condition w_am′(0) = 0 and initial condition $w(x,0)=\dot{w}(x,0)=0$, where w is the transverse deflection of the beam, provides the position vector of any point along a two-dimensional cantilever at any time. Derivatives with respect to x and t are denoted with ( )′ and $(\hspace{0.17em}\dot{}\hspace{0.17em})$, respectively. Inertial force is

$$F_i=m_{\text{d}}\ddot{\mathbf{\text{r}}},$$ 2.2

where m_d is the drop mass calculated by assuming the drop has a spherical curvature above the cantilever and r is the position vector of drop’s centre of mass. Here, r can be calculated as

$$\mathbf{\text{r}}(t)=[x_{\text{d}}-\delta \sin w^{\prime }(x_0,t)]\hat{i}+[y_{\text{d}}+\delta \cos w^{\prime }(x_0,t)]\hat{j},$$ 2.3

where x_d and y_d are the cantilever–drop interface location at any time t. To eject a drop, this inertial force must overcome liquid–solid adhesion defined as [42]

$$F_{\text{ad}}=kR\sigma (\cos {\theta }_{\text{r}}-\cos {\theta }_{\text{a}}),$$ 2.4

where R is the spherical drop radius, σ is the surface tension and θ_a and θ_r are the drop’s advancing and receding contact angles respectively. For normal ejection, there is no advancing contact angle. Therefore, θ_a is functionally 180^° in equation (2.4). The factor k is a mode-dependent experimental factor and the force inequality for each ejection mode is schematized in figure 4a. For normal and sliding ejection, both cohesion force (F_coh) and inertial force (F_i) overcome adhesion force (F_ad). In pinch-off, F_i overcomes F_coh but adhesion keeps some portion of the liquid attached to the cantilever. Cohesion force must be overcome to split the drop into multiple entities and a force for which there is no closed-form representation due to the unpredictable nature of drop shape during sloshing. Kymographs for each ejection mode show temporal drop deformation and cantilever amplitude pre- and post-ejection in figure 4b.

Figure 4.

(a) Schematics and (b) kymographs of drop release modes. (Online version in colour.)

Experimental data shown in figure 3 are fed to machine learning algorithms to build the classification and regression models. The selection of appropriate learning algorithms is critical to the performance of any classification or regression problem [63–65]. Each ensemble learning algorithm is comprised of four base learners. For classification, the ensemble consists of random forests (RFs), gradient boosting machine (GBM), k-nearest neighbour (KNN) and support vector classification (SVC). For regression, the ensemble consists of RFs, GBM, ridge regression (RR) and support vector regression (SVR). The selection of base learners [63] is guided by data distribution (figure 3). RF and GBM are advantageous when data have a high degree of nonlinearity and complex relationships between variables [66]. Small data size and small number of features motivate the use of KNN and support vector machine (SVM) [67]. All the physical system features influence inertial force, and the relatively few we have identified motivates the use of RR, an L2 regularization [68] that ensures no elimination of features. More details about the base learners are provided in the electronic supplementary material.

Ensemble learning is typically more effective compared to individual base learners by lowering error and cross-fitting [69] when individual base learners are selected properly [70]. To combine multiple machine learning algorithms, we use bagging classification which is a combination of bootstrapping and aggregating. In bagging, base learners are selected to predict the class. For each base learner, we provide a sample dataset to all the base learners, re-sampling the training set each time for each base learner using row sampling with a replacement technique [71]. All the base learners are trained on a particular subset of the total dataset, the method of bootstrapping. The predictions from base learners are aggregated by obtaining a majority vote via application of a meta-classifier. A computational scheme of the bagging algorithm is shown in electronic supplementary material, figure S1. Bagging classifiers reduce overfitting compared to single independent models [71]. Sample size is also an important factor to consider in bagging, but the additional data points may not improve accuracy [72]. Our approach in bagging classification is to use small subsamples (starting with 10% of the total dataset) and obtain the accuracy from the meta-classifier. Our algorithm accepts the result from the meta-classifier if the accuracy is higher than the individual algorithms. If the accuracy is lower than the individual model, a re-sampling process is carried out with larger subsamples.

We apply the non-negative least-squares (NNLS) method to combine all the base learners to make the ensemble. NNLS was first introduced by Lawson & Hanson [73], formulated as

$$\begin{array}{rl}& \text{min }f(\alpha )=\frac{1}{2}\parallel G\alpha -\beta {\parallel }^2,\\ & \text{subject to}\hspace{0.17em}\alpha \ge 0,\end{array}$$ 2.5

where α is a weight vector, G is the training set such that G ∈ R^m×n with m features and n number of observations, and β is the unknown sample. The weight vector is distributed among the individual learning algorithms. Predicted values Pr from each base learner are multiplied by a respective weight in the regression to predict the actual outcome. The procedure of predicting outcomes using ensemble learning is schematized in electronic supplementary material, figure S2. The final predictions from the ensemble learner are made by multiplying each base learner’s prediction by its respective weight vector and summing values,

$$\text{Pr}(\text{ensemble})={\alpha }_{\text{RF}}\text{Pr}(\text{RF})+{\alpha }_{\text{GBM}}\text{Pr}(\text{GBM})+{\alpha }_{\text{RR}}\text{Pr}(\text{RR})+{\alpha }_{\text{SVR}}\text{Pr}(\text{SVR}).$$ 2.6

3. Results and discussion

(a). Ensemble prediction of drop ejection modes

Applying the grid search method we find the optimum parameters for each base learner shown in electronic supplementary material, figure S3. We employ a bagging classifier to the dataset and see that it provides slightly better accuracy than the individual learners. Using 10-fold cross validation, we evaluate the accuracy of the learning algorithms and obtain an accuracy of 85% which is only slightly higher than any of the individual classifiers (table 1). We extend our analysis on ensemble learning for classification by analysing the area under the curve (AUC) and the receiver operating characteristics (ROC) curve. To estimate the performance of a classification model, ROC–AUC is an effective classification matrix. ROC is a simple way to summarize the classification accuracy where a large number of confusion matrices are required to summarize the classification accuracy. The ROC curve summarizes all confusion matrices and the AUC represents separability, determining how the algorithm effectively differentiates between the classes. AUC–ROC curves of the ejection modes using bagging classification for our study are shown in figure 5. Higher values of the AUC represent higher model performance for differentiating ejection modes. A model ensures separability if the AUC has a value close to unity. If the ROC passes close to the random guess line, the algorithm is unable to separate different classes. Accuracy, precision, recall, and F-scores for all classification learners are given in table 1.

Table 1.

Performance of models in prediction of drop ejection modes.

algorithm	accuracy	precision	recall	F-score
RFs	0.83	0.84	0.80	0.82
GBM	0.78	0.78	0.75	0.76
KNN	0.83	0.82	0.79	0.80
SVC	0.80	0.79	0.79	0.79
ensemble	0.85	0.84	0.82	0.83

Figure 5.

AUC–ROC curve of the bagging classifier. (Online version in colour.)

The significance of system variables on ejection mode occurrence is computed using permutation feature importance [74], a method of measuring relative importance scores, independent of base learners. Feature importance is measured by the decrease of model accuracy obtained by shuffling a feature’s value. Feature importance scores are plotted in figure 6. For both classification of ejection modes (figure 6a) and predicting ejection force through regression (figure 6b), t and θ_e have higher importance scores than μ, x₀ and R. From the experiments, we witness that if θ_e ≤ 136^°, drops are more likely to eject via the pinch-off mode [42]. Contact line motion begins rapidly, ≈6 ms from base motion inception, when drops are close to the cantilever tip as the inertial force at the tip rises quickly. Drops closer to the base require more time to eject. Feature importance scores likewise reveal role of viscosity, which is a larger factor in determining the type of ejection than in the inertial force at ejection. F_ad is independent of μ (equation (2.4)), but does indirectly influence cohesive failure. Greater viscosity inhibits the drop elongation that creates favourable conditions for pinch-off, and thus greater viscosity promotes sliding from cantilevers undergoing large displacement.

Figure 6.

Variable importance scores for the predictive model of (a) drop ejection modes and (b) minimum inertial force required to eject a drop. (Online version in colour.)

(b). Ensemble prediction of minimum inertial force required for drop ejection

Similar to the classification problem, before combining base learners into an ensemble, we determine the optimum parameters for each base learner using the grid search method shown in electronic supplementary material, figure S4. We train the predictive models to predict the minimum inertial force required to eject a drop and use 10-fold cross validation to validate the performance of the models. The predicted versus actual inertial force for each base learner is plotted in figure 7. RF and GBM outperform the others with low RMSE values, as shown in table 2. The ensemble improves RMSE by 11.9% compared to the best base learner, GBM.

Figure 7.

Observed versus predicted inertial force for (a) RF, (b) GBM (c) RR and (d) SVR algorithm. (Online version in colour.)

Table 2.

Performance of the predictive model of minimum inertial force required to eject a drop.

algorithm	α	RMSE
RFs	0.3707	0.045
GBM	0.4051	0.042
RR	0.2059	0.070
SVR	0.0183	0.058
ensemble	—	0.037

The weight assigned to each individual base learner is calculated by solving the NNLS problem described in §2b and given in table 2. To visualize the performance of ensemble-based learning, we use regression error characteristic (REC) curves with the individual learning algorithm simultaneously shown in figure 8a. RECs plot the accuracy of the regression model with respect to the absolute deviation. The relative position of the REC curve reveals which model is superior compared to others. Models that reach 100% accuracy with a smaller value of error tolerance provide better predictions. In other words, models with higher area under the curve have less error. Predicted inertial force versus the actual force generated from the ensemble learning model is shown in figure 8b.

Figure 8.

(a) REC curve to visualize the performance of regression models and (b) observed versus predicted inertial force for ensemble learning. (Online version in colour.)

We plot the predicted inertial force F_i required to eject a drop from the cantilever surface versus equilibrium contact angle θ_e in figure 9. Globally, larger drops require greater F_i to eject. In the complete liquid removal zone, the inertial force developed by the cantilever must overcome F_ad (equation (2.4)). However, in the partial liquid removal zone, drops need to overcome both F_ad, to get the contact line moving, and an as yet unquantified cohesive force F_coh to perform a pinch-off ejection. Partial liquid removal or pinch-off is associated with lower θ_e and higher F_i. In the ensemble model, we fix drop position to the mean value of $\overline{x_{\text{0}}}=7.2\hspace{0.17em}\text{mm}$ and vary drop size R to produce the inertial force predictions of figure 9a,b. We rationalize the relation between F_i and θ_e in the partial liquid removal zone by noting that lower θ_e values promote flatter drops with centres of mass closer to the substrate, increasing the inertial force required to deform the drop into a shape that can pinch-off. Such deformation necessitates a relatively long time history to slosh to an elongated shape which can eject, a notion supported by figure 6b. The relative values of F_i in the partial removal zone for different drop radii likewise provide insight to value of F_coh, for which there is no theoretical measure. Indeed, $F_{\text{coh}}\propto ̸R$, but this prediction verifies larger drops carry higher cohesive forces.

Figure 9.

Prediction of inertial force for water (a,c) and 1:1 water–glycerin solution (b,d), for fixed drop location (a,b) and fixed drop size (c,d). (Online version in colour.)

Next, we fix drop radius $\overline{R}=0.75\hspace{0.17em}\text{mm}$ to predict F_i for various drop locations. For a fixed drop radius, inertial force in the partial liquid removal zone is greater for drops closer to the cantilever tip. At the tip, inertia forces temporally increase most rapidly at the onset of base vibration. Towards the base, the ejection inertial forces are unrealized for a relatively longer period of time, over which the cantilever imparts a time-history of deformation to the drop. We thus surmise that drops ejected closer to the base have reached a state of enhanced deformation by being allowed to slosh for more vibration cycles; drop elongation decreases F_coh. Predictions of F_i in all panels of figure 9 show a rapid change from the zone of partial to complete liquid removal where inertial force is independent of drop location, as expected by observation of equation (2.4). Above θ_e = 138^° we witness no pinch-off ejections, which agrees with our previous prediction of the transition contact angle [42]. Curves in figure 9 extrapolate to contact angles beyond our highest measured value of 155^°. Physically, we expect F_i → 0 as θ_e → 180^°, but note our cantilever is not capable of producing F_i lower than 0.03 mN when drops finally leave the surface.

By changing R and x₀ in our model, we predict changes to F_i but are unable to move from one removal zone to another, a characteristic of figure 9 that supports our analysis of feature importance scores (figure 6b). Though not captured in our plots, we posit complete liquid removal closer to the base requires longer ejection times at a fixed θ_e. Though θ_e is not fixed in figure 3, the plot of t versus x₀ shows complete removal is more rapid than partial removal. Sliding, though complete removal, takes the longest ejection duration because the drop must travel x₀ + R to leave the substrate.

4. Conclusion

In this study, we develop an ensemble learning algorithm to classify drop ejection modes and predict the minimum inertial force required to remove water drops and drops of glycerin solution from a dampened, damped cantilever. The ensemble model predicts the ejection modes with an accuracy of 85%, which is not a vast improvement over the individual base learners. With the data-driven predictive modelling approach, we identify the importance of system variables—ejection time, contact angle, drop size, and drop location—on ejection mode occurrence and the associated inertial force at ejection. Ejection time and contact angle are the dominant features for predicting both ejection modes and inertial force. In the prediction of inertial force at ejection, a regression problem, tree-based learning algorithms show better performance over kernel- and regularization-based algorithms. The ensemble improves RMSE by 11.9% over the best individual base learner, gradient boosting. Algorithmic predictions reveal that in the zone of partial liquid removal or pinch-off, drop size and location govern ejection inertial force, whereas inertial force in the zone of complete liquid removal is independent of drop size and location.

Data accessibility

Code, raw videos and .xls files containing experimental measurements are available in perpetuity via Open Science Framework: https://osf.io/3xnwq/.

Authors' contributions

M.D.E.A. performed experiments, analysed data, wrote machine learning algorithms and drafted the manuscript. D.W. consulted on machine learning and critically revised the manuscript. A.K.D. secured funding, managed the project and critically revised the manuscript. All authors gave final approval for publication and agree to be held accountable for the work performed therein.

Competing interests

We declare we have no competing interest.

Funding

We thank the NSF 1CBET-1941341 for support.