Automated calibration of 3D-printed microfluidic devices based on computer vision

Abstract

With the development of 3D printing techniques, the application of it in microfluidic/Lab-on-a-Chip (LoC) fabrication is becoming more and more attractive. However, to achieve a satisfying printing quality of the target devices, researchers usually require quite an amount of work in calibration trials even for high-end 3D printers. To increase the calibration efficiency of the average priced printers and promote the application of 3D printing technology in the microfluidic community, this work has presented a computer vision (CV)-based method for rapid and precise 3D printing calibration with examples on cylindrical hole/post diameters of 0.2–2.4 mm and rectangular hole/post widths of 0.2–1.0 mm by a stereolithography-based 3D printer. Our method is fully automated, which contains five steps and only needs a camera at hand to provide photos for convolutional neural network recognition. The experimental results showed that our CV-based method could provide calibrated dimensions with just one print of the specific calibration ruler to meet user desire. The higher resolution of the photo provides a higher precision in calibration. Subsequently, only one more print for the target device is needed after the calibration process. Overall, this work has provided a quick and precise calibration tool for researchers to apply 3D printing in the fabrication of their microfluidic/LoC devices with average price printers. Besides, with our open source calibration software and calibration ruler design file, researchers can modify the specific setting based on customized needs and conduct calibration on any type of 3D printer.

I. INTRODUCTION

Currently, 3D printing, or additive manufacturing, is widely used in the fabrication of microfluidic devices.^1–3 Before the emergence of 3D printing techniques, the dominant methods of microfluidic fabrication were photolithography and chemical etching.^4–7 These methods can provide precise features down to the sub-micrometer or even nanoscale but were not cost-effective or time-effective. For example, polydimethylsiloxane (PDMS) is one of the most popular elastomers used in soft lithography for microfluidic fabrication, whose process is made of a series of complex processes including molding, casting, curing, drilling, plasma treatment, and bonding. Not only is the fabrication process labor-intensive, but also the user interface of the PDMS microfluidics is cumbersome, which requires highly skilled operators, expensive equipment, and a clean room.⁸ Besides, despite the acceptable cost of PDMS, the expensive silicon wafer processing using lithography or chemical etching is restraining researchers from conducting free trial-and-error process.⁹

In contrast, 3D printing has a chance to revolutionize the microfluidic fabrication process. First, thanks to its layer by layer additive process, 3D printing allows users to print the target device using the appropriate materials without paying attention to the corresponding complicated 3D structure.^10–13 It gains followers due to the ease of use, low cost, and, most importantly, time efficiency.⁹ Among all 3D printing techniques, stereolithography (SLA)-based 3D printing has developed quickly in the past few years and now plays an important role in the fabrication of microfluidics due to its relatively high-resolution,² custom resin materials,^14,15 biocompatibility,^16,17 and affordable price (e.g., we can buy an SLA 3D printer on amazon.com with a price as low as 200 USD¹⁸). This SLA-based popular 3D printing method has encouraged researchers to explore applications in different fields. Piironen et al. studied the long-term cell survival in the presence of custom-designed microfluidic shear forces and confirmed the advantage of SLA-based 3D printing in the manufacturing of continuous flow cell culture platforms.¹⁶ Kim et al. reported the use of an unpowered, microfluidic sample preparation device for on-site molecular diagnosis without the use of bulky equipment. Their study has shown that the device was precise and reliable in DNA extraction and purification, which could process low concentrations of E. coli down to 10${}^3$ CFU/ml in the blood sample.¹⁹ Wang et al. printed resin molds using SLA-based 3D printing for the fabrication of PDMS microfluidic devices, and the device was successfully used in detecting trace levels of human pluripotent stem cells labeled with magnetic nanoparticles.²⁰ It is promising that the SLA-based 3D printing technique can conveniently benefit the researchers.

However, the resolution of different 3D printers can be varied, while printing errors exist even in high-end 3D printers.^21–23 To minimize these errors, researchers usually have to calibrate their printer several times before finding an optimal input value to obtain the desired output (e.g., to obtain a printed 1 mm width channel by designing a 1.1 mm width channel for the STL file). There is no doubt that the whole calibration process would take a few hours, days, or even weeks depending on the quality of the printer,² the complexity of the structure,^24,25 and the chemical and physical properties of the printing material.^26,27 Consequently, it is a battle against wasted money, various types of materials, and days of a trial-and-error process. What is more, the frustrating process cannot guarantee a precise result with acceptable deviations in different situations. For example, even if we have found an optimized configuration to print a 1 mm width channel, we likely need to find another configuration to print a 0.5 mm width channel by repeating the trial-and-error process mentioned above.

Where do these errors come from? Errors exist in different aspects and are caused by various factors. Except for the factors described previously, there are also others related to equipment and supplies such as the mechanical structure of the 3D printer,² the property of the resin,²⁸ and the configuration of the printing process.²⁹ Those types of errors are called absolute errors, which could only be eliminated through the quality increment of the commercial products themselves. Although these errors cannot be eliminated by users, is there a way to automate the trial-and-error process to minimize the errors? Also, is it possible that this process can be done once for universal printing situations? More specifically, how do we 3D print our microfluidic devices more efficiently and effectively by minimizing those absolute errors from the very beginning of building the models? The traditional 3D printing calibration method usually includes dimension measurement of the printed 3D body in order to compare with the desired dimension for the adjustment of the next 3D printing. This dimension measurement task is usually undertaken by a caliper or a microscope. This process might not take a long time but the following calibration procedure is quite complicated and the dimension measurement might introduce errors, which could be misleading and increase the calibration steps. With more and more applications of computer vision (CV) techniques, we are given a promising tool to solve the above issues. Among all CV techniques, Convolutional Neural Networks (CNNs) have been shown as a powerful tool undertaking image classification and recognition tasks,^30,31 such as medical image analysis,^32,33 object recognition,^34–36 action recognition,^37,38 or even face recognition with a mask on it.^39,40

To address the issues mentioned above, we present a CV-based automatic calibration method to minimize the absolute errors, which provides a convenient, efficient, and effective solution for 3D printing of microfluidic/lab-on-a-chip devices. In theory, our method can be applied to any brand of 3D printer and any kind of printing material. Briefly, two calibration rulers with pre-designed cylindrical and rectangular features were printed using a custom 3D printer, and waiting to be calibrated. Then, the actual dimension of the two rulers was obtained using our CV-based automatic calibration algorithm. Finally, a custom Python script was provided to investigate the difference between the designed dimension and actual dimension of both rulers, and a series of calibration functions were established based on the statistical underlying relationship. This automatic calibration method guarantees a one-time calibration of 3D printers for the target fabrication and improves accuracy effectively, which has been optimized for the fabrication of microfluidic and lab-on-a-chip devices.

II. THEORY OF AUTOMATED CALIBRATION

A. Automated measurement process of the calibration rulers

The automatic measurement and calibration process diagram of the calibration ruler is shown in Fig. 1. There are five steps in completing this calibration process. First, we need to have our pre-designed calibration rulers printed out, and proceed to actual dimension measurement in step 2. Our method was designed to couple with cameras at hand for convenience and efficiency. The camera we used in this study was Canon EOS Rebel T3i (Canon U.S.A., Inc., Long Island, NY), and a coin with known size has to be present as a reference. The picture has to be clear and cover the whole ruler in order to be used in our automated measurement process in step 3 with pre-trained CNN. After the four vertices of the rulers are manually selected as reference points, our CNN could intelligently outline all the present features on the pictures and report their dimension parameters in pixel. With the dimension results, our scripts written in Python could process those values provided by step 3 automatically and convert the pixel parameters into millimeters with the reference coin and output regression equations in step 4. Last, in step 5, users can calculate the new feature dimensions for printing based on the calibration equations and obtain new designs in a short time. This process is fast and requires fewer trials (a movie illustrating the overall workflow of the calibration process is available in the supplementary material).

FIG. 1.

The automatic measurement and calibration process diagram of the calibration rulers. Step 1: Print two calibration rulers. Step 2: Take the photo of the calibration rulers with the reference coin (RMB 0.1 yuan, and its diameter is 19 mm). Step 3: CNN recognition. Step 4: Analyze recognition data with Python and provide calibration equations. Step 5: New designs with calibrated features.

B. Design and fabrication of the calibration rulers

Photolithography technique is able to fabricate microfluidic devices with features down to a few micrometers even the sub-micrometer level, while in comparison, 3D printing is more advantageous with the features that are larger than the size of a mammalian cell.⁹ Though there are manufacturers such as BMF Precision Technology (Boston, MA) claiming that they can print features with a resolution of 2 $\mu$m, the price for printing a commercial $1\times 1\times 4{\mathrm{cm}}^3$ sample is as high as 300 dollars, which is not affordable for many researchers. Normally, 3D printers with X, Y, and Z resolution of 25–100 $\mu$m are the popular choices equipped in most of the microfluidic labs. Therefore, our goal is to find the best way of quick and precise 3D printing calibration of these types of printers, in order to benefit a wide range of researchers. To achieve this goal, we developed a standard process including the printing of a standard ruler that can fit for the calibration of any type of 3D printer. Specifically, the standard ruler is designed for the precise fabrication of microfluidic devices.

To cover the range of the microchannel shapes and sizes that are usually used in microfluidic chips, we have designed and fabricated a group of 3D printing calibration rulers for the development of a CV-based automatic method for quick and precise 3D printing feature measurement. The basic structural units of microfluidic chips are channels and tube-connecting regions. Channels usually have rectangular cross sections while connecting regions are generally in cylindrical shapes in order to match with tubes. With the development of 3D printing techniques, there are two popular methods that emerged in the past decades for the fabrication of microfluidic/lab-on-a-chip devices. One of them is to print the device directly,⁴¹ while the other one is to print the mask first and then make the device with PDMS.^42,43 Obviously, the two popular methods have different requirements for 3D printing techniques. Considering that 3D printers can print not only negative features but also positive features, the accurate printer calibration to meet each need is quite essential. The method of printing chips directly requires precise calibration on negative channels and connecting regions, while the method of printing mold requires precise calibration on positive rectangular walls and cylindrical posts to represent channels and connecting region sizes, respectively, for the following procedure of soft lithography. Therefore, the group of two calibration rulers with cylindrical holes and posts on one ruler [$66.0\times 10.0\times 2.0{\mathrm{mm}}^3$ of $\mathrm{L}\times \mathrm{W}\times \mathrm{T}$, Fig. 2(a)], and rectangular holes and posts on the other ruler [$50.5\times 10.0\times 2.0{\mathrm{mm}}^3$ of $\mathrm{L}\times \mathrm{W}\times \mathrm{T}$, Fig. 2(b)] were designed.

FIG. 2.

The design diagram and photos of calibration rulers. Design diagram of calibration ruler with cylindrical holes and posts (a), and rectangular holes and posts (b). Photo of 3D printed calibration ruler with cylindrical holes and posts (c), and rectangular holes and posts (d).

The radii of the cylindrical holes were set to be 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, and 1.2 mm in order to cover most of the microfluidic feature sizes. Therefore, the diameters of the cylindrical holes were 0.2, 0.4, 0.6, 0.8, 1.0, 1.2, 1.4, 1.6, 1.8, 2.0, 2.2, and 2.4 mm. The diameters of the corresponding cylindrical posts were the same as the cylindrical holes at the same relative position. The depth of the holes was 2.0 mm, which was the same as the thickness of the calibration ruler. The height of the cylindrical posts was all 1.0 mm. In order to be comparable with the cylindrical holes and posts, the width of the rectangular holes was set to be 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, and 1.0 mm. The size of 0.1 mm is the limit of most average price printers for the printing of a functional feature (e.g., a post or a wall). Although the leading brands like Formlabs (Formlabs Inc., Somerville, MA) provide 3D printers with a resolution of less than 0.1  mm, the price is above the average affordable range for most researchers. Besides, our calibration code and ruler diagram are open to the public (please refer to the supplementary material for details), available for researchers to modify based on their needs if they have high-resolution 3D printers. Therefore, the calibration of 0.1 mm features was not included in this study. As for the rectangular features, the length of the rectangular holes was all set to be 2.0 mm. The width of the corresponding rectangular posts was the same as the rectangular holes at the same relative position, and the length was also set as 2.0 mm. The depth of the rectangular holes was 2.0 mm, the same as the thickness of the calibration ruler, while the height of the rectangular posts was all 1.0 mm. The printing job of the two calibration rulers was conducted by an HP Jet Fusion printer (Hewlett-Packard Company, Palo Alto, CA) using a transparent resin, and the two printed rulers are shown in Figs. 2(c) and 2(d).

C. Theory of predicting microfluidic feature dimensions using CNN

One of the fundamental requirements for training the CNN is large amounts of labeled data. To build up the dataset for the following training process, ten different rulers (five for rectangular and five for cylindrical) were printed with transparent and white resin. Then, the photos of these rulers were taken before the cylindrical/rectangular features were cropped from the photos, and transformed into $50\times 50$ pixel images. Totally, 153 cylindrical hole/post images [Fig. 3(a)], and 77 rectangular hole/post images [Fig. 3(c)] were obtained. Next, the cylindrical hole/post images were labeled manually with their diameters and center coordinates. While the rectangular hole/post images were labeled manually with their widths, lengths, and center coordinates. However, the dataset for the training of CNN is not abundant enough. Therefore, a custom script was developed for the data argumentation by leveraging the OPENCV library.⁴⁴ Specifically, the cylindrical/rectangular hole/post features were cropped with the OPENCV mask operation and merged the cylindrical/rectangular hole/post features with a randomly selected $50\times 50$ pixel background image. During the merging process, the size (diameter) of the cylindrical features was randomly zoomed in or out, while the coordinates of the cylindrical features were randomly shifted in the $50\times 50$ backgrounds. Similar operations were applied to the rectangular features, and both width and length were resized with the same scale factor. Besides, the Poisson fusion operation was applied for better merging quality.⁴⁵ Finally, 27 027 cylindrical hole/post images [Fig. 3(b)], and 10 377 rectangular hole/post images [Fig. 3(d)] were generated in the data augmentation process. The specific script used for data augmentation is available in the supplementary material.

FIG. 3.

(a) Examples of original manually labeled cylindrical hole/post images. (b) Examples of randomly generated cylindrical hole/post images in the data augmentation process. (c) Examples of original manually labeled rectangular hole/post images. (d) Examples of randomly generated rectangular hole/post images in the data augmentation process.

As shown in Figs. 1 and 3, no matter the feature is a cylindrical hole or post, the picture taken always presents circular features or circles. Therefore, to outline the cylindrical features of the calibration ruler, the circle parameter CNN (cCNN) was constructed to convert ruler images into $50\times 50$ matrices as inputs, and the outlined circle parameters as outputs. Those circle parameters include the coordinate of each circle center and the diameter of each circle. The structure of CNN for predicting rectangular feature dimensions (rCNN) was similar to cCNN. The inputs were still original ruler images and the outputs were the outlined rectangle parameters, which include the coordinate of each rectangle center as well as the length and width of each rectangle. It has to be pointed out that all the parameters obtained in the CNN process were in pixel. The features in the above generated 27 027 cylindrical hole/post images and 10 377 rectangular hole/post images were labeled as circles and rectangles, respectively. Within those images, 20 271 circles and 8302 rectangles were used as the training set, and 6756 circles and 2075 rectangles were used as the test set. During the training process, the mean square error (MSE) was used to calculate the loss rate of cCNN and rCNN. The accuracy rates of the test set of cCNN and rCNN were calculated by Eqs. (1) and (2), respectively. The proposed method was implemented in Python 3.7 using Pytorch⁴⁶ and tested on a 2.8 GHz 20-core Intel Xeon server with 96 GB memory and an NVIDIA GTX 1080Ti GPU,

$$\begin{array}{rl}cCN{N}_{\mathrm{accuracy}}& =1-\frac{1}{n}\cdot {\displaystyle \sum _{k=1}^n\cdot \frac{1}{3}\cdot (\frac{|c_{d,k}-c_{d,k}^{\prime }|}{c_{d,k}}}\\ & +\frac{|c_{x,k}-c_{x,k}^{\prime }|}{c_{x,k}}+\frac{|c_{y,k}-c_{y,k}^{\prime }|}{c_{y,k}}),\end{array}$$ (1)

where $k$ indicates the index of each item in the training or test set; $n$ indicates the value of total items in the training or test set; $c_{d,k}$, $c_{x,k}$, and $c_{y,k}$ indicates the labeled diameter, x-coordinate, and y-coordinate of one circle image, respectively; $c_{d,k}^{\prime }$, $c_{x,k}^{\prime }$, and $c_{y,k}^{\prime }$ indicates the CNN-predicted diameter, x-coordinate, and y-coordinate of one circle image, respectively,

$$\begin{array}{rl}rCN{N}_{\mathrm{accuracy}}& =1-\frac{1}{n}\cdot {\displaystyle \sum _{k=1}^n\cdot \frac{1}{4}\cdot (\frac{|r_{w,k}-r_{w,k}^{\prime }|}{r_{w,k}}}\\ & +\frac{|r_{l,k}-r_{l,k}^{\prime }|}{r_{l,k}}+\frac{|r_{x,k}-r_{x,k}^{\prime }|}{r_{x,k}}+\frac{|r_{y,k}-r_{y,k}^{\prime }|}{r_{y,k}}),\end{array}$$ (2)

where $k$ indicates the index of each item in the training or test set; $n$ indicates the value of total items in the training or test set; $r_{w,k}$, $r_{l,k}$, $r_{x,k}$, and $r_{y,k}$ indicates the labeled width, length, x-coordinate, and y-coordinate of one rectangle image, respectively; $r_{w,k}^{\prime }$, $r_{l,k}^{\prime }$, $r_{x,k}^{\prime }$, and $r_{y,k}^{\prime }$ indicates the CNN-predicted width, length, x-coordinate, and y-coordinate of one rectangle image, respectively.

As it is shown in step 2 in Fig. 1, the photo taken has included not only the calibration ruler but also the reference coin. The coin can be of any face value but has to be State issued and have a standard dimension, which is an accurate reference object for calculating ruler feature dimensions. The photo is encouraged to be taken from the top of the ruler and has to be clear and cover the whole ruler in order to be used in our automated measurement process. With the pre-scripted codes written in Python, the program would process those values automatically, convert the pixel parameters into millimeters with the reference coin, and provide plots as well as regression equation outputs. The calibration functions for cylindrical holes and posts (circles), as well as rectangular holes and posts (rectangles), were then calculated in a simple step for providing calibrated parameters in order to obtain desired 3D objects with just one more printing.

To examine the measurement result of our CV-based method, a microscope (Olympus BX51, Tokyo, Japan) was used to measure the dimensions of holes and posts on each calibration rulers. This measurement result was compared with the CNN recognition result for accuracy analysis.

III. RESULTS

A. Investigation of the suitable structures of CNNs

Though data augmentation has been used to amplify the dataset for the training process, 27 027 cylindrical hole/post images and 10 377 rectangular hole/post images were still not enough to train deep neural networks. Therefore, the classic CNN model LeNet-5 was chosen as the starting point for hole/post recognition⁴⁷ with a small kernel size.^30,48 Two custom scripts were written to investigate the suitable CNN structures as well as the activation functions for cCNN and rCNN, which is available in the supplementary material. As shown in Fig. 4, the cylindrical recognition performance of four to seven convolutional layers with one to three linear layers was investigated. Among all the combinations, the CNN structure with five convolutional layers and two linear layers [Fig. 4(e)] has the highest accuracy rates for both training set (98.0%) and test set (96.9%) and the lowest loss rate of the training set (0.181). In addition, five different activation functions, ReLU, Leaky ReLU, Sigmoid, Softplus, and Tanh, were investigated to find out which one was more suitable for the CNN structure with five convolutional layers and two linear layers. Based on training results in Fig. 5, ReLU activation and Leaky ReLU activation were two obvious candidates as the performance of these two was significantly better than Sigmoid, Softplus, and Tanh activation. Then, the accuracy rate for the training set of ReLU activation is 0.7% higher than Leaky ReLU activation, while the accuracy rate for the test set is 0.2% lower than Leaky ReLU activation. Considering that the loss rate of ReLU activation is 47.4% lower than Leaky ReLU activation, ReLU activation was chosen as the activation function for the CNNs. Finally, the configurations of our cCNN and rCNN were determined and shown in Tables I and II, respectively.

FIG. 4.

The optimization process of different CNN structures. The training curves of (a) four convolutional layers with one linear layer, (b) four convolutional layers with two linear layers, (c) four convolutional layers with three linear layers, (d) five convolutional layers with one linear layer, (e) five convolutional layers with two linear layers, (f) five convolutional layers with three linear layers, (g) six convolutional layers with one linear layer, (h) six convolutional layers with two linear layers, (i) six convolutional layers with three linear layers, (j) seven convolutional layers with one linear layer, (k) seven convolutional layers with two linear layers, and (l) seven convolutional layers with three linear layers.

FIG. 5.

The optimization process of different activation functions. The training curves of applying (a) ReLU activation, (b) leaky ReLU activation, (c) sigmoid activation, (d) Softplus activation, and (e) Tanh activation on the five convolutional layers with two linear layers CNN structure.

TABLE I.

The cCNN configuration.

Layer	Type	Depth	Activation	Stride	Padding
1	Convolution	6	ReLU	(1,1)	1
2	Convolution	12	ReLU	(1,1)	1
3	Convolution	36	ReLU	(1,1)	1
4	Convolution	36	ReLU	(1,1)	1
5	Convolution	36	ReLU	(1,1)	1
6	AdaptiveMaxPool2d	36	ReLU	N/A	N/A
7 (Dense1)	Fully connected	2500	ReLU	N/A	N/A
8	Dropout(0.1)	N/A	N/A	N/A	N/A
9	Fully connected	1 (diameter)	ReLU	N/A	N/A
7 (Dense2)	Fully connected	2500	ReLU	N/A	N/A
8	Dropout(0.1)	N/A	N/A	N/A	N/A
9	Fully connected	2 (X,Y coordinates)	ReLU	N/A	N/A

TABLE II.

The rCNN configuration.

Layer	Type	Depth	Activation	Stride	Padding
1	Convolution	6	ReLU	(1,1)	1
2	Convolution	12	ReLU	(1,1)	1
3	Convolution	36	ReLU	(1,1)	1
4	Convolution	36	ReLU	(1,1)	1
5	Convolution	36	ReLU	(1,1)	1
6	AdaptiveMaxPool2d	36	ReLU	N/A	N/A
7 (Dense1)	Fully connected	2500	ReLU	N/A	N/A
8	Dropout(0.1)	N/A	N/A	N/A	N/A
9	Fully connected	2 (width, length)	ReLU	N/A	N/A
7 (Dense2)	Fully connected	2500	ReLU	N/A	N/A
8	Dropout(0.1)	N/A	N/A	N/A	N/A
9	Fully connected	2 (X,Y coordinates)	ReLU	N/A	N/A

B. Training of CNN

The training process and the performance of cCNN and rCNN are shown in Fig. 6. As we can see from Fig. 6(a), the accuracy rates of cCNN training set and test set are 98.7% and 95.9%, respectively, after training of 800 epochs. The loss rate of cCNN is 0.089. Both accuracy curves and the loss rate curve are in a smooth trend and converge to a stable state after 400 epochs. Figure 6(b) shows the absolute error of the diameters and center coordinates of 6756 circles in the test set, where all three groups have more than 97.8% of absolute error less than or equal to 2 pixels, indicating high accuracy. Figure 6(c) presents the accuracy rate and loss rate of rCNN after training of 1000 epochs. The accuracy rates of rCNN training set and test set are 97.0% and 90.1%, respectively, and the loss rate of rCNN is 0.214. Both accuracy curves and the loss rate curve show intense peaks during the epochs of 0–200, indicating fluctuation of updated weights during the CNN process. After that, all three curves show rapid convergence and become stable after about 600 epochs. Figure 6(d) shows the absolute error of width and length as well as center coordinates of 2075 rectangles in the test set, where the four groups of absolute errors are quite similar. The absolute errors of the width and length of the rectangles are 98.4% and 97.3%, respectively, that are less than or equal to 4 pixels. While the absolute errors of the X-coordinate and Y-coordinate are 99.5% and 99.0%, respectively, that are less than or equal to 4 pixels.

FIG. 6.

CNN training results. (a) The training curve of cCNN during 800 epochs. (b) The histogram of absolute error (pixel) of diameters and center coordinates of 6756 circles in the test set. (c) The training curve of rCNN during 1000 epochs. (d) The histogram of absolute error (pixel) of width and length as well as center coordinates of 2075 rectangles in the test set. Note: in (a) and (c), the left y axis indicates the accuracy rate (blue curve) and the right y axis indicates the loss rate (red curve).

C. Feature dimension measurement and analysis

The measurement of the calibration rulers is shown in Fig. 7. CNN automatically outlined the features on each calibration ruler and separated every single feature using green squares [Figs. 7(a) and 7(d)]. Besides, the numbers of features were counted and listed on top of the output image. As we can see from Fig. 7(a), the green circles mostly tracked the outlines of the real features and all the holes and posts were printed out with recognizable dimensions. Figures 7(b) and 7(c) present the comparison of measured diameter and the desired diameter of cylindrical holes and posts, respectively. As Fig. 7(b) shows that the hole diameters measured by cCNN and microscope are close to each other and both regression curves are highly consistent to the perfectly matched data line. The linear regression equations for cCNN and microscope measurement results are $y_{cnn}=1.0136x-0.0575$ and $y_{\mathrm{microscope}}=1.0425x-0.0494$, respectively. Figure 7(c) shows that the post diameters measured by cCNN and microscope are also close to each other. The two regression curves are close to each other, with linear regression equations for cCNN and microscope measurement results of $y_{cnn}=1.0142x+0.1436$ and $y_{\mathrm{microscope}}=1.0009x+0.1164$, respectively. For the calibration ruler with rectangular features, the green rectangles could still track most of the outlines of the real features, and most of the holes and posts were printed out with recognizable dimensions, except the hole #9, post #16, and post #18 [Fig. 7(d)]. Therefore, the above-mentioned three features were not included in the following measurement process. Compared to the diameters of the cylindrical holes and posts, the interesting parameter of the rectangular feature is the width of rectangular holes and posts. The hole width measured by rCNN and microscope remain similar and are mostly smaller than the perfectly matched data [Fig. 7(e)]. The two regression curves are similar as well, with linear regression equations for cCNN and microscope measurement results of $y_{cnn}=0.9226x+0.0041$ and $y_{\mathrm{microscope}}=0.8950x-0.0126$, respectively. The post width measured by rCNN and microscope, however, are mostly bigger than the perfectly matched data and width differences are larger when compared to the rectangular holes [Fig. 7(f)]. The linear regression equations for rCNN and microscope measurement results are $y_{cnn}=0.6844x+0.3864$ and $y_{\mathrm{microscope}}=0.9167x+0.1900$, respectively. Although the slopes of two regression curves do not match as well as Figs. 5(b), 5(c), and 5(e), the slope of the microscope regression curve is similar to that of the perfectly matched data line, indicating the consistency of errors during rectangular post printing and high possibility in precise error correction, while the CNN regression curve still points out the overall trend of the differences between the desired dimension and the actual measured dimension, showing the calibration necessity for specific features.

FIG. 7.

The measured dimension of the ruler features. (a) Outlines of cylindrical features using cCNN. The comparison of measured diameter and desired diameter of cylindrical holes (b) and posts (c). (d) Outlines of rectangular features using rCNN. The comparison of measured width and desired width of rectangular holes (e) and posts (f). Note: the numbers labeled in (a) and (d) are used to list the outputs in order.

D. Feature dimension calibration

The feature dimension measurement result has shown that the CNN measured feature size and microscope measured feature size are not identical but close to each other, with high consistency between the CNN regression curves and microscope regression curves. Therefore, our CNN method can be used in the calibration of feature dimensions. With the CNN regression curves obtained from Fig. 7, we could simply calculate the calibrated dimension to meet user desire. Figure 8(a) shows the newly printed calibration ruler with calibrated design dimension for cylindrical holes and posts to obtain the desired dimension. The “desired dimension” refers to the user-wanted real dimension after 3D printing, while the “design dimension” refers to the CNN calibrated dimension for 3D printing in order to obtain the “desired dimension.” It can be seen from Fig. 8(b) that the printed cylindrical hole diameter, which was measured by the microscope, was almost the same as the desired diameter and fell onto the perfectly matched data line. This result was obtained with the precise CNN calibration, and the relationship between the desired cylindrical hole diameter ($x$) and CNN calibrated design diameter ($y$) can be represented by the equation of $y=0.9866x+0.0567$. For example, if the desired diameter is 1.0 mm, the design diameter for 3D printing should be set to 1.0433 mm to obtain a D=1.0 mm cylindrical hole. Similarly, the printed cylindrical post diameter was also close to the desired diameter and fell onto the perfectly matched data line [Fig. 8(c)]. The relationship between the desired cylindrical post diameter ($x$) and CNN calibrated design diameter ($y$) can be represented by the equation of $y=0.9860x-0.1416$. Compared to the cylindrical hole, the CNN calibrated design diameter of the cylindrical post was slightly smaller than the desired diameter. For example, if the desired diameter is 1.0 mm, the design diameter for 3D printing should be set to 0.8444 mm to obtain a D=1.0 mm cylindrical post.

FIG. 8.

The calibrated dimension of the ruler features. (a) Calibration rulers printed with the adjusted design dimension of cylindrical features. The comparison of measured diameter after calibration and desired diameter of cylindrical holes (b) and posts (c). (d) Calibration rulers printed with the adjusted design dimension of rectangular features. The comparison of measured width after calibration and the desired width of rectangular holes (e) and posts (f).

The situation was more complex for the calibration ruler with rectangular holes and posts. Figure 8(d) shows the newly printed ruler with calibrated design dimension for rectangular holes and posts to obtain the desired dimension. It can be seen from Fig. 8(e) that the printed rectangular hole width was close to the desired diameter. The relationship between the desired hole width ($x$) and CNN calibrated design width ($y$) can be represented by the equation of $y=1.0839x-0.0044$. For example, if the desired hole width is 1.0 mm, the design hole width for 3D printing should be set to 1.0795 mm to obtain a $\mathrm{W}=1.0$ mm rectangular hole. However, when it comes to the cylindrical post, the printing parameters are not optimal [Fig. 8(f)]. The relationship between the desired post width ($x$) and CNN calibrated design width ($y$) can be represented by the equation of $y=1.4611x-0.5645$. Compared to the rectangular hole, the CNN calibrated design width of the rectangular post was much smaller than the desired width, which could be too small to be printed. For example, if the desired post width is 0.4 mm, the design post width for 3D printing should be set to 0.0199 mm to obtain a $\mathrm{W}=0.4$ mm rectangular post. The width of 0.0199 mm could be barely printed, but when it comes to a smaller desired post width such as 0.3 mm, the design post width is a negative number based on the equation, which could not be printed in reality. Therefore, the desired post width of 0.2 and 0.3 mm were not printed, and the corresponding microscope measurement data points were not shown in Fig. 8(f). This is consistent with Fig. 8(d) that the two rectangular posts on the right side of the calibration ruler do not exist.

IV. DISCUSSIONS

A. Training of CNN

Due to the image-analysis characters of CNN, the recognition of the feature dimension has to be processed in pixel and then converted to the millimeter with the coin reference. The training process of cCNN and rCNN was similar but their performance was different. As Fig. 6 shows, the accuracy rates of cCNN training set and test set were all slightly higher than the corresponding accuracy rates of rCNN training set and test set, respectively, while the loss rate of cCNN training set was much smaller than the loss rate of rCNN training set. The three curves of cCNN converged faster than the three curves of rCNN. There were significant fluctuations of updated weights during the rCNN training process, but with all the above factors, the accuracy and loss rate of rCNN were rather close to cCNN, indicating good structure and high reliability of our CNN training method. When comparing the absolute error of cCNN with rCNN test set, the absolute errors of three circle parameters are mostly less than or equal to 2 pixels, while the absolute errors of the four rectangle parameters are mostly less than or equal to 4 pixels. The above-mentioned differences are mainly caused by the shape difference of circles and rectangles, which have three parameters and four parameters, respectively. The inclusion of one more parameter could induce more errors during the training process. However, we cannot ignore the fact that there were almost two-times more training set and test set of cylindrical features for cCNN than the rectangular features for rCNN. In theory, the more data groups provided in the training process, the better the training results.

B. Feature dimension measurement and analysis

The high accuracy of both CNN during the training process enables the following procedure of real feature measurement. CNN automatically outlined the features on each calibration ruler and the circle tracking result was slightly better than the rectangle tracking result [Figs. 7(a) and 7(d)]. Most holes and posts were printed out with recognizable dimensions; however, during the CNN recognition process, some features were recognized with dimensions far off the reality and were not included in the analysis as well as the following calibration procedure. Those features include the diameters of the smallest cylindrical holes #11 and 12 and cylindrical posts #23 and 24, as well as the smallest rectangular hole of #9 and rectangular posts #16 and 18. This data processing step is included in our program, allowing users to remove those data that are wrong with huge errors, in order to obtain a higher calibration accuracy. Results showed that the corresponding circle diameters measured by cCNN and microscope are highly consistent and both regression curves are very close to the perfectly matched data line. Although there are some points not included in the figure, the regression lines clearly show the trend of feature sizes. In comparison, the difference between rCNN measured and microscope measured rectangular feature is bigger than that of the circular feature. The post width differences measured by rCNN and microscope are larger when compared to the rectangular holes [Fig. 7(f)]. This could be caused by the measurement method and printing errors. During the printing process, the zigzag direction of printing could affect the final shape of the features, including the angle of the hole from the bottom to the top, which might not be exactly straight. Therefore, users should be familiar with their printers about the best printing direction and configurations in order to receive the most precise calibration results.

C. Feature dimension calibration

It has to be pointed out that during the photo-taking step shown in Fig. 1, the photo is recommended to be taken right on top of the ruler and coin reference, but not required. Before the CNN recognition process, the four vertices of the calibration ruler will be selected manually, which will provide basic location information of both rulers. Then, CNN will conduct two-dimensional affine transformation, which is included in the algorithm, and stretch the ruler picture to the right aspect ratio. The following procedures are continued with the corrected picture and all the results are reliable. The calibrated dimension measurement results have confirmed that our CNN method can be used in the calibration of feature dimensions. Although the CNN calibrated “design dimension” could be far off the perfectly matched data line (the “desired dimension’), the actual printed feature dimensions are very close to the “desired dimension” [Figs. 8(b), 8(c), 8(e), and 8(f)]. Compared to the measured dimension of the ruler before calibration, the obtained dimension of cylindrical and rectangular features after calibration are all closer to the “desired dimension,” indicating successful calibration. It is noticeable that two of the smallest rectangular posts were not printed out after calibration due to the negative “design dimension” provided [Fig. 8(f)]. Although the originally designed widths for rectangular holes and posts were all 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, and 1.0 mm, it was interesting to see that their 3D printing results were significantly different. Rectangular holes with desired widths of 0.2 and 0.3 mm were printed out after calibration but not the rectangular posts with the same desired widths. This could be caused by the different printing mechanisms of holes and posts. To print the holes, the resin was added to the surrounding area layer by layer in order to leave the rectangles blank. Even though the resin could be squeezed beyond the designed outline, the calibration process could amend this and leave the rectangles blank with the “desired dimension.” However, to print the posts, the resin was added to make the rectangular shape, which was more like a thin line. If coupled with the condition that the resin was always squeezed a little bit, the thin line would always be printed to a thicker line with the wrong dimension. Therefore, during the calibration process, in order to correct this, the “design dimension” has to be amended to meet the “desired dimension,” and if the squeezed amount is bigger than half of the desired width, the calibrated dimension would be negative numbers, which cannot be recognized by 3D printers. It is also interesting to see that the cylindrical post with the diameter of 0.2 mm could be printed out successfully after calibration. Although the dimension was the same, the printing result was different. It can be seen from Figs. 7(c) and 7(f) that the measured dimensions of cylindrical posts and rectangular posts were all higher than their corresponding desired dimension. However, the difference between the measured dimension and the desired dimension of cylindrical posts was much smaller than that of the rectangular posts. This led to a bigger revision of rectangular posts compared to cylindrical posts, which guaranteed the positive dimension of cylindrical posts after calibration but not for rectangular posts. Besides, the slicing of printing a cylindrical post and a rectangular post might be slightly different due to the setup of the printer. We noticed that the long thin wall-shaped rectangular post was easier to collapse during printing when compared to the cylindrical post with the same diameter size as the width of the wall.

D. The robustness of calibration

Most 3D printers have their built-in calibration protocol, which is optimized for printing general items. The feature size of these general items is usually millimeter scale or even centimeter-scale, which is at least one order of magnitude larger than the claimed resolution (e.g., 25 $\mu$m) of the printer. That means that the post-calibration settings may not be efficacious and effective for microfluidics applications. In our experimental tests, we used an HP Jet Fusion printer to print ten different calibration rulers (one cylindrical ruler and one rectangular ruler per time, totally five batches) with the same calibration settings and observed no significant difference (equal or smaller than 2x of the resolution) between each batch by inspecting the features using a microscope, indicating one-time calibration can last for at least five printing jobs for the HP printer we used. Based on that, we suggest executing the re-calibration process after every five printing jobs. After that, inspecting the features of printed items/devices is recommended to determine whether re-calibration between five printing jobs is conservative or aggressive. In addition, re-calibration is always recommended after introducing new interference to the printers such as adding/changing the resin of the container, applying new settings, etc. However, if one printer is not robust enough and cannot provide consistent calibration rulers between different printing jobs, our calibration process will become meaningless because the printer cannot be competent to print any microfluidic/lab-on-a-chip devices in nature no matter what kinds of calibration process have been applied. In other words, our proposed method can extend the application scenarios of the commercial 3D-printers from printing general items to printing microfluidic/lab-on-a-chip devices but cannot increase the design/built-in resolution of the target printers.

V. CONCLUSION

The application of 3D printing in microfluidic chip fabrication is still in its infancy; however, the huge potential of 3D printing is attractive and waiting to be explored by researchers. This work has presented a CV-based method for automated precise 3D printing calibration with microfluidic features on cylindrical holes/posts (diameters of 0.2–2.4 mm) and rectangular holes/posts (widths of 0.2–1.0 mm). The experimental results confirmed that our CV-based method could provide the right “design dimension” with just one print of the calibration ruler to meet the user “desired dimension” of cylindrical holes and posts, as well as rectangular holes and posts. In addition, with our open source calibration code and ruler diagram, researchers can modify their design based on needs if they have high-resolution 3D printers. Since the pre-trained cCNN and rCNN are available in the supplementary material, this method can be applied to not only the resin-based 3D printers but also any other types of 3D printers by leveraging machine learning techniques such as transfer learning.^49,50 Overall, this work has provided a user-friendly tool for researchers to apply 3D printing in the fabrication of their microfluidic chips in a quick and precise way with average price printers.

SUPPLEMENTARY MATERIAL

See the supplementary material for the complete guideline to deploy the calibration process and the associated CNN training scripts.

DATA AVAILABILITY

The data that support the findings of this study are available within the article and its supplementary material.