Smartphone Scanning and Machine Learning for Automated Presurgical 3D-printed Plate Fabrication From Cleft Impressions

Summary:

Capturing accurate palatal impressions in newborns with cleft lip and palate is essential for early intervention, yet traditional methods using plaster casts are often challenging and uncomfortable for infants. We present an innovative digital workflow utilizing a smartphone scanning application and a machine learning tool for presurgical plate generation. This protocol employs an open-source mobile app to capture high-resolution 3-dimensional scans of the infant’s palate directly at the point of care. The process includes positioning of the infant, scanning technique, and file exportation. The resulting 3-dimensional model is then processed through a custom machine learning tool, which automates the design of a presurgical orthopedic plate with nasal stents. The method is simple, portable, and cost-effective, making it feasible in outpatient settings and especially advantageous for resource-limited environments. By eliminating the need for stone models and expensive intraoral scanners, this workflow streamlines the production of presurgical plates while enhancing patient comfort and safety. This digital approach introduces a new standard in early cleft care, offering a reproducible, accessible, and efficient protocol for clinicians aiming to modernize and simplify presurgical treatment.

Takeaways

Question: How can smartphone-based digital impressions be effectively implemented in outpatient cleft care for neonates, and what benefits do they offer compared with traditional methods?

Findings: This technical note presents a step-by-step protocol for using a smartphone and open-source app to scan palatal impressions in infants with cleft lip and palate. The scans are processed using a machine learning tool to automate 3-dimensional-printed passive plate design. The workflow is safe, accurate, low-cost, and feasible in resource-limited settings.

Meaning: Using a smartphone to scan impressions and design plates helps make cleft care for infants easy and more affordable, especially where resources are limited.

STEP-BY-STEP SMARTPHONE SCANNING

The patient with complete unilateral cleft lip and palate (CLP) undergoes a palatal impression in an outpatient setting, without general anesthesia. The infant is seated comfortably on the caregiver’s lap while being gently restrained. For the impression, a 3-dimensional (3D)-printed customized tray is selected based on the type of cleft and the infant’s size¹ (Fig. 1A). The impression is taken using a soft putty Vinly Polysiloxane impression material (3M ESPE, MN) that resists distortion and has a good viscosity to minimize leakage as alginate, reducing the risk of overflow into the oropharynx or nasal cavity.

Fig. 1.

Process of using impression tray for cleft care using a silicone material and a smartphone. A, Customized impression tray for a right unilateral complete CLP, before (left) and after the impression using silicone (right). B, Impression scanning process begins from the top, covering as many views of the object as possible until no more stripes appear, following the app instructions.

Once the impression is taken, it is scanned using a smartphone iPhone 12 (Apple, Inc.) equipped with a 2-camera system: a 12-megapixel main and ultrawide camera, with main ƒ/1.6 aperture, ultra-wide ƒ/1.6 aperture and 120-degree field of view, featuring 100% Focus Pixels with an open-source app (Scaniverse 2024 Niantic, Inc.) (Fig. 1B). Scaniverse can perform scanning and generate 3D meshes by light detection and ranging, photogrammetry, and Gaussian splatting. For scanning, the impression is placed on a flat, well-lit surface, although direct light is not required. The scan is conducted using the photogrammetry method for small objects, which captures images at 15 frames per second (fps) at the full resolution of the device. The impression is centered on the surface, and scanning begins with an occlusal view. The device is then moved at an angle of 30–45 degrees to capture the model from multiple views, ensuring complete coverage, while maintaining a consistent distance of 20–30 cm from the impression. The scanning process is completed in under 60 seconds, and a 3D model is generated, ready for export in a variety of formats for postprocessing and presurgical passive plate generation. (See Video [online], which displays impression scanning process from commencement to final file exportation.)

Video 1. This video demonstrates the step-by-step process of scanning a cleft palate impression using a smartphone-based 3D scanning app, highlighting positioning, camera angles, and model coverage for optimal capture of the impression. This scanning process serves as a crucial precursor for AI-driven plate design by providing accurate 3D models essential for automated landmark identification.

Download video file ^{(38.8MB, mp4)}

The exported negative 3D model is first postprocessed using a software for mesh inversion due to its lightweight and efficient handling of surface reconstruction (MeshLab v.2023.12). Although inversion can be done in Blender, we selected MeshLab to streamline processing for clinical teams unfamiliar with full 3D environments. A machine learning tool, hosted as a plugin in Blender and developed using a trained DiffusionNet model, is used to predict anatomical landmarks and create an automatically generated presurgical passive plate. The identified landmarks are automatically generated by trained model and are not manually placed or altered. A detailed explanation of the algorithm, point slection, and visual ouputs is available in our previous work.² This tool enables recognition of smartphone scans similar to intraoral scans. (See figure, Supplemental Digital Content 1, which displays the automatically generated anatomy-based landmarks [left side] and the corresponding presurgical passive plate ready for export and 3D printing [right side], https://links.lww.com/PRSGO/E348.) After plate generation, the file is exported and 3D-printed using biocompatible resin (NextDent Ortho Rigid, Vertex-Dental B.V., Soesterberg, The Netherlands). In our clinical implementation (Fig. 2), newborns usually needed an average of 5 outpatient visits throughtout the entire presurgical plate therapy process before surgery.

Fig. 2.

Clinical implementation in a complete unilateral CLP case. Patient receiving a 3D-printed passive plate generated from a smartphone scan (A) and after plate therapy (B).

DISCUSSION

The current study demonstrates a significant advancement in the management of CLP through the introduction of a smartphone-based 3D scanning workflow for the fabrication of presurgical passive plates. Traditional methods, including plaster casts, present challenges such as inconsistency, logistical difficulties, and discomfort for patients while taking impressions.³ Digital impressions effectively eliminate many of these drawbacks, offering improved accuracy and patient comfort, yet they can remain costly alternatives.⁴ The introduction of smartphone-based tools has transformed healthcare by accelerating processes and improving accessibility, particularly in resource-limited settings where high-cost intraoral scanners may not be affordable.⁵

Our workflow provides a reliable, fast, and more accessible method for scanning, designing, and delivering automated presurgical plates to newborns. In our study, the newborns received a quick point-of-care treatment, needing to attend only a mean of 5 visits throughout the whole plate therapy before surgery. These metrics demonstrate a notable degree of efficiency compared with traditional methods, which necessitate 11–14 visits.^6,7 The reduced appointments and fast treatment start are indicative of the simplified nature of our approach. This approach offers the dual benefit of enhancing both clinical efficiency and reducing the burden of care while also enhancing patient comfort.

In terms of clinical outcomes, the initial fitting of the 3D-printed plates was remarkably successful, with no need for trimming. This contrasts sharply with traditional methods, where studies have reported significant modifications of the plate at each visit.⁸ The high accuracy of the smartphone scanning process and the robustness of the machine learning tool used for plate design are underscored by this success. Such precision is particularly crucial in pediatric care, where patient comfort and rapid adaptation are vital.

Previous studies have validated the feasibility of smartphone-based scanning apps on phantoms and models of newborns with CLP^5,9 with efforts toward clinical implementation.¹⁰ However, no smartphone app has been clinically applied for treatment, making this work a pioneer of the clinical transposition of smartphone-based scanning technology. Smartphone-based 3D scanning applications offer a cost-effective and free solution for cleft documentation and presurgical orthopedics therapy.^5,9,10 The ability to process these digital models using open-source apps and software further streamlines the workflow, making it accessible in low- and middle-income settings.

A key innovation in our clinical implementation is the integration of machine learning tools for the automated design of presurgical passive plates. This reduces the time and expertise required to produce customized appliances for presurgical therapy.² By combining smartphone technology, open-source software, and 3D printing, our workflow presents a sustainable and accessible alternative to traditional methods. The digital models are easy to store, retrieve, and share, enabling long-term treatment planning and multidisciplinary collaboration.¹⁰ In future implementations, this digital workflow could also support centralized databasing for long-term research, patient tracking, and outcome assessment in cleft care.

Although this method is limited by the lack of detailed anatomical scans, our findings indicate that the machine learning tool has demonstrated robust anatomical recognition capabilities even with smartphone-derived inputs, as corroborated by a previous study.⁵ Furthermore, this workflow provides a foundation for the integration of digital technologies in cleft care, including a machine learning–driven tool and treatment planning. As these technologies continue to develop, such workflows will probably become more prevalent in cleft care, enabling more expedient documentation and the optimization of clinical workflows. Although this workflow currently involves taking a conventional impression, future developments in smartphone-based intraoral scanning could make it possible to work without taking impressions at all. However, technical challenges such as limited access to the oral cavity and motion sensitivity in neonates still need to be addressed. Future research will also compare this method with traditional and intraoral digital scanning workflows, evaluating differences in accuracy, efficiency and clinical performance.

In conclusion, smartphone-based digital impressions offer a practical, accessible, and efficient alternative to traditional plaster casts for capturing palatal morphology in infants with CLP. This workflow improves patient safety, reduces the need for clinical visits and enhances cleft care, particularly in areas with limited resources.

DISCLOSURES

The authors have no financial interest to declare in relation to the content of this article. This research was supported by the Research Fund for Excellent Junior Researchers of the University of Basel, Switzerland. The work was further supported by the Basel Research Centre for Child Health, supporting the Research Consortium for Pediatric Digital Health with a Multi-Investigator Project, “Burden-Reduced Cleft Lip and Palate Care and Healing.” The funding sources were not involved in the study design, the collection, analysis, or interpretation of data, the preparation of the manuscript, or the decision to submit the manuscript for publication.

PATIENT CONSENT

Informed consent was secured from the parents of the infant.

ETHICS APPROVAL

The case report received ethical approval from the Research and Ethics Committee of GSR Institute of Craniofacial Surgery (GSR ICFS/2023/001-I) for collection and further use of the clinical data.

DECLARATION OF HELSINKI

All procedures were conducted in accordance with relevant ethical guidelines and regulations (such as the Declaration of Helsinki).