Reliability of Measurements on Plaster and Digital Models of Patients with a Cleft Lip and Palate

Abstract

Objective

The purpose of this study was to determine (1) the more and less reliable measurements/methods and (2) the influence of knowledge and skill on the inter-examiner, intra-examiner, and inter-method reliability of nasolabial measurements on plaster casts and three dimensional (3D) stereophotogrammetric images of casts in infants with an unrepaired unilateral cleft lip and palate (UUCLP).

Methods

Preoperative extraoral plaster casts from 42 patients with UUCLP were measured with a digital caliper, and the image acquisition of casts was performed with the 3dMDface stereophotogrammetry system (3dMD, Atlanta, GA). Two examiners (one postgraduate student, one lecturer) evaluated 19 nasolabial measurements in two separate sessions.

Results

Intra-rater, inter-rater, and inter-method reliability was lower in measurements of nasal, philtral, and nasal floor width. Almost all of the interclass correlation coefficients (ICC) for measurements performed by the lecturer were above 0.75, whereas the intra-examiner reliability of some measurements performed by the postgraduate student showed low ICC (<0.75).

Conclusion

Measurements of curving slopes, such as nasal width, of small dimensions, such as nostril floor width, and deformity-affected anatomic parts, such as philtrum width, presented a low reliability. Measurements on 3D images showed a higher reliability compared to plaster model measurements performed by the postgraduate student. Therefore, it may be recommended to use 3D digital images of infants with CLP for nasolabial measurements especially if performed in postgraduate settings.

INTRODUCTION

Cleft lip and palate (CLP) is the second most common congenital anomaly with the incidence of 0.6%–1% (1, 2). The treatment protocol of patients with CLP consists of interventions in special time periods over approximately 18–20 years. Therefore, records are not only used for diagnosis and fabrication of plate for presurgical orthopedic treatment in infancy, but also to evaluate the treatment progress, growth changes, and treatment outcomes over years. Moreover, records are required to communicate and transfer the history of the individual to the forthcoming specialist (3). Briefly, taking and archiving of the records of these cases is much more important than of traditional orthodontic cases.

The assessment and recording of the cleft deformity is performed using different methods. Photography, one of the oldest two-dimensional (2D) recording methods, needs training and effort of the professionals for standardization (4). Furthermore, this technique loses the three-dimensional (3D) nature of the anatomy (5). Facial anthropometry may deliver the most precise data; nevertheless, it unfortunately has shortcomings, such as the difficulty and a long duration during direct measurements on the face, particularly in infants and small children. The other disadvantage is the lack of communication of professionals without the presence of the patient (6).

Recently, the most frequently used 3D recoding method in cleft clinics is impression taking and cast model production. These methods are superior over photography and direct anthropometry, regarding the 3D evaluation and ease of communication and data transfer between specialists. However, tissue deformation due to the weight of the impression material, the risks of the impression-taking procedure, duration of the model production, the storage requirements, probable data loss due to model fragility, and difficulties in the analyzing of anatomic structures of models are the shortcomings of this method that could not be overcome for years (7–9).

Currently, thanks to the advancements in medical technology, 3D imaging systems including photo-optical, laser-optical scanning (10–12), and stereophotogrammetry (13–15) are introduced to enable the digitalization, even of former cast models and full computer-based management of patient records. Stereophotogrammetry, obtaining images by taking multiple photographs simultaneously, is usually used for facial soft tissue assessments; however, researchers suggest that it is also proper for imaging of plaster casts (16, 17). The inter-method measurement reliability between 3D images and anthropometric assessment (18, 19), as well as 3D virtual models and intraoral stone casts, was satisfying (17). In addition, several studies emphasized that the manipulation of 3D images is easy and uncomplicated (18–20). Certainly, identification of reliable 3D landmarks, and also performing of reproducible measurements, is related with the observer’s familiarity (involving knowledge and skill) to 3D images and software programs. As in every manipulation skill, training in 3D image visualization and analysis is required. In a study by Radeke et al. (21), three examiners with different degrees of expertise in dentistry measured the mesio-distal width of each tooth on cast models manually and on 3D images digitally. They concluded that the measurements from software-based methods did not diverge from conventional manual methods if performed even by observer who have a weaker background in dentistry. Nevertheless, the tooth forms assessed in the aforementioned study were more precise compared with abnormal anatomical variations such as a cleft lip and palate. In fact, to the best of our knowledge, none evaluated the effect of experience about the cleft anatomy on the reliability of plaster model and also 3D image assessments. Furthermore, no evaluation of the intra-reliability and shortcomings of lecturer in this topic were evident. Overall, the determination of less reliable measurements and the more reliable method in evaluating patients with craniofacial anomalies will enable to make up a checklist and integrate courses into the educational curriculum in postgraduate settings for the measurements and the method, respectively.

Therefore, the purpose of this study was to determine (1) the more and less reliable measurements/methods and (2) the influence of knowledge and skill on the inter-examiner, intra-examiner, and inter-method reliability of nasolabial measurements on plaster casts and three dimensional (3D) stereophotogrammetric images of casts in infants with unrepaired unilateral cleft lip and palate (UUCLP).

METHODS

This study was carried out on facial models of infants with UUCLP from the archive of the Orthodontic Department of Yeditepe University School of Dentistry. Patient data were handled according to the requirements and recommendations of the Declaration of Helsinki. Ethical approval (no.58/490) was obtained from the institutional review board of Yeditepe University.

Facial plaster models of 42 infants with UUCLP were selected from the archive. The models that were broken or had deficient representation of the anatomical morphology were excluded from the study. Then, the 3D sterephotogrammetric acquisition of the plaster models was performed with the 3dMDface system (3dMD, Atlanta, GA). The stereophotogrammetric system is composed of two modular units of six medical-grade machine vision cameras and a flash system. The models were placed 1 m away from the cameras, and images were captured in 1.5 miliseconds. All 3D images were imported to the 3dMD patient software program (3dMD, Atlanta, GA) for measurements.

Two examiners (R.B.N.Y. and M.A.) performed the measurements. One of the raters (R.B.N.Y.) was a lecturer experienced in patients with CLP and their variable anatomical structures and an active staff member in the cleft clinic over 10 years. She had the experience in handling of both the facial plaster model and 3D stereophotogrammetric images. The second rater (M.A.) was a postgraduate student in the orthodontic department, in the fifth semester. Although, she was theoretically familiar with the cleft lip and palate anatomy and manual measurements of teeth on plaster models, she never performed any measurements on facial plaster models as well as on 3D images of the models. However, she assisted regularly in the cleft clinic upon her first semester and was postgraduate student chef in the cleft clinic. Lecturer gave instruction lessons about not only the use of digital calipers and the 3D software program, but also the definition of the anatomical landmarks to the student. After training, both examiners located the anatomical landmarks and performed the measurements in the nasolabial areas on plaster models and 3D digital images.

Eighteen landmarks were identified to perform 19 linear measurements, consisting of 12 vertical and five horizontal measurements (Figure 1, 2; Table 1, 2). A digital caliper (Opto-Rs 232 simplex/duplex, Sylvac/Fowler, Crissier, Switzerland) was used for the measurements on plaster models, whereas the caliper function of the software program (3dMD, Atlanta, GA) was used for the measurements on 3D digital images.

Figure 1.

Nasolabial landmarks

Figure 2.

Nasolabial measurements

Table 1.

The definition of the nasolabial landmarks

Landmark	Definition
Pronasale (prn)	The most anterior midtip point of the nasal tip
Subnasale (sn)	The midpoint on the nasolabial soft tissue contour between the columella crest and the upper lip
Subnasale CS (snCS)	The point at the margin of the midportion of the columella crest at CS
Subnasale NCS (snNCS)	The point at the margin of the midportion of the columella crest at NCS
Alare CS (alCS)	The most lateral point on the alar contour at CS
Alare NCS (alNCS)	The most lateral point on the alar contour at NCS
Labiale superior (ls)	The midpoint of the vermillion line of the upper lip
Crista philtri CS (cphCS)	The point on crossing of the vermillion line and the elevated margin of the philtrum at CS
Crista philtri NCS (cphNCS)	The point on crossing of the vermillion line and the elevated margin of the philtrum at NCS
Christa philtri’ NCS’ (cph’NCS’)	The point at the noncleft side crossing the vermillion line and the elevated margin of the philtrum, corresponding the same point
Lateral subnasale inferior CS (sbnCS)	The lowest point of the lateral internal outer contour of nostril at CS
Lateral subnasale inferior NCS (sbnNCS)	The lowest point of the lateral internal outer contour of nostril at NCS
Lateral subnasale superior CS (zCS)	The highest point of the lateral internal outer contour of nostril at CS
Lateral subnasale superior NCS (zNCS)	The highest point of the lateral internal outer contour of nostril at NCS
Medial nostril superior CS (tCS)	The highest point of the medial internal outer contour of nostril at CS
Medial nostril superior NCS (tNCS)	The highest point of the medial internal outer contour of nostril at NCS
Nostril top point CS (cCS)	The highest point between lateral subnasale superior and medial nostril superior point on CS
Nostril top point NCS (cNCS)	The highest point between lateral subnasale superior and medial nostril superior point on NCS

CS, cleft side; NCS, noncleft side

Table 2.

The definition of the nasolabial measurements

Measurements	Definition
Cleft lip gap	Distance between cph’NCS’ and cphCS
Philtrum median height	Distance between sn and ls
Philtrum lateral height (CS)	Distance between snCS and cph’NCS’
Philtrum lateral height (NCS)	Distance between snNCS and cphNCS
Philtral width (NCS)	Distance between cph’NCS’ and cphNCS
Lateral lip height (CS)	Distance between cphCS and sbnCS
Lateral lip height (NCS)	Distance between cphNCS and sbnNCS
Nasal width	Distance between alCS and alNCS
Nostril floor width (CS)	Distance between sbnCS and snCS
Nostril floor width (NCS)	Distance between sbaNCS and snNCS
Nasal tip protrusion	Distance between sn and prn
Nostril height (CS)	Distance between snCS and cCS
Nostril height (NCS)	Distance between snNCS and cNCS
Medial nostril height (CS)	Distance between snCS and tCS
Medial nostril height (NCS)	Distance between snNCS and tNCS
Lateral nostril height (CS)	Distance between sbnCS and zCS
Lateral nostril height (NCS)	Distance between sbnNCS and zNCS
Nostril diameter (CS)	Distance between snCS and zCS
Nostril diameter (NCS)	Distance between snNCS and zNCS

CS: cleft side, NCS: noncleft side

Statistical Analysis

All plaster models and 3D images were remeasured within a 3-week interval by both examiners. Statistical analyses were performed using the Statistical Package for Social Sciences version 22 (IBM Corp.; Armonk, NY, USA) for Windows. Intraclass correlation coefficients (ICC) were used to determine intra-examiner, inter-examiner agreement for each measurement. ICC has a maximum value of 1 when there is total homogeneity. On the other hand, ICC values above 0.75 and 0.9 are considered as good and excellent, respectively.

RESULTS

The intra-examiner assessment of the plaster model measurements showed that the lecturer was consistent in the repeated measurements (ICC were greater than 0.90 for almost all measurements and had a lower boundary of 0.804), whereas the ICC of the half of the measurements performed by the postgraduate student were greater than 0.75 (Table 3).

Table 3.

Interclass correlation and 95 percent confidence interval for intra-examiner agreement of experienced and inexperienced operator

	Intra-examiner reliability
	Plaster models		3D images
	Experienced operator	Inexperienced operator	Experienced operator	Inexperienced operator
Cleft lip gap	0.976 (0.957–0.987)	0.917 (0.850–0.954)	0.997 (0.994–0.998)	0.949 (0.907–0.972)
Philtrum median height	0.929 (0.872–0.961)	0.594 (0.357–0.759)	0.970 (0.946–0.984)	0.882 (0.790–0.935)
Philtrum lateral height (CS)	0.977 (0.958–0.988)	0.891 (0.805–0.940)	0.999 (0.997–0.999)	0.944 (0.898–0.969)
Philtrum lateral height (NCS)	0.955 (0.918–0.976)	0.705 (0.512–0.830)	0.996 (0.993–0.998)	0.783 (0.630–0.877)
Philtral width (NCS)	0.814 (0.680–0.896)	0.466 (0.192–0.672)	0.947 (0.904–0.971)	0.547 (0.294–0.728)
Lateral lip height (CS)	0.957 (0.921–0.977)	0.808 (0.670–0.892)	0.969 (0.943–0.983)	0.866 (0.764–0.925)
Lateral lip height (NCS)	0.927 (0.868–0.960)	0.770 (0.611–0.870)	0.994 (0.988–0.997)	0.915 (0.847–0.953)
Nasal width	0.988 (0.978–0.994)	0.942 (0.894–0.968)	0.509 (0.246–0.702)	0.970 (0.945–0.984)
Nostril floor width (CS)	0.984 (0.970–0.991)	0.935 (0.883–0.965)	0.890 (0.805–0.939)	0.892 (0.808–0.940)
Nostril floor width (NCS)	0.904 (0.829–0.947)	0.574 (0.331–0.746)	0.748 (0.577–0.856)	0.530 (0.272–0.716)
Nasal tip protrusion	0.961 (0.929–0.979)	0.688 (0.488–0.819)	0.995 (0.990–0.997)	0.698 (0.503–0.826)
Nostril height (CS)	0.834 (0.712–0.907)	0.694 (0.497–0.823)	0.997 (0.995–0.998)	0.896 (0.815–0.943)
Nostril height (NCS)	0.946 (0.902–0.971)	0.445 (0.167–0.658)	0.990 (0.982–0.995)	0.914 (0.846–0.953)
Medial nostril height (CS)	0.804 (0.664–0.890)	0.692 (0.495–0.822)	0.966 (0.938–0.982)	0.831 (0.707–0.906)
Medial nostril height (NCS)	0.895 (0.813–0.942)	0.704 (0.512–0.829)	0.929 (0.872–0.961)	0.853 (0.741–0.919)
Lateral nostril height (CS)	0.989 (0.979–0.994)	0.919 (0.854–0.955)	0.987 (0.977–0.993)	0.792 (0.642–0.883)
Lateral nostril height (NCS)	0.920 (0.855–0.956)	0.791 (0.643–0.882)	0.982 (0.966–0.990)	0.691 (0.492–0.821)
Nostril diameter (CS)	0.872 (0.774–0.929)	0.747 (0.575–0.855)	0.998 (0.997–0.999)	0.963 (0.932–0.980)
Nostril diameter (NCS)	0.898 (0.818–0.944)	0.430 (0.149–0.647)	0.996 (0.993–0.998)	0.861 (0.755–0.923)

The intra-examiner reliability of all the 3D digital measurements of the lecturer was good (ICC greater than 0.75) except for the nasal width. Similarly, the intra-examiner reliability for most of the measurements carried out by the postgraduate student was good. The ICC values of only the philtral width, nasal tip protrusion, noncleft side nostril floor width, and lateral nostril height measurements were below 0.75 (Table 3).

An inter-examiner agreement was not present for the philtral width, nasal tip protrusion, nostril floor width, and the non-cleft side medial nostril height and cleft-side lateral nostril height measured on both plaster models and 3D digital images. An inter-examiner agreement was identified in more 3D digital measurements compared to those on plaster models (Table 4). Overall, the measurement performed by the lecturer showed a good inter-method agreement (Table 4).

Table 4.

Interclass correlation and 95 percent confidence interval for inter-examiner and inter-method agreement of experienced and inexperienced operator

	Inter-examiner reliability		Inter-method reliability
	Plaster models	3D images	Experienced operator	Inexperienced operator
Cleft lip gap	0.859 (0.753–0.922)	0.934 (0.880–0.964)	0.961 (0.929–0.979)	0.816 (0.683–0.897)
Philtrum median height	0.648 (0.432–0.794)	0.792 (0.645–0.883)	0.947 (0.904–0.971)	0.576 (0.333–0.748)
Philtrum lateral height (CS)	0.849 (0.736–0.916)	0.903 (0.827–0.947)	0.960 (0.927–0.978)	0.783 (0.631–0.877)
Philtrum lateral height (NCS)	0.760 (0.596–0.864)	0.770 (0.624–0.875)	0.884 (0.795–0.936)	0.653 (0.438–0.797)
Philtral width (NCS)	0.504 (0.239–0.699)	0.388 (0.098–0.616)	0.710 (0.521–0.833)	0.403 (0.117–0.628)
Lateral lip height (CS)	0.772 (0.614–0.871)	0.772 (0.613–0.871)	0.931 (0.875–0.962)	0.646 (0.428–0.792)
Lateral lip height (NCS)	0.824 (0.695–0.901)	0.840 (0.721–0.911)	0.929 (0.872–0.961)	0.691 (0.493–0.821)
Nasal width	0.925 (0.865–0.959)	0.296 (0.005–0.548)	0.270 (0.033–0.528)	0.874 (0.777–0.930)
Nostril floor width (CS)	0.941 (0.893–0.968)	0.901 (0.823–0.945)	0.977 (0.958–0.988)	0.820 (0.690–0.899)
Nostril floor width (NCS)	0.623 (0.397–0.778)	0.382 (0.091–0.612)	0.793 (0.647–0.883)	0.370 (0.078–0.604)
Nasal tip protrusion	0.737 (0.561–0.850)	0.583 (0.342–0.752)	0.790 (0.642–0.881)	0.590 (0.352–0.757)
Nostril height (CS)	0.663 (0.452–0.803)	0.833 (0.710–0.906)	0.841 (0.724–0.912)	0.704 (0.512–0.829)
Nostril height (NCS)	0.783 (0.631–0.877)	0.787 (0.636–0.879)	0.930 (0.873–0.962)	0.680 (0.477–0.815)
Medial nostril height (CS)	0.549 (0.297–0.729)	0.802 (0.660–0.888)	0.722 (0.538–0.840)	0.636 (0.415–0.787)
Medial nostril height (NCS)	0.407 (0.121–0.631)	0.467 (0.189–0.675)	0.753 (0.584–0.859)	0.415 (0.127–0.639)
Lateral nostril height (CS)	0.433 (0.152–0.649)	0.659 (0.447–0.801)	0.954 (0.916–0.975)	0.380 (0.089–0.611)
Lateral nostril height (NCS)	0.353 (0.059–0.591)	0.733 (0.554–0.847)	0.887 (0.800–0.938)	0.245 (0.060–0.508)
Nostril diameter (CS)	0.902 (0.825–0.946)	0.922 (0.859–0.957)	0.982 (0.967–0.990)	0.866 (0.764–0.926)
Nostril diameter (NCS)	0.818 (0.686–0.898)	0.827 (0.700–0.903)	0.806 (0.666–0.891)	0.686 (0.486–0.818)

DISCUSSION

Patients with impaired facial appearances such as a cleft lip and palate have a long treatment period, and the follow-ups are frequently difficult to manage; therefore, reliable, user-friendly, and easy-to-achieve documentation methods are necessary (22). Certainly, direct clinical evaluation and anthropometry is the golden standard in documentation (6). However, performing the measurements directly on the face to classify the deformity, to determine the treatment plan, to evaluate the treatment progress, as well as outcomes, is not easy, particularly in infants and children, or patients with mental retardation. Consequently, impression taking has been used more frequently to remodel the facial anatomy. Visually, the plaster models accumulate in the archives of clinicians over years, insomuch that some of the oldest ones have to be trashed. In addition to the storage requirement, fragile cast models are also prone to damage. Nowadays, more and more centers digitize the plaster models and transfer them into software programs to avoid data loss. Additionally, these virtual models allow easier communication between professionals due to the convenience of sharing files (23).

Virtual models may be an advantageous tool in converting the physical archives into digital ones; however, the reliability of the measurements performed on 3D models needs to be evaluated. Fleming et al. (24) compared the reliability of measurements performed on plaster and digital models in their systematic review and concluded that the use of digital models as an alternative to plaster models can be recommended. However, they also added that the reliability is based on various variables. One of the most important factors in the assessment of the performance of any new system, or in other words any invention introduced into a workflow, is the users’ experience (21). In addition, after determination of the reliability of measurements and evaluation methods performed by postgraduate students, a lecturer may make up guidelines for the students and integrate courses into the dental educational curriculum. Therefore, the purpose of this study was to determine (1) the more and less reliable measurements/methods and (2) the influence of knowledge and skill on the inter-examiner, intra-examiner, and inter-method reliability of nasolabial measurements on plaster casts and 3D stereophotogrammetric images of casts in infants with UUCLP.

The intra-examiner agreement of all plaster model measurements and all 3D digital measurements (except for the nasal width) performed by the lecturer were good or excellent (ICC equal or greater than 0.75 and 0.9, respectively). For the postgraduate student, most of the digital measurements showed a good reliability, whereas only half of the plaster model measurements showed an ICC above 0.75. The reliability of measurements carried out on plaster models depends on the ability of landmark identification, knowledge about the anatomy, and exact transfer of quantitative data to the computer. Furthermore, the operator has to deal sensitively with the plaster models, to avoid any breakage or deformation of anatomical structures during measurements (25, 26). Similarly, the reliability of measurements on 3D images are bases on a 3D landmark identification, the morphology of the anatomical structure, and image quality. Radeke et al. (21) compared the tooth-width measurements of operators with different levels of experience or even without dentistry background. They concluded that the measurements revealed no statistically significant differences between examiners. However, because the cleft anatomy is much more complicated for an inexperienced examiner, the intra-examiner as well as inter-examiner reliability showed differences between examiners in our study. Overall, another important factor affecting the reliability of measurements in both methods is the examiners experience not only regarding the anatomy of the observed structures, but also in handling of both measurement methods. Othman et al. (27), emphasized that the reproducibility of the identification of landmarks on 3D images by one operator is acceptable, but they concluded that further research of the inter-examiner reproducibility is required. Indeed, the familiarity of the examiner with 3D images and software programs plays a major role in the accuracy and repeatability of the measurements. The familiarity of the experienced examiner with 3D images and also the cleft anatomy may be the reason for the acceptable reliability of measurements.

On 3D facial scans, landmark identification on well-defined borders is easier, and therefore the reproducibility is higher. On the other hand, points located on curving slopes such as the alare point are difficult to determine (20). Accordingly, in our study, we found that the nasal width measurement was not reliable. In addition, it appeared that the experience factor did not matter. The ICC for the philtral width, nostril floor width, and medial nostril height (on NCS) measurements on 3D images done by the student were below 0.75. Anatomical areas, which show individual variations in cleft cases, such as the lateral subnasale inferior (sbnNCS), and areas most affected from the deformity, such as christa philtri (cph’NCS), have to be inspected with attention. The lateral subnasale inferior point, defined as the lowest point of the lateral, internal, and outer contour of the nostril, may be placed on different levels at the vertical plane depending on the shape of the nostril. If the examiner does not have enough experience about the cleft anatomy and the aforementioned anatomical variations, a divergence of measurements may occur (21). The nostril area on the noncleft side in cases with UCLP is also a small area so that validity is more difficult to achieve (28).

The intra-examiner reliability of the measurements performed by the lecturer on virtual models was higher than on the plaster models. Furthermore, the inter-examiner reliability was also higher for 3D images. The caliper manipulation requires experience and training. Sforza et al. (17) mentioned that the tip of the caliper may contact the plaster and afterwards landmarks cancelled the dot, inducing impression in the values of measurements. If measurements are performed on plaster models, the caliper has to be manipulated sensitively so that no anatomical structure is deformed. On the other hand, cancelling the dot on 3D images is not possible. In addition, 3D images enhance accurate measurements by enabling the researcher to rotate and to zoom into the image (29–31). In other words, software programs used in the 3D imaging technology may facilitate the manipulation skill of the operator and may be user-friendly, especially for inexperienced operators. Thus, 3D imaging may be used for training of postgraduate students.

CONCLUSION

• Measurements of curving slopes such as the nasal width, of small dimensions such as nostril floor width, and deformity-affected anatomic parts such as philtrum width presented low reliability.

• The reliability of measurements performed by the experienced examiner was high for both methods, whereas the intra-examiner reliability of some measurements performed by the inexperienced examiner showed low ICC.

• The reliability of a number of 3D digital measurements performed by the inexperienced examiner was found to be higher than plaster model measurements. Therefore, it may be recommended to use 3D digital images of infants with CLP for nasolabial measurements, especially if performed by inexperienced users.