Use of non-clinical smile images for human identification: a systematic review

Abstract

Human identification using Forensic Dentistry occurs through comparative analysis of ante-mortem (AM) and post-mortem (PM) data. With the constant improvement of technology, photographs became a common source of AM data. When clinical dental records are not available, images showing the smile can be useful in human identification. The aim of this study was to investigate human identification techniques through the analysis of smile images in the available literature. Studies on human identification through the analysis of smile images were searched in the scientific literature. The search resulted in 4,043 studies. After screening, 14 studies were considered eligible. Eleven were case reports, two were pilot studies and one a technical note. From the eligible studies, in addition to the methodological data, information about the Use of non-clinical smile images for human identification: a systematic reviewsample, used techniques and results regarding human identification were extracted. Three techniques were detected: direct comparison of morphological characteristics, AM/PM image overlap, and the analysis of smile lines. One or more associated techniques were used for human identification. Authors highlighted as a common limitation of the techniques the quality of the available images, the difficulty in reproducing PM the same images AM, and the eventual image modifications performed by the victim before posting in social media. Advantages included the low-cost aspect of the technique, as well as a potential fast and accurate procedure (depending on the quantity and quality of evidence). In general, studies considered the technique useful and adjuvant for human identification.

INTRODUCTION

Forensic Dentistry is one of the primary methods of identification. (1) The area of ​​investigation is traditionally the oral cavity. In this field, teeth are the main objects of study. Because teeth are considered the most stable and resistant structures in the human body, (2) they provide a good source of information. In skeletal remains, and victims with advanced decomposition or charred, teeth are often the better-preserved structure of the body. (3)

Basically, the forensic exam is performed through the analysis of comparative ante-mortem (AM) and post-mortem (PM) data. (4, 5) Traditionally, human identification through dentistry is performed using data from the dental records. Radiographs, clinical forms, dental casts and, more recently, intraoral photographs are examples of data that can be evaluated in the reconciliation phase. (6, 7)

With the improvement of technology, smartphones became tools capable of taking self-portraits. The frontal camera is a device that works as a kind of mirror, which allows the user to see and self-evaluate before taking the photograph. (8) The act self-photographic registration become popular in recent years. Worldwide, millions of social media users such as Facebook^TM, Instagram^TM and TikTok^TM feed their profiles with images of their faces and their daily activities. The so-called selfies (photographs of the individual taken by him/herself), became more popular. The word “selfie”, a reduction of the term “self-portrait photograph”, was considered, in 2013, the word of the year by the Oxford dictionary. (4) Taken mainly focusing and framing the face, (9) selfies can be useful in the forensic context. Moreover, in specific cases, casual photographs and videos showing the smile might be the only available data to identify a missing person.

Therefore, the present work aimed to investigate the existing literature on techniques to analyze smile photographs as a comparative method for human identification.

MATERIAL AND METHODS

Study Design

Studies on human identification through smile analysis have been sought in the literature. The nature of the present study was qualitative, consisting of a systematic review. The review was carried out according to the parameters of Preferred Reporting Items for Systematic Reviews and Meta-Analyzes (PRISMA) (10) and based on Cochrane standards. (11)

Eligibility

The present study was conducted based on the following guiding question: “What is the expert relevance of non-clinical images of the smile for human identification?”. To structure this question, the PICO strategy was used, in which P (person) = comparative human identification techniques in Forensic Dentistry; I (intervention) = non-clinical photographs of the smile; C (comparison) = post-mortem dental data and O (outcome) = limitations and advantages of the technique.

Studies human identification cases by smile analysis and studies describing the techniques were included. Literature reviews, letters to the editor, abstracts for conference proceedings and non-scientific articles were excluded, as well as original studies that did not clarify the methodology used.

Variables and sources

Pubmed, Lilacs, Web of Science, Scielo, Scopus, Embase, Open Gray and Open Acess Thesis and Dissertation were the databases used for primary data collection. The terms to be searched were first searched in the Medical Subject Headings (MeSH) and Descriptors in Health Sciences (DeCS) and the Boolean operators “AND” and “OR” were used. The terms were divided into two groups with word variations. The first group included the terms “smile identification”, “selfie identification”, “smile photography” and “selfie photography”. In the second group, the terms “smiling”, “human identification”, “dental photography”, “forensic” and “dental records” were included. The research was carried out on April, 2020, without restriction of year or language of publication of the study.

The studies found in the mentioned databases were imported into Mendeley^TM (Mendeley Ltd., London, UK), software used to organize the volume of studies and exclude duplicates. As soon as they were imported into the software, duplicates were automatically deleted. Studies that remained duplicated were deleted manually, after the first filtering.

Selection of studies

The survey of studies was carried out in three phases. In each selection phase, the process was supervised by a second examiner. The first phase was the selection of study titles. Studies that did not have titles related to the objective of this study were excluded. The names of the authors of the studies and the journals in which they were published were not blinded. In the second phase, the study abstracts were read and included or excluded, based on the eligibility criteria. Studies with abstracts with insufficient data were maintained for the next phase. In the third phase, the study was read in its entirety. The studies excluded in this phase had their reasons recorded.

Data extraction

General data were extracted from each study, such as the name of the authors, year, country, and journal of publication. Were registered the number of individuals reported, the type of smile image (selfies, photos or videos), technique used for the analysis, software used for the comparison and the type study (case report, case series, pilot study or technical note).

Risk of bias in the included studies

The Joanna Briggs Institute (JBI) checklist was used as a tool to assess the risk of bias in case reports, case reports in series and cross-sectional observational. (12) Eligible studies were assessed based on the percentage of positive responses to the requirements of the JBI checklists. Studies with less than 50% positive responses to the checklist were considered as high risk of bias. A moderate risk of bias was considered when 50 to 69% of the questions in the checklist were positive and a low risk of bias was considered when more than 70% of the questions in the checklist were positively marked.

RESULTS

Selection of studies

The initial search in the databases, with the terms described, showed 4,053 studies. The studies found in each database, as well as the terms used in the search are described in Table 1. The studies found in the Open Gray and Open Access Thesis and Dissertation databases were not exported, as they were not scientific articles. In the remaining databases, after being exported to the Mendeley^TM platform (Mendeley Ltd., London, UK), the studies were automatically checked for duplicates and reduced to 4,028.

Table 1. Results of studies found using specific terms in the search platforms.

Database	Terms	Results
PubMed	((((smile OR smiling OR selfie OR records)) AND (dental OR oral OR teeth OR tooth)) AND (identification OR photography)) AND forensic	484
Lilacs	(records OR photography) AND (smiling OR dental OR oral OR teeth OR tooth) AND (identification OR forensic)	183
Web of Science	ALL=((smile OR selfie OR records OR smiling) AND (dental OR oral OR teeth OR tooth) AND (identification OR photography) AND (forensic))	245
Scielo	(records OR photography) AND (smiling OR dental OR oral OR teeth OR tooth) AND (identification OR forensic)	42
Scopus	ALL ((smile OR selfie OR records OR smiling) AND (dental OR oral OR teeth OR tooth) AND (identification OR photography) AND (forensic)) AND (LIMIT-TO (DOCTYPE, "ar"))	2697
Embase	('smile' OR 'smile'/exp OR smile OR selfie OR 'records' OR 'records'/exp OR records OR 'smiling' OR 'smiling'/exp OR smiling) AND ('dental' OR 'dental'/exp OR dental OR oral OR 'teeth' OR 'teeth'/exp OR teeth OR 'tooth' OR 'tooth'/exp OR tooth) AND ('identification' OR 'identification'/exp OR identification OR 'photography' OR 'photography'/exp OR photography) AND forensic AND [article]/lim AND [embase]/lim	391
Open Grey	(smile OR smiling OR selfie OR records) AND (dental OR oral OR teeth OR tooth) AND (identification OR photography) AND (forensic)	1
Open access theses and dissertations	(smile OR smiling OR selfie OR records) AND (deal OR oral OR teeth OR tooth) AND (identification OR photography) AND (forensic)	10
TOTAL		4053

The databases were first organized into directories. The exclusion by reading the title was performed in each folder, removing 3,644 studies in this phase. 384 articles were left for reading the summary in all folders. These were grouped into a single folder and checked for duplicates again. After checking, the number of studies for reading the summary was 274. In this phase, 214 studies were excluded, leaving 50 studies for complete reading. The studies were carefully read and, of these, 35 were excluded because they did not meet the proposed eligibility criteria. Studies that did not use the smile image for human identification, did not describe which identification technique was used or that presented different designs from the one proposed were excluded. At each stage of the filtering process, in case of doubt, the study was continued for the next phase. Thus, 15 eligible studies remained.

Of the remaining 15 studies, four of them were not obtained in full; they were requested from the respective corresponding author, but without success. During the data extraction phase, in the analysis of the references used in the eleven available eligible studies, three more studies that met the eligibility criteria were found and attached. Thus, this systematic review was conducted with fourteen eligible articles (Figure 1).

Figure 1.

Screening process for eligible studies

Characteristics of selected studies

The studies were published between the years 1994 and 2019. The most prevalent countries were Brazil (n = 6) and Italy (n = 3), followed by the United States (n = 2), Australia (n = 1), Malaysia (n = 1) and the United Kingdom (n = 1) (Table 2).

Table 2. Eligible studies, selected from the initial search and organized in chronological order of publication in scientific journals.

#	Autors	Year	Country		Journal	Title
1	Phrabhakaran (13)	1994	Malaysia	Medical Journal of Malaysia		Identification from dental characteristics
2	Marks et al. (14)	1997	United States	Forensic Science International		Digital Video Image Capture in Establishing Positive Identification
3	Whittaker et al. (15)	1998	United Kingdom	British Journal of Orthodontics		Orthodontic Reconstruction in a Victim of Murder
4	Al-Amad et al. 16	2006	Australia	Journal of Forensic Odonto-Stomatology		Craniofacial identification by computer-mediated superimposition
5	De Angelis et al. (17)	2007	Italy	International Journal of Legal Medicine		Dental superimposition: a pilot study for standardising the method
6	Silva et al. (6)	2008	Brazil	Journal of Forensic Odonto-Stomatology		Forensic odontology identification using smile photograph analysis – case reports
7	Bollinger et al. (18)	2009	United States	Journal of Forensic Sciences		GrinLine Identification Using Digital Imaging and Adobe Photoshop
8	Tinoco et al. (19)	2010	Brazil	Journal of Forensic Odonto-Stomatology		Dental anomalies and their value in human identification: a case report
9	Terada et al. (20)	2011	Brazil	Revista de Odontologia da UNESP		Human identification in forensic dentistry from a photographic record of smile: a case report
10	Silva et al. (5)	2015	Brazil	American Journal of Forensic Medical and Pathology		Human Identification Through the Analysis of Smile Photographs
11	Miranda et al. (21)	2016	Brazil	Forensic Science International		An unusual forensic of human identification: Use of selfie photographs
12	Silva et al. (7)	2016	Brazil	Revista Brasileira de Odontologia Legal		Computerized dental delimitation of incisal edges in smile photographs with forensic purposes
13	Olivieri et al. (22)	2018	Italy	Forensic Science International		Challenges in the identification of dead migrants in the Mediterranean: the case study of the Lampedusa shipwreck of October 3rd 2013
14	Santoro et al. (23)	2019	Italy	Journal of Forensic Odonto-Stomatology		Personal identification through digital photo superimposition of dental profile: a pilot study

* Number assigned to each study found, according to the chronological order of publication in a scientific journal.

Except for study #2, (14) related to a child, all individuals reported were adults.

The studies were developed with volunteers (patients) (n = 3) and with cadavers (n = 11). Most studies were case reports (n = 11), followed by a pilot study (n = 2) and technical note (n = 1).

Except for study #2, (14) all studies used AM photographs for comparative analysis. Of those studies that used photographs, only study #11 (21) specified that the photographs used were selfies. Study #2¹⁴ used images captured from a video (frames) filmed by the victim's family.

Three techniques of comparative analysis of AM / PM or simulated data were performed. The techniques were: direct comparative analysis of morphological characteristics, overlapping images and incisal smile line. Three studies used two combined techniques: incisal smile line and image overlay. (5, 14, 21)

The tools used for comparative analysis were Adobe Photoshop^® (n = 8), PowerPoint^® (n = 1) and CorelDRAW X71^® (n = 1). In four studies, tools were not used or specified, and in study #14 (23) a comparative metric formula created for the study was used.

In all studies, the result of the analysis was decisive. In the case reports, the analysis resulted in positive identifications of the individuals. In the pilot and technical note studies, the simulated technique used was considered satisfactory to be used in real cases of human identification. All the methodological data described above, as well as the studies in which they were used, are detailed in Table 3.

Table 3. Methodological data extracted from the eligible studies.

*	Sample	Analysis Technic	Image	Tool	Identification	Type of study
1	-Charred body (female) -Charred body (female)	Direct comparison of morphological characteristics in AM / PM images	Non-clinical photographs taken by third parties (provided by the family)	-	Positive	Cases Series
2	Infant skeletal remains (female)	Smile lines and AM / PM image overlay comparing morphological characteristics	Video frames recorded by third parties (provided by the family)	Adobe® Photoshop®	Positive	Case Report
3	Skeletal remains (female)	Overlapping AM / PM images comparing morphological characteristics	Non-clinical photographs taken by third parties (provided by the family)	-	Positive	Case Report
4	Putrified body (male)	Overlapping AM / PM images comparing morphological characteristics	Non-clinical photographs taken by third parties (provided by the family)	Adobe® Photoshop®	Positive	Case Report
5	Pictures, plaster casts and volunteers	Image overlay (provided and simulated) and comparison with your plaster casts	Photographs taken by the authors for the tests	Adobe® Photoshop®	Satisfactory	Pilot Study
6	- Putrified body (male) - Skeletal remains (female) - Charred body (male)	Overlapping AM / PM images comparing morphological characteristics	Non-clinical photographs taken by third parties (provided by the family)	-	Positives	Cases Series
7	Volunteers	Overlapping images (provided and simulated) comparing morphological characteristics	Photographs taken by the authors for the tests	Adobe® Photoshop®	Satisfactory	Technical Note
8	Charred body (female)	Direct comparison of morphological characteristics in AM / PM images	Non-clinical photographs taken by third parties (provided by the family)	Adobe® Photoshop®	Positive	Case Report
9	Skeletal remains (male)	Direct comparison of morphological characteristics in AM / PM images	Non-clinical photographs taken by third parties (provided by the family)	Adobe® Photoshop®	Positive	Case Report
10	Charred body (male)	Smile lines and AM / PM image overlay comparing morphological characteristics	Non-clinical photographs taken by third parties (provided by the family)	Adobe® Photoshop®	Positive	Case Report
11	Charred body (male)	Smile lines and AM / PM image overlay comparing morphological characteristics	Selfies	CorelDRAW X71®	Positive	Case Report
12	Charred and putrefied body (male)	Smile line (lower arch)	Non-clinical photographs taken by third parties (provided by the family)	PowerPoint®	Positive	Case Report
13	Shipwreck (8)	Overlapping AM / PM images comparing morphological characteristics	Non-clinical photographs taken by third parties (provided by the family)	-	Positive	Cases Series
14	Volunteers	Image overlay (simulated)	Photographs taken by the authors for the tests	Adobe® Photoshop®	Satisfactory	Pilot Study

* Number assigned to each study found, in chronological order of publication as shown in table 2. Adobe^® Photoshop^® (Adobe Inc.^TM, San Jose, California, USA); CorelDRAW X71^® (Corel Corporation^TM Ottawa, Ontario, Canada); PowerPoint^® (Microsoft Corporation^TM Redmond, Washington, USA).

Risk of bias in the included studies

All eligible studies were considered to be at low risk of bias. In the eight case reports, seven of them were rated 100% based on the checklist responses, (5, 7, 14, 16, 19-21) while one study had 90% of positive responses. (15) Of the three case series, two of them had 80% of positive responses (6, 13) and one obtained 70% of the positive responses. (22) When it comes to the two detected pilot studies, one obtained 100% of the positive responses (23) and one obtained 90% of the positive responses. (17) For the technical note study type, the checklist was used for studies of the case report type; this study obtained 90% positive responses. (18)

DISCUSSION

In human identification, the availability, quantity and quality of AM data is fundamental. Challenging cases, however, include victims with no clinical/dental AM records, such as clandestine migrants, (22, 23) victims that never had dental appointments, or victims that were treated by dentists that could not be tracked. (21, 24) The analysis of smile photographs emerges as an alternative tool for the process of human identification by forensic odontology. (22)

Depending on the photograph available, the analysis of smile images can be performed in different ways. When the record is of good quality, visualization of the anterior teeth may be sufficient to detect distinctive features of the dentition. (21) In this type of analysis, morphological characteristics are evaluated, such as the shape of the crowns, gingival contours, incisal edges, dental anomalies and distance and alignment between the teeth. (6, 21) In the pioneering study proposed by McKenna (1986), (25) 1,000 individuals had their smiles evaluated from photographs. Of these, 76.7% had distinctive dental characteristics that would identify them.

The images posted on users’ profile of social media tend to show the best version of the individual. Happiness, in this context, may be expressed through the smile. Eventually, dental features of interest for human identification may be detected. (26) This scenario endorses the importance of forensic odontologists not only during PM data collection and reconciliation, but also during the AM data search analysis. (4)

In summary, the eligible studies detected in this review presented three main methods for evaluating teeth in photographs - direct morphological analysis, image overlap and comparison of smile lines. Each method has advantages and disadvantages and is adopted depending on the case. Direct morphological analysis is performed pairwise between the AM photograph of the victim and the PM examination of the deceased. A direct comparison consists of visualizing dental features simultaneously AM and PM – similarly to the use of an atlas. In general, AM photographs must have quality high enough to enable magnification with pixelization. The direct comparison focuses on exploring an overview of the available AM evidence (that could include distinctive shape and angulation of teeth, for instance). (6) The analysis of details in size and minor morphological features of teeth is hampered in the direct comparison. Image overlap, on the other hand, allows the visualization of the AM and PM data in the same spatial position. In this context, PM photographs reproducing the AM position and photographic frame is necessary. Taking anatomic references between AM and PM images (e.g. canines), computerized superimposition is performed. Normally, the reference points used are based on hard tissue (bones and teeth) because soft tissue is usually damaged or dehydrated PM. A drawback in image overlap is the need of basic knowledge in computer software for image manipulation. Preferably, image overlay-based software packages are preferred. Alternatively, software packages that allow the manipulation of image transparency are used. (18, 21) The analysis of smile lines focus on the incisal edges of the anterior teeth and may lead to a more visual representation of the similarities and discrepancies between AM and PM data, but this technique is restricted by the visualization of the anterior teeth on the photograph, image quality, and malalignment of anterior teeth. More evident distinctiveness is found in the deceased that present crowding, while more challenging cases are found within victims that have well-aligned teeth. (5, 6) It must be noted that the analysis of the smile line is based on a considerably reduced part of the crowns (incisal edges only) – and, so far, the uniqueness of the anatomy of anterior teeth is disputable in the scientific literature. (27) Hence, these techniques should be used in combination with other means for human identification. (28) Even the different techniques for the analysis of non-clinical photographs can be combined in a single case. (5, 6, 14, 21)

According to Miranda et al. (2016), (21) the smile analysis technique using photographs shows good results. As disadvantages, on the other hand, the authors highlight the limited number of visible teeth, the low quality of the available images and the potential manipulation of the image by the victim while still alive. (6) On limitations of the technique for the analysis of smile lines, Silva et al. (2016) (7) highlight three points that require the attention of the expert: The first is in relation to the quality of the images, in which, in low quality, that can influence the expert to erroneously trace the smile line; Secondly, aesthetic modifications that can “correct” a possibly individualizing characteristic of the victims, such as crowding, fractures and rotations. The third point refers to peri or PM tooth modifications. In cases of charred or body remains, tooth loss and alterations are common, which can impair the analysis of the smile line.

As photographs are two-dimensional images of a three-dimensional structure, PM images should try to reproduce the angulation and characteristics of the AM collection. To this end, experts should strive for AM data collection before the cadaveric examination. (24) It is also important to pay attention to the laterality of the images. (29) Some types of photographs, especially selfies, may be mirrored, creating confusion about the position in which the features appear in each photograph.

Although the existing limitations, the analysis of non-clinical photographs is considered an adjuvant for human identification in the current scientific literature.

CONCLUSIONS

Studies found in the literature on the analysis of non-clinical images for human identification have shown useful application of this technique in practice. The modalities of the technique were considered fast, accurate and low cost, and should be associated with other existing methods with known scientific reliability.