Skip to main content
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2020 Oct 23;9(4):159–163. doi: 10.1016/j.ejwf.2020.10.003

The impact of COVID-19 pandemic on patients receiving orthodontic treatment: An online questionnaire cross-sectional study

Nour Bustati 1,, Nada Rajeh 1
PMCID: PMC7584421  PMID: 33221177

Abstract

Background

The spread of Coronavirus Disease 2019 (COVID-19) has led to a major public health issue; most dental clinics were closed and millions of orthodontic patients were unable to complete their treatment. This study aimed to assess the challenges faced by patients receiving orthodontic treatment and their preferred solutions to overcoming these challenges during this pandemic.

Methods

An online questionnaire was developed and sent to patients receiving orthodontic treatment at a public or private clinic.

Results

A total of 388 responses were analyzed: mean age 20.4 ± 4 years, 75% (291) female, and 58% (226) received their treatment at a public clinic. Of all participants, 27.3% (106) were still unable to attend their appointments and 69% (244) stated that closing of the clinic was the main reason for missing their appointments. Depending on their type of appliance, the patients faced different problems and chose multiple ways to deal with them. Most participants had fixed appliances, 84% (327), and only 21% (64) of them stated that they had no problem compared with 39% (11) and 36% (8) for removable appliance and clear aligner groups, respectively.

Conclusion

The COVID-19 pandemic has had a significant impact on orthodontic treatments. Almost every orthodontic patient had to stop attending their appointments, which put them in complicated situations and in fear of delayed treatment. Patients from a public clinic and patients with fixed appliances reported more problems than others. More attention should be giving to teleorthodontics; also orthodontists should prepare their patients to deal with some of the problems related to their appliances when possible.

Keywords: COVID-19, Orthodontic emergencies, Fixed orthodontic appliances, Delayed appointments, Teleorthodontics

1. Introduction

At the end of 2019, an outbreak of a novel coronavirus (Coronavirus Disease 2019 [COVID-19]) started from Wuhan, China, and later on spread widely around the globe and led to a major public health issue [1]. To the day of writing this article, more than 30 million confirmed cases of COVID-19, including more than 940,000 deaths globally, have been reported to the World Health Organization, and the number is rising [2]. Different policies and strategies were taken by each affected country to face this pandemic and minimize the number of cases, from partial to complete lockdown, giving that in the absence of a vaccine, the World Health Organization has recommended some precautions to control infection, such as regular hand hygiene, the use of face masks, following cough etiquette, and most importantly keeping social distancing [3]. The main way of transmission is through close contact with an infected individual; however, numbers of asymptomatic carriers were reported, which increases the possibility of transmission without cautious actions being taken [4]. Although most governments encouraged people to stay and work from home, society is in need of health care workers in everyday life and during this pandemic specifically, and this put them at high risk. In particular, of all health care systems, dental procedures have more direct contact with aerosols, with higher possibility of contracting infection [5]. Despite numerous modifications and guidelines that were recommended to dental practice during this pandemic, several dental clinics were closed as it was suggested that all other regular treatments should be postponed and only patients requiring emergency care could visit their dentist under strict precautions [6,7].

On the other hand, patients receiving orthodontic treatment require multiple checkup visits to their orthodontist to adjust their appliances over a longer period that sometimes may take more than 2 years. Therefore, millions of orthodontic patients were not able to receive the care they needed during this pandemic and the closing of clinics that followed [8]. This led to higher level of anxiety and mental distress among orthodontic patients in comparison with patients receiving other types of dental treatments [9]. The aim of this study was to assess the challenges faced by orthodontic patients during the COVID-19 pandemic and to understand what they went through and their attitudes toward it.

2. Materials and methods

This cross-sectional study was carried out with a Web-based questionnaire using google forms (docs.google.com/forms). The target population was patients within the age range of 15 to 30 years, and currently undergoing orthodontic treatment at a public or private clinic. Ethical approval for this study was obtained from the ethical committee of Damascus University, Damascus, Syria.

The questionnaire was developed following CHERRIES (Checklist for Reporting Results of Internet E-Surveys) [10], and was structured into four sections. The first section consisted of personal information (age, gender, type of clinic and type of appliance). The second section had one question regarding the period of time the respondent had missed their visits during this pandemic. Those who answered with “did not miss any visit” were directed into the submit page, whereas the other respondents were able to proceed to the third section (the reason for missing the appointments, communication with the orthodontist, main concern, degree of agreement toward closing of clinics). The last section was about the problems patients faced and their ways of addressing those problems.

To validate the questionnaire, it was evaluated by an experienced orthodontist to check whether the questions effectively capture the purpose of the study, and a questionnaire construction expert for presence of confusing and leading questions.

For the pilot study, the questionnaire was sent to 30 orthodontic patients. They were asked to specify if any question was unclear, and an open option was added at the end of every question so that they can add more answers if needed. Then the questionnaire was modified in accordance and finalized.

Because some answers may vary over a short period, test-retest correlation was inapplicable. To test reliability, a question (type of appliance) was repeated twice.

The questionnaire was uploaded to the website, and the generated link was sent to patients from one public clinic (University clinic) and five private clinics via mobile phone application (WhatsApp Messenger). Responses were received over a period of 3 weeks. Sample size was determined by considering the fact that the population of patients receiving orthodontic treatment in the Syria country as 3 million with 95% confidence level and 5% confidence interval. The required sample size was 384 participants.

This being an open voluntary questionnaire, answers to all the questions were mandatory. Adaptive questioning was used, hence some questions were on display in accordance with the previous answer. Also, respondents were able to click on the “back” option to change their answer if they wish to before submitting. Uncompleted questionnaires were not submitted.

To obtain informed consent, in the description bar on the first page of the questionnaire, detailed information was given about the aim of the study, that it would take 5 to 10 minutes to complete, and that all the responses will be confidential and anonymous. Consent was obtained if the respondent clicked on “agree to participate.”

For data protection, no sign-in process was required from the respondents. All the responses were anonymous and only the authors had access to the collected data.

For data analysis, responses were automatically linked to a spreadsheet on Google Sheets. Then the data were downloaded to Microsoft Excel 2019. Statistical analyses were performed using IBM SPSS Statistics for Windows, version 25 (IBM Corp., Chicago, IL).

3. Results

A total of 400 responses were received; 4 did not agree to participate, giving a participation rate of 99%. However, 12 respondents were excluded because they were inconsistent and answered the repeated question differently. The final number of responses that were analyzed was 388. The average age of participants was 20.4 ± 4.0 years. Descriptive statistics of responses from section 1 are shown in Table 1 .

Table 1.

Descriptive statistics of section 1 responses

Gender, % (n) Clinic, % (n) Appliance, % (n)
Male 25 (97) Public clinic 58 (226) Fixed appliance 84 (327)
Female 75 (291) Private clinic 42 (162) Removable appliance 8 (30)
Clear aligners 8 (31)

Regarding the second section, 9% (35) of participants reported that they did not miss any visit to their orthodontists and were directed to the submit page.

Table 2 shows descriptive statistics of section 2 and section 3, and were dichotomized according to those who attended public or private clinics.

Table 2.

Descriptive statistics of section 2, 3 responses

Question Item Public clinic patients, % (n) Private clinic patients, % (n) Total, % (n)
During the COVID-19 pandemic, for how long did you stop visiting your orthodontist? I didn't stop visiting my orthodontist 6 (2) 94 (33) 9 (35)
For 1 to 2 months 25 (31) 75 (95) 32.5 (126)
More than 2 months 77 (93) 23 (28) 31.2 (121)
I'm still not visiting my orthodontist 94 (100) 6 (6) 27.3 (106)
What was the reason you stopped visiting your orthodontist? Clinic was closed 73 (179) 27 (65) 69 (244)
I was afraid of the spread of COVID-19 27 (15) 73 (41) 16 (56)
Both previous reasons 62.5 (30) 37.5 (18) 14 (48)
I was out of the country 0 (0) 100 (5) 1 (5)
How many times did you communicate with your orthodontist? I didn't communicate 47 (43) 53 (48) 26 (91)
Once or twice 63 (113) 37 (65) 50 (178)
3 or 4 times 76 (45) 24 (14) 17 (59)
More than 4 times 92 (23) 8 (2) 7 (25)
How did you communicate with your orthodontist? Voice call 51 (41) 49 (40) 31 (81)
Mobile phone application 76 (116) 24 (37) 58 (153)
Calling and application 83 (20) 17 (4) 9 (24)
SMS 100 (4) 0 (0) 2 (4)
What was your main concern during this period? Treatment time will be extended 62 (127) 38 (79) 58 (206)
My teeth would relapse 66 (97) 34 (50) 42 (147)
To what extent do you agree or disagree about closing the dental clinics to minimize the spread of COVID-19? Strongly disagree (1) 75 (53) 25 (18) 20 (71)
Disagree (2) 73 (19) 27 (7) 7 (26)
Somewhat disagree (3) 78 (35) 22 (10) 13 (45)
Neutral (4) 67 (54) 33 (27) 23 (81)
Somewhat agree (5) 51 (20) 49 (19) 11 (39)
Agree (6) 48 (12) 52 (13) 7 (25)
Strongly agree (7) 47 (31) 53 (35) 19 (66)

Also, responses from the third section were divided according to the type of appliance the patient was using. Table 3 shows descriptive statistics of section 3 regarding the type of appliance.

Table 3.

Descriptive statistics of section 3 responses

Question Item FA, % (n) RA, % (n) CA, % (n)
What was the reason you stopped visiting your orthodontist? Clinic was closed 87.7 (214) 7.8 (19) 4.55 (11)
I was afraid of the spread of COVID-19 86 (48) 9 (5) 5 (3)
Both previous reasons 81.4 (39) 8.3 (4) 10.4 (5)
I was out of the country 40 (2) 0 (0) 60 (3)
How many times did you communicate with your orthodontist? I didn't communicate 80.2 (73) 14.3 (13) 5.5 (5)
Once or twice 87 (155) 6 (10) 7 (13)
3 or 4 times 88 (52) 7 (4) 5 (3)
More than 4 times 92 (23) 4 (1) 4 (1)
How did you communicate with your orthodontist? Voice call 92.6 (75) 3.7 (3) 3.7 (3)
Mobile phone application 85 (130) 7 (10) 8 (13)
Calling and application 92 (22) 8 (2) 0 (0)
SMS 75 (3) 0 (0) 25 (1)
What was your main concern during this period? Treatment time will be extended 87 (179) 6 (13) 7 (14)
My teeth would relapse 84.4 (124) 10.2 (15) 5.4 (8)
To what extent do you agree or disagree about closing the dental clinics to minimize the spread of COVID-19? Strongly disagree (1) 89 (63) 3 (2) 8 (6)
Disagree (2) 69.2 (18) 15.4 (4) 15.4 (4)
Somewhat disagree (3) 87 (39) 11 (5) 2 (1)
Neutral (4) 89 (72) 7 (6) 4 (3)
Somewhat agree (5) 85 (33) 10 (4) 5 (2)
Agree (6) 92 (23) 0 (0) 8 (2)
Strongly agree (7) 83 (55) 11 (7) 6 (4)

CA, clear aligners; FA, fixed appliances; RA, removable appliances.

About the final section of the survey, patients were divided into three groups according to their type of appliance. Twenty-one percent (64) of patients with fixed appliances (FAs) stated that they had no problem compared with 39% (11) and 36% (8) of patients with removable appliances (RAs) and clear aligners (CA), respectively.

Table 4 demonstrates the problems and the way to address them by each group.

Table 4.

Descriptive statistics of section 4 responses

Type of appliance Problems, % (n)a Way to address the problems, % (n)a
Fixed appliance Exposed end of wire, 44 (106) Calling my orthodontist, 38 (90)
Bracket/band off, 31 (73) Ignoring the problem, 27 (64)
Appearing of spaces, 29 (69) Orthodontic wax, 19 (46)
Gum swelling, 23 (55) Sending a picture to my orthodontist, 13 (30)
Orthodontic pain, 14 (34) Visiting a nearby dentist, 12 (29)
Broken metal piece, 14 (34) Requesting an emergency appointment, 10 (24)
Lost elastic/spring, 14 (33) Searching for a solution over the Internet, 6 (15)
Run out of elastic band, 13 (32)
Removable appliance Orthodontic pain, 35 (6) Ignoring the problem, 35 (6)
Loose appliance, 29 (5) Searching for solution over the Internet, 35 (6)
Broken part, 24 (4) Calling my orthodontist, 18 (3)
Appearing of spaces, 12 (2) Sending a picture to my orthodontist, 12 (2)
Gum swelling, 6 (1) Visiting a nearby dentist, 6 (1)
Requesting an emergency appointment, 6 (1)
Clear aligner Loose aligner, 86 (12) Calling my orthodontist, 57 (8)
Broken part, 14 (2) Requesting an emergency appointment, 14 (2)
Orthodontic pain, 14 (2) Ignoring the problem, 14 (2)
Gum swelling, 7 (1) Visiting a nearby dentist, 14 (2)
Appearing of spaces, (0) Searching for solution over the Internet, 7 (1)
a

More than one answer can be chosen.

4. Discussion

Globally, the COVID-19 pandemic has been the main concern among societies because of many limitations that have occurred to everyday life. Being in high risk of infection, most dental clinics were closed, which put the patients who were receiving orthodontic treatment in a complicated situation, mainly due to the hindrance that occured to their regular visits to their orthodontist. This study aimed to assess the challenges faced by these patients and their preferred solutions to overcoming these challenges during this pandemic. Over the period of 3 weeks, the link for the questionnaire was sent to patients via WhatsApp Messenger because it is used by most patients, and made it easier to reach them during lock-down period.

The main focus was to access the percentage of patients who were still unable to complete their treatment. Most of them were receiving their treatment at the public clinic of the university and reported that the clinic being closed was the main reason for that. This can be justified by the governmental decision of mandatory closing of schools and universities and the patients being unable to afford the cost of their treatment somewhere else.

Most of the participants kept in touch with their orthodontist using many different ways, from voice calling to SMS, but more than half chose to use a mobile phone application, which is in line with a recent study assessing orthodontists’ experience that found most patients got in touch with their orthodontist using WhatsApp Messenger [11]. This reveals the importance of teleorthodontics in general and during this period specifically, to reduce unnecessary visits to the clinic while still monitoring patients from a distance [12]. Also, it might be an easier way to communicate using images instead of words and more appropriate to illustrate the problem they were facing; in addition, there are variable options of applications nowadays which can be utilized for this purpose [13].

More patients were concerned about the duration of treatment rather than the fact that their treatment might relapse; and this result is in concordance with a previous study in this regard [14]. This is expected, as the patients’ first question, even before initiating treatment is about the length of treatment time [15]. However, participants from public and private clinics had different opinions toward the closing of dental clinics, which might be linked to the difference of what each participant went through during this period and the main reason that led them to stop attending their appointments.

Participants with FAs had more problems than the RA and CA groups. This could be linked to FA causing discomfort due to the archwire poking points and irritation to the inner cheek or the tongue. FAs, also have multiple parts that could break and cause problems [16]. These two problems were the most reported issues among participants from this group and were consistent with the findings of recent similar studies [11,17]. Also, the number of these participants chose the appearance of new spaces between teeth as their main concern, as this was not explained or anticipated by them. In addition, presence of gum swelling was a common answer from this group. This is normal because gingival enlargement can accompany treatment with FAs, especially in absence of good oral hygiene [18]. On the other hand, the main problem for the RA group was orthodontic pain, although several studies have reported a lower level of orthodontic pain among these patients [19]. Most of the CA group chose the loosening of the aligner as their main problem. This led to the suggestion that patients should change to the next aligner after 2 weeks [20].

Also, participants differ between groups regarding the way of addressing their problems. FA participants used orthodontic wax to cover sharp ends, which was not an option in the other groups. Although most patients with FAs or CAs consulted with their orthodontist before managing their problems, most of the RA patients chose to look for a solution online, giving that the Internet has become an alternative way for some patients to seek information and find experiences of people who went through their same situation [21]. A considerable number of participants in all groups reported that they ignored their problems; and this could be justified by their fear of contracting COVID-19 infection and choosing not to visit any clinics.

In light of these findings, more attention should be given toward orthodontic patients and their long journey with treatment. Many acceleration methods have been reported in the literature and should be considered in clinical practice to help in shortening treatment time and avoiding problems associated with this long period [22]. Also, the type of appliance may have an important role in making the treatment process easier. In addition to their aesthetic appearance, CAs seem to be more comfortable and less complicated than FAs [23], and practitioners around the world are widening the use of CAs to treat different cases while limited cases can be treated with RAs. Thus, it is recommended to choose CA as a treatment when possible, especially during this pandemic.

On the other hand, many of the problems that patients were facing could have been solved by providing them with emergency guidelines like the British Orthodontic Society Covid-19 Orthodontic Emergency Protocol [24] and train patients to address these problems when possible. Also, orthodontists can benefit from many recommendations and guidelines that have been suggested in the literature [8] to be able to provide their patients with proper care and protect themselves as well.

The main limitation of this study is that most participants had FA and this might have affected the results derived from RA and CA groups.

5. Conclusions

The COVID-19 pandemic has significant impact on orthodontic treatments. Almost each orthodontic patient had to stop attending their appointments at some point during lock-down periods, which put them in complicated situations and fear of their treatment being delayed. Most problems were reported by patients receiving their treatment at a public clinic and by those with FAs. More attention should be given to teleorthodontics, especially during this period and for the foreseeable future; also, orthodontists should prepare their patients to deal with problems linked to their appliances when possible.

Footnotes

Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interest: Authors have completed and submitted the ICMJE Form for Disclosure of potential conflicts of interest. None declared.

Provenance and peer review: Not commissioned; Externally peer reviewed.

References

  • 1.Wu Y.C., Chen C.S., Chan Y.J. The outbreak of COVID-19: an overview. J Chin Med Assoc. 2020;83:217–220. doi: 10.1097/JCMA.0000000000000270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.World Health Organization WHO Coronavirus Disease (COVID-19) Dashboard. 2020. https://covid19.who.int/ Available at:
  • 3.Chakraborty I., Maity P. COVID-19 outbreak: migration, effects on society, global environment and prevention. Sci Total Environ. 2020;728:138882. doi: 10.1016/j.scitotenv.2020.138882. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Lai C.C., Shih T.P., Ko W.C. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus disease-2019 (COVID-19): the epidemic and the challenges. Int J Antimicrob Agents. 2020;55:105924. doi: 10.1016/j.ijantimicag.2020.105924. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Ahmed M.A., Jouhar R., Ahmed N. Fear and practice modifications among dentists to combat novel coronavirus disease (COVID-19) outbreak. Int J Environ Res Public Health. 2020;17:2821. doi: 10.3390/ijerph17082821. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Alharbi A., Alharbi S., Alqaidi S. Guidelines for dental care provision during the COVID-19 pandemic. Saudi Dent J. 2020;32:181–186. doi: 10.1016/j.sdentj.2020.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Gurzawska-Comis K., Becker K., Brunello G. Recommendations for dental care during COVID-19 pandemic. J Clin Med. 2020;9:1833. doi: 10.3390/jcm9061833. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Suri S., Vandersluis Y.R., Kochhar A.S. Clinical orthodontic management during the COVID-19 pandemic. Angle Orthod. 2020 doi: 10.2319/033120-236.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Peloso R.M., Pini N.I.P., Sundfeld Neto D. How does the quarantine resulting from COVID-19 impact dental appointments and patient anxiety levels? Braz Oral Res. 2020;34:e84. doi: 10.1590/1807-3107bor-2020.vol34.0084. [DOI] [PubMed] [Google Scholar]
  • 10.Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) J Med Internet Res. 2004;6:e34. doi: 10.2196/jmir.6.3.e34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Cotrin P., Peloso R.M., Pini N.I.P. Urgencies and emergencies in orthodontics during the coronavirus disease 2019 pandemic: Brazilian orthodontists' experience. Am J Orthod Dentofacial Orthop. 2020 doi: 10.1016/j.ajodo.2020.06.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Maspero C., Abate A., Cavagnetto D. Available technologies, applications and benefits of teleorthodontics. A literature review and possible applications during the COVID-19 pandemic. J Clin Med. 2020;9:1891. doi: 10.3390/jcm9061891. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Zotti F., Dalessandri D., Salgarello S. Usefulness of an app in improving oral hygiene compliance in adolescent orthodontic patients. Angle Orthod. 2016;86:101–107. doi: 10.2319/010915-19.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Cotrin P., Peloso R.M., Oliveira R.C. Impact of coronavirus pandemic in appointments and anxiety/concerns of patients regarding orthodontic treatment. Orthod Craniofac Res. 2020;23:455–461. doi: 10.1111/ocr.12395. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Skidmore K.J., Brook K.J., Thomson W.M. Factors influencing treatment time in orthodontic patients. Am J Orthod Dentofacial Orthop. 2006;129:230–238. doi: 10.1016/j.ajodo.2005.10.003. [DOI] [PubMed] [Google Scholar]
  • 16.Kluemper G.T., Hiser D.G., Rayens M.K. Efficacy of a wax containing benzocaine in the relief of oral mucosal pain caused by orthodontic appliances. Am J Orthod Dentofacial Orthop. 2002;122:359–365. doi: 10.1067/mod.2002.126405. [DOI] [PubMed] [Google Scholar]
  • 17.Turkistani K.A. Impact of delayed orthodontic care during COVID-19 pandemic: emergency, disability, and pain. J World Fed Orthod. 2020;9:106–111. doi: 10.1016/j.ejwf.2020.07.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.van Gastel J., Quirynen M., Teughels W. Longitudinal changes in microbiology and clinical periodontal variables after placement of fixed orthodontic appliances. J Periodontol. 2008;79:2078–2086. doi: 10.1902/jop.2008.080153. [DOI] [PubMed] [Google Scholar]
  • 19.Sergl H.G., Klages U., Zentner A. Pain and discomfort during orthodontic treatment: causative factors and effects on compliance. Am J Orthod Dentofacial Orthop. 1998;114:684–691. doi: 10.1016/s0889-5406(98)70201-x. [DOI] [PubMed] [Google Scholar]
  • 20.Drake C.T., McGorray S.P., Dolce C. Orthodontic tooth movement with clear aligners. ISRN Dent. 2012;2012:657973. doi: 10.5402/2012/657973. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Bratucu R., Gheorghe I.R., Purcarea R.M. Cause and effect: the linkage between the health information seeking behavior and the online environment—a review. J Med Life. 2014;7:310–316. [PMC free article] [PubMed] [Google Scholar]
  • 22.Miles P. Accelerated orthodontic treatment—what's the evidence? Aust Dent J. 2017;62(Suppl 1):63–70. doi: 10.1111/adj.12477. [DOI] [PubMed] [Google Scholar]
  • 23.Gao M., Yan X., Zhao R. Comparison of pain perception, anxiety, and impacts on oral health-related quality of life between patients receiving clear aligners and fixed appliances during the initial stage of orthodontic treatment. Eur J Orthod. 2020 doi: 10.1093/ejo/cjaa037. [DOI] [PubMed] [Google Scholar]
  • 24.British Orthodontic Society BOS COVID-19 Orthodontic Emergencies Protocol. 2020. https://www.bos.org.uk/COVID19-BOS-Advice/COVID19-BOS-Advice/BOS-COVID-19-Orthodontic-Emergencies-Protocol Available at:

Articles from Journal of the World Federation of Orthodontists are provided here courtesy of Elsevier

RESOURCES