The impact of socioeconomic status on telemedicine utilization during the COVID-19 pandemic among surgical clinics at an academic tertiary care center

Abstract

Background

The COVID-19 pandemic caused many surgical providers to conduct outpatient evaluations using remote audiovisual conferencing technology (i.e., telemedicine) for the first time in 2020. We describe our year-long institutional experience with telemedicine in several general surgery clinics at an academic tertiary care center and examine the relationship between area-based socioeconomic measures and the likelihood of telemedicine participation.

Methods

We performed a retrospective review of our outpatient telemedicine utilization among four subspecialty clinics (including two acute care and two elective surgery clinics). Geocoding was used to link patient visit data to area-based socioeconomic measures and a multivariable analysis was performed to examine the relationship between socioeconomic indicators and patient participation in telemedicine.

Results

While total outpatient visits per month reached a nadir in April 2020 (65% decrease in patient visits when compared to January 2020), there was a sharp increase in telemedicine utilization during the same month (38% of all visits compared to 0.8% of all visits in the month prior). Higher rates of telemedicine utilization were observed in the two elective surgery clinics (61% and 54%) compared to the two acute care surgery clinics (14% and 9%). A multivariable analysis demonstrated a borderline-significant linear trend (p = 0.07) between decreasing socioeconomic status and decreasing odds of telemedicine participation among elective surgery visits. A sensitivity analysis to examine the reliability of this trend showed similar results.

Conclusion

Telemedicine has many patient-centered benefits, and this study demonstrates that for certain elective subspecialty clinics, telemedicine may be utilized as the preferred method for surgical consultations. However, to ensure the equitable adoption and advancement of telemedicine services, healthcare providers will need to focus on mitigating the socioeconomic barriers to telemedicine participation.

Throughout 2020 and continuing into 2021, the COVID-19 pandemic has caused unprecedented delays and disruptions in patients’ access to care. In response to the pandemic, many surgical providers began offering telemedicine appointments in the outpatient setting to mitigate the spread of COVID-19 while maintaining access to care for patients. Telemedicine has allowed patients to consult with surgeons remotely even as elective operations were being suspended across the USA. There are many established benefits to telemedicine including the reduction in travel costs [1–4], the potential to increase access to subspecialty care for rural communities [3–6], and the ease of including family members in important medical conversations [1, 7, 8]. However, there remain many potential barriers in the equitable and effective delivery of telemedicine services. Telemedicine requires patients to have access to devices with audiovisual conferencing capabilities as well as stable high-speed internet, both of which may not be readily available to socioeconomically disadvantaged communities [8, 9].

The initial widespread adoption of telemedicine was a necessary response to the global pandemic. However, it remains to be seen how telemedicine utilization among patients and surgeons will change as communities ease pandemic-related restrictions, and whether these patterns will differ between separate subspecialties. Furthermore, new adoption and advancements in medical technology have historically resulted in disparities in access and health outcomes for marginalized populations [9–13]. In this study, we describe patterns in telemedicine utilization during the first year of the COVID-19 pandemic and examine how socioeconomic status may impact likelihood of telemedicine participation among several acute care and elective surgery outpatient clinics at a large tertiary care center.

Methods

This study is a retrospective review of our institutional experience with telemedicine in the University of Washington Department of Surgery. We reviewed all patient visits that were completed in 2020 among four subspecialty clinics. At the beginning of the COVID-19 pandemic, we observed that our minimally invasive surgery and bariatric clinics were among the highest utilizers of telemedicine at our institution. To contextualize the variable use of telemedicine among differing surgical subspecialties, we chose to also review telemedicine utilization in two acute care & general surgery clinics, which were the two surgical clinics that conducted the highest number of patient visits at our academic tertiary center in 2020. Thus, the four subspecialty clinics included in this study were: (1) a university hospital minimally invasive surgery clinic (MIS); (2) a university hospital bariatric surgery clinic (Bariatrics); (3) a university hospital acute care & general surgery clinic (University ACS); and (4) a satellite campus acute care & general surgery clinic (Satellite ACS). All in-person and telemedicine patient visits conducted with a surgical provider in these four clinics from January 1st, 2020 through December 31st, 2020 were included. Telemedicine visits were defined as clinic appointments that were completed remotely using audiovisual conferencing capabilities. Patient phone calls, phone visits, procedural visits, and encounters in which patients were not seen by a surgical provider were excluded.

To examine the association between socioeconomic status (SES) and patient participation in telemedicine we needed a suitable indicator for socioeconomic status. While prior studies have identified several individual measures of SES including education, occupation, income, and wealth [14, 15], these individual measures are not readily accessible in most health systems, and their use in public health research is limited [15, 16]. Community socioeconomic measures (such area-based poverty rate and median family income), however, are widely available and have been shown to reliably detect socioeconomic gradients in a wide variety of population health outcomes [13, 17–19]. To determine area-based socioeconomic measures for each patient visit, patient addresses were collected and matched to the designated US census tract, through a process known as geocoding [20]. Census tracts are relatively permanent county subdivisions that generally contain between 2500 and 8000 residents and have a unique 11-digit geographic code assigned by the U.S. Census Bureau [21]. Thus an 11-digit tract code was matched to each patient visit, and this geocoded data were then used to link patient visits to area-based socioeconomic measures including tract-level poverty percentage (available through the U.S. Census Bureau’s 2015 American Community Survey) [20] and tract-level estimated median family income (available through the 2020 Federal Financial Institutions Examination Council) [22]. Postal office boxes could not be accurately geocoded and thus were excluded from this analysis.

Tract-level poverty percentage (PP), or poverty rate, is defined as the percent of individuals within a census tract whose income falls below the poverty line. For this study, tract-level poverty percentage was grouped into the following a priori categories: 0–4.9%, 5–9.9%, 10.0–14.9%, 15–19.9%, and ≥ 20% based on previously published methodology [15, 17]. Tract-level estimated median family income (eMFI) is the expected median family income of a specific census tract as determined annually by the Federal Financial Institutions Examination Council [22]. For this study, tract-level eMFI was grouped empirically into the following quintiles to perform a sensitivity analysis (described in the following paragraph): ≥ 150.0 K, 125.0–149.9 K, 100.0–124.9 K, 75.0–99.9 K, and < 75.0 K.

Demographic data including age, sex, race, primary language, insurance type, and residential address were collected for the third quarter of the 2020 calendar year (i.e., July 1st, 2020, through September 30th, 2020) for all visits. This three-month period was selected to capture a representative sample of patients participating in telemedicine after a period of acclimation following the initial implementation of telemedicine at our institution in April 2020. At the beginning of the pandemic, the adoption of telemedicine (including staff training, technological troubleshooting, and scheduling processes) was individualized for each clinic. After the first few months, best practices began emerging and more standardized approaches were implemented such as the embedding of secure telemedicine links for each patient visit within our electronic medical record. Patient addresses were geocoded to the census tract level as previously described, and also used to determine the driving distance from the associated clinic for each patient visit.

Statistical analyses were performed using Stata/SE version 16.1 (Stata Corp LP, College Station, TX). Patient characteristics were summarized using frequency distributions for categorical variables, means for normally distributed continuous variables, and medians for non-normally distributed continuous variables. A multivariable regression was performed to determine the association between telemedicine participation and census tract poverty category while adjusting for age, sex, race, and distance traveled. To determine the reliability of potential trends between census tract poverty category and telemedicine participation, a sensitivity analysis was performed using median tract family income as the predictive socioeconomic measure, while also adjusting age, sex, race, and distance traveled. For both adjusted analyses, a test of linear trend was used to test the hypothesis that lower socioeconomic status (defined as increasing poverty tract percentage and decreasing tract median income) would be associated with a lower odds of telemedicine participation. Two-sided p values of less than 0.05 were considered statistically significant.

Results

Patient visit characteristics

During the 2020 calendar year, 9332 total patient visits (combined average of 778 visits per month) were conducted among the four surgical clinics we examined. Of these, 7165 occurred between the two general and acute care surgery clinics (University ACS and Satellite ACS) while the two elective surgical clinics (MIS and Bariatrics) totaled 2167 visits. Examining demographic data from the 2190 patient visits in the third quarter of 2020, similar distributions in age, insurance type, and tract-level PP were seen between in-person visits and telemedicine visits (Table 1). The median age was 55 for both groups. Using geocoded area-based socioeconomic data from the 2015 U.S. Census, we found that in-person visits and telemedicine visits had similar median tract-level PPs (10.3% vs. 10.7%, respectively; p = 0.37). Median tract-level eMFI, however, was higher for in-person patient visits compared to telemedicine patient visits ($102.6 K vs. $95.0 K, respectively; p < 0.01).

Table 1.

Patient & visit characteristics (n = 2190)

	In-Person (n = 1742)		Telemedicine (n = 448)
		95% CI		95% CI
Clinic type, n (%)
Acute care & general surgery	1477 (85)	82–86	139 (31)	27–36
MIS & bariatric surgery	265 (15)	13–17	309 (69)	64–73
Age (years), n (%)
16–29	155 (9)	8–10	25 (6)	4–8
30–39	284 (16)	15–18	69 (15)	12–19
40–49	271 (16)	14–17	84 (19)	15–23
50–59	334 (19)	17–21	87 (19)	16–23
60–69	362 (21)	19–23	110 (25)	21–29
70–79	336 (19)	17–21	73 (16)	13–20
Age (years), median (SD)	55 (39–67)	–	55 (42–66)	–
Female, n (%)	806 (46)	44–49	248 (55)	51–60
Race/ethnicity, n (%)
White	1286 (74)	72–76	342 (76)	72–80
Black or African American	127 (7)	6–9	29 (6)	4–9
Asian	168 (10)	8–11	18 (4)	2–6
Hispanic/latinx	91 (5)	4–6	28 (6)	4–9
All other races	27 (1)	1–2	6 (1)	0–3
Missing	43 (2)	2–3	25 (6)	3–8
Preferred Language, n (%)
English	1647 (95)	93–96	440 (98)	97–99
Spanish	40 (2)	2–3	2 (0)	0–2
All others	55 (3)	2–4	6 (1)	0–3
Insurance type, n (%)
Private	840 (48)	46–51	224 (50)	45–55
Medicare	544 (31)	29–33	153 (34)	30–39
Medicaid/financial assistance	325 (19)	17–21	62 (14)	11–17
Other	26 (1)	1–2	9 (2)	1–4
Driving distance (mi), median (IQR)	11.8 (4.3–33.9)	–	34.4 (14.3–83.2)	–
Tract poverty percentage, n (%)
< 5.0%	218 (13)	11–14	50 (11)	8–14
5.0–9.9%	593 (34)	32–36	140 (31)	27–36
10.0–14.9%	382 (22)	20–24	109 (24)	20–29
15.0–19.9%	205 (12)	10–13	47 (11)	8–14
≥ 20.0%	260 (15)	13–17	66 (15)	12–18
Missing	84 (5)	4–6	36 (8)	6–11
% Below poverty line, median (IQR)	10.3 (6.5–15.8)	–	10.7 (7.0–15.9)	–
Tract median family income, $, n (%)
≥ 150 K	290 (17)	15–18	52 (12)	9–15
125.0–149.9 K	294 (17)	15–19	62 (14)	11–17
100.0–124.9 K	299 (17)	15–19	62 (14)	11–17
75.0–99.9 K	440 (25)	23–27	124 (28)	24–32
< 75.0 K	333 (19)	17–21	112 (25)	21–29
Missing	86 (5)	4–6	36 (8)	6–10
Tract median income, median (IQR)	102,639 (79,805–138,169)		95,025 (73,320–129,577)

The median driving distance associated with in-person visits was 11.8 miles, compared to 34.4 miles for telemedicine visits (Table 1). Female patients comprised a smaller proportion of in-person visits when compared to telemedicine visits (46% vs. 55%, respectively). Race characteristics were largely similar between the two groups, except Asian patients comprised a larger proportion of in-person visits when compared to telemedicine visits (10% vs. 4%). Patients who spoke English as a primary language also comprised a smaller percentage of in-person visits when compared to telemedicine visits (95% vs. 98%).

Telemedicine utilization patterns

The highest monthly volume of patient visits occurred in January 2020, in which 995 visits were conducted among the four clinics prior to the implementation of pandemic-related restrictions. Following the first announcements of statewide and national restrictions in February and March of 2020, including recommendations to limit all non-essential surgeries from the U.S. Center for Medicare and Medicaid Services on March 18th [23], total visits among all four clinics reached a nadir of 352 in the month of April, representing a 65% decrease when compared to visits in January (Fig. 1).

Fig. 1.

At the start of the COVID-19 pandemic, the total number of patient visits across four academically affiliated surgical clinics declined by 65% between January and April of 2020 (solid line). The subsets for general and acute care surgery visits (dashed line), and elective MIS & bariatric surgery visits (dotted line) are depicted

While April had the lowest monthly total of outpatient visits, it also marked a sharp increase in telemedicine utilization with 38% of all patient visits from this month employing remote audiovisual conferencing capabilities (Fig. 2a). Prior to this, there were no telemedicine visits in January or February, and only 0.4% of outpatient visits in March utilized telemedicine. From April through December, higher rates of telemedicine utilization were seen in the MIS and Bariatrics clinics (61% and 54%, respectively) when compared to the University ACS and Satellite ACS clinics (14% and 9%, respectively) (Fig. 2b). This discrepancy was even more pronounced when examining only new patient consultations in the third quarter of 2020. The MIS and Bariatrics clinics utilized telemedicine for 61% and 76% of new patient visits, while the University ACS and Satellite ACS clinics both utilized telemedicine in only 4% of new patient consultations. In reviewing each clinic’s approach to adopting telemedicine, it is important to note that both the elective MIS and Bariatric clinics adopted a non-selective approach that strongly encouraged all newly referred patients to participate in telemedicine while the two acute care and general surgery clinics adopted a selective approach to utilizing telemedicine that screened patients based on primary complaint and surgical diagnosis.

Fig. 2.

Telemedicine utilization increased sharply in April of 2020 (A). The MIS and bariatric surgery clinics utilized telemedicine for a higher percentage of patient visits when compared to the University ACS and Satellite ACS clinics (B)

Effect of area-based socioeconomic measures on telemedicine participation

94.5% of the 2190 patient visits from the third quarter of 2020 were successfully geocoded and linked to area-based tract-level poverty measures. The odds of telemedicine participation were calculated for each tract-level poverty category, adjusting for age, sex, race, and driving distance (Table 2). Due to the aforementioned discrepancy in telemedicine utilization rates, the acute care and general surgery patient visits were analyzed separately from the elective MIS and bariatric patient visits. Acute care and general surgery patient visits showed no significant linear association between census tract-level PP and odds of telemedicine participation (p = 0.65). For MIS and bariatric patient visits, there was a non-significant trend toward increasing tract-level PPs and decreasing odds of telemedicine participation (p = 0.07). Stated differently, in the two elective surgical clinics that had the highest telemedicine utilization rates, there was some indication that likelihood of telemedicine participation seemed to decrease with increasing tract-level poverty percentage.

Table 2.

Adjusted odds of participating in telemedicine by poverty percentage categories

	Acute care & general surgery (n = 1616)		MIS & bariatric surgery (n = 574)
	OR	95% CI	OR	95% CI
Age categories (years)
16–29	Ref.	Ref.	Ref.	Ref.
30–39	0.90	0.40–2.01	0.99	0.36–2.75
40–49	1.15	0.52–2.52	1.17	0.43–3.23
50–59	1.15	0.54–2.44	0.78	0.29–2.11
60–69	0.94	0.44–2.01	1.02	0.38–2.70
70–79	0.98	0.46–2.1	0.85	0.31–2.32
Sex
Male	1.10	0.75–1.60	1.06	0.73–1.56
Female	Ref.	Ref.	Ref.	Ref.
Race/ethnicity
White	Ref.	Ref.	Ref.	Ref.
Black or African American	0.24	0.06–0.96	0.68	0.37–1.24
Asian	0.51	0.24–1.08	0.98	0.37–2.56
Hispanic/Latinx	0.53	0.19–1.50	1.76	0.80–3.89
All other races	1.71	0.47–6.21	0.41	0.11–1.58

	OR*	95% CI	OR**	95% CI
Tract Poverty Percentage
< 5.0%	Ref.	Ref.	Ref.	Ref.
5.0–9.9%	1.12	0.61–2.09	0.87	0.45–1.66
10.0–14.9%	1.46	0.77–2.77	0.98	0.50–1.93
15.0–19.9%	0.93	0.41–2.07	0.74	0.35–1.58
≥ 20.0%	1.32	0.64–2.72	0.53	0.26–1.09
Driving distance (10 mi)	1.01	1.002–1.02	1.00	0.99–1.00

Test of linear trend for tract-level poverty percentage categories not significant for acute care & general surgery visits (*p = 0.65) or elective MIS & bariatric surgery visits (**p = 0.07)

To test the reliability of the trends seen above, a sensitivity analysis was performed utilizing tract-level eMFI as the area-based socioeconomic measure. The odds of telemedicine participation were calculated for each tract-level eMFI category, adjusting for age, sex, race, and driving distance (Table 3). No significant association was seen between eMFI and likelihood of telemedicine participation in the acute care and general surgery patient visits. Similar to the primary analysis that employed PP categories, a statistically non-significant association between lower eMFI and lower odds of telemedicine participation was seen in the MIS and bariatric patient visits (p = 0.08).

Table 3.

Adjusted odds of participating in telemedicine by median family income categories

	Acute care & general surgery (n = 1616)		MIS & bariatric surgery (n = 574)
	OR	95% CI	OR	95% CI
Age categories (years)
16–29	Ref.	Ref.	Ref.	Ref.
30–39	0.90	0.41–2.00	0.95	0.35–2.60
40–49	1.10	0.50–2.42	1.08	0.40–2.96
50–59	1.20	0.53–2.38	0.79	0.30–2.12
60–69	0.90	0.42–1.93	1.00	0.38–2.64
70–79	0.95	0.45–2.00	0.89	0.33–2.41
Sex
Male	1.14	0.78–1.67	1.05	0.71–1.53
Female	Ref.	Ref.	Ref.	Ref.
Race/ethnicity
White	Ref.	Ref.	Ref.	Ref.
Black or African American	0.24	0.07–0.91	0.63	0.35–1.16
Asian	0.51	0.24–1.10	0.95	0.37–2.42
Hispanic/Latinx	0.49	0.17–1.41	1.76	0.80–3.87
All other races	1.77	0.49–6.34	0.39	0.10–1.55

	OR*	95% CI	OR**	95% CI
Tract median family income, $, n (%)
≥ 150 K	Ref.	Ref.	Ref.	Ref.
125.0–149.9 K	1.22	0.64–2.31	1.67	0.75–3.72
100.0–124.9 K	1.01	0.51–2.00	0.93	0.46–1.87
75.0–99.9 K	1.27	0.69–2.32	1.07	0.57–2.02
< 75.0 K	1.80	0.98–3.32	0.81	0.43–1.52
Driving distance (10 mi)	1.01	1.00–1.01	1.00	0.99–1.01

Test of linear trend for tract median income categories not significant for acute care & general surgery visits (*p = 0.23) or elective MIS & bariatric surgery visits (**p = 0.08)

Though it was not the primary aim of this study, both the PP-adjusted and the eMFI-adjusted analyses showed a statistically significant association between Black/African American race and lower odds of telemedicine participation in the acute care and general surgery patient visits (OR 0.24, CI 0.06–0.96) (Table 2). This was an unexpected finding. No significant association between race and telemedicine participation was seen in the MIS and bariatric patient visits. Furthermore, neither age nor sex was significantly associated with telemedicine participation (Tables 2, 3).

Discussion

We examined patterns of telemedicine utilization among four surgical clinics affiliated with a large academic tertiary care center during the COVID-19 pandemic. We found that our elective minimally invasive and bariatric surgery clinics utilized telemedicine at a much higher rate than our acute care and general surgery clinics. Additionally, these two elective surgery clinics maintained high rates of telemedicine utilization throughout 2020, even as restrictions on elective surgery and inpatient evaluations eased during the summer and fall.

The discrepancy in telemedicine utilization among the clinics we examined is likely due to a variety of reasons related to the patients, clinic staff, and surgical providers. As mentioned previously, the elective surgery clinics adopted a policy early during the pandemic that all new referrals would be offered and encouraged to participate in telemedicine appointments. This was facilitated by the fact that the MIS and bariatric clinics were staffed by fewer attending surgeons (2–3 surgeons each) with a focus on treating hernias, obesity, benign foregut problems, and foregut malignancies. For many uncomplicated hernia diagnoses as well as most benign and malignant foregut diagnoses, the MIS surgeons found that preoperative evaluation and counseling could be conducted completely virtually, leaving the surgical consent to be signed on the day of surgery when patients would be seen in-person for the first time. In the bariatric surgery clinic, patients that were enrolled in the surgical weight loss program following a virtual consultation would be seen at least once in-person to document an initial weight and undergo a physical exam. For patients that completed a multi-disciplinary bariatric workup and were felt to be appropriate candidates for a weight loss procedure, the final preoperative visit was generally conducted virtually through telemedicine. The acute care and general surgery clinics, on the other hand, were staffed by a larger team of surgeons (5–6 surgeons each) that combined to see not only a greater volume of patients, but also a broader variety of illnesses. Each surgeon submitted individual preferences outlining the diagnoses they were comfortable evaluating virtually. This eventually led to the development of a complicated matrix used to guide the office staff in offering and scheduling telemedicine appointments, which resulted in inconsistent telemedicine utilization, even for those diagnoses initially deemed appropriate for telemedicine.

Furthermore, during the course of this study we found that the importance of providing office staff with the time and tools to advocate for telemedicine cannot be overstated. The office staff in the elective MIS and bariatric clinics became remarkable champions of telemedicine during the COVID-19 pandemic. One of the schedulers took on the responsibility of coaching new patients through the installation and use of our teleconferencing platform, routinely conducting technological trial runs with patients prior to their scheduled appointments to identify and troubleshoot potential problems. In contrast, office staff in the acute care and general surgery clinics generally felt they had neither the time nor expertise to promote telemedicine beyond offering it to patients as an alternative to in-person visits.

Using a geocoding approach to link data from our medical records with area-based socioeconomic measures, we saw a trend toward lower socioeconomic status being associated with decreased likelihood of telemedicine participation in our elective surgery clinics where telemedicine was utilized for over half of all patient visits. Though not statistically significant, this trend was consistent across two analyses utilizing different area-based socioeconomic measures (census tract PP and eMFI). This data raises the concern that patients from areas of lower socioeconomic status may experience more barriers to accessing non-urgent surgical care via telemedicine, perhaps related to lack of access to necessary technology such as stable high-speed internet or devices with audio-visual conferencing capabilities.

Additionally, even with the appropriate technology patients from communities with higher levels of poverty may be more reluctant to engage in telemedicine, as this technology allows providers to peer into their home and living environment [1, 8, 24]. This is an important consideration when evaluating the many social and cultural implications of utilizing telemedicine in the outpatient setting. The risk of potentiating implicit bias through viewing a patient’s home environment may be especially relevant in specialties such as bariatric and transplant surgery where management decisions are based in part on social factors.

Although it was not the intended aim of our study, we found evidence of racial disparities in telemedicine utilization in our acute care and general surgery clinics. When adjusted for age, sex, socioeconomic status, and distance traveled, Black or African American patients had significantly lower odds of telemedicine participation compared to White patients. This trend, although not statistically significant, was also seen with Hispanic and Asian patient visits. We recognize that systemic bias is a major problem in healthcare, and these findings may highlight bias within our system. It is important for us to further investigate these findings to delineate the reasons for these disparities in telemedicine utilization.

It is important to note that the patient population examined in this study may not reflect the population for other regions and healthcare systems. The patients captured in this study were predominantly White, English-speaking, and tended to reside in census tracts with estimated median family incomes higher than the national average (Table 1). In 2020, the national median family income was $78.5 K while the median eFMI in this study was $102.6 K and $95.0 K among in-person and telemedicine visits, respectively. This disproportionate representation of higher socioeconomic groups may have limited our ability to detect a significant association between socioeconomic measures and telemedicine utilization in both of our adjusted analyses.

The COVID-19 pandemic has caused massive shifts in the way surgeons utilize remote teleconferencing to work and interact with patients. While this technology was initially implemented to promote social distancing, there are many patient-centered benefits to telemedicine that support its continued use and advancement. This study demonstrates that for certain elective subspecialty clinics, telemedicine can be utilized as a preferred method for meeting and consulting with surgical patients. Moving forward, we intend to continue offering and advancing telemedicine services for our patients with an enhanced focus on how we can promote equity in telemedicine utilization.

Declarations

Disclosures

Baraka Gitonga and Jay Zhu, Alex W. Lois, Judy Y. Chen-Meekin, Estell J. Williams, Saurabh Khandelwal, Rocio Carrera Ceron, Brant K. Oelschlager, and Andrew S. Wright have no conflicts of interest or financial ties to disclose.