Abstract
Objective
Persisting neurological symptoms after COVID‐19 affect up to 10% of patients and can manifest in fatigue and cognitive complaints. Based on recent evidence, we evaluated whether cerebral hemodynamic changes contribute to post‐COVID syndrome (PCS).
Methods
Using resting‐state functional magnetic resonance imaging, we investigated brain perfusion and oxygen level estimates in 47 patients (44.4 ± 11.6 years; F:M = 38:9) and 47 individually matched healthy control participants. Group differences were calculated using two‐sample t‐tests. Multivariable linear regression was used for associations of each regional perfusion and oxygen level measure with cognition and sleepiness measures. Exploratory hazard ratios were calculated for each brain metric with clinical measures.
Results
Patients presented with high levels of fatigue (79%) and daytime sleepiness (45%). We found widespread decreased brain oxygen levels, most evident in the white matter (false discovery rate adjusted‐p‐value (p‐FDR) = 0.038) and cortical grey matter (p‐FDR = 0.015). Brain perfusion did not differ between patients and healthy participants. However, delayed patient caudate nucleus perfusion was associated with better executive function (p‐FDR = 0.008). Delayed perfusion in the cortical grey matter and hippocampus were associated with a reduced risk of daytime sleepiness (hazard ratio (HR) = 0.07, p = 0.037 and HR = 0.06, p = 0.034). Decreased putamen oxygen levels were associated with a reduced risk of poor cognitive outcome (HR = 0.22, p = 0.019). Meanwhile, lower thalamic oxygen levels were associated with a higher risk of cognitive fatigue (HR = 6.29, p = 0.017).
Interpretation
Our findings of lower regional brain blood oxygen levels suggest increased cerebral metabolism in PCS, which potentially holds a compensatory function. These hemodynamic changes were related to symptom severity, possibly representing metabolic adaptations.
Introduction
Coronavirus disease 2019 (COVID‐19) can present with neurological complications during the acute infection, 1 but also lead to lasting neurological manifestations (>3 months). Known as post‐COVID syndrome (PCS), recent studies found that these long‐term symptoms include clinically relevant fatigue, 2 , 3 , 4 , 5 and sleep disorders. 3 , 6 Cognitive functions were also frequently affected in PCS on both screening 2 and extensive neuropsychological testing, with marked cognitive slowing 7 and impairments of memory, attention, executive functions and language. 3 , 6 , 8
To develop targeted treatment strategies, it is essential to understand the neurobiological underpinnings of PCS. During the acute infection, pathological effects of increased vascular inflammation, system‐wide vascular dysfunction and COVID‐19 spike protein‐binding sites in intracranial blood vessels leading to blood–brain barrier dysfunction are thought to play a role. 9
In PCS, potential brain mechanisms include vascular dysfunction, hypoxemia, hypometabolism and functional changes in brain networks. 8 , 10 , 11 , 12 Recently, COVID‐19‐related effects on the cerebral blood system have gained more attention in the attempt to explain post‐COVID symptoms. 13 For instance, endothelial damage observed in acute COVID‐19 infection and PCS 14 can result in perturbations of cerebral perfusion and oxygen levels. An overall reduced cerebral blood flow has also been found in patients with non‐COVID chronic fatigue syndrome. 15
Information about cerebral haemodynamics, including tissue perfusion, can be extracted noninvasively from resting‐state functional MRI (rs‐fMRI) without requiring exogenous contrast agents. Cerebral perfusion is reflected in delays in systemic low‐frequency oscillations (sLFOs) of the blood oxygenation level‐dependent (BOLD) signal between the brain parenchyma and a reference region, such as the dural venous sinuses. 16 Other properties of these sLFOs are closely related to blood oxygenation and cerebral blood volume. 17
Given that post‐COVID patients frequently have neurological sequelae, robust MRI measures are needed to monitor persistent symptoms and gain further insight into long‐term outcomes. In this exploratory study, we therefore aim to (1) characterise the cerebrovascular effects of PCS using rs‐fMRI and (2) examine how changes to cerebral perfusion and estimated blood oxygen levels relate to clinical measures, including post‐COVID cognitive symptoms, fatigue and daytime sleepiness.
Methods
Participants
The CAMINO study 3 recruited 50 patients from two neurological post‐COVID outpatient clinics at Charité–Universitätsmedizin Berlin and 51 age‐, sex‐ and education‐matched healthy participants (HP) without previous COVID‐19 infection or history of neurological or psychiatric diseases between April and November 2021. Inclusion criteria for PCS patients were (1) a confirmed SARS‐CoV‐2 infection, (2) post‐infectious neurological symptoms ≥3 months and (3) no history of relevant neurological or psychiatric disease prior to COVID‐19. Three patients and four HP were excluded from the analysis due to lack of fulfilling all inclusion criteria (Table 1), resulting in a final cohort of 47 PCS patients and 47 HP. The study was approved by the local ethics committee (EA2/007/21) and conducted in accordance with the Declaration of Helsinki. All participants received neuropsychological testing and MRI scanning on the same clinical visit date and provided written informed consent.
Table 1.
Reason for exclusion | Study group | N |
---|---|---|
MRI quality issues | Healthy participants | 3 |
Incomplete clinical data | Healthy participants | 1 |
Received neurological diagnosis relevant to current analysis during study participation | Patient | 1 |
Post‐COVID neurological symptoms for less than 3 months | Patient | 1 |
No resting‐state fMRI scan | Patient | 1 |
fMRI, functional magnetic resonance imaging; MRI, magnetic resonance imaging.
Cognitive, daytime sleepiness and fatigue assessments
We evaluated neuropsychiatric and cognitive functions as detailed previously, 3 including the Montreal Cognitive Assessment (MoCA) as a screening tool for cognitive impairment and the Trail Making Test‐B (TMT‐B) for executive function and visual attention. Additionally, we assessed self‐reported daytime sleepiness using the Epworth Sleepiness Scale (ESS) and fatigue using the Fatigue Scale for Motor and Cognitive Function (FSMC).
MRI analysis
Image acquisition and preprocessing
MRI sequences were acquired on a 3T Siemens Magnetom PRISMA scanner (Erlangen, Germany) using a 64‐channel head coil, including (1) a high‐resolution 3D T1‐weighted magnetization prepared rapid gradient echo (MPRAGE) and (2) a multiband resting‐state fMRI, 18 , 19 , 20 and two spin‐echo sequences for field map distortion correction (Table 2).
Table 2.
Sequence | Imaging parameters |
---|---|
3D T1‐weighted magnetisation prepared rapid gradient echo (MPRAGE) | 0.9 mm isotropic voxel resolution, matrix size = 256 × 256, 192 slices, TR = 2200 ms, TE = 2.3 ms, TI = 900 ms, flip angle = 8 degrees |
Resting‐state fMRI (BOLD) | 2 mm isotropic voxel resolution, matrix size = 104 × 104, 72 slices, TR = 800 ms, TE = 37 ms, FA = 52 degrees, 720 volumes, multiband acceleration factor = 8 |
Spin echo | Two acquisitions with opposite phase encoding directions, voxel size = 2 × 2 × 2 mm, TR = 8000 ms, TE = 66 ms, flip angle = 90 |
BOLD, blood oxygen level dependent; fMRI, functional magnetic resonance imaging; TE, echo time; TI, inversion time; TR, repetition time.
Preprocessing of rs‐fMRI data were performed with FSL (https://fsl.fmrib.ox.ac.uk/fsl) and AFNI (https://afni.nimh.nih.gov/afni), which included discarding the first four volumes for magnetization equilibrium, realigning the volume to time series mean and regressing the effect of three rigid body translations and three rotations out of the data. Subsequently, images were despiked, spatially smoothed (6 mm Gaussian kernel) and band‐pass filtered (0.01–0.15 Hz).
Brain perfusion maps
For extracting rs‐fMRI blood perfusion and oxygen level estimates, we used ‘rapidtide’, a set of Python tools for finding time‐lagged correlations (https://github.com/bbfrederick/rapidtide), to generate the BOLD delay maps.
Each voxel of the BOLD delay maps was assigned the value of the time shift that achieves maximum cross‐correlation (MCC) between the BOLD signal in the large venous sinuses (the reference signal) and the voxel's time series. 21 In these maps, the coefficient reflects the degree of similarity between a voxel's BOLD signal and the BOLD signal in venous blood. Rapidtide sets significance thresholds for the cross‐correlation using a shuffling procedure (10,000 times) to calculate the distribution of null correlation values. To determine the highest cross‐correlation coefficient, we shifted the time course from −5 to +5 s, where the peak of the correlation function between these lag times is fitted. This time course encompasses physiological to minimally pathological hemodynamic lags in the BOLD signal. 16
As BOLD delay is a measurement of the time, it requires for the BOLD sLFOs to reach a region in the brain in comparison with the venous sinus signal, we normalised each participant's regional delay time to their own cortical grey matter BOLD delay time using the following equation:
(1) |
Each regional BOLD delay time measure per person is in relation to the individual's perfusion time, giving the opportunity for better group comparison evaluations, as inter‐individual fluctuations are minimized.
The cortical grey matter BOLD delay time is then naturally not subtracted by itself, rather it is effectively used as a threshold for the rest of the regional BOLD delay times. Thus, negative BOLD delay times are observed when the BOLD signal is detected in a region slower than its detection in the cortical GM (i.e. perfusion of blood reaches a region after reaching the cortical GM). To keep the sign convention (i.e. negative delay times = slower arrival of blood), we multiplied all cortical GM values by −1 to allow for consistent interpretation of BOLD delay times. Therefore, for the rest of the study, we refer to more negative delay times as ‘slower’ and more positive delay times as ‘faster’ perfusion.
Oxygen level estimates (OLE)
The MCC coefficient between the reference signal and the voxel's time series reflects the degree of similarity between the voxel's BOLD signal and the BOLD signal in venous blood (from the venous sinus reference region). 22 Voxel‐wise maps of MCC coefficients were created from rapidtide. The MCC coefficients would be the equivalent of the ‘R’ value in a linear regression model. Thus, as a method to evaluate the goodness of fit of the BOLD delay signal with the venous blood BOLD delay signal, we extracted maps of the MCC coefficient values from each voxel.
Squaring the MCC metric is the equivalent to an R 2 value, or goodness of fit, between the voxel and reference time series. These squared values were then multiplied by 100% (Equation 2) to give a relative percent estimate of oxygen in the blood vessels in each brain region and is referred to in this study as the ‘oxygen level estimate’ (OLE).
(2) |
OLE measures how closely related the regional blood oxygen signal and the venous blood oxygen signal are. It is therefore closely related to the proportion of deoxygenated blood signal in that voxel. Calculating mean OLE per parcellation represents the percent signal of deoxygenated blood in each parcellation. Therefore, in this study, we refer to higher mean OLE values as ‘less oxygenated’ and lower mean OLE values as ‘oxygenated’.
Brain parcellation and co‐registration
Following reorientation and co‐registration of the MPRAGEs to MNI‐152 standard space with fslreorient2std and FSL flirt, we parcellated whole brain white (WM) and grey matter (GM) using CAT12 (https://neuro‐jena.github.io/cat/) and deep grey matter structures using FSL flirt. We used these parcellations and fslutils to create masks of the cortical grey matter and total deep grey matter (DGM, also included the brainstem).
Lastly, we calculated field maps (fsl_prepare_fieldmap), applied EPI distortion correction to each individual BOLD delay and OLE maps. These were subsequently co‐registered to each MPRAGE, and extracted regional mean metrics for BOLD delay (in seconds) and OLE (as percentages) were calculated (Fig. 1).
Statistical analysis
Cohort demographics were statistically evaluated using the appropriate parametric and nonparametric tests. Regional BOLD delay and OLE metrics were compared between groups using Welch two‐sample t‐tests and Wilcoxon rank‐sum tests using Cohen's d as effect size. To test for associations, we used multivariable linear regression models with age and sex as covariates. Additionally, we examined whether these associations differed between patients and HP using group as interaction effect and report p group. Two‐tailed p‐values were adjusted for multiple comparisons using false discovery rate (p‐FDR).
Explorative multivariable Cox regression analyses were performed to determine which imaging metrics were associated with an increased risk of one of the following in post‐COVID patients: (1) high daytime sleepiness (ESS ≥ 13), (2) high fatigue (FSMC ≥ 53 + 1SD, cognitive ≥ 34 + 1SD, motor ≥ 32 + 1SD), (3) poor MoCA scores (≤28), 23 or (4) poor attention (TMT‐B ≥ 59 s + 1SD). 24 Hazard ratios were first calculated separately for only BOLD Delay (HR D ) and only OLE (HR o ), to evaluate each regional mean metric without possible overlaps in regional metrics (i.e. collinearity) prior to investigating full hazard ratio (HR) models using all significant metrics from the HR D and HRo analyses.
For explorative multivariable Cox regression analyses, initial Cox proportional hazards regression models were performed with post‐acute COVID infection duration (in months) as the time dependent strata, using the Andersen and Gill method. 25
(3) |
where t is the post‐acute COVID infection duration, h 0 is the baseline (healthy group) hazard, h(t) is the hazard function determined by a set of z covariates (x 1, x 2, …, x z ) and the coefficients (b 1, b 2, …, b z ) measure the effect size of covariates (hazard ratios).
The hazard ratios (HRs) of each covariate multiply the effects on the baseline hazard. Thus, each HR is calculated by holding all covariates constant versus the covariate of interest. 26
For the full HR models, regional oxygen levels were binarised to give labels based on OLE values greater than the mean OLE of the patient cohort (less oxygenated) and OLE of less than or equal to the mean were denoted as normal (oxygenated). This is also performed in the same way for BOLD delay binarised labels. Regional BOLD delay normalized values greater than the regional mean BOLD delay of the patient cohort was deemed to be ‘faster’ perfused (less delayed) versus lower or equal to the mean regional BOLD delay values were denoted as ‘slower’ (more delayed).
Patients with missing clinical scores were excluded from each Cox regression modelling. For all analyses, p < 0.05 was considered statistically significant.
All data were analysed and plotted using Rv4.3.0 (https://www.r‐project.org/).
Results
Cohort demographics
Table 3 illustrates the well‐matched PCS patient and HP cohorts. No significant differences in group‐wise comparisons of sex or age were found. However, higher levels of fatigue and sleepiness scores were found in PCS compared to HP. It is evident by the percentages of hospitalizations and oxygen treatment, that some of our PCS patients did suffer a severe clinical disease course, however in line with other larger studies. 27 , 28 PCS symptoms in our cohort, during the post‐recovery stage, showed relatively average incidence of dyspnoea and a high percentage of patients with dysosmia. 27 , 29
Table 3.
PCS patients (n = 47) | Healthy participants (n = 47) | Statistics | |
---|---|---|---|
Age [years] (mean ± SD) | 44.4 ± 11.6 | 44.3 ± 13.7 | t = 0.178, p = 0.859 |
Sex [F:M] (%) | 38:9 (81:19) | 36:11 (77: 23) | W = 1152, p = 0.620 |
Hospitalization due to COVID‐19 [n] (%) | 7 (15) | NA | |
ICU admittance due to COVID‐19 [n] (%) | 3 (6.4) | NA | |
Received oxygen treatment for COVID‐19 [n] (%) | 3 (6.4) | NA | |
On ventilator due to COVID‐19 [n] (%) | 1 (2.1) | NA | |
Post‐COVID‐19 dyspnoea [n] (%) | 11 (23.4) | NA | |
Post‐COVID‐19 dysosmia [n] (%) | 23 (48.9) | NA | |
FSMC total score [median ± SD] | 77 ± 15.5 | 36 ± 13.5 | W = 1836, p = 4.20e‐13 |
ESS [median ± SD] | 11 ± 5.5 | 5 ± 3.1 | W = 1553, p = 3.15e‐6 |
Time since infection [months] (mean ± SD) | 8.4 ± 3.0 | NA |
Bolded text indicates statistical significance.
COVID, coronavirus disease 2019; ESS, Epworth sleepiness scale; FSMC, fatigue scale for motor and cognitive functions; ICU, intensive care unit; MRI, magnetic resonance imaging.
Oxygen level estimates (OLE) and brain perfusion
Group comparisons
Compared to HP, patients with PCS showed higher OLE values, that is, decreased oxygen levels, in the whole brain WM, cortical GM, whole DGM, hippocampus, thalamus and putamen (Fig. 2A,B). With FDR‐correction for multiple comparisons, significant group differences remained for whole brain WM (two‐sample t‐test, t(78.7) = 2.77, p‐FDR = 0.038, d = 0.572) and cortical GM (t(87.7) = 3.30, p‐FDR = 0.015, d = 0.681). However, the putamen OLE measures showed a moderate effect size difference between the two groups (t(78.7) = 2.50, p‐FDR = 0.05, d = 0.52). Thus, on a group level, PCS patients seem to show more decreased blood oxygen levels in several brain regions compared to HP. Meanwhile, perfusion (BOLD delay) did not differ significantly between groups before FDR‐adjustment (except in the cortical GM, please see Fig. S1 for details on all comparisons).
Associations with post‐COVID symptoms
Cognitive symptoms
Decreased whole DGM and thalamic oxygen levels were associated with better MoCA scores (ß = 0.29, p = 0.031, p‐FDR = 0.111 and ß = 0.30, p = 0.022, p‐FDR = 0.111, respectively). We observed the same pattern for the caudate nucleus (MoCA: ß = 0.30, p = 0.036, p‐FDR = 0.111) and brainstem (MoCA: ß = 0.29, p = 0.040, p‐FDR = 0.111; TMT‐B: ß = −0.35, p = 0.025, p‐FDR = 0.277), although these regions did not differ between patients and HP. Meanwhile, slower perfusion times of the putamen (ß = −0.38, p = 0.014, p‐FDR = 0.078) and caudate nucleus (ß = −0.50, p = 0.0007, p‐FDR = 0.008, Fig. 3A) were associated with better attention (TMT‐B).
No significant associations with MoCA remained after FDR‐adjustment of the multivariable linear regression models. While we observed no significant differences between group associations of MoCA scores with oxygen levels or perfusion times (p group), the interaction effects between caudate BOLD delay times and TMT‐B were significantly different between groups.
PCS patients with lower putamen oxygen levels had a reduced risk of experiencing poor MoCA scores (≤28) during their post‐COVID disease course (HR O = 0.22, 95% CI 0.06–0.78, p = 0.019). The Cox regression HR analysis for BOLD delay revealed a relationship between slower putamen perfusion and an increased risk of poor MoCA scores (HR D = 3.53, 95% CI 1.29–9.70, p = 0.035).
Risk analysis did not show any regional BOLD delay or OLE measures with significant HRs relating to poor TMT‐B.
Daytime sleepiness
Fourteen out of 42 (33%; 5 missing) patients reported high levels of daytime sleepiness (ESS ≥ 13). Decreased putamen oxygen levels were associated with lower ESS in patients (ß = −0.35, p = 0.014, p‐FDR = 0.124, Fig. 3B, Table 4). Additionally, higher globus pallidus OLE were associated with less sleepiness in patients (ß = −0.33, p = 0.023, p‐FDR = 0.124, Fig. 3B), but not in HP. However, the globus pallidus and putamen associations were not different between groups (p group = 0.251 and p group = 0.250, respectively). ESS was associated with age and sex in the multivariable linear regression models, which did not survive FDR multiple comparisons adjustment. However, significant associations of the post‐COVID OLE measures in the globus pallidus and putamen with ESS were observed, increasing the fit of the full model (R 2 adj = 0.254 and R 2 adj = 0.271, respectively).
Table 4.
OLE | Age | Sex | Full model fit | |
---|---|---|---|---|
Brainstem | ß = −0.16, p = 0.300 | ß = 0.44, p = 0.009* | ß = 0.49, p = 0.006* | R 2 adj = 0.168 |
Deep grey matter | ß = −0.25, p = 0.085 | ß = 0.43, p = 0.010* | ß = 0.49, p = 0.004* | R 2 adj = 0.209 |
Amygdala | ß = −0.14, p = 0.348 | ß = 0.46, p = 0.007* | ß = −0.44, p = 0.010* | R 2 adj = 0.163 |
Nucleus accumbens | ß = −0.27, p = 0.065 | ß = 0.47, p = 0.005* | ß = 0.42, p = 0.010* | R 2 adj = 0.218 |
Hippocampus | ß = −0.16, p = 0.271 | ß = 0.47, p = 0.007* | ß = 0.44, p = 0.009* | R 2 adj = 0.171 |
Thalamus | ß = −0.23, p = 0.115 | ß = 0.44, p = 0.009* | ß = 0.48, p = 0.005* | R 2 adj = 0.198 |
Globus pallidus | ß = −0.33, p = 0.023* | ß = 0.39, p = 0.016* | ß = 0.47, p = 0.004* | R 2 adj = 0.254* |
Putamen | ß = −0.35, p = 0.014* | ß = 0.39, p = 0.015* | ß = 0.43, p = 0.007* | R 2 adj = 0.271* |
Caudate nucleus | ß = −0.14, p = 0.381 | ß = 0.42, p = 0.013* | ß = 0.47, p = 0.007* | R 2 adj = 0.161 |
Total white matter | ß = −0.13, p = 0.366 | ß = 0.46, p = 0.008* | ß = 0.46, p = 0.008* | R 2 adj = 0.164 |
Cortical grey matter | ß = −0.23, p = 0.097 | ß = 0.47, p = 0.005* | ß = 0.43, p = 0.010* | R 2 adj = 0.204 |
Note that p‐values are not adjusted for multiple comparisons for false discovery rate in this table. Bolded text indicates statistically significant associations (*p < 0.05).
ESS, Epworth Sleepiness Scale; OLE, oxygen level estimate.
Pertaining to brain perfusion, BOLD delay times in the WM were associated with ESS in patients prior to FDR‐adjustment (β = −0.31, p = 0.038, p‐FDR = 0.415, p group = 0.047); while in HPs, whole DGM (ß = 0.42, p = 0.005, p‐FDR = 0.034, p group = 0.251), nucleus accumbens (ß = 0.38, p = 0.015, p‐FDR = 0.056, p group = 0.877), putamen (ß = 0.33, p = 0.030, p‐FDR = 0.082, p group = 0.65) and thalamus (ß = 0.42, p = 0.006, p‐FDR = 0.034, p group = 0.418) Normalized BOLD delay times were found to be associated with ESS scores. Only in the WM were the group associations of BOLD delay times with ESS significantly different. Associations between daytime sleepiness and whole DGM and thalamus regional BOLD delay times in HPs remained significant after FDR p‐adjustment for multiple comparisons, however did not significantly differ from post‐COVID patient associations.
Decreased nucleus accumbens, putamen, brainstem and cortical GM oxygen levels also showed a reduced risk of daytime sleepiness (Table 5). Conversely, increased age, female sex and decreased oxygen levels in the whole DGM complex showed an increased HR for higher daytime sleepiness.
Table 5.
Z‐score | 95% CI | Hazard ratio | p‐value | |
---|---|---|---|---|
Age | 2.985 | 1.145–1.922 | 1.484 | 2.84e‐3* |
Sex | 2.034 | 1.164–3.548e3 | 64.27 | 0.042* |
Brainstem | −2.671 | 0.011–0.499 | 0.074 | 7.56e‐3* |
Deep grey matter | 2.272 | 1.813–3.226e3 | 76.48 | 0.023* |
Amygdala | 1.724 | 0.968–1.652 | 1.265 | 0.085 |
Nucleus accumbens | −2.621 | 0.570–0.922 | 0.725 | 8.77e‐3* |
Hippocampus | −0.795 | 0.520–1.319 | 0.828 | 0.427 |
Thalamus | −1.599 | 0.307–1.127 | 0.588 | 0.110 |
Globus pallidus | 1.050 | 0.772–2.355 | 1.348 | 0.294 |
Putamen | −2.518 | 0.210–0.823 | 0.416 | 0.012* |
Caudate nucleus | −0.577 | 0.631–1.285 | 0.901 | 0.563 |
Total white matter | 1.440 | 0.902–1.960 | 1.330 | 0.150 |
Cortical grey matter | −2.855 | 0.167–0.717 | 0.346 | 4.30e‐3* |
Note that p‐values are not adjusted for multiple comparisons for false discovery rate in this table. Bolded text indicates statistically significant associations (*p < 0.05).
CI, confidence interval; ESS, Epworth Sleepiness Scale; HR O, hazard ratio calculated from Cox proportional hazard models using only OLE metrics; OLE, oxygen level estimate.
We also observed a lower risk of experiencing high levels of daytime sleepiness in patients with more negative BOLD delay times in the cortical GM (HR = 0.066, 95% CI 0.005–0.85, p = 0.037) and hippocampus (HR = 0.06, 95% CI 0.005–0.81, p = 0.034, Fig. 4).
In contrast, decreased oxygen levels of the whole DGM was found to be related to an increased risk of experiencing high daytime sleepiness during the PCS disease course (HR O = 76.48, 95% CI 1.81–3,226, p = 0.023), even when accounting for perfusion factors in the full HR model (HR = 1,300, 95% CI 3.95–410,000, p = 0.015, Fig. 4).
Fatigue
Out of 43 patients, 37 (86%; 4 missing) reported severe fatigue (FSMC ≥ 63). No associations were found between any OLE or BOLD delay regional metrics with motor fatigue scores. Also, multivariable linear models did not show any significant associations between any fatigue scores and oxygen levels or perfusion in PCS patients or HP.
Increased risk of experiencing high fatigue during the post‐COVID disease course was related to slower BOLD delay times in the cortical GM (HR D = 8.61, 95% CI 1.47–50.39, p = 0.017), putamen (HR D = 115.10, 95% CI 2.44–5,435, p = 0.016) and hippocampus (HR D = 56.02, 95% CI 2.46–1,276, p = 0.012).
Patients with higher thalamic OLE showed an increased risk of experiencing cognitive fatigue (cognitive FSMC ≥ 28, HR = 6.29, 95% CI 1.39–28.52, p = 0.017). However, higher OLE of the nucleus accumbens was related to a reduced risk (HR = 0.20, 95% CI 0.06–0.76, p = 0.018).
Discussion
Our exploratory study used rs‐fMRI to evaluate oxygen level estimates (OLE) and cerebral perfusion in PCS and led to several new findings.
PCS patients showed higher OLE (lower oxygen levels) in the total white and cortical grey matter, with implications of decreased oxygen levels in the thalamus, putamen and hippocampus, compared to healthy participants (HP) at a median of 8.4 months after first COVID‐19 diagnosis.
These widespread cerebral metabolic differences were related to neurological symptoms. Higher OLE measures in the globus pallidus and putamen of PCS patients were associated with lower daytime sleepiness scores in the ESS. Faster BOLD delay times in the putamen and caudate nucleus were related to worse TMT‐B scores. While no group differences were seen in regional BOLD delay times (blood perfusion) between patients and HP, the association of faster putamen perfusion with worse TMT‐B scores were significantly different than that in healthy participants. Thus, indicating that this association is unique to post‐COVID and likely disease‐induced.
Interestingly, slower cortical GM and hippocampal perfusion times, potentially representing an adaptive increase in metabolism, were associated with a reduced risk of daytime sleepiness. Patients with lower putamen oxygen levels showed a reduced risk of daytime sleepiness and along with less risk of cognitive impairment (MoCA). In contrast, higher whole deep GM OLE measures were associated with increased risk of high daytime sleepiness. Also, lower thalamic oxygen levels pointed to a higher risk of experiencing cognitive fatigue.
Impaired blood flow is hypothesised as one pathomechanism in PCS, given the strong evidence that internalisation of angiotensin‐converting enzyme‐2 induced by acute SARS‐CoV‐2 infection causes a cascade of protein activations related to capillary constriction. 30
The often higher OLE we observed in post‐COVID patients suggests that cerebral oxygen metabolism could be increased due to adaptation of blood perfusion changes in specific regions. We found evidence of lower oxygen levels in the white matter, deep and cortical grey matter. This is in‐line with other smaller studies using positron emission tomography (PET) observing altered glucose metabolism in the DGM. Metabolic changes (both hyper‐ and hypometabolism) measured on PET scans persisted over time based on inflammation status 31 and were related to cognitive impairment and functional complaints. 12
Our results show higher OLE (or more signal from deoxygenated blood was detected) largely in the DGM, including the putamen, thalamus and hippocampus. Potential hypermetabolism in these regions may lead to decreased oxygen levels in the white matter and cortical GM (patients had a mean increase of 4% and 6% in OLE compared to HP in those regions, respectively). As dynamic changes in brain metabolism are linked to levels of oxygen concentration, our findings could indicate that higher sustained metabolism occurs in PCS patients compared to HP even during the resting state. Autopsies of COVID‐19 patient brains have shown that astrocytes infected by the virus had higher oxygen consumption 32 and upregulation of inflammatory substrates in perivascular brainstem regions were found. 33 A recent small study found 8 out of 34 (24%) PCS patients, had cerebral hypoxia measured by frequency domain near‐infrared spectroscopy, relating to reduced oxyhaemoglobin and cognitive function. 34 As even slight exposure to hypoxia of the human brain leads to cerebral blood flow changes, 35 but not to increased oxygen consumption in healthy people, 36 it stands to reason that decreased regional oxygenation found in PCS may be due to a disease‐specific mechanism.
No overall BOLD delay differences were found between PCS and healthy participants in our study. Meanwhile, another study using arterial spin labelling MRI found PCS with significant hypoperfusion in the frontal, parietal and temporal cortices compared to healthy controls. 37 However, this smaller study included patients suffering from moderate‐to‐severe COVID‐19 disease and a minimum of 4 weeks after a positive polymerase chain reaction test for the virus. This is a more severe disease course and shorter time period post‐COVID than our cohort, possibly indicating our findings relate more to the longer term adaptations in PCS patients.
Interestingly, we found in our patients that slower blood perfusion in the caudate nucleus and the putamen were associated with better visual attention. The TMT also evaluates processing speed, working memory and executive functions, that is, cognitive abilities linked to the dorsal striatum. 38 Our findings could therefore indicate retained normal vasodilation or recovery from capillary damage, resulting in better TMT‐B scores. Meanwhile, those with faster perfusion, possibly due to reduced oxyhaemoglobin or compensatory vascular reactivity based on reduced tissue oxygenation, 39 fared less well on this task.
We observed a reduced risk of daytime sleepiness in patients with slower cortical GM and hippocampal perfusion. This could indicate that perfusion is altered to effectively deliver oxygen to DGM regions and faster than normal perfusion is required to maintain wakefulness. Interestingly, another study showed that sleep deprived HP have increased cerebral blood flow in the deep and cortical GM, 40 suggesting a normal vascular mechanism in adults, where lack of sleep drives increased perfusion and oxygen metabolism.
Moreover, the risk of experiencing severe fatigue during the disease course was related to perfusion times in the cortical GM, putamen and hippocampus. Lower thalamic oxygen levels were specifically associated with an increased risk of cognitive fatigue. A previous study of the same cohort observed reduced diffusion‐related integrity, volume loss and shape changes of the thalamus, putamen and pallidum, underlying fatigue symptoms. 3 Aside from structural changes, altered brain blood perfusion and oxygen metabolism could also contribute to post‐COVID symptoms.
Our study has some limitations, including the lack of physiological data monitoring during the scan, for example, pulse oximetry and respiratory measurement, which may have helped in interpreting the results. While BOLD delay has been established as a measure of perfusion in several clinical contexts, 20 , 41 rs‐fMRI‐based oxygen level estimates (OLE) are a relatively novel postprocessing method. Nevertheless, we were able to find consistent differences between patients and well‐matched HPs. Our study shows that the correct application of this method provides a useful tool to assess blood oxygen metabolism in the brain, particularly in patients that may have no clinically pathological MRI findings, but with subtle vascular changes. 13 Other in vivo imaging modalities that probe metabolism in the brain, such as positron emission tomography (PET)‐MRI, PET‐single photon emission computed tomography, or magnetic resonance spectroscopy may aid in evaluating energetics in PCS. 42 , 43 Although these techniques are promising, clinical availability/applicability of these methods are lacking. 44 Thus, our use of the BOLD rs‐fMRI signal as an indicator of vascular health is more accessible 45 to many clinics and the open‐source software from our analysis pipeline may lead to better replication of our findings in other PCS patients. Finally, with a relatively small cohort, the full HR models should be interpreted cautiously. The extremely large HR we calculated for the association between less oxygenated DGM with high daytime sleepiness indicates that there is most likely some overfitting of the Cox proportional hazard models. As there were many covariates included in the analyses, including categorical splitting to create binary labels, the possibility of overfitting may not be negligible. However, looking at the regions that are indicated in our analyses, and understanding that Cox proportional hazard models calculate compounded effects of each variable on the outcome measure (high daytime sleepiness); one could postulate that in our older, female PCS patients, increased oxygen metabolism in subregions of the DGM (e.g. the nucleus accumbens, amygdala and putamen), may lead to a less oxygenated whole DGM structure, which relates to higher risk of daytime sleepiness. Thus, our findings will allow for more hypothesis‐driven research in the highlighted brain regions of PCS patients, especially in terms of cerebral blood perfusion and oxygen levels.
In conclusion, our study shows widespread reductions in cerebral blood oxygen levels in PCS that are related to symptoms of daytime dysfunction and cognitive impairment. Changes in oxygen metabolism and blood perfusion may serve as an adaptive mechanism to mediate brain vascular damage and/or as a mode of maintaining normal daily functioning.
Author Contributions
Claudia Chien was responsible for conceptualization, data curation, formal analysis, methodology, resources, software, visualisation, interpretation, writing of the original draft, and review and editing. Josephine Heine was responsible for conceptualization, data collection and curation, methodology, resources, project administration, and draft review and editing. Ahmed Khalil was responsible for formal analysis, methodology, resources, software, visualisation, writing of the original draft, and review and editing. Lars Schlenker, Tim Hartung and Fabian Boesl were responsible for data curation, resources and draft review. Katia Schwichtenberg was responsible for data collection and curation, resources and draft review. Rebekka Rust was responsible for data collection and curation, and resources. Judith Bellmann‐Strobl was responsible for data collection and curation, resources and draft review. Christiana Franke was responsible for data collection and curation, resources and draft review. Friedemann Paul was responsible for resources and draft review. Carsten Finke was responsible for conceptualization, data collection, curation, interpretation, resources and draft review and editing.
Funding Information
Deutsche Forschungsgemeinschaft (DFG) and German Ministry of Education and Research (BMBF).
Conflict of Interest Statement
Dr. Chien has received writing honoraria from the British Society for Immunology, is a Standing Committee on Science Member for the Canadian Institutes of Health Research (CIHR) and has received research support from Novartis and Alexion, outside of the submitted work. Dr. Franke has received speaking honoraria from Pfizer, Bristol Myers Squibb and Boehringer Ingelheim, outside of the submitted work. Dr. Rust reported speaking honoraria from Roche unrelated to this study. Dr. Bellmann‐Strobl reports personal fees from Bayer Healthcare, personal fees from Sanofi‐Aventis/Genzyme, personal fees from Roche, outside the submitted work. Dr. Paul reports personal fees and nonfinancial support from SanofiGenzyme, personal fees, nonfinancial support and other from BiogenIdec, personal fees and non‐financial support from MedImmune, personal fees and nonfinancial support from Shire, personal fees and nonfinancial support from Alexion, grants, personal fees and nonfinancial support from Bayer, grants and personal fees from Novartis, grants and personal fees from Teva, grants and personal fees from Merck Serono, personal fees from Actelion, personal fees from Chugai, personal fees from Roche, personal fees from Celgene, grants from Sanofi‐Aventis/Genzyme, grants from Alexion, grants from German Research Council (DFG Exc 257), grants from Werth Stiftung of the City of Cologne, grants from German Ministry of Education and Research (BMBF Competence Network Multiple Sclerosis), grants from Arthur Arnstein Stiftung Berlin, grants from EU FP7 Framework Program (combims.eu), grants from Guthy Jackson Charitable Foundation and grants from National Multiple Sclerosis Society of the USA, outside the submitted work. All other authors declare no competing interests.
Supporting information
Acknowledgements
The authors thank the University of Minnesota Center for Magnetic Resonance Research (CMRR) for providing the multiband echo planar imaging rs‐fMRI sequence used in this study. Open Access funding enabled and organized by Projekt DEAL.
Funding Statement
This work was funded by German Ministry of Education and Research (BMBF) ; Deutsche Forschungsgemeinschaft (DFG) .
Data Availability Statement
Data from this study can be shared at the request of other investigators for purposes of replicating procedures and results.
References
- 1. Cho S‐M, White N, Premraj L, et al. Neurological manifestations of COVID‐19 in adults and children. Brain J Neurol. 2023;146(4):1648‐1661. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Hartung TJ, Neumann C, Bahmer T, et al. Fatigue and cognitive impairment after COVID‐19: a prospective multicentre study. EClinicalMedicine. 2022;53:101651. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Heine J, Schwichtenberg K, Hartung TJ, et al. Structural brain changes in patients with post‐COVID fatigue: a prospective observational study. EClinicalMedicine. 2023;58:101874. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Ceban F, Ling S, Lui LMW, et al. Fatigue and cognitive impairment in post‐COVID‐19 syndrome: a systematic review and meta‐analysis. Brain Behav Immun. 2022;101:93‐135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Hartung TJ, Bahmer T, Chaplinskaya‐Sobol I, et al. Predictors of non‐recovery from fatigue and cognitive deficits after COVID‐19: a prospective, longitudinal, population‐based study [Internet]. eClinicalMedicine. 2024;69:102456. [cited 2024 Feb 28]. https://www.thelancet.com/journals/eclinm/article/PIIS2589‐5370(24)00035‐X/fulltext [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Lauria A, Carfì A, Benvenuto F, et al. Neuropsychological measures of long COVID‐19 fog in older subjects. Clin Geriatr Med. 2022;38(3):593‐603. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Zhao S, Martin EM, Reuken PA, et al. Long COVID is associated with severe cognitive slowing [Internet]. 2023;2023.12.03.23299331. [cited 2024 Feb 28]. doi: 10.1101/2023.12.03.23299331v1 [DOI] [PMC free article] [PubMed]
- 8. Díez‐Cirarda M, Yus M, Gómez‐Ruiz N, et al. Multimodal neuroimaging in post‐COVID syndrome and correlation with cognition. Brain J Neurol. 2023;146(5):2142‐2152. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Xie Y, Xu E, Bowe B, Al‐Aly Z. Long‐term cardiovascular outcomes of COVID‐19. Nat Med. 2022;28(3):583‐590. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Davis HE, McCorkell L, Vogel JM, Topol EJ. Long COVID: major findings, mechanisms and recommendations. Nat Rev Microbiol. 2023;21(3):133‐146. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Franke C, Boesl F, Goereci Y, et al. Association of cerebrospinal fluid brain‐binding autoantibodies with cognitive impairment in post‐COVID‐19 syndrome. Brain Behav Immun. 2023;109:139‐143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Guedj E, Campion JY, Dudouet P, et al. 18F‐FDG brain PET hypometabolism in patients with long COVID. Eur J Nucl Med Mol Imaging. 2021;48(9):2823‐2833. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Greene C, Connolly R, Brennan D, et al. Blood–brain barrier disruption and sustained systemic inflammation in individuals with long COVID‐associated cognitive impairment. Nat Neurosci. 2024;27(3):421‐432. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Haffke M, Freitag H, Rudolf G, et al. Endothelial dysfunction and altered endothelial biomarkers in patients with post‐COVID‐19 syndrome and chronic fatigue syndrome (ME/CFS). J Transl Med. 2022;20(1):138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Yoshiuchi K, Farkas J, Natelson BH. Patients with chronic fatigue syndrome have reduced absolute cortical blood flow. Clin Physiol Funct Imaging. 2006;26(2):83‐86. [DOI] [PubMed] [Google Scholar]
- 16. Tong Y, Lindsey KP, Hocke LM, Vitaliano G, Mintzopoulos D, Frederick BB. Perfusion information extracted from resting state functional magnetic resonance imaging. J Cereb Blood Flow Metab. 2017;37(2):564‐576. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Pelphrey KA. Blood‐Oxygen‐Level‐Dependent (BOLD) Signal [Internet]. In: Volkmar FR, ed. Encyclopedia of Autism Spectrum Disorders. Springer; 2013:465‐466. [cited 2023 Jan 5]. doi: 10.1007/978-1-4419-1698-3_550 [DOI] [Google Scholar]
- 18. Moeller S, Yacoub E, Olman CA, et al. Multiband multislice GE‐EPI at 7 tesla, with 16‐fold acceleration using partial parallel imaging with application to high spatial and temporal whole‐brain fMRI. Magn Reson Med. 2010;63(5):1144‐1153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Xu J, Moeller S, Auerbach EJ, et al. Evaluation of slice accelerations using multiband echo planar imaging at 3 T. NeuroImage. 2013;83:991‐1001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Feinberg DA, Moeller S, Smith SM, et al. Multiplexed Echo planar imaging for sub‐second whole brain FMRI and fast diffusion imaging. PLoS One. 2010;5(12):e15710. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Tanrıtanır AC, Villringer K, Galinovic I, et al. The effect of scan length on the assessment of BOLD delay in ischemic stroke [Internet]. Front Neurol. 2020;11:381. [cited 2023 Jan 5]. doi: 10.3389/fneur.2020.00381 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Tong Y, Yao JF, Chen JJ, deB Frederick B. The resting‐state fMRI arterial signal predicts differential blood transit time through the brain. J Cereb Blood Flow Metab. 2019;39(6):1148‐1160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Thomann AE, Goettel N, Monsch RJ, et al. The Montreal cognitive assessment: normative data from a German‐speaking cohort and comparison with international normative samples. J Alzheimers Dis. 2018;64(2):643‐655. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Tombaugh TN. Trail making test a and B: normative data stratified by age and education. Arch Clin Neuropsychol. 2004;19(2):203‐214. [DOI] [PubMed] [Google Scholar]
- 25. Therneau TM, Grambsch PM. Modeling Survival Data: Extending the Cox Model [Internet]. Springer; 2000. [cited 2024 Feb 5]. doi: 10.1007/978-1-4757-3294-8 [DOI] [Google Scholar]
- 26. Majeed K, Eliyas JK. Chapter 91—design and analysis [Internet]. In: Eltorai AEM, Bakal JA, Newell PC, Osband AJ, eds. Translational Surgery. Academic Press; 2023:575‐589. [cited 2024 Feb 5]. https://www.sciencedirect.com/science/article/pii/B9780323903004001002 [Google Scholar]
- 27. Pérez‐González A, Araújo‐Ameijeiras A, Fernández‐Villar A, et al. Long COVID in hospitalized and non‐hospitalized patients in a large cohort in Northwest Spain, a prospective cohort study. Sci Rep. 2022;12:3369. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Mateu L, Tebe C, Loste C, et al. Determinants of the onset and prognosis of the post‐COVID‐19 condition: a 2‐year prospective observational cohort study [Internet]. Lancet Reg Health Eur. 2023;33:100724. [cited 2024 May 23]. https://www.thelancet.com/journals/lanepe/article/PIIS2666‐7762(23)00143‐6/fulltext [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Abdel‐Aziz M, Azab N. Dysosmia in recovered COVID‐19 patients. J Craniofac Surg. 2023;34(3):843‐844. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Hirunpattarasilp C, James G, Kwanthongdee J, et al. SARS‐CoV‐2 triggers pericyte‐mediated cerebral capillary constriction. Brain. 2023;146(2):727‐738. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Martini AL, Carli G, Kiferle L, et al. Time‐dependent recovery of brain hypometabolism in neuro‐COVID‐19 patients. Eur J Nucl Med Mol Imaging. 2022;50(1):90‐102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Crunfli F, Carregari VC, Veras FP, et al. Morphological, cellular, and molecular basis of brain infection in COVID‐19 patients. Proc Natl Acad Sci. 2022;119(35):e2200960119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Radke J, Meinhardt J, Aschman T, et al. Proteomic and transcriptomic profiling of brainstem, cerebellum and olfactory tissues in early‐ and late‐phase COVID‐19. Nat Neurosci. 2024;27(3):409‐420. [DOI] [PubMed] [Google Scholar]
- 34. Adingupu DD, Soroush A, Hansen A, Twomey R, Dunn JF. Brain hypoxia, neurocognitive impairment, and quality of life in people post‐COVID‐19. J Neurol. 2023;270(7):3303‐3314. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Vestergaard MB, Lindberg U, Aachmann‐Andersen NJ, et al. Acute hypoxia increases the cerebral metabolic rate – a magnetic resonance imaging study. J Cereb Blood Flow Metab. 2016;36(6):1046‐1058. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Vestergaard MB, Ghanizada H, Lindberg U, et al. Human cerebral perfusion, oxygen consumption, and lactate production in response to hypoxic exposure. Cereb Cortex. 2022;32(6):1295‐1306. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Ajčević M, Iscra K, Furlanis G, et al. Cerebral hypoperfusion in post‐COVID‐19 cognitively impaired subjects revealed by arterial spin labeling MRI. Sci Rep. 2023;13(1):5808. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Graff‐Radford J, Williams L, Jones DT, Benarroch EE. Caudate nucleus as a component of networks controlling behavior. Neurology. 2017;89(21):2192‐2197. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Claassen JAHR, Thijssen DHJ, Panerai RB, Faraci FM. Regulation of cerebral blood flow in humans: physiology and clinical implications of autoregulation. Physiol Rev. 2021;101(4):1487‐1559. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Elvsåshagen T, Mutsaerts HJMM, Zak N, et al. Cerebral blood flow changes after a day of wake, sleep, and sleep deprivation. NeuroImage. 2019;186:497‐509. [DOI] [PubMed] [Google Scholar]
- 41. Khalil AA, Villringer K, Filleböck V, et al. Non‐invasive monitoring of longitudinal changes in cerebral hemodynamics in acute ischemic stroke using BOLD signal delay. J Cereb Blood Flow Metab. 2020;40(1):23‐34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Kwon SJ, Ha S, Yoo S‐W, et al. Comparison of early F‐18 Florbetaben PET/CT to Tc‐99m ECD SPECT using voxel, regional, and network analysis. Sci Rep. 2021;11(1):16738. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Zhu X‐H, Lu M, Chen W. Quantitative imaging of brain energy metabolisms and neuroenergetics using in vivo X‐nuclear 2H, 17O and 31P MRS at ultra‐high field. J Magn Reson San Diego Calif. 1997;2018(292):155‐170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Prasuhn J, Kunert L, Brüggemann N. Neuroimaging methods to map in vivo changes of OXPHOS and oxidative stress in neurodegenerative disorders. Int J Mol Sci. 2022;23(13):7263. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. Zhong XZ, Tong Y, Chen JJ. Assessment of the macrovascular contribution to resting‐state fMRI functional connectivity at 3 Tesla. Imaging Neurosci. 2024;2:1‐20. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Data from this study can be shared at the request of other investigators for purposes of replicating procedures and results.