Skip to main content
F1000Research logoLink to F1000Research
. 2018 Jul 9;7:1028. [Version 1] doi: 10.12688/f1000research.15142.1

A three minutes supine position test reveals higher risk of spinal anesthesia induced hypotension during cesarean delivery. An observational study.

Markos Erango 1, Arnoldo Frigessi 2, Leiv Arne Rosseland 3,4,a
PMCID: PMC6085602  PMID: 30135733

Abstract

Background: Cesarean delivery is performed under spinal anesthesia, and vasodilation is the main cause for a drop in blood pressure. The compression of the aorta and inferior vena cava by the gravid uterus is of additional clinical importance. Hypotension may occur during cesarean delivery even if prophylactic infusion of phenylephrine is practiced. We have tested if a 3 minute supine observation, can identify a subset of women with decreasing systolic arterial pressure (SAP) under spinal anesthesia.

Methods: We performed a prospective observational study at Oslo University Hospital on healthy pregnant women for planned cesarean delivery. Continuous measurements of calibrated invasive SAP and estimated cardiac output were recorded for 76 women in a 3 minutes measurement with the woman in the left lateral position, followed by supine position for 3 minutes. Using functional data clustering, principal component analysis and curve smoothing, to filter way noise and reduce the dimensionality of the signal, we clustered the women into separate SAP groups.  

Results: We identified two significantly different groups of women during supine position; one characterized by initial drop in SAP, the other showed initial increase. After spinal anesthesia, the mean SAP curve of the women in the first group showed a drop in blood pressure, which was more rapid than for the other women. A minor difference in cardiac output was observed between the two groups of women with the mean cardiac output curve for the first group being higher.

Conclusions: This work indicates that supine position affect clinically relevant cardiovascular measurements in pregnant women. A simple test may identify patients with increased risk of spinal anesthesia induced hypotension.

Keywords: Blood pressure, spinal anesthesia, cardiac output, cesarean delivery

List of abbreviations

SAP; systolic arterial pressure

Hb; hemoglobin concentration

BMI; body mass index

DAP; diastolic arterial pressure

MAP; mean arterial pressure

HR; heart rate

SV; stroke volume

CO; cardiac output

SVR; systemic vascular resistance

Background

Spinal anesthesia is a standard method in cesarean deliveries and is regarded as being safe for the mother and the baby, even though significant maternal hypotension often occur 1. The vasodilatory effect of spinal anesthesia is the main cause for this drop in blood pressure 2. The compression of the aorta and inferior vena cava by the gravid uterus is known to reduce venous return and this is estimated to be of clinical importance after approximately 20 weeks of pregnancy. Pregnant women may find the supine position uncomfortable and avoid this position close to delivery. During cesarean delivery left lateral tilt is recommended to oppose this effect, and theoretically this should reduce the negative effects on maternal circulation. However, in clinical practice the incidence of spinal induced hypotension is high if no prophylactic measures to oppose the vasodilatation are included 1. We have previously compared two different prophylactic methods, lower leg compression and phenylephrine 3. In that study continuous infusion of phenylephrine was superior to lower leg compression and placebo. Lower leg compression was found to increase heart stroke volume compared to placebo indicating a significant effect on venous return. Some women have hypotension during cesarean delivery even if prophylactic infusion of phenylephrine and/or lower leg compression is practiced. Phenylephrine continuous infusion is not included as standard prophylaxis globally 4. Previous attempts to predict risk of spinal induced hypotension and to individualize the prophylactic treatment have not yet changed clinical practice.

Aims

We tested if a 3 minutes supine observation prior to spinal anesthesia with continuous invasive hemodynamic measurements

a) Allows identification of a subset of women with decreasing blood pressure and/or cardiac output;

b) If this subset of women could be identified in left lateral positon;

c) If this subset of women showed any special properties in terms of cardiac output and systolic arterial pressure during cesarean delivery with phenylephrine prophylaxis under spinal anesthesia.

Methods

The patients participated in a randomized, double-blinded, placebo-controlled, parallel-group comparison of carbetocin and oxytocin given intravenously during elective cesarean delivery under spinal anesthesia. The study was conducted at the Department of Anesthesiology, Division of Emergency and Critical Care, Oslo University Hospital, Rikshospitalet between November 2009 and September 2011. The Birth Clinic at Rikshospitalet is a tertiary care center, but the majority of the laboring women are healthy and representative of the general population in central Norway. The protocol was approved by the Data Inspectorate’s local representative at Oslo University Hospital, the Regional Committee for Medical and Health Research Ethics of Southern Norway (Oslo, Norway)(2009/130), and the Norwegian Medicines Agency (Oslo, Norway)(09/07301–7). This approval covered both the RCT and sub-study. The randomized controlled trial was registered at clinicaltrials.gov ( NCT00977769) and was conducted according to Good Clinical Trial practice, the CONSORT guidelines, and the principles of the Declaration of Helsinki. The results from the randomized controlled trial are published previously 5.

In addition to the RCT we performed an observational study. Detailed information about the observational study was integrated in the patient information and consent form. The observational study was conducted and reported according to STROBE guidelines 6.

Eligible participants were screened by the senior author (LAR) for inclusion at their last midwife consultation before their scheduled delivery. Oral and written information was given to each woman at least 24 hours before her delivery. Written informed consent was gained before inclusion. The women were informed about the intervention in the randomized controlled trial and the observational study being performed the last minutes before spinal anesthesia. They were asked to assess specifically the extra pain caused by to the arterial cannulation. The inclusion criteria included being in good health, a singleton pregnancy, being aged 18 years or older, and a scheduled planned cesarean section at a minimum gestational age of 36 weeks. The exclusion criteria included pre-eclampsia, placenta previa, placenta accreta, von Willebrand disease or other bleeding disorder, and systolic arterial pressure (SAP) < 90 mmHg. A total of 185 patients were screened, and 76 were included in the observational study. The indications for elective cesarean section were maternal request 43%, previous cesarean delivery 25%, breech presentation 8%, other obstetric reasons 13%, and other non-obstetric maternal or neonatal medical conditions 11%.

In addition to continuous beat-to-beat hemodynamic variables that were collected and stored electronically during left and supine position, we registered each patient’s age, height, weight, gestational age, preoperative hemoglobin concentration (Hb), hours fasting (liquids and solids), and the baby’s birth weight. High body mass index (BMI) may influence the hemodynamic response to both supine position and spinal anesthesia. Maternal weight upon inclusion was used to calculate the BMI (weight in kg / (height in meters) 2). An arterial catheter was placed into the radial artery after infiltration of lidocaine (10–20 mg) immediately after arrival at the operating theatre. Peripheral intravenous (i.v.) catheters were placed on both forearms. Invasive blood pressure were calibrated and continuous beat-to-beat measurements of invasive SAP, diastolic arterial pressure (DAP), mean arterial pressure (MAP), and heart rate (HR) were recorded in 3 minutes with the woman in a left lateral position. Stroke volume (SV), cardiac output (CO), systemic vascular resistance (SVR), and other estimated variables based on continuous arterial waveform analysis system were recorded using PulseCO (PulseCO TM, Cambridge, United Kingdom), an integrated part of the LiDCO plus monitoring system (LiDCO Ltd., Cambridge, United Kingdom). We omitted the calibration of CO with the lithium dilution technique since the primary outcome was change in repeated measurements of hemodynamic variables. The hemodynamic observations were performed before spinal anesthesia was given. After the 3 minutes measurements in the left lateral position the women were turned into supine position for the 3 minutes test of aorto-caval compression. This experiment was not part of standard clinical procedure, and the women were encouraged to discontinue the test if they felt unwell.

With the woman in a right lateral position, spinal anesthesia was induced in the L2-L3 vertebral interspace, and bupivacaine 10 mg + fentanyl 20 µg was injected through a 25-G non-traumatic needle (Pencan®, B. Braun, Melsungen, Germany). Concomitantly, we started a rapid intravenous infusion of saline 0.9 mg/ml (37°C, 10 ml/kg) and a phenylephrine bolus (0.25 µg/kg). This was followed with a phenylephrine infusion (0.25 µg/kg/min). During surgery, the patient was supine with an operating wedge under her right hip (19° Tempur pillow). Hypotension (SAP< 90 mmHg) was treated with an extra i.v. bolus of phenylephrine 30 µg if the HR was above 60 beats per minute (bpm) or with i.v. ephedrine 5–10 mg if the HR was 60 bpm or below.

The hemodynamic data were stored prepared for statistical analysis according to methods described in detail previously 5. The data set used in the analyses of the supine test is open access available for analyses ( Dataset 1).

Phenylephrine prophylaxis (25 µg/kg/min i.v.) was started when spinal anesthesia was given. Due to the phenylephrine prophylaxis we expected only a minor decrease in SAP after induction of spinal anesthesia. The analyses of spinal induced hypotension were based on the beat-to-beat measurements during the first 5 minutes, guided by inspection of the blood pressure curves. Based on a previously published study we expected a maximum decrease in SAP 3–4 minutes after spinal anesthesia 3, 7.

Statistical analyses

In order to capture the change of blood pressure, we considered the first order difference between two consecutive measurements d i ( t) = b i ( t + 1) – b i ( t), where b i ( t) is the average blood pressure of woman i measured in the 5 seconds before time t. The time t is discrete, so that t = 1, …, T = 40 corresponds to the three minutes measurement. Because data were still noisy, we smoothed the first order difference blood pressure curve d i ( t), t = 1, …, T of every woman, using a roughness penalty method 8, which led to a smoothed first order difference blood pressure curve denoted s i ( t), t = 1,…, T – 1. The method depends on a smoothing parameter λ. The results of the analysis below depend on the choice of λ. However, when clustering the women in separate groups, only a few women changed cluster when we changed the value of λ. To document such robustness, in the Supplementary material, we present results obtained for many values of λ ( Supplementary Table 1, Supplementary File 1 and Supplementary Figure 1). We found that λ=10000 allows to filter noise and capture details in the best way and in the rest we show the results for λ=10000.

We computed the principal components of the smoothed first order difference blood pressure curves s i ( t) for all women. The first two principal components represented in total 76.9% of the total variation of the data and were used for clustering. We clustered the 76 women using these two first principal components, into two or three groups. We used the k-means method 9, implemented in cclust package (Version 0.6–21) 10. More details and an intuitive description of the statistical methods are given in the Supplementary material ( Supplementary File 1).

We used the multivariate paired Hotelling’s T 2 test 11 to test the null hypothesis that the mean curves of the two clusters are equal for all time points against the alternative that the mean vectors of the two groups are not equal for all time points (H 0: μ 1 = μ 2 vs. H 1: μ 1μ 2) where μ 1 is the mean curve vector for S1 and μ 2 is the mean curve vector for S2. For comparison of the three clusters, we used the multivariate MANOVA to test the null hypothesis that the mean curves of are equal for all time points against the alternative that the mean vectors of the three groups are not equal for all time points (H 0: μ 1 = μ 2 = μ 3 vs. H 1: μ 1μ 2μ 3), where μ 1 is the mean curve vector for T1, μ 2 is the mean curve vector for T2 and μ 3 is the mean curve vector for T3. Statistical analyses were performed with R-software versions 3.2 to 3.3.2.

Results

Baseline measurements of HR, SAP, MAP, DAP, BMI, age, parity, gestational age, hours in fasting for solids, and estimated intake of liquids for the 76 included women are presented in Table 1. Mean supine SAP curves of the women with increasing values are available as Supplementary Figures 2. The principle component analysis identified two groups of women, by clustering the smoothed blood pressure difference curves during supine position into two groups as described in the methods. The first group, denoted S1, included 30 women, the second group, S2, the remaining 46 women. There were no statistically significant differences between the two clusters in baseline characteristics (see Supplementary Table 2).

Table 1. Patient characteristics.

Age (years) 34.5 (21, 43)
BMI (kg/m 2) 28.5 (SD 4.3)
Nulliparous 17 (of 76)
GA (weeks) 39 (36, 40)
Preop Hb (g/100 mL) 11.3 (SD 1.1)
Preop fasting (h) 12.0 (SD 2.8)
Baseline a:
Syst AP (mmHg) 134 (SD 14.0)
Mean AP (mmHg) 91 (SD 9.2)
Diast AP (mmHg) 68 (SD 7.5)
HR (bpm) 77 (SD 11.6)

Data presented as mean (SD, range, or total number).

BMI; body mass index, GA; gestational age,

a; baseline values representing a 60 sec mean of intra-arterial blood pressure (AP), and heart rate (HR) with the patient in left lateral position.

Figure 1 shows the mean SAP curves for the two groups, which are characterized by different trajectories. During supine position the mean SAP of the S2 patients dropped by about 10 mmHg in the first 100 seconds. After that the blood pressure was roughly constant. In the S1 group, the estimated mean SAP increased in the first 50 seconds of about 5 mmHg, and then started to decrease. The decrease was however less sharp than for group S2 (T2 = 216.89, p-value <0.001). The raw data curves of SAP for S1 and S2 groups are presented as Supplementary Figure 3.

Figure 1.

Figure 1.

Mean smoothed systolic arterial pressure (SAP) curves for S1 and S2 during supine position (mmHg) (panel a) and mean smoothed SAP curves for S1 and S2 during left lateral position (mmHg) (panel b). S1 in red, S2 in black.

We looked further into possible characterizations of the groups S1 and S2. We computed the standard deviation (SD) of the two groups for each time point. We found that S1 had minimum variation after 35 seconds, and then from the 100th seconds onwards had stable SD, whereas the SD in group S2 had a maximum at start and a minimum after 85 seconds (see Supplementary Figure 4).

In a second analysis, we clustered the 76 women into three groups. The reason for this was that the percentage (60%) of women in S2, characterized by a mean blood pressure drop, appeared to be larger than expected from clinical experience 12. When repeating the same analysis with three groups, we found a first group T1 with 28 women (36.8%) and a second group T2 with 15 women (19.7%) for which the average blood pressure increased in the first 30–50 seconds before then starting to drop. The difference between these two groups was in the different rate of decay, stronger for the women in T2. The third group T3 (43.5%) was characterized by a sharp drop in the average blood pressure right away, of approximately 10 mmHg in the first 100 seconds. After that the mean blood pressure was roughly constant. The three groups are not equal (Wilks Lambda= 0.033481, p-value<0.001). Mean hemodynamic measurements in left lateral and supine position defined by the principle components are presented in Supplementary Table 3.

Comparison between supine and left lateral position

We investigated if the clusters defined on the basis of the measurements while in supine position could be associated with blood pressure patterns while on left lateral position. This was not the case, as the mean blood pressures while on left lateral position for groups S1 and S2 were not significantly different ( Figure 1). The present data indicated that the drop in blood pressure when put in supine position for one group of women (S2), was not reflected in a blood pressure pattern when on the side which was different from the women in S1. Figure 2 shows how the first two principal components allow clustering the women into two groups (S1 and S2). The clustering algorithm puts each woman in one of two groups, so to divide them in the best way. The two clusters are nicely distinct, though there are borderline women.

Figure 2. Principal component analysis of continuously measured systolic arterial pressure (SAP) from 76 women in supine position into two groups S1 (black) and S2 (red).

Figure 2.

There is one point per woman, with coordinates given with respect to the first and second principal component. PC= principal component.

Comparison between supine position and during spinal anesthesia

When comparing the two clusters (S1 and S2) in terms of their mean SAP curves, we found that for all women in both groups mean SAP increased roughly 8 mmHg in the first 80–100 seconds after spinal anesthesia. After that, mean SAP started to decrease in both groups (S1 and S2), but the blood pressure drop for the women in S2 was clearly sharper than for the women in S1 ( Figure 3). We found that the group of women who experience a drop of blood pressure when put in supine position (S2) also have a drop which is faster and towards lower values compared to the women (in S1) who in supine position have a non-dropping blood pressure (S1) ( Figure 3a). Mean cardiac output decreased in both groups but more in the group S1 than in S2 ( Figure 3b). The mean SAP curves during anesthesia of the two groups were different (T2 = 370.5, p-value <0.001) as were CO (T2=294.3, p-value <0.001). Due to hypotension (SAP< 90 mmHg) three group S2 patients were given phenylephrine bolus 30 µg. None in group S1 received extra bolus phenylephrine.

Figure 3.

Figure 3.

Mean systolic arterial pressure (SAP)(mmHg) (panel a) and cardiac output (CO)(L/min) (panel b) after spinal anesthesia in the two clusters (S1 and S2).

When we considered SAP difference under spinal anesthesia (SAP (time=250 sec) – SAP (time=10 sec)) compared with SAP difference during the supine period (SAP (time=180 sec) – SAP (time=10 sec)), the correlation in S1 was -0.158 (p-value =0.40) and the correlation in S2 -0.48 (p-value = 0.0008). When we consider SAP difference under spinal anesthesia in the period from 90 sec (peak SAP, see Figure 3a) and 250 sec (SAP (time=250 sec) – SAP (time=90 sec)) compared with SAP during supine (SAP (time=180 sec) – SAP (time=10 sec)), the correlation in S1 was 0.095 (p-value = 0.62) the correlation in S2 was 0.15 (p-value = 0.32).

When repeating this analysis with the three groups division (T1, T2 and T3) we observed a comparable pattern ( Figure 4). Mean SAP increased during the initial 50–100 seconds followed by a decrease which was different for the three groups. The group of women with the most prominent drop in blood pressure when put in supine position (T3), also had the largest drop in average blood pressure under spinal anesthesia, and was still falling after 250 seconds. The group of women with the smallest drop in blood pressure in supine position (T1) did not have any drop in blood pressure under spinal anesthesia, with mean SAP remained constant. The intermediate group of women (T2), with some drop of blood pressure when in supine position, also had some drop under spinal anesthesia, but it was a slower drop than what was observed for T3 ( Figure 4a). Mean cardiac output decreased in all three groups ( Figure 4b). The group differences in mean SAP and CO curves for all time points were statistically significant (p-value < 0.001)(MANOVA test, Wilks’ Lambda= 0.31862, F-value= 31.36, and Wilks’ Lambda = 0.17622, F-value= 48.307, respectively).

Figure 4. Mean systolic arterial pressure (SAP)(mmHg) and CO (L/min) after spinal anesthesia in the three clusters T1, T2, and T3.1.

Figure 4.

Complete continuous invasive hemodynamic measurements in both left lateral and supine positions

Copyright: © 2018 Erango M et al.

Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Discussion

The three minutes supine observation test prior to spinal anesthesia identified women with risk of spinal induced hypotension. This finding is in concordance with previous studies reporting an association between heart rate increase or hypotension in the supine position and subsequent development of spinal hypotension and/or vasopressor requirements 1315. Kinsella et al., studied non-invasive arterial pressure and heart rate during a 5 min supine position test 13, and concluded that it was not possible to identify women with increased risk of spinal induced hypotension based on the observation of changes in arterial pressure. In contrast, the patients with a heart rate increase above 10 beats/min were more likely to develop hypotension. The novelty of our method is the potential benefit of in the future being able to predict hypotension and thereby individualize prophylactic therapy. We also demonstrated that this physiologic phenomenon present in the supine position was absent when observing the same women in left lateral position, supporting the concept of aorto-caval compression as the dominant mechanism of hypotension in the supine position 13.

The cluster of supine women having a falling SAP trend developed lower mean SAP and higher mean CO after spinal anesthesia. The decrease in systemic vascular resistance following spinal anesthesia is the dominant mechanism of spinal hypotension 2. A statistically significant difference in mean CO was observed between the two groups of women and the group with lowest SAP had a higher CO. The difference in CO was probably of limited clinical significance but indicates that the difference in the compensation of spinal anesthesia induced vasodilatation is the mechanism behind this observation. Women are more dependent on sympathetic tone during pregnancy than in non-pregnant state and the effect of spinal anesthesia is a rapid and prominent vasodilatation, leading to a decrease in blood pressure and increase in CO 7, 16. All women in this study received standard phenylephrine bolus and infusion, and this prophylaxis was effective as the mean SAP was stable ( Supplementary Figure 2c). However, three women had a drop in SAP below 90 mmHg and needed extra doses of phenylephrine to keep SAP close to baseline. All three women belonged to the group characterized with drop in SAP during supine position.

It was evident that the principal component analysis could not be replaced by physician observation and visual assessment of the continuous SAP curves, as the variance is large and individual changes difficult to interpret ( Supplementary Figure 2a). Functional data analysis with principal component analysis of continuous invasive measurements of SAP made a distinction between two separate clusters, one of the two characterized by decreasing mean SAP. It is important to notice that the two groups are not created because the women in S1 started with a lower average blood pressure than the women in S2. The two groups were created from the first order difference blood pressures, where the level does not have any influence. Principal component analyses have been used for making predictive models. In this population it would be interesting to identify patients at risk of severe spinal anesthesia induced hypotension and tailor the prophylactic treatment. The reliability of this method has to be proven in a new independent study.

Heart rate variability analysis captures the autonomic nervous activity and has been shown to predict spinal anesthesia induced hypotension in pregnant women 17, 18, and the ratio of the low frequency and the high frequency bands (LF/HF) correlated with spinal induced hypotension. Spinal anesthesia induced hypotension is caused by the vasodilatory effects, and to a lesser extent by aorto-caval compression 5, and the relationship between estimated autonomic activity (LF/HF) and spinal anesthesia induced hypotension is likely. The requirements of heart rate variability analyses limits the feasibility as stable ECG-signals has to be monitored at least 5 minutes to perform the frequency domain analyses 19. Sakata et al., analyzed heart rate variability during postural changes and found a correlation between low-to-high frequency ratio shift and spinal induced hypotension during cesarean delivery 18. Time domain analyses or continuous wavelet transform may be applicable 20 and requires shorter heart rate recordings. However, the method is not yet validated in the obstetric population.

Pulse transit time measures the interval from the electrical activity to the physical pulse wave in the periphery. Pulse transit time provide rapidly available beat-to-beat information of changing hemodynamics and may estimate the real time vasodilatory effect of spinal anesthesia 21, and is not suitable in a prediction model.

The external validity of a classification algorithm based on functional data analysis and clustering, as suggested in this paper, needs to be investigated in similar clinical assays. In this study all patients were treated according to a protocol including phenylephrine starting bolus and continuous infusion and we expected the blood pressure to be quite stable. Treatment protocols without intravenous phenylephrine prophylaxis will be different; Hypotension will be more profound and occur over a longer time sequence 3.

In a randomized placebo-controlled trial we showed that lower leg wrapping was less effective than phenylephrine bolus + infusion regarding spinal induced hypotension 3. Cardiac output increased in all groups. The study confirmed that spinal anesthesia is vasodilatory and that the changes in venous return, represented by heart stroke volume, were minor. Interestingly, heart stroke volume was higher in the lower leg wrapping group indicating some effect on venous return. Individual differences in the ability to compensate a fall in venous return may be the mechanism observed in the three minute supine test. Even though vasodilation is the main effect of spinal anesthesia it would be of interest to study how the prediction model works together with mechanical lower leg wrapping or when no hypotension prophylaxis is administered.

The algorithm should also be evaluated using a non-invasive continuous monitoring device. Non-invasive, continuous, beat-to-beat monitoring of blood pressure is available and included in clinical practice 22 and invasive monitoring is not standard clinical practice in obstetric anesthesia. Two classes of devices have been developed; Tonometric devices and volume clamp devices. The limits-of-agreement analyses of the latter 2 device classes using invasive measures as a reference standard are promising 22. Non-invasive measurement of blood pressure is not yet validated in obstetric anesthesia.

Conclusions

This work indicates the possibility to identify patients with increased risk of spinal anesthesia induced hypotension based on a simple pre-anesthesia test.

Data availability

The data referenced by this article are under copyright with the following copyright statement: Copyright: © 2018 Erango M et al.

Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication). http://creativecommons.org/publicdomain/zero/1.0/

Dataset 1: Complete continuous invasive hemodynamic measurements in both left lateral and supine positions 10.5256/f1000research.15142.d209363 23

Acknowledgements

The authors thank MD PhD Guro Grindheim, Oslo University Hospital, Oslo, Norway, for contributing during planning of the project and for valuable advices in interpretation, and Professor PhD Magne Thoresen, Oslo Centre of Biostatistics and Epidemiology, and University of Oslo, Oslo, Norway for his advices in planning and interpretation of the statistical analyses.

Funding Statement

The author(s) declared that no grants were involved in supporting this work.

[version 1; referees: 3 approved]

Supplementary material

Supplementary File 1- File containing expansion of the statistical methods and the following supplementary tables:

Supplementary Table 1: Baseline demographics in the cluster S1 and S2.

Supplementary Table 2: Assignment of each woman to one of the two groups for various values of λ

Supplementary File 2: File containing the following supplementary table:

Supplementary Table 3: Hemodynamic variables in clusters defined by principle components

Supplementary Figure 1: plot for each of the values of λ which were tested, the mean supine SAP of the women in each of the two groups. This is the mean of the smoothed supine SAP curves of each woman, and therefore depends on λ. With increasing values of λ, the two curves become more regular. There is little change, but noise is progressively reduced. By smoothing the data, noise is filtered out allowing for a better signal. The important observation is that the results are very stable for λ larger than 5000. This confirms that the choice of λ=10000 is appropriate, and that results would be qualitatively unchanged for other values of λ around 10000.

Supplementary Figure 2: SAP of all women during lateral position (a), supine position (b) and during anesthesia (c), and (in bold red) average SAP at each time point. We did not notice any general clear trend, though the average SAP had a weak negative decay. Notice the large variance among the women.

Supplementary Figure 3: Smoothed SAP curves for S1 and S2 during supine position and mean SAP (mmHg). Notice the large variability. The scale of the y-axis is much larger than in Figure 2 in the paper, to permit the plot of all women data.

Supplementary Figure 4: Standard deviation (SD) of the smoothed SAP curves for S1 and S2 during three minutes supine position, time point per time point. There was a high variability in each group, clearly more in the S2 group. We therefore tried with three groups, so that variance in each groups will be less. Notice that the principle component analysis allows filtering out noise.

References

  • 1. Cyna AM, Andrew M, Emmett RS, et al. : Techniques for preventing hypotension during spinal anaesthesia for caesarean section. Cochrane Database Syst Rev. 2006; (4):Cd002251. 10.1002/14651858.CD002251.pub2 [DOI] [PubMed] [Google Scholar]
  • 2. Assali NS, Prystowsky H: Studies on autonomic blockade. I. Comparison between the effects of tetraethylammonium chloride (TEAC) and high selective spinal anesthesia on blood pressure of normal and toxemic pregnancy. J Clin Invest. 1950;29(10):1354–1366. 10.1172/JCI102373 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Kuhn JC, Hauge TH, Rosseland LA, et al. : Hemodynamics of Phenylephrine Infusion Versus Lower Extremity Compression During Spinal Anesthesia for Cesarean Delivery: A Randomized, Double-Blind, Placebo-Controlled Study. Anesth Analg. 2016;122(4):1120–1129. 10.1213/ANE.0000000000001174 [DOI] [PubMed] [Google Scholar]
  • 4. Ngan Kee WD: The use of vasopressors during spinal anaesthesia for caesarean section. Curr Opin Anaesthesiol. 2017;30(3):319–325. 10.1097/ACO.0000000000000453 [DOI] [PubMed] [Google Scholar]
  • 5. Rosseland LA, Hauge TH, Grindheim G, et al. : Changes in blood pressure and cardiac output during cesarean delivery: the effects of oxytocin and carbetocin compared with placebo. Anesthesiology. 2013;119(3):541–551. 10.1097/ALN.0b013e31829416dd [DOI] [PubMed] [Google Scholar]
  • 6. von Elm E, Altman DG, Egger M, et al. : The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453–1457. 10.1016/S0140-6736(07)61602-X [DOI] [PubMed] [Google Scholar]
  • 7. Langesaeter E, Rosseland LA, Stubhaug A: Continuous invasive blood pressure and cardiac output monitoring during cesarean delivery: a randomized, double-blind comparison of low-dose versus high-dose spinal anesthesia with intravenous phenylephrine or placebo infusion. Anesthesiology. 2008;109(5):856–863. 10.1097/ALN.0b013e31818a401f [DOI] [PubMed] [Google Scholar]
  • 8. Ramsay J, Silvermann B: Functional Data Analysis. Springer Series in Statistics.In: Wiley Online Library,1998. [Google Scholar]
  • 9. Kanungo T, Mount DM, Netanyahu NS, et al. : An efficient k-means clustering algorithm: Analysis and implementation. IEEE transactions on pattern analysis and machine intelligence. 2002;24:881–892. 10.1109/TPAMI.2002.1017616 [DOI] [Google Scholar]
  • 10. Dimitriadou E, Hornik K, Hornik MK: Package ‘cclust’.2015. Reference Source [Google Scholar]
  • 11. Anderson T: An Introduction to Multivariate Analysis.John Wiley & Sons (New York).2003. Reference Source [Google Scholar]
  • 12. Kinsella SM, Lohmann G: Supine hypotensive syndrome. Obstet Gynecol. 1994;83(5 Pt 1):774–788. [PubMed] [Google Scholar]
  • 13. Kinsella SM, Norris MC: Advance prediction of hypotension at cesarean delivery under spinal anesthesia. Int J Obstet Anesth. 1996;5(1):3–7. 10.1016/S0959-289X(96)80067-7 [DOI] [PubMed] [Google Scholar]
  • 14. Dahlgren G, Granath F, Wessel H, et al. : Prediction of hypotension during spinal anesthesia for Cesarean section and its relation to the effect of crystalloid or colloid preload. Int J Obstet Anesth. 2007;16(2):128–134. 10.1016/j.ijoa.2006.10.006 [DOI] [PubMed] [Google Scholar]
  • 15. Jeon YT, Hwang JW, Kim MH, et al. : Positional blood pressure change and the risk of hypotension during spinal anesthesia for cesarean delivery: an observational study. Anesth Analg. 2010;111(3):712–715. 10.1213/ANE.0b013e3181e8137b [DOI] [PubMed] [Google Scholar]
  • 16. Sharwood-Smith G, Drummond GB: Hypotension in obstetric spinal anaesthesia: a lesson from pre-eclampsia. Br J Anaesth. 2009;102(3):291–294. 10.1093/bja/aep003 [DOI] [PubMed] [Google Scholar]
  • 17. Hanss R, Bein B, Francksen H, et al. : Heart rate variability-guided prophylactic treatment of severe hypotension after subarachnoid block for elective cesarean delivery. Anesthesiology. 2006;104(4):635–643. [DOI] [PubMed] [Google Scholar]
  • 18. Sakata K, Yoshimura N, Tanabe K, et al. : Prediction of hypotension during spinal anesthesia for elective cesarean section by altered heart rate variability induced by postural change. Int J Obstet Anesth. 2017;29:34–38. 10.1016/j.ijoa.2016.09.004 [DOI] [PubMed] [Google Scholar]
  • 19. Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. Circulation. 1996;93(5):1043–1065. 10.1161/01.CIR.93.5.1043 [DOI] [PubMed] [Google Scholar]
  • 20. Wachowiak M, Hay D, Johnson M: Quantification of Wavelet Band Metrics for Assessing Heart Rate Variability.In: World Congress on Medical Physics and Biomedical Engineering, June 7-12, 2015, Toronto, Canada Springer,2015:1026–1029. 10.1007/978-3-319-19387-8_250 [DOI] [Google Scholar]
  • 21. Sharwood-Smith G, Bruce J, Drummond G: Assessment of pulse transit time to indicate cardiovascular changes during obstetric spinal anaesthesia. Br J Anaesth. 2006;96(1):100–105. 10.1093/bja/aei266 [DOI] [PubMed] [Google Scholar]
  • 22. Bartels K, Esper SA, Thiele RH: Blood Pressure Monitoring for the Anesthesiologist: A Practical Review. Anesth Analg. 2016;122(6):1866–1879. 10.1213/ANE.0000000000001340 [DOI] [PubMed] [Google Scholar]
  • 23. Erango M, Frigessi A, Rosseland LA: Dataset 1 in: A three minutes supine position test reveals higher risk of spinal anesthesia induced hypotension during cesarean delivery. An observational study. F1000Research. 2018. 10.5256/f1000research.15142.d209363 [DOI] [PMC free article] [PubMed] [Google Scholar]
F1000Res. 2018 Aug 13. doi: 10.5256/f1000research.16496.r36887

Referee response for version 1

Julian Stander 1

I am pleased to have the opportunity to comment on this interesting article that has important implications for clinical practice.  I am a statistician and will restrict my comments almost entirely to the statistical analysis.  

Comments on the Statistical Analysis

I find that the statistical analysis presented to be sound and appropriate. 

I very much appreciate the use of techniques from functional data analysis to extract understanding from these data.  I was curious as to how the smoothing parameter lambda was selected.  I understand that you have performed a thorough study to document the robustness of the results to the choice of lambda.  I would, however, like to see a very brief discussion about whether lambda could be selected automatically, perhaps working in the Bayesian framework.  Implementing and performing such an analysis is clearly beyond the scope of this paper, and I am not asking for it to be done.

I did not understand from the paragraph beginning "We used the multivariate paired Hotelling's T^2 test..." on page 4 to what S1 and S2 refer.  I think that S1 and S2 should be defined in the previous paragraph.

For the test H_0: mu_1 = mu_2 = mu_3, I think that H_1 is not precisely stated.  I would state it simply as H_1: not H_0.  

I have no objection to you performing Hotelling's T^2 tests.  I was, however, wondering whether there were specific tests for  use with functional data.  Perhaps your tests are specific to functional data.  If they are not, a little discussion of this issue would be helpful.

On page 6 you state that "The clustering algorithm puts each woman in one of two groups, so to divide them in the best way".  In what way is it best please?  It is not quite clear to me from Figure 2, how you would choose the number of groups/clusters.  I was wondering whether this could be done in some automatic way.  Again, implementing and performing such automation is beyond the scope of this paper, but a brief discussion about it would be helpful and interesting.

I was left a little perplexed by the paragraph beginning "It was evident that the principal component analysis...".  I think that that paragraph needs to be revised with perhaps some more detail.  There could be a relationship here with the comments above about automating certain choices. Further discussion would be valuable.

General Comment

In the Abstract, there should be no comma between "observation" and "can".

I do not think that the Aims read like aims. I would like to see these re-written and then better referred to later in the article in a way that allows me to see that they have been satisfied.

I think that it would be helpful if you were to remind us about the meaning of the colours on page 7.

I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

F1000Res. 2018 Aug 8. doi: 10.5256/f1000research.16496.r36886

Referee response for version 1

Gianpaolo Scalia Tomba 1

An interesting study with potential clinical relevance for the identification of women possibly needing reinforced antihypotensive treatment during spinal anesthesia. The study identifies a group of women with stronger SAP reaction to spinal anesthesia based on pre-anesthesia SAP measurement dynamics. The study uses advanced statistical techniques such as smoothing, PCA and clustering to identify the relevant data patterns. Suppl Fig 2 shows how difficult this task would be without the statistical instruments. In the subsequent analysis, smoothing allows the signal to become clear, PCA to reduce the complexity of data and clustering to define groups. In conclusion, a study with a relevant aim, relevant data and relevant statistical analysis.

Detailed comments

In the Discussion, the authors point out that a new independent study will be needed to show the reliability of the findings, a wise comment when interesting patterns are found in one data set... Maybe the authors could also comment on how this group finding could be used for an individual classification method... They hint at a individual classification algorithm, but maybe a possible realization of such an algorithm should be at least sketched, especially for a non-statistical readership.

Otherwise, just minor details:

  • Abstract,Background: "...women with decreasing ...(SAP)..." According to later data, all women eventually had decreasing SAP, the problem being approaching a hypotension limit...

  • Abstract, Methods: way -> away

  • Abstract, Conclusions: As commented above, here a "simple test" is mentioned, some more details would be useful later on...

  • Background, para 1,l2: occur-> occurs

  • Methods, para 1, l10: are published -> have been published

  • Methods, para 3, l5: by to -> by

  • Statistical analyses: Maybe the last sentence (about R) should be first...

  • Statistical analyses, para 1, l10: in the rest -> in the rest of the paper

  • Statistical analyses, para 2, l2: represented in total -> represented

  • Statistical analyses, para 3, l6: the negation of H0 is that at least one is different, not that all are different...)

  • Results, para 1, l3: why "with increasing values"?

  • Results, para 1, l3: principle -> principal

  • Results, para 1, l3: The whole sentence beginning on this line should be reformulated, since it is the clustering of the data (blood pressure difference curves) reduced in dimensionality by principal component analysis that allowed the identification of two or three groups of women.

  • Results, Fig 1: it would be better, for comparability, if the two panes had the same y-axis

  • Results, para 3: the relevance of the analysis of standard deviations is unclear, also whether these are computed on raw SAP or on first order differences. In the Suppl Fig 4, the group S2 seems clearly lower than S1. Does this mean anything? Apparently, these results lead to the clustering in 3 groups...

  • Results, para 4: what are the relations between these 3 groups and S1/S2?

  • Are the women in T1+T2 = S1 and T3=S2?

  • Results, Comparison...supine/left, para 1, l7: maybe "borderline points" is better than "borderline women"...

  • Results, Comparison...supine/left, para 1, l7: maybe Fig 2 (and the related last 3 sentences) should be in the preceding subsection...

  • Discussion, para 1, l4: no comma (,) after Kinsella et al.

  • Discussion, para 4, l9: requires -> require

  • Discussion, para 5, l2: provide -> provides

  • Discussion, para 6, l4: Hypotension -> hypotension

I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

F1000Res. 2018 Jul 23. doi: 10.5256/f1000research.16496.r35875

Referee response for version 1

Kim Ekelund 1

The present paper is a fine example of how simple questions can be answered by simple studies. The authors have observed that although a standardized approach is used to treat the spinal caused vasodilation during a cesarean section (CS) some women will still experience hypotension despite prophylactic treatment with phenylephrine. They try to identify subsets of women with different susceptibility to vasodilation by register the hemodynamic response after placing them in a supine position for a few minutes. The hypothesis is that some women may need a more aggressive anti-hypotensive treatment during CS and these women might react more when placed on the back, compressing the inferior caval vein and aorta.

The paper is very interesting because it presents new clinical relevant knowledge on a frequent challenge. 

The paper is well written, most of the time easy to understand, but is also at some points cut to tight, and I feel as a reader, I am missing information, and this stays with me, despite several attempts to understand. 

Abstract

The abstract is maybe to condense, (maybe due to restriction from the publisher)… I have added a few comments to make the text easier to read… 

"Cesarean delivery is performed under spinal anesthesia, and vasodilation is the main cause for a drop in blood pressure. [During supine position] The compression of the aorta and inferior vena cava by the gravid uterus is of additional clinical importance. [The spinal caused] Hypotension may occur during cesarean delivery even if prophylactic infusion of phenylephrine is practiced. We have tested if a 3 minute supine observation, can identify a subset of women with decreasing systolic arterial pressure (SAP) under spinal anesthesia”

Background

“The compression of the aorta and inferior vena cava by the gravid uterus is known to reduce venous return and this is estimated to be of clinical importance after approximately 20 weeks of pregnancy. Pregnant women may find the supine position uncomfortable and avoid this position close to delivery. During cesarean delivery left lateral tilt is recommended to oppose this effect, and theoretically this should reduce the negative effects on maternal circulation”

Q: Consider moving this paragraph. The paragraphs before and after is about vasodilation – this is not.

Methods

”They were asked to assess specifically the extra pain caused by to the arterial cannulation”

Q: What does this mean?

“The indications for elective cesarean section were maternal request 43%, previous cesarean delivery 25%, breech presentation 8%, other obstetric reasons 13%, and other non-obstetric maternal or neonatal medical conditions 11%.”

Q: Why is this important in the present study?

“Concomitantly, we started a rapid intravenous infusion of saline 0.9 mg/ml (37°C, 10 ml/kg) and a phenylephrine bolus (0.25 µg/kg). This was followed with a phenylephrine infusion (0.25 µg/kg/min)”

  • and later in the same column:

”Phenylephrine prophylaxis (25 µg/kg/min i.v.) was started when spinal anesthesia was given. Due to the phenylephrine prophylaxis we expected only a minor decrease in SAP after induction of spinal anesthesia”.

Q: I read this as identical information. And if so – one of the paragraphs should be deleted.

“The analyses of spinal induced hypotension were based on the beat-to-beat measurements during the first 5 minutes, guided by inspection of the blood pressure curves. Based on a previously published study we expected a maximum decrease in SAP 3–4 minutes after spinal anesthesia 3 , 7.”

Q: I believe this should be moved to the Discussion. And be part of an explanation regarding why “three minutes” were chosen, over two, four, five or…. This is missing.

Statistical analysis:

Q: Although it seems to be a thorough explanation, I do not understand the majority. But I was intensively looking for a description of an algorithm, since this is mentioned later in the paper:

P6: “The clustering algorithm puts each woman in one of two groups, so to divide them in the best way.W

P8:  ”The external validity of a classification algorithm based on functional data analysis and clustering, as suggested in this paper,…”

P8: “The algorithm should also be evaluated using a non-invasive continuous monitoring device.”

Q: What algorithm are the authors referring to?

Results

I have read and re-read the paper the sections regarding the three clusters. I have a hard time understanding the reasons why this subdivision is necessary and more important clinical relevant.

Q: The authors compare the three clusters in supine position and after spinal. But in the present paper I am not presented with data from the supine position.

Figure 4.

Q: What are T1, T2 and T3, color-explanation is missing.

Q: The. Red and black curves in Figure 3b and 4b are identical. Explanation?

Q: Figure 4 text: “…T3.1”? Explanation of the colors should be added, please.

Discussion

”A statistically significant difference in mean CO was observed between the two groups of women and the group with lowest SAP had a higher CO. The difference in CO was probably of limited clinical significance but indicates that the difference in the compensation of spinal anesthesia induced vasodilatation is

the mechanism behind this observation. Women are more dependent on sympathetic tone during pregnancy than in non-pregnant state and the effect of spinal anesthesia is a rapid and prominent

vasodilatation, leading to a decrease in blood pressure and increase in CO7,16. All women in this study received standard phenylephrine bolus and infusion, and this prophylaxis was effective as the mean SAP was stable.”

Q: Treat the patient not the picture! How were the women doing? Was this clinical relevant?

“Principal component analyses have been used for making predictive models. In this.  population it would be interesting to identify patients at risk of severe spinal anesthesia induced hypotension and tailor the

prophylactic treatment.”

Q: I would be interesting if the authors would share some ideas on how prevent hypotension – an anti-hypotensive adjusted prophylactic treatment?

Heart rate variability, time domain analysis, and pulse transit time measures are all mentioned in the discussion, and maybe potentially relevant in this clinical setting.

Q: However, as a reader, I cannot see the obvious link between these investigations and the three minutes supine position test. I would like to have more information, linking to the right population, and to the results of the present study in the discussion, if this information should be considered.

Q: Finally, I would have preferred if the authors have been more strict on answering their Aims (P5) in the discussion.

I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

F1000Res. 2018 Jul 23.
Leiv Arne Rosseland 1

Thank you for the positive review of the manuscript. We will respond to the comments and questions and when we have received more referee assessments a revised manuscript will be prepared, including the editing of repeated phrases and the missing information in figure legends.

Specific questions/answers:

”They were asked to assess specifically the extra pain caused by to the arterial cannulation”

Q: What does this mean?

A: The parturients were informed about that arterial cannulation is painful. They were asked to consider if this extra pain could be accepted before considering giving their consent. In the revised manuscript we will describe this better.

“The indications for elective cesarean section were maternal request 43%, previous cesarean delivery 25%, breech presentation 8%, other obstetric reasons 13%, and other non-obstetric maternal or neonatal medical conditions 11%.”

Q: Why is this important in the present study?

A: The generalizability of the study relies on the sample being representative of the population and the information about the included women and the indication for cesarean is included for this reason.

Statistical analysis:

Q: Although it seems to be a thorough explanation, I do not understand the majority. But I was intensively looking for a description of an algorithm, since this is mentioned later in the paper:

P6: “The clustering algorithm puts each woman in one of two groups, so to divide them in the best way.W

P8:  ”The external validity of a classification algorithm based on functional data analysis and clustering, as suggested in this paper,…”

P8: “The algorithm should also be evaluated using a non-invasive continuous monitoring device.”

Q: What algorithm are the authors referring to?

In our paper we have used one after the other, three statistical procedures on our data, three algorithms, implemented as packages in the software language R. First we recognise that the measurements made for each women, though they come as a sequence of measures taken in discrete points, represent a function: measures of the pressure exist at any time point, and must be quite smooth. This means that measures made after each other, cannot change too much. The first algorithm estimates such a function for every women. The data for every women are then represented by such a function. The next procedure is finding the most important components of such function, by calculating the principal components and by using the first two components. At this point we have reduced the dimension of the very much, as every women is described by just two numbers (one per principal component). The third procedure is

clustering: the women are now assigned to one of two clusters, so that within a cluster they are quite similar and across clusters they are quite different. The three procedures realise our algorithm.

Results

I have read and re-read the paper the sections regarding the three clusters. I have a hard time understanding the reasons why this subdivision is necessary and more important clinical relevant.

Q: The authors compare the three clusters in supine position and after spinal. But in the present paper I am not presented with data from the supine position.

A: The data from supine and left lateral position in the three clusters are presented in Supplementary Table 3.We agree that the clinical relevance of this information (the three clusters) is not obvious. Without this information one might question why we have chosen only the PCA analysis with two clusters. Was there more information hidden that would be revealed when applying three? Or four? Hence, this is kept for methodological reasons.

Discussion

”A statistically significant difference in mean CO was observed between the two groups of women and the group with lowest SAP had a higher CO. The difference in CO was probably of limited clinical significance but indicates that the difference in the compensation of spinal anesthesia induced vasodilatation is

the mechanism behind this observation. Women are more dependent on sympathetic tone during pregnancy than in non-pregnant state and the effect of spinal anesthesia is a rapid and prominent

vasodilatation, leading to a decrease in blood pressure and increase in CO7,16. All women in this study received standard phenylephrine bolus and infusion, and this prophylaxis was effective as the mean SAP was stable.”

Q: Treat the patient not the picture! How were the women doing? Was this clinical relevant?

A: On average, the women were well treated with the combination of intra-venous fluid + phenylephrine prophylaxis/treatment. However, some patients will develop hypotension and the idea was to analyze if the simple supine position test could predict hypotension. The study should be regarded as an explorative assay, not a pragmatic clinical study. We believe we have demonstrated both the existence of a specific physiologic effect of supine position and that modern signal analyses may be useful in the development of prediction models. The clinical relevance of this has to be tested and proven in future trials.

“Principal component analyses have been used for making predictive models. In this. population it would be interesting to identify patients at risk of severe spinal anesthesia induced hypotension and tailor the

prophylactic treatment.”

Q: I would be interesting if the authors would share some ideas on how prevent hypotension – an anti-hypotensive adjusted prophylactic treatment?

A: Patients at risk according to this test (or further developments of the test, or other tests) should be treated state of the art to prevent hypotension. This includes lowering the local anesthetic dose injected spinally, increasing the dose of prophylactic phenylephrine (or at least included phenylephrine prophylaxis), left tilting the operating table adequately, and last but not least, consider using beat-to-beat monitoring, invasive or non-invasive monitor. We will include this information  in the revised discussion.

Heart rate variability, time domain analysis, and pulse transit time measures are all mentioned in the discussion, and maybe potentially relevant in this clinical setting.

Q: However, as a reader, I cannot see the obvious link between these investigations and the three minutes supine position test. I would like to have more information, linking to the right population, and to the results of the present study in the discussion, if this information should be considered.

A: We will like to keep this information in the discussion of different methods used and studied for prevention of spinal induced hypotension. If any of these methods prove sensitive and specific as prediction models, it is clinically relevant in the future. We will revise the discussion to make this clearer.

Q: Finally, I would have preferred if the authors have been more strict on answering their Aims (P5).

A: Thank you. We will try to indicate more specifically the answers to the aims in results and discussion.

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    Complete continuous invasive hemodynamic measurements in both left lateral and supine positions

    Copyright: © 2018 Erango M et al.

    Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

    Data Availability Statement

    The data referenced by this article are under copyright with the following copyright statement: Copyright: © 2018 Erango M et al.

    Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication). http://creativecommons.org/publicdomain/zero/1.0/

    Dataset 1: Complete continuous invasive hemodynamic measurements in both left lateral and supine positions 10.5256/f1000research.15142.d209363 23


    Articles from F1000Research are provided here courtesy of F1000 Research Ltd

    RESOURCES