Validity, reliability and stability of the portable Cortex Metamax 3B gas analysis system

Abstract

This study investigated the performance of the portable Cortex Metamax 3B (MM3B) automated gas analysis system during both simulated and human exercise using adolescents. Repeated measures using a Gas Exchange System Validator (GESV) across a range of simulated metabolic rates, showed the MM3B to be adequately reliable (both percentage errors, and percentage technical error of measurements <2%) for measuring expired ventilation (V _E), oxygen consumption (VO₂), and carbon dioxide production (VCO₂). Over a 3 h period, the MM3B was shown to be acceptably stable in measuring gas fractions, as well as V _E, VO₂, and VCO₂ generated by the GESV, especially at moderate and high metabolic rates (drifts <2% and of minor physiological significance). Using eight healthy adolescents during rest, moderate, and vigorous cycle ergometry, the validity of the MM3B was tested against the primary criterion Douglas bag method (DBM) and a secondary criterion machine known to be accurate, the Jaeger Oxycon Pro system. No significant errors in V _E were noted, yet the MM3B significantly overestimated both VO₂ and VCO₂ by approximately 10–17% at moderate and vigorous exercise as compared to the DBM and at all exercise levels compared to the Oxycon Pro. No significant differences were seen in any metabolic variable between the two criterion systems (DBM and Oxycon Pro). It is concluded the MM3B produces acceptably stable and reliable results, but is not adequately valid during moderate and vigorous exercise without some further correction of VO₂ and VCO₂.

Introduction

The measurement of oxygen uptake (VO₂) and its associated variables, carbon dioxide production (VCO₂) and expired ventilation (V _E), are now commonly performed in many laboratories around the world in order to assess cardiorespiratory fitness and the metabolic demands of various activities. Traditionally these measurements were undertaken in controlled laboratory conditions via open-circuit calorimetry using developments of the original Douglas bag method (DBM) (Douglas 1911). The advent of automated computerized metabolic gas analysis systems has generally overtaken the time-consuming and skill-dependent DBM in most labs (Macfarlane 2001). The DBM can be used in field trials (Daniels 1971), but it remains very limited due to its bulk, added air-resistance, and inability to easily undertake sequential measurements (Durnin and Passmore 1967). Consequently, a number of portable systems have been designed to acquire metabolic gas measurements during field studies.

One of the earliest portable systems was the fully mechanical Max-Planck respirometer, developed during the Second World War and often referred to as the Kofranyi–Michaelis respirometer (Johnson et al. 1967) after the authors of an early publication (Kofranyi and Michaelis 1949). The early automated portable systems (e.g., “Oxycon”, “Oxylog” and “K2”) were restricted to only the measurement of V _E and VO₂ (not VCO₂) (Macfarlane 2001). Modern technologies now permit portable systems weighing less than 2 kg to possess virtually all the data collection powers of their lab-based counterparts (often recording or telemetering breath-by-breath and heart rate data). One such system is the Metamax 3B (MM3B) system (Cortex, Leipzig, Germany), that is also marketed as the VmaxST in many countries, which supersedes the earlier Metamax I and II models that were shown to be valid and reliable (Medbo et al. 2002).

To be able to be used in discipline-specific field studies, these portable gas analysis systems have to be not only small and unobtrusive, but also reliable and valid. There is a lack of consensus on not only the level of precision and accuracy expected for measures of VO₂ and V _E (Macfarlane 2001), but also which method is most appropriate to assess both reliability and validity. For example, studies have reported the reliability of the MM3B by comparing data acquired from human participants twice on different days (Perkins et al. 2004), yet this analysis will include not only the technical error but also the biological within-subject variation. Only reliability comparisons of dual measurements using a gas exchange simulator will remove the within-subject variation, and this has been reported recently with the MM3B (Vogler et al. 2010). Three validation studies on the MM3B have been performed against the DBM (Brehm et al. 2004; Prieur et al. 2003; Vogler et al. 2010), and despite its limitations the DBM is often considered to be the criterion (Hodges et al. 2005), especially when precise metabolic calibrators are not available (Gore et al. 1997). Yet other validation studies on the MM3B have only used a previously validated automated system as its criterion measure (Laurent et al. 2008; Perkins et al. 2004), a method that has been questioned as not being a true gold standard (Meyer et al. 2005). Ideally, any metabolic gas analysis system, such as the MM3B, should have its reliability reported using a gas exchange simulator and its validity reported compared to at least the DBM. Only one extensive study has published such data on the MM3B (Vogler et al. 2010); however, this study focused on elite athletes at relatively high levels of performance and did not include resting or light activity. In order to determine if the MM3B could be applied to the common non-elite populations such as children, validity and reliability trials were needed at much lower physiological levels.

Furthermore, it is important for any metabolic gas analysis system to also demonstrate that, once calibrated, it does not drift significantly (i.e., is stable) over the usual data-acquisition period. For measurements taken in or near a laboratory this may only require determining the stability over a 20 min period (Prieur et al. 2003), but for some studies, the calibration of a portable system in the field may be problematic due to the need for carrying bulky calibration equipment, including large pressurized bottles of alpha-standard calibration gases. Thus, some portable systems may need to be initially calibrated in a laboratory and then transported and used at a significant time later for field studies. The identification of how each gas analyzer and resultant VO₂ and VCO₂ drifts in these situations is rarely reported (Atkinson et al. 2005). Only one study has reported on the stability of the Cortex Metamax 3B/VmaxST system, but this was for only a maximum of 20 min (Prieur et al. 2003) and many field studies may exceed this duration. Furthermore, it is important to match the type of validation as to how the testing system will be used (Unnithan et al. 1994), and although many automated metabolic systems are used on pediatric/adolescent populations, few systems have been actually validated using these groups (Unnithan et al. 1994).

The aim of this study was therefore to report on the suitability of the MM3B to measure variables over non-elite physiological ranges, especially those commonly found in pediatric/adolescent or elderly groups, as well as its ability to be used in prolonged field situations. We therefore studied the performance of the MM3B system for (1) reliability, using a commercially available gas exchange simulator; (2) stability/drift over a 3 h period; (3) validity, compared to both the criterion DBM as well as a previously validated gas analysis system (Jaeger Oxycon Pro) using an adolescent sample.

Methods

Participants

Eight healthy young volunteers (3 boys, 5 girls) were recruited with the following characteristics (mean ± SD): age 12.9 ± 3.6 years; height 150.1 ± 13.1 cm; mass 44.6 ± 11.5 kg. All subjects, as well as their legal guardians, provided a written informed consent after the project was approved by the Research Ethics Committee of the University of Hong Kong.

Equipment

Metamax 3B (breath-by-breath system)

The MM3B is a portable metabolic system composed of a measurement module and a battery module. These two parts are of the same size (120 × 110 × 45 mm) and designed to be worn on the chest with a harness, with a total weight of 1.40 kg. The MM3B measures volume using a bidirectional digital turbine. A 60 cm length of Nafion/Permapure sampling tube is attached to the turbine to permit analysis of the O₂ and CO₂ concentrations using an electrochemical cell and an infrared analyzer, respectively. VO₂ and VCO₂ were calculated using standard metabolic algorithms (Wasserman et al. 1999) employing the Haldane transformation, but with F_IO₂ and F_ICO₂ continuously measured, rather than assumed to be constant, in order to correct for changes in ambient conditions. The breath-by-breath data of respiratory volume and gas concentrations can be stored in on-board memory for later downloading to a PC, or sent immediately via telemetry to a PC. The system was tested using Metasoft 3 software, version 3.7.0 SR2.

Prior to using, the system was turned on for at least 20 min, and then calibrated prior to every test according to the manufacturer’s recommendations. This involves first calibrating the gas analysers by using a reference gas (14.97% O₂, 4.96% CO₂, balance N₂: ±0.02% absolute, Hong Kong Specialty Gases), and then verifying the calibration against ambient air. Secondly, a volume calibration was performed using a standardized 3-L syringe (5530 series, Hans Rudolph, Inc., MO, USA). For avoiding potential gas leakages known to be problematic with facemasks, all participants wore a nose clip and had a mouthpiece attached to the MM3B turbine.

Gas Exchange System Validator (GESV)

The GESV (MedGraphics, MA, USA; a similar, but updated, GESV is now sold by Vacumed, CA, USA) was a mechanical gas exchange system simulating human breathing and has a reported accuracy in producing VO₂ and VCO₂ of ±2% (Huszczuk et al. 1990). When a gas of known CO₂ concentration (recommended 21.00%) was added to the inspirate, the GESV expired gas of constant expired fractions at ambient temperature and pressure that could be used to simulate a range of VO₂ and VCO₂. The GESV could be adjusted so that it simulated breathing over a wide range of tidal volumes (0.5, 1.0, 1.5, 2.0, 2.5 and 3.0 L) at various respiratory rates (low = 10 breaths min⁻¹; medium = 20 breaths min⁻¹; and high = 40 breaths min⁻¹). These resulted in minute ventilations ranging from 5 to 104 L min⁻¹; VO₂ ranging from 0.30 to 2.81 L min⁻¹; and VCO₂ ranging from 0.29 to 2.69 L min⁻¹.

Douglas bag method (DBM)

With each participant wearing a nose clip and mouthpiece attached to a Radiax valve (dead space ~28 ml), all expired gases were collected in 250 L Douglas bags (WE Collins, Braintree, USA) using a 1 m length of Collins spiral tubing (38 mm ID) and a Collins 3-way stopcock. The mixed expirates were analyzed within 15 min using an S-3A oxygen and CD-3A carbon dioxide analyzer (Applied Electrochemistry, Sunnyvale, CA: now AEI Technologies) that had been previously calibrated using two alpha/primary-reference gases (26.13% O₂ and 0.00% CO₂; 13.94% O₂ and 5.96% CO₂; all gases ±0.02% absolute, Hong Kong Specialty Gases) and checked against ambient air (analyser linearity was checked using 0.00, 13.94 and 26.13% O₂, and 0.00, 4.96 and 5.96% CO₂ as well as ambient air). The volume of the Douglas bag was measured by a dry gas meter (Harvard, USA), with the aid of a vacuum pump on the exit port. The temperature of the expired gas was monitored with a telethermometer (YSI, Ohio, USA) placed at the outlet of the gas meter for later correction to “standard temperature pressure dry” (STPD), and “body temperature pressure saturated” (BTPS). All bags were checked for an absence of leaks and diffusion (no change in volume or composition of mixed expirate noted over a 30 min period), and each one flushed with expired gases before use to reduce the dilution effect of dead-space gas trapped in the bag and any rigid tubing.

Jaeger Oxycon Pro

The Oxycon Pro (Jaeger—now CareFusion, Germany; running JLab Software version 4.66.0) was used in its “mixing-chamber mode” with participants, wearing a nose clip, breathing via a mouthpiece and Radiax valve. The system was turned on for at least 30 min prior to use, and then fully calibrated for gases (14.00% O₂, 6.00% CO₂: ±0.02% absolute, Hong Kong Specialty Gases) and volumes before every test according to the manufacturer’s recommendations.

Studies

Study 1—reliability

Reliability trials of V _E, VO₂ and VCO₂ measured by the MM3B were examined by attaching a known CO₂ gas supply (20.62%) to the inspired port of the GESV as recommended by the manufacturer. Each trial consisted of the GESV working at 1.0 L at low respiratory rate, 1.5 L at medium respiratory rate, and 1.5 L at high respiratory rate, with the GESV inspiratory/expiratory port connected directly in-series to the MM3B turbine. Each trial was repeated twice at each level of V _E, VO₂, and VCO₂ in order to assess the reliability of this measure, all during the same day, with re-calibration of the MM3B between each trial.

Study 2—stability

To measure the stability/drift of two components of the MM3B (a) the gas analysers; a known gas (15.83% O₂ and 4.05% CO₂) was introduced to the sampling line of the MM3B at 0, 20, 40, 60, 120 and 180 min; (b) the full system; simulated VO₂ and VCO₂ were introduced by attaching a 20.62% CO₂ gas to the inspired port of the GESV as recommended by the manufacturer. A trial consisted of the GESV working at 1.0 L at low respiratory rate, 1.5 L at medium respiratory rate, and 1.5 L at high respiratory rate, with the GESV inspiratory/expiratory port connected directly to the MM3B turbine for 30 complete “breaths” at 20-min intervals until 180 min had elapsed. The 180-min time period was considered as the maximum period the MM3B would be used during field measures once it had been calibrated prior to data collection.

Study 3—validity

The eight participants performed one trial comprising stages of resting and incremental cycling exercise (Corival 400, Lode, The Netherlands). Each exercise stage lasted 13 min and measurements were made in-parallel (sequentially) with the DBM, the MM3B, and an Oxycon Pro system. The Oxycon Pro system acted as a “secondary criterion” as it has been shown to be a valid metabolic system in its mixing-chamber mode (Foss and Hallen 2005). The DBM collections of expired gas were used as the primary criterion and the order of measurement system followed a counterbalanced Latin-Square process so as to avoid any order effect (Bradley 1958). The expired gas collected by the DBM was immediately measured by the calibrated S-3A oxygen and CD-3A carbon dioxide analyzers and then passed through the dry gas meter whose accuracy (0.8% error) had been previously checked using multiple pumps of a 3-L calibration syringe (Hans Rudolph).

Each trial using the DBM, MM3B, and the Oxycon Pro began with a 10-min rest period, with gas collection over the final 5 min. Each participant then pedaled at a constant rate of 50 rpm in three or four stages, starting from 50W and with a 25W or 50W increment according to their perceived fitness and body size. Each exercise stage lasted for a total of 13 continuous minutes. Each participant exercised for 5 min whilst breathing into the first instrument, and then kept pedaling at the same rate for 1 min during which the experimenter quickly changed to the second measurement system. As the participant was already in steady-state exercise, the participant continued exercise for 3 min with the second measurement system, and then the procedure was repeated for the third measurement system. The three for four intensity levels were later classified as being representative of resting, moderate or vigorous exercise. The data used in calculation of all variables was the mean of the final 2 min of measurement with each device, which was considered to be during the steady state (supported by visual inspection of the heart rate). All trials were performed in a quiet laboratory in environmentally stable conditions; any variations in F_IO₂ and F_ICO₂ during DBM collections were noted from the Applied Electrochemistry analysers and appropriate corrections made in the DBM calculations.

Data analysis

Tests to examine any differences between the dependent variables of V _E, VO₂, VCO₂, F_EO₂, and F_ECO₂ included percentage differences, repeated-measures ANOVA, and the Bland–Altman analysis (Bland and Altman 1986). Univariate Intra-Class Correlation Coefficients (ICC), and t tests (for shift in the mean scores) were also used in the reliability tests between repeated measures. Technical Errors of Measurement (TEM) were generated for both the between-device validity comparisons (inter-TEM), as well as within-device reliability comparisons (intra-TEM), following similar procedures reported by Gore et al. (2000). SPSS (8.0) was used for most analyses, with the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ {\text{TEM}} = \sqrt {\sum {D^{2} /2N} } $$\end{document} where D is the difference between the pairs of measurement, and N is the number of measurement pairs.

Results

In the reliability trials of the MM3B in measuring V _E, VO₂, and VCO₂ with a re-calibration in-between (Study 1), significant but small differences were found for VO₂ and VCO₂ at all levels of frequencies. As shown in Table 1, the percentage differences between the two trials were consistency less than 2.5%, whilst the average intra-device TEM across all pumping frequencies was 0.2, 1.4 and 1.1% for V _E, VO₂ and VCO₂, respectively. Ideal ICC values (r = 1.00) were also obtained by the MM3B in the repeated measurements of V _E, VO₂, and VCO₂.

Table 1.

Reliability results, showing the mean ± SD values, percentage differences, and intra-device TEM (%) of V _E, VO₂ and VCO₂ (L min⁻¹) of repeated trials by the MetaMax 3B across a range of low, moderate, high metabolic rates generated by the Gas Exchange System Validator

	Metabolic rate	Trial 1	Trial 2	% difference#	Intra-device TEM (%)
V E (BTPS, L min−1)	Low	9.94 ± 0.06	9.93 ± 0.08	−0.1	0.1
	Medium	29.53 ± 0.28	29.69 ± 0.23	0.5	0.4
	High	58.06 ± 0.61	58.10 ± 0.65	0.1	0.0
VO2 (STPD, L min−1)	Low	0.30 ± 0.01	0.30 ± 0.00	1.9*	1.3
	Medium	1.62 ± 0.01	1.65 ± 0.01	1.8*	1.3
	High	2.74 ± 0.03	2.80 ± 0.03	2.3**	1.6
VCO2 (STPD, L min−1)	Low	0.29 ± 0.01	0.29 ± 0.00	2.5*	1.7
	Medium	1.57 ± 0.02	1.59 ± 0.02	1.2*	0.8
	High	2.66 ± 0.04	2.69 ± 0.04	1.1*	0.8

TEM% technical error of measurement expressed as a percentage of mean

^#Percentage differences were calculated from mean data using four decimal places, but the data are reported here to two decimal places only

* Significant difference (p < 0.05)

** Significant difference (p < 0.01) from t test

The stability/drift trials of the MM3B gas analysers (Study 2) showed that when compared to the baseline (0 min), significant but small differences were found for F_EO₂ and F_ECO₂ at each subsequent measurement time (20, 40, 60, 120 and 180 min). The differences ranged from 0.01 to 0.09% for F_EO₂ and −0.05 to −0.16% for F_ECO₂. Table 2 shows the descriptive statistics for the gas concentration in percentages and comparisons between each time point and the baseline by the MM3B. F_ECO₂ immediately dropped nearly 4% at the first 20-min interval and this under-measurement slowly improved until 120 min where only a 1% difference from the baseline existed. Very small, yet statistically significant, increases were also seen in F_EO₂ as it slowly drifted upwards.

Table 2.

Stability/drift results, showing the mean ± SD values measured by MetaMax 3B of F_EO_2, and F_ECO_2, as recorded from the calibration gas bottle, as well as VO₂ and VCO₂ at different metabolic rates (low, medium, high) as simulated by the Gas Exchange System Validator, over a 180-min period

Time (min)	FEO2 (%)	FECO2 (%)	VO2 (L min−1)			VCO2 (L min−1)
			Low	Med	High	Low	Med	High
0	15.84 ± 0.00	4.10 ± 0.02	0.30 ± 0.00	1.67 ± 0.02	2.82 ± 0.03	0.28 ± 0.00	1.57 ± 0.02	2.66 ± 0.04
20	15.85 ± 0.00 (0.1***)	3.94 ± 0.04 (−3.9***)	0.30 ± 0.00 (−0.1)	1.66 ± 0.01 (−0.4)	2.81 ± 0.04 (−0.5)	0.29 ± 0.00 (1.5***)	1.58 ± 0.02 (0.7)	2.68 ± 0.04 (1.1*)
40	15.87 ± 0.00 (0.2***)	3.99 ± 0.03 (−2.7***)	0.30 ± 0.00 (−0.7)	1.67 ± 0.01 (0.1)	2.82 ± 0.03 (−0.2)	0.28 ± 0.01 (0.7)	1.58 ± 0.01 (1.1*)	2.68 ± 0.03 (0.8)
60	15.88 ± 0.00 (0.3***)	3.97 ± 0.03 (−3.2***)	0.31 ± 0.00 (2.0***)	1.67 ± 0.02 (−0.2)	2.83 ± 0.03 (0.3)	0.29 ± 0.00 (3.4***)	1.58 ± 0.02 (0.8)	2.67 ± 0.03 (0.4)
80	–	–	0.31 ± 0.00 (2.8***)	1.68 ± 0.02 (0.6)	2.84 ± 0.03 (0.9)	0.29 ± 0.00 (3.8***)	1.59 ± 0.02 (1.3*)	2.68 ± 0.04 (1.1)
100	–	–	0.31 ± 0.00 (2.7***)	1.68 ± 0.01 (0.6)	2.85 ± 0.03 (1.0*)	0.29 ± 0.00 (3.5***)	1.58 ± 0.01 (0.6)	2.68 ± 0.04 (1.0)
120	15.91 ± 0.00 (0.4***)	4.05 ± 0.03 (−1.2***)	0.31 ± 0.00 (2.6***)	1.69 ± 0.02 (1.1*)	2.84 ± 0.03 (0.8)	0.29 ± 0.00 (3.2***)	1.58 ± 0.02 (1.0)	2.67 ± 0.05 (0.4**)
140	–	–	0.31 ± 0.00 (2.7***)	1.69 ± 0.01 (1.0**)	2.86 ± 0.03 (1.5***)	0.29 ± 0.00 (3.2***)	1.58 ± 0.02 (0.8)	2.69 ± 0.03 (1.3)
160	–	–	0.31 ± 0.00 (2.6***)	1.69 ± 0.02 (1.3**)	2.85 ± 0.03 (1.0*)	0.29 ± 0.00 (2.8***)	1.58 ± 0.02 (0.8)	2.68 ± 0.04 (0.8*)
180	15.93 ± 0.00 (0.6***)	4.05 ± 0.03 (−1.2***)	0.31 ± 0.00 (2.1***)	1.69 ± 0.01 (1.3***)	2.87 ± 0.03 (1.8***)	0.29 ± 0.00 (2.1***)	1.58 ± 0.01 (0.7)	2.69 ± 0.04 (1.1)

The resultant relative percentage error in comparison with baseline, at time = 0, is shown in brackets

Significantly different to time at 0 min (ANOVA)

Error (%)  difference between F_EO_2, F_ECO₂, VO₂ or VCO₂ values as [each time (min)−baseline (0 min)]/value of baseline × 100%, and was calculated from mean data expressed as four decimal places

* p < 0.05

** p < 0.01

*** p < 0.001

Results of the full system analysis of how VO₂ and VCO₂ drifted over time (Study 2, see Table 2) showed that at low metabolic rate, VO₂ started to drift upwards from the baseline at the 60th minute onwards and resulted in an average of 2% difference. A significant drift of VCO₂ was found at the 20th minute and continued to deviate significantly from the baseline at the 60th minute onwards and this resulted in an average of 3% difference. At medium and high metabolic rates, some values slightly and significantly higher than the baseline VO₂ were found around the 100–120 min mark. For VCO₂ at medium and high metabolic rates, there were few significant or consistent trends to the drift.

The descriptive statistics for differences between the MM3B as compared to the primary criterion (DBM) and the secondary criterion (Oxycon Pro) are shown in Table 3 (Study 3), whilst detailed comparisons for the resting, moderate, and vigorous activity between the DBM and MM3B using data from the Bland–Altman analysis, percentage differences, as well as TEM are shown in Table 4. The combined data for all activities are shown in Fig. 1, where no systematic error with some proportional random error is seen in V _E, whilst both VO₂ and VCO₂ show evidence of both proportional systematic and random errors (Atkinson et al. 2005). No statistically significant differences in V _E were seen between the DBM and MM3B despite the overall TEM% averaging around 11%, and the MM3B underestimating resting V _E by nearly 9%. However, significant differences were seen in both VO₂ and VCO₂ for moderate and vigorous activity, with the MM3B overestimating these variables by more than 10% and producing TEM% values that varied from 9 to 18%. When compared to the Oxycon Pro, the MM3B varied significantly at all levels of VO₂ and VCO₂, but only for moderate V _E (Table 3). Interestingly, Table 3 also shows that when compared to the primary criterion DBM, the secondary criterion Oxycon Pro showed no significant differences in V _E, VO₂ or VCO₂ at rest, moderate or vigorous activity.

Table 3.

Validity results, showing the mean ± SD values of physiological variables measured by the three metabolic systems during parallel data collection at rest, moderate, and vigorous cycle exercise

	Activity	Douglas bag	MetaMax 3B	Oxycon Pro
V E (BTPS, L min−1)	Rest	8.72 ± 1.03	7.93 ± 1.73	7.72 ± 1.82
	Moderate	29.89 ± 8.19	30.61 ± 9.78^	28.73 ± 8.31
	Vigorous	66.00 ± 20.52	66.25 ± 19.16	62.54 ± 18.26
VO2 (STPD, L min−1)	Rest	0.27 ± 0.04	0.30 ± 0.08^	0.25 ± 0.06
	Moderate	1.12 ± 0.33	1.24 ± 0.41*^	1.12 ± 0.36
	Vigorous	2.32 ± 0.46	2.59 ± 0.52*^	2.37 ± 0.44
VCO2 (STPD, L min−1)	Rest	0.22 ± 0.04	0.26 ± 0.07^	0.20 ± 0.06
	Moderate	1.03 ± 0.31	1.17 ± 0.40*^	1.01 ± 0.33
	Vigorous	2.25 ± 0.52	2.64 ± 0.66*^	2.17 ± 0.43

* Significantly different to Douglas bag measurements (ANOVA, p < 0.05)

^ Significantly different to Oxycon Pro measurements (ANOVA, p < 0.05)

Table 4.

Validity analyses of the Metamax 3B data against the Douglas bag (primary criterion) during parallel data collection at rest, moderate, and vigorous cycle exercise

Variable	Activity level	Bland–Altman bias (L min−1)	% difference	Inter-device TEM (%)
		Mean ± SD (95% LOA)	Mean ± SD	Mean
V E (BTPS, L min−1)	Rest	−0.79 ± 1.62 (2.45, −4.04)	−8.8 ± 19.3	17.0
	Moderate	0.72 ± 2.76 (6.24, −4.80)	1.5 ± 9.8	7.0
	Vigorous	0.25 ± 10.02 (20.28, −19.79)	2.2 ± 13.3	9.7
VO2 (STPD, L min−1)	Rest	0.03 ± 0.05 (0.14, −0.07)	10.6 ± 19.3	14.0
	Moderate	0.12 ± 0.14** (0.41, −0.16)	9.7 ± 13.2**	10.9
	Vigorous	0.27 ± 0.16** (0.59, −0.05)	11.8 ± 7.6**	9.4
VCO2 (STPD, L min−1)	Rest	0.04 ± 0.05 (0.14, −0.06)	17.3 ± 21.8	18.0
	Moderate	0.14 ± 0.11** (0.36, −0.08)	12.5 ± 9.8**	10.6
	Vigorous	0.39 ± 0.23** (0.85, −0.07)	17.4 ± 8.1**	12.8

Data shown from Bland–Altman analyses (mean ± SD of bias, and 95% limits of agreement, LOA), percentage differences between means, and the inter-device TEM (%)

Bias and differences computed for data calculated from the Metamax 3B–Douglas bag

TEM% technical error of measurement expressed as percentage of mean value

** Metamax 3B significantly different to Douglas bag measurements (ANOVA, p < 0.01)

Fig. 1.

Modified Bland–Altman plots showing agreement between measurements taken across all activities combined (rest, moderate, vigorous) for all eight subjects from the Metamax 3B (MM3B) and criterion Douglas bag method (DBM). Y axes showing the absolute differences in V _E (BTPS), VO₂ (STPD) and VCO₂ (STPD) plotted against the average values of both methods (X axes)

Discussion

This is the first study to comprehensively examine the reliability and prolonged stability of the Cortex Metamax 3B over multiple ranges of simulated exercise conditions, as well as performing validity comparisons against both a primary criterion DBM, and a secondary criterion (Oxycon Pro) over metabolic ranges suited to non-elite participants (e.g., adolescents/elderly).

Meyer et al. (2005) reported that despite the need for establishing the reliability of portable gas analysis devices, few studies have addressed this issue; however, three studies appear to have reported data on the reliability of the Metamax 3B. The study by Perkins et al. (2004) examined reliability of the MM3B but used repeated measurements on human participants, which inflates the variability as it combines the relatively large biological error of the participants and the smaller technical error of the machine [their contributions to the total variability has been estimated to be 90% and 10% respectively (Macfarlane 2001)]. Even so, Perkins et al. (2004) reported the MM3B had extremely high single and multiple trial reliabilities and with narrow confidence intervals. Prieur et al. (2003) reported data from a stability trial using a comprehensive gas exchange simulator that showed the MM3B to be very reliable, although data were provided for only a single metabolic rate (VO₂ = 2.6 L min⁻¹). Data from the Vogler study (Vogler et al. 2010) also showed that the MM3B to be extremely reliable, with typical errors that ranged from 2.0% (VO₂) and 3.6% (V _E), which were superior to the reliability of their criterion Douglas bag system.

The reliability data from our study that used the GESV to simulate a wide range of conditions likely to be experienced in normal field trials (e.g., low, medium, and high metabolic rates), showed that the technical variability of the MM3B measurements was adequately low. The relative percentage errors for V _E, VO₂ and VCO₂ all being typically less than 2% between tests, with the TEM% generally less than 1.5%. These reliability results compare favorably with a 1% relative error generated from a complex automated calibration system (Gore et al. 1997), and is below the TEM reliability limit of 3% as recommended by the Australian Sports Commission (Gore 2000) for these variables.

An important aim of this study was to examine the stability (or resistance to drift) of key variables measured by the MM3B (gas fractions, VO₂, VCO₂) over a 3-h period that would reflect the longest likely time the MM3B would be used in the field after calibration in the lab (e.g., 60 min travel time to destination, 30 min preparation on-site, then allowing up to 90 min of episodic data collection). In countries like Hong Kong, where some field measures are undertaken in remote locations with no parking for private vehicles, it is often not feasible to transport calibration equipment to the site, necessitating prior calibration in the lab and reliance that the equipment is adequately stable over time. Our analysis, although limited to static laboratory conditions, therefore partially addresses the comment by Atkinson et al. (2005) that insufficient data is available on how stable specific gas analysis systems are, as does the paper by Eriksson et al. (2011).

The results of this study agrees with that of Prieur et al. (2003) that the MM3B shows some statistically significant drift, however, the absolute magnitude of this drift is small. The relative errors appear to be large (2–3%) only when the comparative original value is small (i.e., low denominator in the resting conditions), but under typical moderate-to-vigorous exercise conditions, the absolute error is relatively minor and is seen more in VCO₂ and likely due to a greater drift in the F_ECO₂ measurement. As these drifts in VO₂ and VCO₂ during simulated intermittent exercise lasting 180 min were all below 2%, there are unlikely to be of physiological importance, to the extent that the MM3B can be considered very stable.

During moderate and vigorous exercise the MM3B significantly overestimated VO₂ and VCO₂, but not V _E, by 10–17% when compared to the primary criterion DBM, but at all VO₂ and VCO₂ values during exercise when compared to the secondary criterion Oxycon Pro. In comparison, there were no significant differences in VO₂, VCO₂ or V _E across all conditions between the primary (DBM) and secondary (Oxycon) criterion machines, which support a previous study showing the Oxycon Pro’s mixing-chamber mode to be very accurate (Foss and Hallen 2005). However, for the key VO₂ variable, previous validation studies on the MM3B have produced inconsistent findings, with both overestimates (Perkins et al. 2004; Vogler et al. 2010) and underestimates (Brehm et al. 2004; Laurent et al. 2008; Prieur et al. 2003) being reported. The relatively large percentage errors reported in our study for VO₂ and V _E by the MM3B are generally higher than those reported in the above validation studies, and also exceed the 4–10% guidelines recommended by some (Brehm et al. 2004; Laurent et al. 2008; Vogler et al. 2010), although these limits are not universally agreed upon (Macfarlane 2001). It is worth noting that some percentage differences (in the means) between the MM3B and DBM in Table 4 were relatively small (e.g., V _E during moderate and vigorous exercise, <2.5%), yet were associated with relatively large TEM% scores of >7%. This was possibly due to large (quasi-symmetric) variation in the pairs of data around the mean, as the TEM% is sensitive to the degree of variability in the data pairs (and systematic error), yet this variability is not reflected in the percentage difference of the mean scores as it is only sensitive to systematic error.

It is unclear why our relative errors in VO₂ and VCO₂ are higher than other MM3B validation studies, as it is unlikely this was due to errors made in the Douglas bag assessments, as there was good agreement between the primary DBM and secondary Oxycon Pro criterions. Nor was greater variation likely to be due to our small number of participants (n = 8), as other validation studies have also used comparable numbers (n = 8–11) of participants (Brehm et al. 2004; Laurent et al. 2008; Prieur et al. 2003; Vogler et al. 2010). The slow drift upwards in VO₂ and VCO₂ over time by <4% at a low metabolic rate and <2% at the two higher metabolic rates, as reported in Table 2, may partially account for some, but not all, of the error. It would appear that the additional challenges of measurement during dynamic human exercise, such as movement shocks, gas leakages, saliva entrapment, and variations in gas and flow waves as suggested by Prieur et al. (2003), might incur greater measurement errors that are not seen during static testing using a mechanical simulator.

This study contains several limitations. Ideally, validations should be done using a serial method so that all expired gases passes sequentially through the MM3B and then into the Douglas bag; however, as outlined by others (Prieur et al. 2003; Vogler et al. 2010), and confirmed in our lab, this was not possible due to interference during simultaneous measurements, and instead we used separate (parallel) trials. The stability/drift analysis was performed only under stable laboratory conditions and without the device being subjected to regular movements and environmental changes as might occur during transport outdoors to a remote venue. It is unknown how changing environmental conditions might influence the validity and reliability of the MM3B. As we were interested in evaluating the MM3B for prolonged field studies on children, the participants we recruited for the validation phase of this study were not highly trained athletes, hence the upper range of their metabolic responses even during vigorous exercise are unlikely to be cover the range necessary to study elite athletes.

Summary

In many situations the MM3B will be used in field studies and when used at remote sites the results of this study suggest this device remains acceptably stable (significant variations were of minor physiological importance) for periods up to 3 h. The MM3B is also very reliable, but appears to be insufficiently valid when measuring VO₂ and VCO₂ during moderate and high intensities (evidence of proportional systematic and random errors), although these errors may be mitigated using a simple linear regression equation.