Validation of algorithms used in commercial off-the-shelf dive computer

Abstract

Introduction

Whilst the US Navy has been very systematic about validating Navy dive computer algorithms, there has been little documented or published evidence of rigorous testing of the algorithms in commercial off-the-shelf dive computers. This paper reports the evaluation of four algorithms used in these − Bühlmann ZHL-16C; VPM-B; Suunto-RGBM; EMC-20H − by comparison with US Navy experimental dives with known decompression sickness outcomes.

Methods

Three specific tests were developed to test the algorithms' ability to mitigate decompression sickness: Total decompression time; no stop times and first stop depth. Output of commercial decompression algorithms were compared to either the probability of decompression sickness (P_DCS) results from US Navy man-trials or statistical models derived from P_DCS data. The algorithms were first tested with default conservative factors, then these factors were adjusted if the tests were not initially passed. The last verification step was to compare the output of the wrist computer with that of the full desktop algorithm.

Results

This testing indicated that, whilst none of the four passed all of the proposed tests with factory-default conservatism, ZHL-16C and Suunto-RGBM could be made to pass by adjusting user-defined settings.

Conclusions

Man-trial data on P_DCS is available to the non-US Navy scientific community for testing of commercial decompression algorithms. This type of validation testing can be very informative on how to best use available commercial dive computers to improve diver safety.

Introduction

In the last few decades, there has been considerable work on the development of new decompression algorithms, as well as variations on existing algorithms. Many of these algorithms are used in commercial off-the-shelf dive computers. During this time, the US Navy Experimental Diving Unit (NEDU) has systematically validated Navy dive computer algorithms against the incidence of decompression sickness. However, there has been little documented or published evidence of rigorous testing of the algorithms in commercial off-the-shelf dive computers used by recreational divers. In a meeting abstract, testing was reported of a wide sampling of commercial dive computers against the few dive profiles for which there are relatively good estimates of the probability of decompression sickness (P_DCS).[ 1] Since that brief report, there have been comparative studies that have simply compared different commercial off-the-shelf dive computers against each other using the same depths and bottom times (defined as the difference between the time the diver leaves the surface and the time at which the diver leaves the maximum depth).[ 2 , 3] These two studies showed a wide variation of results in terms of no-stop times (NST, defined as the maximum bottom time for a given depth for which a direct ascent is allowed with no staged decompression) and total decompression time (TDT, defined as the sum of times spent at decompression stops plus the time to travel from depth to surface) varying by up to a factor of two. However, the issue of whether any specific computer (or algorithm) provided too little decompression time or too much was not addressed. This paper assesses some of the algorithms used in commercial off-the-shelf dive computers.

Methods

The recommendations from reference 4 were used to develop the testing process: 1) define requirements; 2) validate each algorithm against those requirements; 3) verify to the best extent possible that the algorithms evaluated are the same as those implemented in the chosen dive computers.4 The main requirements were that the algorithms prescribe decompression schedules that result in an acceptably low P_DCSS. In addition to this main requirement, one would like this dive profile to be efficient, or not significantly longer than needed as determined by TDT for a given P_DCS. Any algorithm can always result in lower P_DCS by arbitrarily increasing shallow-depth stop times. However, this can introduce other hazards to the diver, such as running out of breathing gas or, in cold water, hypothermia. Since the ultimate objective was to reduce PDCS in divers, this study included higher-risk dive profiles, where the product of the pressure (in bar) and the square root of the bottom time (minutes) or PRT was greater than 25.[ 5]

Four different decompression algorithms were tested (Table 1). These algorithms were chosen for evaluation owing to the availability of implementation in desktop computer decompression planning software. More details of these algorithms are available in the literature; some algorithms are better documented in the open literature than others.[ 6 - 8] Some details on EMC-20H came directly from Cochran Undersea Technology (Corso J, personal communication, 2016). For purposes of this study, the different algorithms were tested as ‘black boxes’ with known inputs and outputs, using desktop versions of the algorithms.

Table 1. Overview of decompression algorithms assessed; Exp – exponential .

Algorithm	Category	On-gas/off-gas	Number of compartments	Desktop software	Wrist unit used for verification
ZHL-16C	Dissolved gas (Haldanean)	Exp/exp	16	MultiDeco 4.12	Shearwater Perdix
EMC-20H	Dissolved gas	Exp/linear	20	Analyst 4.01	N/A
VPM-B	Dual phase	Exp/exp	16	MultiDeco 4.12	Shearwater Perdix
Suunto-RGBM	Dual phase	Exp/delayed exp	9	DM5 V1.2.47	Zoop-Novo

The four algorithms tested were all deterministic in nature, based on either gas loading or bubble formation in multiple tissue-type compartments. In this study, the four algorithms were validated against predictions made by probabilistic models derived from man-trial P_DCS data and by using the data directly. Since we know from the previous comparative studies that these different algorithms will produce markedly different results, the basic differences between the various approaches were looked at first. Two of the algorithms (ZHL-16C and EMC-20H) were in the category of 'dissolved gas' or 'tissue loading algorithms' (Haldanean) and the other two (VPM-B and Suunto-RGBM) were dual-phase (bubble) algorithms. Also, within each of these two physiological-based paradigms, there are still major differences, not just the number of compartments, but more importantly the functional form of the tissue loading process.

The algorithms were evaluated as implemented in the following software packages: MultiDeco 4.12 (Bühlmann ZHL-16C and VPM-B), DM5V1.2.47 (Suunto-RGBM) and Analyst 4.01 (EMC-20H). All dive computers tested employed some form of conservatism factor, a user-adjustable parameter that changed the computed decompression requirements. The ZHL-16C algorithm employed a two-dimensional conservatism factor, referred to as a 'gradient factor' (GF). GFs, ranging from 0% to 100%, were added to the ZHL-16C algorithm such that they modify the M-value equations in the Bühlmann model, and hence alter the prescribed decompression profile.[ 9] The lower gradient factor (GF-Lo) controls the depth of the first stop. The higher gradient factor (GF-Hi) affects total decompression time. More details on this topic can be found elsewhere.[ 9] Setting values of GF-Hi = GF-Lo = 100 results in the original ZHL-16C model. EMC-20H used a single-value conservatism factor, input as a percentage ranging from 0% (default) to 50%. Suunto-RGBM utilised a single-value conservatism factor which could be set to 0 (default), +1 or +2. The VPM-B implementation used in MultiDeco enabled a similar single-value conservatism factor, but with a range from 0 (default) to +5. All algorithms were initially evaluated with their default conservatism factors: GF-Lo = GF-Hi = 100% for ZHL-16C, 0% for EMC-20H, and 0 for VPM-B and Suunto-RGBM. If any algorithm failed a test then its conservatism factor(s) was/were adjusted iteratively to determine whether a suitable setting could be found to allow the algorithm to pass the test.

TOTAL DECOMPRESSION TIME

To assess these algorithms against the requirement of low PDCS, we first re-visited methods and models designed to specifically assess the suitability of the US Navy's decompression schedule (used at that time). Data on single-level, non-repetitive, nitrogen-oxygen dives from the US Navy Decompression Database were fitted to a logistic regression that resulted in PDCS isopleths as a function of bottom time and TDT.[ 10 , 11] As in the original paper, it was postulated that after depth and bottom time, TDT is a strong candidate for the most influential variable in modeling DCS (as compared to profile/stop-time combinations),[ 10] and has been corroborated by other studies.[ 12 , 13]

The domain of applicability of the current study was limited to 13 < PRT < 37, so the 'StandAir' model, which is based on data from standard air dives which had depths of less than 190 feet’ sea water (fsw) and bottom times of less than 720 minutes (min), was utilised.[ 10] This model was considered to be reasonable except at the two depth extremes (nominally < 60 fsw and > 190 fsw). In the original study, the TDT required by the algorithm-under-test was compared to the P_DCS isopleths from the statistical model. It was found that the TDT required by the algorithm-under-test lay between the 2% and 3% P_DCS isopleths and thus the algorithm was deemed acceptable for US Navy use.[ 10] Figure 1 (reproduced from Figure 8D from an additional reference[ 11]) shows graphically the relationship between the computed PDCS isopleths from the StandAir model to the original data underlying the model.[ 11] The symbols show grouped data from Navy trials that resulted in DCS incidence at depths between 145 and 154 fsw and bottom times were rounded to the nearest 5 min. The figure/symbology is described as follows:

Figure 1.

Reproduction of Figure 8D from reference 11. P_DCS isopleth contours from the StandAir model plotted with the data the model was derived from; depth is 150 fsw; see text for explanation of symbols

"Triangles locate the bottom times and TDTs of particular dive trials that resulted in at least one case of DCS. Circles show trials that produced no DCS in any divers."[ 11]

In the current study, we selected the 3% P_DCS isopleth as an initial standard of comparison as a compromise between managing DCS risk while not requiring excessive total decompression times.

NO-STOP TIMES

As a special/limiting case these algorithms were also evaluated for NST, i.e., for a given depth what is the maximum bottom time for the algorithm that allows a direct ascent?. For the NST limit test, extensive research performed by NEDU in 2009 was leveraged.[ 14] In this research, the basic methodology used was to compare no-stop decompression data from many well-documented experimental man-trials and fit a logistic model of the P_(CNS)DCS to the data.[ 14] The resultant model, 'Model2' was used to generate P_(CNS)DCS isopleth curves. Figure 2 (reproduced from Figure 5 from reference [ 14]) shows graphically the relationship between the computed 0.2% P(CNS)DCS isopleth to data symbols that represent dive profile summaries. The data covered a range of depths from 30 to 260 fsw. From Figure 2 we observed that a large fraction of the data is between 50 and 200 fsw and thus covers the domain of applicability of this study. The authors' chi-square goodness-of-fit analysis "motivates retention of the hypothesis that Model2 provides a valid summary of results from all the dive trials across all depths."[ 14] This 0.2% P_(CNS)DCS isopleth is used as the NST test in the present study. This special case serves to test the various algorithms at the lower range of PRT (∼17 to 20) and thus is the most relevant for recreational divers. For both NST and TDT, statistical regression models, derived from man-trial data as a standard of comparison, were used rather than the individual data points themselves.

Figure 2.

Reproduction of Figure 5 from reference 14. 0.2% P_(CNS)DCS isopleth curve from Model2 plotted with data symbols the model was derived from

Figure 5.

No-stop times versus depth for the tested algorithms

FIRST SIGNIFICANT STOP

While it has been shown that TDT was more important than how that total time was distributed amongst the stops, the second important variable was "how deep the schedule starts."[ 13] To assess this, data were used from a NEDU controlled experiment which compared PDCS outcomes of dives of the same depth, bottom time and same TDT but with different stop profiles.[ 15] This NEDU study was performed specifically to assess the effect of deep versus shallow profiles as dictated by dual-phase versus tissue-loading algorithms. In the NEDU study, the maximum depth was 170 fsw, the bottom time was 30 min, the ascent rate was 30 fsw·min_-1 and the TDT for both profiles was 180 min. The group of divers who started their first stop at 40 fsw (for 9 min) had significantly lower PDCS (P = 0.0489 one-sided, Fisher Exact Test) than the group that stopped first at 70 fsw for 12 min. Since there is not sufficient information to know exactly where between the two tested depths might be optimal, this test case used the two depths as a maximum and minimum for the first stop criterion. In applying this test, care must be taken owing to the inter-relationship between the effects of ascent rate and first stop depth. So this last test is better called a first 'significant' stop depth test, where significant is tentatively defined as ≥ 1 min.

VERIFICATION

In the verification step, the objective was to assess whether the algorithms as implemented into the wrist computer hardware are a faithful representation of the full/baseline algorithm used in the desktop planning software. The specific wrist units under test are listed in Table 1. But not all wrist computers have a dive planning mode. If they do not, then the best approach would be to test the wrist unit in a hyperbaric chamber, reproducing a particular dive-pressure profile.[ 1] This type of testing was beyond the scope of the present research and verification will be limited to comparing desktop numerical results to wrist-unit implementation where the wrist computers have a dive planning mode. We compared numerical results (output) of TDT (or NST) between that given by the desktop planning version and the actual wrist unit, given the same inputs. While this was not an exhaustive/conclusive test, this criterion is necessary but not sufficient for good verification, and, thus, it is a logical first step in the verification process.

Both ZHL-16C and VPM-B were easily tested by comparing the desktop version in MultiDeco to the wrist computer implementation of the Shearwater Perdix unit operated in dive planning mode. We verified Suunto-RGBM by comparison to the DM5 software. The EMC-20H algorithm could not be verified using this method because the wrist dive computer does not have a dive planning mode.[ 8]

TOTAL DECOMPRESSION TIME

Total decompression times required by the various algorithms under test (at default conservatism levels) are plotted versus bottom time for the two depths (Figure 3 and Figure 4). The 3% PDCS isopleth as predicted by the StandAir model of Reference 10 is included as a guideline. From Figures 3 and 4, it is reasonable to tentatively infer the following:

Figure 3.

Total decompression time (TDT) vs. bottom time; results for 120 fsw dives

Figure 4.

Total decompression time (TDT) vs. bottom time; results for 150 fsw dives

• Suunto-RGBM seems to prescribe sufficient TDT over a fairly wide range of bottom times at both depths;

• VPM-B appears to prescribe adequate TDT for bottom times up to 30 min at 120 fsw and 29 min at 150 fsw (corresponding to PRT values of 25 and 30);

• ZHL-16C seems to prescribe insufficient TDT for bottom times of 28 min (120fsw) and 21 min (150 fsw) which equates to a PRT value of 25;

• EMC-20H seems to prescribe insufficient TDT for bottom times of greater than 26 min (120 fsw) and 19 min (150fsw), which roughly corresponds to a PRT value of 24

Through iterative adjustment of the conservatism settings, the three algorithms ZHL-16C, VPM-B and EMC-20H could be tuned to require adequate TDT, but for all three, the conservatism factor required depends on the PRT of the dive. Using the most conservative setting of 50%, EMC-20H could be made to provide sufficient TDT up to a PRT of about 30. At the high end of the domain of applicability in this study of 37, VPM-B could be made to provide sufficient TDT according to the requirements of the test as described here by setting the conservatism factor to +5. ZHL-16C should use a GF-Hi (which mainly affects TDT) < = 70.

NO-STOP TIMES

The results for NST are shown in Figure 5. From this figure, we can observe that both ZHL-16C GF-Hi=GF-Lo = 100 and VPM-B (conservatism = 0) specify sufficiently short NST over this fairly wide range of depths. EMC-20H seems to prescribe adequate NST for depths less than 80 fsw, but at greater depths, the NST may be too long. The Suunto-RGBM algorithm is more difficult to interpret for this metric, since it adds a 3 min/10 fsw 'safety stop' to all dives with a depth greater than 30 fsw.[ 16] Therefore, it was assumed the NST for Suunto-RGBM using the DM5 software to be the bottom time at which the prescribed 10 fsw stop time was increased from 3 to 4 minutes, or if an additional stop was added deeper than 10 fsw. Suunto-RGBM appears to prescribe sufficiently short NST at the shallower depths, then approaches the NEDU-suggested limit around 130 fsw. Suunto-RGBM may be too conservative over the lower range of depths. EMC-20H was adjusted to require adequately low NST with a conservatism factor of > 25%.

FIRST SIGNIFICANT STOP

Given that the conservatism factors of all of these algorithms have been modified to provide adequate TDT, the various profiles differ considerably with respect to the profile or depths of the various decompression stops. As described above, the four decompression algorithms were run to calculate the depth of the first significant stop using the depth-bottom time from reference 15.[ 15] VPM-B with the default conservatism of 0 calls for a first stop of 90 fsw, which is deeper than the 70 fsw depth that resulted in higher PDCS in the trial. Increasing the conservatism factor did not help; the first significant stop increased to 100 fsw for values of four or greater. EMC-20H was first evaluated with a conservatism of 25% as suggested by the NST test. At this level, the first significant stop was 70 fsw. This first stop can be brought down to 60 fsw with a conservatism value of 5% or less. However, at this conservatism value, the algorithm does not prescribe sufficient TDT and NST.

For Suunto-RGBM in default mode of conservatism factor of 0 and using the ‘deep-stop’ option yields a first stop depth of 113 fsw, which is significantly deeper than both of the NEDU profiles. Turning off the deep-stop mode reduces the first stop depth to 59 fsw.

We first evaluated ZHL-16C using GF-Hi = GF-Lo = 70, since the TDT test suggested lowering GF-Hi to 70. With these GF values, the first significant stop was at 60 fsw, which is inside the desired range. Keeping GF-Hi at 70, we can lower GF-Lo down to 55 and still maintain a first significant stop depth of shallower than 70 fsw.

VERIFICATION

For the verification of ZHL-16C and VPM-B, the computed TDT for two depth-time combinations (120 fsw/50 min and 150 fsw/40 min) agreed within 4.5 to 6.8%, which is within the inherent uncertainty of the validation.

Verifying the Suunto-RGBM algorithm was more complex. While the desktop dive planning software DM5 enables the user to plan decompression dives with the Suunto-RGBM version of the algorithm, the low-end wrist computers that use this algorithm (in this study, Zoop-Novo) only enable the planning of non-decompression dives. As mentioned above, Suunto-RGBM adds a 10 fsw/3 min safety stop to all dives with a depth greater than 30 fsw. The manual states that this is an optional safety stop.[ 16] As before, NST time using the DM5 software was assumed to be the bottom time at which the 10 fsw stop time was increased from 3 to 4 minutes, or if an additional stop was added deeper than 10 fsw. Useful data were only obtained if the deep stop option in the DM5 planner was turned off; see Table 2 for a summary of the verification test performed on Suunto-RGBM.

Table 2. Verification results of the no-stop times using the Suunto-RGBM algorithm; fsw – feet sea water .

Depth (fsw)	Depth (fsw)	DM5-deep stop on (min)	DM5- deep stop off (min)	Difference (%)
50	63	71	71	11
70	33	Added stop @ 35 fsw	38	13
90	19	Added deeper stop	24	21
110	11	Added deeper stop	17	35
130	7	Added deeper stop	13	46
150	5	Added deeper stop	10	50

OVERALL ASSESSMENT

The assessment results are summarised in Table 3. Based on an initial analysis, it can be inferred reasonably that none of the four algorithms evaluated passed all of the tests with default settings. ZHL-16C could be adjusted to pass all of the tests with GF-Hi < = 70 and GF-Lo > = 55. Suunto-RGBM could be made to pass all of the tests by simply turning off the deep stop option, which is easily done on the wrist unit tested. VPM-B could be adjusted to prescribe sufficient TDT (the required conservatism factor depends on the depth-bottom time of the dive) but could not be adjusted to pass the first significant stop test; the first stop was always too deep. EMC-20H could be tuned to pass the TDT test (conservatism ∼ 50%) and the first stop test (conservatism < 5%) independently, but not simultaneously.

Table 3. Summary of preliminary findings; Cons – conservatism setting; GF – gradient factor; NST – no stop time; PRT – see text (p.251) for definition; TDT – total dive time .

Algorithm	TDT	NST	Profile	Verification
ZHL-16C	Default OK for PRT ≤ ∼25; suggest GF-Hi ≤ 70 for higher	OK with GF 100/100	Suggest GF-Lo ≥ 55	Good; agreement better than 10%
VPM-B	Default OK for PRT < 28; Above this may need to increase Cons. with PRT	OK	Cannot tune to address this	Good; agreement better than 10%
Suunto-RGBM	Default OK for wide range of PRT	OK	Suggest turn off deep stop option	Partial test; only able to compare NST; agreement 11−50%
EMC-20H	Default OK for PRT < 24; above this, may need to increase Cons. with PRT	Default not sufficient > 70 fsw; okay with Cons > 25%	Default OK, but marginal with Cons ≥ 5%	N/A; computer does not have plan-mode

Discussion

The domain of applicability of the current study was dives with PRT values in the range of 13 to 37, which covers no-stop dives (generally PRT < 20), and low-risk decompression dives (20 < PRT < 25), as well as some historically higher risk dives (25 < PRT < 37). One obvious limitation of this study is that we used US Navy test data and there are significantly different risk factors between Navy dives and recreational dives, such as bottom work-load, potentially low water temperature and high currents. One complicating factor in this study was the lack of software configuration control in these algorithms. Variants exist and these different variants are not well identified or documented, which impedes the validation and verification process.[ 17 , 18] In the future, the concepts and procedures of model configuration management and verification should be more rigorously implemented into algorithms used for commercial off-the-shelf dive computers. The initial verification tests in this study only cover a few pairs of possible tests that should be performed.

This study presents how man-trial data with known P_DCS can be used by the non-Navy scientific community for testing of commercial decompression algorithms. This type of validation testing informs how to best use available commercial dive computers to improve diver safety. More research on how to structure and improve these tests is needed, specifically on the first significant stop test. Lastly, for algorithms used in commercial dive computers where desktop dive planners are not available, similar testing as described here can be performed by simulating dives with wrist units in hyperbaric chambers.

Summary

Commercial off-the-shelf dive computer algorithms were evaluated by comparison with US Navy experimental dives with known decompression sickness outcomes and resultant statistical models. Four algorithms were evaluated: Bühlmann ZHL-16C, VPM-B, Suunto-RGBM and EMC-20C. This preliminary testing indicates that while none of the four passed all of these proposed tests with factory default settings, ZHL-16C and Suunto-RGBM could be made to pass by adjusting user-defined settings.