Clarification of methods used to validate surface EMG decomposition algorithms as described by Farina et al. (2014)

to the editor: We are compelled to clarify some points made by Farina et al. (1). In a previous exchange of letters Farina et al. sought that we “decompose a set of synthetic surface EMG signals that we [Farina et al.] generate with a model” to provide a convincing validation of our sEMG decomposition algorithm. They now state that the use of simulated signals is “more limited than an experimental validation”—a position that we have held for the past decade (2).

As an alternative validation, they contend that the two-source test, originally proposed and used by our group [see detailed history in Kline and De Luca (2)], is the “only current reliable approach to assess the accuracy of a surface EMG decomposition algorithm.” Indeed, we have used the two-source test to validate our decomposition algorithm in different subjects, muscles, and force levels ranging from 20 to 50% MVC and observed on average 92–98% accuracy (3, among others). Further two-source validation by an independent group [described in Kline and De Luca (2)] yielded 95% average accuracy from more than 100 motor unit-action potential trains (MUAPTs) extracted by our algorithm. When compared with our decompose-synthesize-decompose-compare (DSDC), both validations average 95% accuracy—establishing that the validity of the DSDC assessment is at least comparable to that of the two-source test.

Now, as to the usefulness of the two-source test—in its present form, it has a fundamental drawback: its accuracy assessment is limited to a select few MUAPTs. For example, in one study cited by Farina et al. (1) using a convolution kernel compensation (CKC) algorithm, less than one (average 0.7) MUAPT was validated per contraction, with some accuracies <60%. The accuracy of all other MUAPTs remains untested with the two-source test. In contrast, our DSDC validation provides an assessment of all decomposed MUAPTs.

Regarding their statements that our decomposition produces a “minimized” residual that causes the DSDC to yield incorrectly high accuracy values, consider the following. Our four-channel electrode array is specifically designed to record overlapping MUAPs of a relatively large number of motor units in sEMG signal S. Clearly, any algorithm Q that incorrectly decomposes S into nonoverlapping segments (giving zero residual) would be inappropriate for DSDC validation—ours does not work in this fashion.

Now, consider algorithm R that is data driven, as is ours (2, 3), for decomposing the sEMG signal S into overlapping MUAPs of a large number of motor units. Decomposing S with exactitude (zero residual) is “nondeterministic-polynomial (NP) type hard,” or in clearer terms, impossible to practically carry out. Our sEMG decomposition algorithm overcomes NP-hardness by decomposing S to at least within 25% of its average energy (3), resulting in a significant residual. Yet, DSDC validation still produces 95% agreement between its two decomposition results. In practical reality, this can only be explained by the fact that our decomposition algorithm is able to successfully discriminate overlapping action potentials to the extent of the accuracy measures we report.

Artificial models that circumvent physiological realities could produce alternative objections to our DSDC validation algorithm. But Farina et al. (1) state that the use of simulated signals is “more limited than an experimental validation.” We agree.

GRANTS

This work was supported by funds from Delsys Inc.

DISCLOSURES

Carlo J. De Luca is the President and CEO of Delsys Inc., the company that developed the sEMG decomposition technology.

AUTHOR CONTRIBUTIONS

Author contributions: C.J.D.L. conception and design of research; C.J.D.L., S.H.N., and J.C.K. drafted manuscript; C.J.D.L., S.H.N., and J.C.K. edited and revised manuscript; C.J.D.L., S.H.N., and J.C.K. approved final version of manuscript.