Abstract
Background
Botanical reference materials (BRMs) generally account for the species, cultivar, and year and location of harvest that result in variability in the chemical composition that may lead to statistically significant differences using chemometric methods.
Objective
To compare the chemical composition of five species of Actaea root BRMs, four herbal sources of A. racemosa root BRMs, and A. racemosa BRMS, and commercial roots and supplements using chemometric methods and selected pre-processing approaches.
Method
Samples were analyzed by flow injection mass spectrometry (FIMS), principal component analysis (PCA), and factorial multivariate analysis of variance (mANOVA).
Results
Statistically significant (P = 0.05) compositional differences were found between three genera (Actaea, Panax, and Ginkgo), five species of Actaea (A. racemosa, A. cimicifuga, A. dahurica, A. pachypoda, and A. rubra) root BRMs, four herbal sources of A. racemosa root BRMs, and A. racemosa BRMS and commercial roots and supplements. The variability of 6% of the BRM variables was found to be quantitatively conserved and reduced the compositional differences between the four sources of root BRMs. Compositional overlap of A. racemosa and other Actaea BRMs was influenced by variation in technical repeats, pre-processing methods, selection of variables, and selection of confidence limits. Sensitivity ranged from 94 to 97% and specificity ranged from 21 to 89% for the pre-processing protocols tested.
Conclusions
Environmental, genetic, and chemometric factors can influence discrimination between species and authentic botanical reference materials.
Highlights
Frequency distribution plots derived from soft independent modeling of class analogy provide excellent means for understanding the impact of experimental factors.
Verification of botanical identity can be accomplished by comparison of an unknown sample to one or more authenticated botanical reference materials (BRMs). However, all BRMs are not equal. Some are highly characterized either botanically, morphologically, chemically, or genetically, whereas others are more generic. Highly characterized BRMs may account for the species, cultivar, and year and location of harvest. Generic BRMs may be of unknown origin and may have been subjected to only minimal characterization. All of these factors can result in variability in the chemical composition that may lead to statistically significant differences when applying chemometric profiling (1–4). From a medicinal perspective, such differences may be inconsequential if the efficacy of the material is preserved. However, these analytically determined compositional differences make it difficult to develop a chemometric model to objectively determine authenticity and quality. The more composition varies, the more inclusive and less specific a model becomes. Judging similarity is highly dependent on the samples collected, the range of factors that they account for, and the confidence limits selected by the analyst.
In general, variation in BRM chemistry is attributable to genetics (location drift, breeding), environment (location and time of harvest), post-harvest conditions (e.g., drying, initial processing, or storage), age of plant, age of sample, and other processing practices (grinding and extraction). Although breeding may produce different cultivars (genotypes), the changes may or may not be sufficient to warrant description as a new species. Nontargeted chemical profiles (chromatograms and spectra), when coupled with multivariate analysis, are extremely sensitive to subtle but potentially significant changes in the chemical composition (5). For example, a study of grapefruit juice from fruit harvested three times in a season over two years from conventional and organic orchards showed significant differences with respect to all three factors (harvest, season, and farming mode) (6). While all the samples were authentic grapefruit juice, chemometric analysis of their mass spectra showed them to be statistically different (95% confidence limit—this level is used throughout the paper) in ion profiles. Interestingly, the difference in composition related to organic and conventional farming was also reflected in taste studies (7).
Specific observations of chemical differences of BRMs have been reported for a number of species, including black cohosh (Actaea racemosa) (1), American ginseng (Panax quinquefolius) (2), ginkgo (Ginkgo biloba) (3), and cinnamon (Cinnamomum verum) (4), based on nontargeted analyses. With black cohosh, BRMs from four reliable sources were shown to be statistically different at the 95% confidence level, and metadata failed to provide easily distinguishable causes. Samples of American ginseng obtained from Wisconsin, Canada, and China were also statistically different. Ginkgo extracts prepared from freshly collected leaf material were difficult to reconcile with each other and with a historic reference standard characterized by the National Toxicology Program. Recently, it was shown that BRMs for four species of Cinnamomum used as “cinnamon” showed significant interspecies heterogeneity, producing different ion profiles based on nontargeted analyses.
It is necessary to put the compositional differences of BRMs in perspective. Systematic morphological differences allow for differentiation between genera and species. These morphological differences have both a genetic and chemical basis. Thus, compositional differences can be used to distinguish between species, while their compositional similarities may place them in the same genus. Similarly, compositional differences can produce statistically different cultivars, while compositional similarities place them in the same species. Put in statistical terms, the variance between species is less than the variance between genera, and the variance between cultivars is less than the variance between species. Thus, the mean profile of two genera or two species are expected to be statistically different.
Statistical differences between the profiles of BRMs do not prevent constructing an inclusive model for the species (1). Despite differences in some components, there are sufficient similarities to construct a model that is inclusive of the targeted species and exclusive of other species from the same genus. This was true for black cohosh, ginseng, ginkgo, and cinnamon (1–4). However, inclusion of all authentic samples results in a model with a large variance and a lack of homogeneity. These inclusive models may result in poor sensitivity (ability to identify authentic materials as authentic) and/or poor specificity (ability to identify nonauthentic materials as nonauthentic). The lack of homogeneity may also be troublesome for cross-validation where the random selection of subsamples can give rise to poor sensitivity and specificity.
The variability between BRMs is usually not random. That is, some components may change more than others. Truly random variation of all components would produce a homogeneous distribution of random profiles around a species mean and cross-validation would produce consistent results. Thus, the impact of genetic, environmental, and management factors is not uniform for all components and, with sufficient metadata, may be predictable. Unfortunately, this level of characterization is generally unavailable.
The current study has targeted discrimination between species as the metric for judging the adequacy of the analytical method. For method validation, identification of the inclusive population, usually the authenticated population, is obvious. This population should always (95% of the time, P = 0.05) be identified as authentic. Identification of an exclusive population, one that is identified as not authentic, is more difficult. Often, the pharmacological literature makes identification of potential adulterants clear. However, as adulteration becomes more sophisticated, an intentionally adulterated BRM may be used as an exclusive material. The question then becomes which adulterant(s) to use.
Our experience has been that differentiation between species of the same genus is often more difficult than detecting crude adulteration with alien components or materials, especially with nontargeted methods. This is because the chemical components of authentic species are often identical, although the relative concentrations (signal intensities) are different. Introduction of new peaks or disappearance of existing peaks in authentic specimens are a rare occurrence. It is easier to detect the presence of an adulterant compound not usually seen or an existing compound present at an unusually high concentration. Hence, the current study has focused on the more challenging task of discriminating between species.
This study systematically examined the compositional similarities and dissimilarities of genera and species in terms of ion profiles acquired by negative ionization flow injection mass spectrometry (FIMS). Principal component analysis (PCA) and multivariate analysis of variance (mANOVA) of historic results for echinacea, ginseng, and black cohosh were used to determine the variance between genera and for species, within species, between BRM sources, between samples, between replicates, and commercial supplements for A. racemosa (1). Variations in ion intensities were determined and evaluated for consistency of expression in historic data for Actaea racemosa. Selected variables were used to determine the agreement of Actaea racemosa BRMs and the ability to differentiate other Actaea species. The impact of pre-processing methods (sample normalization and variable scaling) on sensitivity and specificity was also investigated. Finally, frequency plots were used to illustrate the distribution of analytical results and clarify the interaction of sensitivity and specificity based on the chosen confidence limit (95% of the time, P = 0.05).
Experimental
Actaea spp. Roots
Sample acquisition was previously described (1). In brief, two classes of BRMs (specific BRMs subjected to a formal identification process and generic BRMs that have been reliably identified by the source organizations) from various species of Actaea were obtained from four sources of authentic herbal materials and from a variety of commercial sources (Table 1). The sources of BRMs were American Herbal Pharmacopoeia (AHP, samples BCR01-BCR24), Strategic Sourcing, Inc. (SS, samples SS01-SS07), the North Carolina Arboretum Germplasm Repository (NCA, samples NCC2 and NC01-NC22), and the National Institutes of Standards and Technology (NIST, samples SRM 3295, 3296, 3297, and 3298). Commercial root samples were purchased from the Internet and local stores in China (samples CA01-CA14). Commercial liquid, tablet, and capsule supplements were purchased from local stores in Maryland (CS01-CS14). NCA samples were collected from the permanent national A. racemosa germplasm collection in collaboration with the USDA NPGS (National Plant Germplasm System). All root materials and finished products were authenticated using DNA barcoding methods at AuthenTechnologies LLC, Richmond, CA using proprietary procedures.
Table 1.
Samples analyzed
| Sample | Plant | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Number | Species | Part | Form | Location | nrDNA | Mix | Hybrid | cpDNA | Mix | |
| AHP | BCR01 | A. racemosa | r/r | pow | NC, USA | A. racemosa | yes | n/a | A. racemosa | no |
| BCR02 | A. pachypoda | r/r | pow | NC, USA | A. pachypoda | no | no | A. pachypoda | no | |
| BCR03 | A. pachypoda | r/r | pow | China | ||||||
| BCR04 | A. racemosa | r/r | pow | Washington, NJ | A. racemosa | little | no | A. racemosa | no | |
| BCR05 | A. pachypoda | r/r | pow | NC, USA | A. pachypoda | no | no | A. pachypoda | no | |
| BCR06 | A. racemosa | r/r | pow | ? | A. racemosa | no | maybe | A. racemosa | no | |
| BCR07 | A. racemosa | r/r | pow | NC, USA | A. racemosa | little | maybe | No ID | yes | |
| BCR08 | A. racemosa | r/r | pow | NC, USA | A. racemosa | little | maybe | No ID | yes | |
| BCR09 | A. racemosa | r/r | pow | NC, USA | A. racemosa | no | maybe | A. racemosa | no | |
| BCR10 | A. podocarpa | r/r | pow | NC, USA | A. podocarpa | no | no | A. podocarpa | no | |
| BCR11 | A. podocarpa | r/r | pow | NC, USA | A. podocarpa | no | no | A. podocarpa | no | |
| BCR12 | A. podocarpa | r/r | pow | NC, USA | A. podocarpa | no | no | A. podocarpa | no | |
| BCR13 | A. dahurica | r/r | pow | China | ||||||
| BCR14 | A. rubra | r/r | pow | Quebec, Canada | A. rubra | no | yes | A. pachypoda | no | |
| BCR15 | A. rubra | r/r | pow | Ashland, OR, USA | A. rubra | no | no | A. rubra | no | |
| BCR16 | A. racemosa | r/r | pow | commercial | A. racemosa | no | maybe | A. racemosa | no | |
| BCR17 | A. racemosa | r/r | pow | commercial | A. racemosa | no | maybe | A. racemosa | no | |
| BCR18 | A. cimicifuga | r/r | pow | commercial | Erotium | yes | n/a | No ID | n/a | |
| BCR19 | A. cimicifuga | r/r | pow | China | Erotium | yes | n/a | No ID | yes | |
| BCR20 | A. cimicifuga | r/r | pow | China | Erotium | yes | n/a | No ID | yes | |
| BCR21 | A. cimicifuga | r/r | pow | commercial | Erotium | yes | n/a | No ID | n/a | |
| BCR22 | A. dahurica | r/r | pow | commercial | A. dahurica | yes | n/a | A. dahurica | no | |
| BCR23 | A. dahurica | r/r | pow | China | A. dahurica | yes | n/a | A. dahurica | no | |
| BCR24 | A. dahurica | r/r | pow | China | A. dahurica | yes | n/a | A. dahurica | no | |
| SS | SS01 | A. racemosa | r/r | r/r | Madison, AL, USA | A. racemosa | no | maybe | A. racemosa | no |
| SS02 | A. racemosa | r/r | r/r | Bell, KT, USA | A. racemosa | no | maybe | A. racemosa | no | |
| SS03 | A. racemosa | r/r | r/r | Logan, WV, USA | A. racemosa | yes | n/a | A. racemosa | no | |
| SS04 | A. racemosa | r/r | r/r | Carter, MO, USA | A. racemosa | yes | n/a | A. racemosa | no | |
| SS05 | A. racemosa | r/r | r/r | Washington, MO, USA | A. racemosa | yes | maybe | A. racemosa | no | |
| SS06 | A. racemosa | r/r | r/r | Clay, KT, USA | A. racemosa | no | no | A. racemosa | no | |
| SS07 | A. racemosa | r/r | r/r | Pike, KT, USA | A. racemosa | yes | maybe | A. racemosa | ||
| NIST | SRM3295 | A. racemosa | r/r | pow | ? | |||||
| SRM3296 | A. racemosa | leaf | pow | ? | ||||||
| SRM3297 | A. racemosa | r/r | pow | ? | ||||||
| SRM3298 | A. racemosa | r/r | pow | ? | ||||||
| CR | CA01 | A. dahurica | r/r | r/r | Liaoning, China | A. dahurica | yes | n/a | A. dahurica | no |
| CA02 | A. dahurica | r/r | r/r | Heilongjiang, China | A. dahurica | yes | n/a | no result | yes | |
| CA03 | ? | r/r | r/r | Sichuan, China | Acanthaceae | no | n/a | Baphicacanthus cusia | no | |
| CA04 | ? | r/r | r/r | Hebei, China | Eurotium sp. | Yes | n/a | no result | n/a | |
| CA05 | A. dahurica | r/r | r/r | suzhou, China | A. dahurica | yes | n/a | A. dahurica | no | |
| CA06 | A. dahurica | r/r | r/r | Hebei, China | no result | yes | n/a | A. dahurica | yes | |
| CA07 | A. dahurica | r/r | r/r | Sichuan, China | Pichia sp. | Yes | n/a | A. dahurica | no | |
| CA08 | A. dahurica | r/r | r/r | North Korea | A. dahurica | yes | no | A. dahurica | no | |
| CA09 | A. brachycarpa | r/r | r/r | North Korea | A. brachycarpa | yes | no | A. brachycarpa | no | |
| CA10 | A. dahurica | r/ | r/r | Yunnan, China | A. dahurica | yes | yes | A. dahurica | no | |
| CA11 | A. dahurica | r/r | r/r | Henan, China | A. dahurica | yes | yes | A. dahurica | yes | |
| CA12 | ? | r/r | r/r | Hebei, China | Eupatorium | no | n/a | E. fortunei | no | |
| CA13 | ? | r/r | r/r | Yunnan, China | Astereae | yes | n/a | E. fortunei | no | |
| CS | CS01 | A. racemosa | — | tab | ? | |||||
| CS02 | A. racemosa | — | tab | ? | ||||||
| CS03 | A. racemosa | — | liq | ? | ||||||
| CS04 | A. racemosa | — | liq | ? | ||||||
| CS05 | A. racemosa | — | liq | ? | ||||||
| CS06 | A. racemosa | — | liq | ? | ||||||
| CS07 | A. racemosa | — | liq | ? | ||||||
| CS08 | A. racemosa | — | cap | ? | A. racemosa | no | no | A. racemosa | no | |
| CS09 | A. racemosa | — | cap | ? | A. racemosa | no | no | A. racemosa | no | |
| CS10 | A. racemosa | — | cap | ? | A. racemosa | no | no | A. racemosa | no | |
| CS11 | ? | — | tab | ? | no DNA | n/a | no | no DNA | n/a | |
| CS12 | A. brachycarpa | — | cap | ? | A. brachycarpa | no | no | poor quality sequence | n/a | |
| CS13 | A. racemosa | — | cap | ? | A, racemosa | yes | no | poor quality sequence | n/a | |
| CS14 | ? | — | cap | ? | Oryza sativa | yes | n/a | Oryza sativa (Rice) | no | |
| NCA | NCC2 | A. racemosa | r/r | pow | composite | A. racemosa | no | no | A. racemosa | no |
| NC01a | A. racemosa | r/r | pow | |||||||
| NC01b | A. racemosa | r/r | pow | |||||||
| NC01c | A. racemosa | r/r | pow | |||||||
| NC02a | A. racemosa | r/r | pow | |||||||
| NC02b | A. racemosa | r/r | pow | |||||||
| NC02c | A. racemosa | r/r | pow | |||||||
| NC03a | A. racemosa | r/r | pow | |||||||
| NC03b | A. racemosa | r/r | pow | |||||||
| NC03c | A. racemosa | r/r | pow | |||||||
| NC04a | A. racemosa | r/r | pow | |||||||
| NC04b | A. racemosa | r/r | pow | |||||||
| NC04c | A. racemosa | r/r | pow | |||||||
| NC05a | A. racemosa | r/r | pow | |||||||
| NC05b | A. racemosa | r/r | pow | |||||||
| NC05c | A. racemosa | r/r | pow | |||||||
| NC06a | A. racemosa | r/r | pow | |||||||
| NC06b | A. racemosa | r/r | pow | |||||||
| NC06c | A. racemosa | r/r | pow | |||||||
| NC07a | A. racemosa | r/r | pow | |||||||
| NC08a | A. racemosa | r/r | pow | |||||||
| NC08b | A. racemosa | r/r | pow | |||||||
| NC08c | A. racemosa | r/r | pow | |||||||
| NC09a | A. racemosa | r/r | pow | |||||||
| NC09b | A. racemosa | r/r | pow | |||||||
| NC09c | A. racemosa | r/r | pow | |||||||
| NC10a | A. racemosa | r/r | pow | |||||||
| NC11a | A. racemosa | r/r | pow | |||||||
| NC11b | A. racemosa | r/r | pow | |||||||
| NC11c | A. racemosa | r/r | pow | |||||||
| NC12a | A. racemosa | r/r | pow | |||||||
| NC12b | A. racemosa | r/r | pow | |||||||
| NC12c | A. racemosa | r/r | pow | |||||||
| NC13a | A. racemosa | r/r | pow | |||||||
| NC13b | A. racemosa | r/r | pow | |||||||
| NC13c | A. racemosa | r/r | pow | |||||||
| NC14a | A. racemosa | r/r | pow | |||||||
| NC14b | A. racemosa | r/r | pow | |||||||
| NC14c | A. racemosa | r/r | pow | |||||||
| NC15a | A. racemosa | r/r | pow | |||||||
| NC15b | A. racemosa | r/r | pow | |||||||
| NC15c | A. racemosa | r/r | pow | |||||||
| NC16a | A. racemosa | r/r | pow | |||||||
| NC16b | A. racemosa | r/r | pow | |||||||
| NC16c | A. racemosa | r/r | pow | |||||||
| NC18a | A. racemosa | r/r | pow | |||||||
| NC19a | A. racemosa | r/r | pow | |||||||
| NC19b | A. racemosa | r/r | pow | |||||||
| NC20a | A. racemosa | r/r | pow | |||||||
| NC20b | A. racemosa | r/r | pow | |||||||
| NC20c | A. racemosa | r/r | pow | |||||||
| NC21a | A. racemosa | r/r | pow | |||||||
| NC21b | A. racemosa | r/r | pow | |||||||
| NC21c | A. racemosa | r/r | pow | |||||||
| NC22a | A. racemosa | r/r | pow |
MS Sample Preparation
As previously described (1), root samples were ground into fine powders. Ten milligrams of each sample were mixed with 5 mL of methanol–water (70–30, v/v) in 15 mL centrifuge tubes and sonicated for 60 min at room temperature. The extracted samples were centrifuged at 5000 × g for 10 min using an IEC Clinical Centrifuge (Danon/IEC Division, Needham H.T.S., MA). The supernatant was diluted 1 to 10 (v/v) with methanol and filtered through a 17 mm (0.45 µm) PVDF syringe filter (VWR Scientific, Seattle, WA) before injection. To avoid errors arising from unexpected degradation of some compounds, the sample analysis was completed within 24 h of the extraction. Tablets were prepared the same as root samples. Capsules were opened and the solid contents were emptied onto a weighing paper. Ten milligrams were mixed with 5 mL of methanol–water (70/30, v/v) in a 15 mL centrifuge tube and then treated in the same manner as the root samples. For liquid supplements, 10 μL were mixed with 5 mL of methanol–water (70/30, v/v) in a 15 mL centrifuge tube and then treated in the same manner as the root samples.
Mass Spectrometry Instrumentation
As previously described (1), the FIMS system consisted of a Q Exactive mass spectrometer (Thermo Fisher Scientific) with an Agilent 1200 HPLC system (a quaternary pump with a vacuum degasser, a thermostated column compartment, an autosampler, and a diode-array detector). The flow injection used a guard column (Adsorbosphere All-Guard Cartridge, C18, 5 μm, 4.6 × 7.5 mm, Alltech Associates, Inc.) to minimize potential contamination of the FIMS system. Mobile phases consisted of 0.1% formic acid in H2O (A) and 0.1% formic acid in acetonitrile (B) with isocratic elution at 50:50 (v/v) at a flow rate of 0.5 mL/min for 2 min. Electrospray ionization was performed in the negative ion mode from m/z 150–1500 to obtain the FIMS fingerprints. The following conditions were used for the mass spectrometer: sheath gas flow rate, 80 (arbitrary units); aux gas flow rate, 10 (arbitrary units); spray voltage, 4.50 kV; heated capillary temperature, 220°C; capillary voltage, 4.0 V; tube lens offset, 25 V. The injection volume for all samples was 10 μL.
Sample Analysis
As previously described (1), the sequence of the samples was randomized and each sample (with the exception of BCR21 which was not run) was run five times for a total of 1140 analyses. After running each sample once in random order, a new random sequence of measurements was made. Spectra were summed over the 1.0 min interval from 0.5 min to 1.5 min of the sample bolus.
Chemometrics
As previously described (1), the FIMS fingerprints of each sample were mass spectra, i.e., ion counts with respect to the mass-to-charge ratio for a range of m/z 151 to 1500. The spectra were exported as Excel files (Microsoft, Inc.), combined, sorted by type and experimental factors, and then imported into Solo (Eigenvector Research, Inc.) for pre-processing, principal component analysis (PCA), and soft independent modeling of class analogy (SIMCA).
PCA
Principal component analysis was performed in the standard manner using Solo (8). All pre-processing was done in Solo. Pre-processing consisted of combinations of sample normalization to unit vector, variable normalization by the square root of the mean, autoscaling (normalization by the standard deviation for each variable), and mean centering. The specific combinations of pre-processing are described in the text. Results were viewed as score plots, loading plots, and Q statistics versus sample (8).
Multivariate Analysis of Variance (mANOVA)
This method has been previously described (2). In brief, classic ANOVA is performed for every variable in the MS spectra and summed across the spectra. The mean and residuals are systematically computed for each experimental factor (genus, species, sample type, technical repeat, sample source, and sample). The summed mean and residual variances were used to compute an overall F value. The sample residuals reflect the analytical variance.
Results and Discussion
PCA of Genera, Species, and Supplements
The difference in composition between genera, species, subsamples, and supplements needs to be put in perspective. Starting at the genera level and working down the taxonomic strata serves to illustrate the expected decrease in variance. Figure 1A provides a PCA score plot comparing FIMS fingerprints for the Actaea, Echinacea, and Panax genera. Clusters for genera are composed of BRMs of multiple species, different plant parts, and commercial supplements: Echinacea (E. purpurea and E. angustifolia), aerial and root parts, and commercial supplements, Panax (P. ginseng, P. quinquefolius, and P. notoginseng, and commercial supplements), and Actaea (A. racemosa, A. cimicifuga, A. dahurica, A. pachypoda, and A. rubra and the commercial supplements listed in Table 1). Figure 1A illustrates that each genera forms a separate cluster despite being composed of multiple species, plant parts, and commercial samples, i.e., the between genera variance exceeds the within genera variance (Table 2, Panel A). Thus, although the Actaea species and commercial roots and supplements may be statistically different from each other, as a group, they can be clearly differentiated from the other genera.
Figure 1.
PCA score plots for (A) collections of Actaea, Echinacea, and Panax BRMs, BRMs for other species, and commercial supplements, (B) all Actaea samples and supplements, (C) only Actaea samples (BRMs and commercial samples), (D) only A. racemosa BRMs from AHP, SS, NCA, and NIST, (E) Cooman’s plot based on independent PCA models for A. racemosa BRMs from AHP and SS, and (F) one-class SIMCA model based on A. racemosa BRMs showing all Actaea samples. Symbols in Figure 1A: (
) all Actaea samples (Actaea species and commercial supplements), (
) all Echinacea samples (E. purpurea and E. angustifolia, aerial and roots, and commercial supplements), and (
) all Panax samples (P. ginseng, P. quinquefolius, and P. notoginseng and commercial supplements). Symbols in Figure 1B–F: (
) A. racemosa BRMs from AHP, (
) A. racemosa BRMs from SS, (
) A. racemosa BRMs from NCA, (
) A. racemosa BRMs from NIST, (
) non-A. racemosa species BRMs from AHP, (
) commercial Actaea roots, and (
) commercial Actaea supplements. The dotted line in Figure 1D provides the 95% confidence limit, P = 0.05).
Table 2.
Multivariate analysis of variance for five levels of data grouping
| Panel A: Actaea/Echinacea/Panax |
Panel B: Actaea |
Panel C: A. racemosa (repeats 1–5, 1350 variables) |
Panel D: A. racemosa (repeats 3–5, 1350 variables) |
Panel E: A. racemosa (repeats 3–5, 86 variables) |
|||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Variance |
Variance |
Variance |
Variance |
Variance |
|||||||||||||||||||||||||||||||
| Source of variance | n | df | Total | Pct Tot | Mean | F | P | n | df | Total | Pct Tot | Mean | F | p | n | df | Total | Pct Tot | Mean | F | p | n | df | Total | Pct Tot | Mean | F | p | n | df | Total | Pct Tot | Mean | F | p |
| Grand mean residuals | 1135 | 669 | 100.0% | ||||||||||||||||||||||||||||||||
| Between species | 3 | 2 | 404 | 60.4% | 201.85 | 763 | <0.001 | ||||||||||||||||||||||||||||
| Within species | 1126 | 1002 | 265 | 39.6% | 0.26 | ||||||||||||||||||||||||||||||
| Echinacea | 350 | 104 | 15.5% | ||||||||||||||||||||||||||||||||
| Panax | 216 | 50 | 7.5% | ||||||||||||||||||||||||||||||||
| Actaea | 560 | 111 | 16.7% | 560 | 111 | 100.0% | |||||||||||||||||||||||||||||
| Between sample types | 4 | 3 | 36 | 32.3% | 11.98 | 88.3 | <0.001 | ||||||||||||||||||||||||||||
| Within sample types | 560 | 556 | 75 | 0.14 | |||||||||||||||||||||||||||||||
| Actaea other species | 90 | 11 | 9.7% | ||||||||||||||||||||||||||||||||
| Commercial roots | 60 | 9 | 8.2% | ||||||||||||||||||||||||||||||||
| Commercial suppl | 70 | 31 | 28.3% | ||||||||||||||||||||||||||||||||
| Actaea racemosa | 340 | 24 | 21.6% | 340 | 24 | 100.0% | 204 | 14.2 | 100.0% | 204 | 3.73 | 100.0% | |||||||||||||||||||||||
| Between repeats | 5 | 4 | 0.08 | 0.3% | 0.0195 | 8 | <0.001 | 3 | 2 | 0.02 | 0.2% | 0.0114 | 32 | <0.001 | 3 | 2 | 0.01 | 0.2% | 0.003 | 1.2 | 0.3 | ||||||||||||||
| Between sources | 4 | 3 | 5.07 | 21.1% | 1.6890 | 674 | <0.001 | 4 | 3 | 2.99 | 21.0% | 0.99633 | 2786 | <0.001 | 4 | 3 | 1.1 | 29.4% | 0.366 | 128.5 | <0.001 | ||||||||||||||
| Between samples | 68 | 67 | 18.2 | 75.8% | 0.2716 | 108 | <0.001 | 68 | 67 | 11.2 | 78.5% | 0.16681 | 466 | <0.001 | 68 | 67 | 2.25 | 60.3% | 0.034 | 11.8 | <0.001 | ||||||||||||||
| Residuals | 340 | 266 | 0.67 | 2.8% | 0.0025 | 204 | 132 | 0.05 | 0.3% | 0.0004 | 204 | 132 | 0.38 | 10.1% | 0.003 | ||||||||||||||||||||
The Actaea cluster is composed of 114 samples of raw roots and commercial supplements of Actaea that were previously analyzed by FIMS and DNA barcoding (Table 1). American Herbal Pharmacopoeia (AHP) provided 24 root BRMs collected around the world: eight root BRMs of A. racemosa and 16 root BRMs of A. cimicifuga, A. dahurica, A. pachypoda, A. podocarpa, and A. rubra. Strategic Sourcing (SS) provided seven root BRMS of A. racemosa collected in the United States and the North Carolina Arboretum Germ Plasm Repository (NCA) provided 54 root BRMs of A. racemosa collected from 22 sites in the eastern United States and one ground and composited sample. The National Institute of Standards and Technology (NIST) developed four Actaea reference standards; SRM 3295 A. racemosa rhizome, SRM 3296 A. racemosa leaf, SRM 3297 A. racemosa extracted rhizome, and SRM3298 A. racemosa pelletized product. In addition, commercial root samples were purchased from the Internet and local stores in China, and commercial liquid, tablet, and capsule supplements were purchased from local stores in Maryland. None of the commercially purchased roots or supplements were BRMs.
Figure 1B presents a PCA scores plot based on FIMS profiles for all the Actaea samples listed in Table 1. The commercial supplements (tablets, liquids, and capsules) show a large degree of variation compared to the BRMs. There was no documentation regarding preparation of the supplements. It can be speculated that their large deviations from the BRMs most likely arose from the preparation process, from the loss and/or concentration of specific components during the extraction process and the possible addition of other components. The commercial preparation process obviously produced chemical profiles different from the preparation of the root BRMs in our laboratory. However, it has been shown that profiles for raw materials are not always statistically similar to supplements due to varied extraction procedures and added components (9). The large deviations between the BRMs and commercial supplements as shown in Figure 1B indicate significant compositional differences between these two sources of botanical materials. However, Figure 1A indicates that the highly divergent commercial supplements in Figure 1B do contain sufficient compositional similarity to be grouped within the Actaea genus.
Figure 1C shows that most of the non-racemosa species, either BRMs or commercially acquired, clustered closer to but still outside the cluster for A. racemosa. The PCA scores plot in Figure 1D shows that the A. racemosa BRMs from the four herbal sources (AHP, NCA, NIST, and SS) appear to be compositionally different. This is also seen in the Cooman’s plot in Figure 1E, which presents a paired Q residuals plot showing the distance of samples to the models for the AHP samples (horizontal axis) and the SS samples (vertical axis). In general, the AHP and SS samples are below and to the right of their respective 95% confidence limits (dashed lines) and the NCA samples are greater than both 95% limits, indicating that all three groups are statistically different. The NIST sample is marginal between SS and NCA.
Figure 1D suggests that the four sources of A. racemosa BRMs have a different composition, and the Cooman’s plot in Figure 1E shows that they are statistically different. The impact of the differences in composition on the authentication of the samples can be seen in the Q residuals plot for the one-class SIMCA model in Figure 1F. Spectra from the A. racemosa BRMs were analyzed by PCA, and Q residuals (distance from the model) were computed for all samples. Inclusion of all the A. racemosa BRMs (samples 1–204) in the model leads to a large variance (vertical dotted line 95% confidence limit) that includes all the A. racemosa BRMs and provides a sensitivity of 95%. However, when other non-A. racemosa species, roots, and supplements are included, a specificity of only 68% is provided. Figure 1F illustrates the tradeoff between sensitivity and specificity based on the selection of the confidence level. Changing the confidence limit (raising or lowering the dotted line) reciprocally increases or decreases the sensitivity and specificity.
Figure 1 provides an interesting perspective regarding PCA modeling and variance. Including spectra for all three genera for PCA results in large overall variance (Figure 1A) with relatively smaller variance for each genus and clear separations in the scores plot. A model for a single genus (Figure 1B) reduces the total variance and is equivalent to magnifying a selected region of the previous PCA plot. The single-genus model in turn shows separation of the different fractions and preparation of the genus samples. Further amplification by removal of different fractions in Figure 1C and D shows what appear to be statistically significant differences between species and within species, respectively. The similarities and differences in Figure 1A–F are all based on the same set of variables. Pre-processing of the variables in a data set has the potential to strongly influence the results and are investigated in more detail in later sections.
mANOVA of Genera, Species, and Supplements
Factorial multivariate analysis of variance (mANOVA) of the FIMS spectra for the samples analyzed in Figure 1 provides statistical validation of the previous observations. Table 2, Panel A verifies that the three genera in Figure 1A are different with a probability <0.001. Table 2, Panel B compares the four major types of Actaea samples shown in Figure 1B and listed in Table 1 (BRMs for A. racemosa and other Actaea species, commercial Actaea roots, and commercial Actaea supplements) and shows they are statistically different (P < 0.001). Table 2, Panel C considers only the A. racemosa BRMs (Figure 1D) and establishes that the four herbal sources (AHP, NCA, NIST, and SS), the 68 samples, and the technical repeats are statistically different (P < 0.001). Table 2, Panel C confirms the observation in Figure 1D and E that there is a compositional difference between the samples from the four sources of A. racemosa BRMs. As expected, the samples are also significantly different.
The statistically significant difference between technical repeats was surprising. The reproducibility of MS data has always been of concern due to the variability of the ionization process with time and sample concentration. The FIMS analyses took less than 5 min per sample. Still, the analysis of 114 samples took more than 8 h. Thus, five repeat analyses required at least 3 days even if samples were run overnight. Consequently, an initial PCA score plot was labeled according to technical repeat and yielded results similar to those shown in Figure 2A, suggesting no difference between repeats. However, the mANOVA results led to further investigation. If only the intensities of ions m/z = 151 to 350 are plotted, a strong deviation is observed for runs 1, 2, and 3–5 (Figure 2B). A PCA score plot for ions m/z = 401 to 550 or higher showed a diminishing impact and a PCA score plot similar to that of Figure 2A. Thus, the statistical significance for technical repeats in Table 2, Panel C can be attributed primarily to masses less than m/z = 350.
Figure 2.
PCA score plots labeled for technical repeat for all vouchered A. racemosa BRMs for (A) m/z = 400–549 and (B) m/z = 150 to 399. Symbols are (
) run 1, (
) run 2, (
) run 3, (
) run 4, and (
) run 5.
Based on the results in Figure 2B, the data for the first two runs were dropped from the mANOVA calculations in Table 2, Panel D. The mean variance attributable to technical repeats decreased by 40%, but the variance between technical repeats was still significant. Some variation between runs 1 and 3 can still be seen in Figure 2B. Analyses based on data from a single run (data not shown) did not result in different conclusions compared to the results for the combination of repeats 3–5. Results from repeat analyses 1 and 2 were dropped from the calculations for the rest of the paper. Interestingly, judicious selection of variables did lead to a reduced probability of significance (Table 2, Panel E) and will be discussed in the next section.
Variance of Individual Components
The previous two sections of this paper focused on the comparison of samples based on their integrated spectra (i.e., nontargeted examination of the 1350 variables) using PCA and mANOVA. An examination of the variance of individual components can often shed light on the sources of variance between samples and can lead to identification of key markers. An examination of the PCA scores plot loadings for Figure 1D failed to identify any components that had a marked impact on the separation of the sample clusters (data not shown).
A more general examination, a plot of the standard deviation versus the average intensity for each variable (for technical repeats 3–5) for the authentic A. racemosa samples from AHP, NCA, NIST, and SS revealed two groups of variables with different relative standard deviations (RSDs) (Figure 3). An arbitrary threshold of 10% was selected as a basis for dividing the two groups of variables. Figure 3 shows that the 86 variables with an RSD less than 10% have a slope of 0.033, i.e., a RSD of 3.3%. The remaining 1264 variables with RSDs greater than 10% have a slope corresponding to a RSD of 33.5%. Both sets of variables have correlation coefficients greater than 0.9. Thus, a subset of 86 variables appears to be highly conserved quantitatively for the A. racemosa BRMs.
Figure 3.
Standard deviation as a function of ion counts for 1350 ions: (
) ions with relative standard deviations (RSDs) less than 10% and (
) ions with RSDs greater than 10%. Trend lines show RSDs of 3.3 and 33.5%, respectively.
Figure 4A compares the intensity profiles of the 86 conserved variables (low RSD in red) with the full profile of 1350 variables (in blue). The fraction of conserved variables is 6.4% and generally falls below m/z = 350 (arbitrarily selected from inspection of Figure 4A) but constitutes 47% of the total spectral counts. Figure 5, a PCA scores plot based on the 86 variables (and technical repeats 3–5), shows a visually more homogeneous mix compared to the plot in Figure 1D. Interestingly, a PCA scores plot based on variables 351–1350 (data not shown) shows the same pattern as Figure 1D. Thus, the higher masses with average RSDs of 33% appear to be primarily responsible for the observed differences in the A. racemosa BRMs. A comparison of the variance for 86 variables (Table 2, Panel E) to 1350 variables (Table 2, Panel D) shows a decrease in the total sum of squares for A. racemosa since the variance is summed for only 86 variables. However, as expected, using the 86 conserved variables dramatically decreased the significance of the variance between technical repeats and between samples.
Figure 4.
Intensity and F value spectra for 1250 ions (out of 1350 ions. (A) Intensities of ions (
) with relative standard deviations (RSDs) less than 10% and intensities of all (
) ions. (B) F values of 1250 ions obtained from mANOVA.
Figure 5.
PCA score plot of vouchered A. racemosa from AHP, SS, NCA, and NIST based on repeats 3–5 and 83 variables with quantitative conservation.
The behavior of individual variables, or individual ions for FIMS, is at the heart of targeted versus nontargeted methods. Can targeted variables, based on a knowledge of the chemistry of the plant, be as robust as nontargeted analysis? Conversely, does the large cumulative variance of 1350 variables obscure significant differences that might occur for a very limited number of variables? Chen et al. (4) used targeted components to discriminate between cinnamon species when PCA proved unreliable. The current study shows an interesting distribution of variables: those with excellent reproducibility (quantitatively conserved) between the A. racemosa BRMs (∼3.3% RD) and those with poor reproducibility (∼33% RSD). It will be interesting to examine MS spectra for other botanical materials to determine if the same trend is found. It will also be interesting to identify the quantitatively conserved ions and to determine if certain families of compounds or metabolic pathways provide stable platforms for recognizing genera, species, and varieties. Can analysis of only a few variables (compounds) provide sufficient information that negates acquisition of the full spectrum? The impact of using only the quantitatively conserved variables is considered in the next section.
The existence of quantitatively conserved compounds also impacts the consideration of how many samples are sufficient to characterize an authentic material. It is possible that conserved compounds will provide greater similarity of authentic samples from different sources with respect to variety, location, weather, year and time of year of harvest. If so, it would be advantageous to only have to collect a comprehensive set of samples once and to use known quantitatively conserved marker compounds for authentication. It should be noted that the conserved compounds may or may not be responsible for the desired activity of the botanical. At present, most marker compounds used in routine quality control are chosen for their relative uniqueness to the plant material, not their stability.
Distribution Profiles of Actaea BRMs
Generically, discrimination between two populations of samples is ultimately dependent on the distribution of each population as characterized by their means, standard deviations, and degree of overlap. Such a comparison requires analysis of an extensive collection of authentic samples for both the authentic and the unknown population. With nontargeted analysis, the multivariate data must be converted to a single analytical parameter. This is readily accomplished in chemometrics using the Q statistic (7), the distance of a sample from the model (Figure 1E and F). In the current study, the A. racemosa BRMs constitute the authentic population, and the BRMs for the other Actaea species constitute the unknown population. Distribution plots for the A. racemosa and the other Actaea species can be obtained by simply rotating the axes of the SIMCA plot (Figure 1F) and plotting frequency of occurrence versus the Q residual.
Figure 6A presents a frequency plot for the A. racemosa BRMs (solid line) and other Actaea species (dashed line) obtained by rotating the axes of Figure 1F. The data in Figures 1 and 6A were computed with a minimum of pre-processing, i.e., only mean centering (MC). The mean and +2σ limit are shown as vertical solid and dashed lines, respectively. It can be seen that the distribution of the A. racemosa BRMs appear to be skewed to higher concentrations and that there are too few non-A. racemosa BRMs to make any judgment regarding their distribution, i.e., with more samples of all species, confidence in the shape of the distributions would be greater. The overlap of the two distributions is 18% for A. racemosa and 54% for the other Actaea species (Table 3). Based on the 2σ limit shown in Figure 6A, 5% of the A. racemosa BRMs were identified as nonauthentic (false negatives) and provided a sensitivity of 95%. Forty-nine percent of the other Actaea species were erroneously classified as A. racemosa (false positives) and provided a specificity of 51%. The poor specificity is readily understood from the high degree of overlap of the two populations.
Figure 6.
Frequency plots for (A) 1350 variables and mc, (B) 1350 variables and norm-sqrt-mc, (C) 86 variables and norm-sqrt-mc, and (D) 1350 variables weighted and norm-sqrt-mc. The solid lines present the frequency distribution of the A. racemosa BRMs, and the dashed lines present the combined frequency distribution of the BRMs for the other Actaea species (cimicifuga, dahurica, pachypoda, and rubra). Vertical solid and dashed lines present the mean and 95% confidence limit for the A. racemosa BRMs, respectively.
Table 3.
Characteristics of data set as a function of treatment and pre-processing
| SIMCA |
|||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Percent Overlap |
Specificity |
Cross-Validation |
|||||||||
| Variables | Pre-Processing | AR | OR | NAR | Sensitivity | OR | NAR | Training | Test | NAR | Skewness |
| In silico | MC | — | — | — | 96 | 96 | 96 | 96 | 96 | — | 0.18 |
| m/z 151-1500 | MC | 15 | 54 | 27 | 95 | 21 | 68 | 92 | 83 | 92 | 0.73 |
| m/z 151-1500 | N-MC | 12 | 42 | 28 | 96 | 70 | 85 | 92 | 87 | 92 | 0.93 |
| m/z 151-1500 | N-SQRT-MC | 5 | 18 | 9 | 97 | 89 | 96 | 88 | 74 | 96 | 0.81 |
| m/z 151-1500 | Auto | 3 | 11 | 8 | 96 | 89 | 96 | 89 | 85 | 96 | 0.68 |
| 86 (<10% variability) | N-SQRT-MC | 14 | 48 | 21 | 94 | 67 | 84 | 89 | 73 | 96 | 1.68 |
| m/z 151-1500 | N-F Weighted-MC | 10 | 37 | 17 | 97 | 89 | 92 | 90 | 84 | 98 | 0.48 |
Table 3 also presents the overlap for all the non-A. racemosa samples (other Actaea species, commercial root roots, and commercial supplements) analyzed in this study. Since the total population was larger and provided higher Q values (plots not shown), the overlap for the non-A. racemosa samples was smaller in terms of percentage. Table 3 also presents the SIMCA values obtained by computing the 2σ limit for the A. racemosa samples in Figure 6A and determining the sensitivity and specificity as described above. Finally, the A. racemosa samples were subjected to cross-validation, and the sensitivity for the training and test sets and the specificity for all the non-A. racemosa samples were computed. Cross-validation will be discussed in more detail below.
Chemometric Pre-Processing
Pre-processing of data transforms it into a form that is more easily analyzed and provides more accurate and/or meaningful results. Common forms of pre-processing in chemometrics are normalization, filtering, scaling, and mean centering. Pre-processing can also alter the frequency plots as shown in Figure 6A–D. Mean centering (MC) of variables is standard so that the midpoint of a PCA plot (0,0) provides the center of mass for all the samples. All the data in this study were mean centered. A common form of normalization is unit vector normalization (N), where each sample spectra is multiplied by a normalization constant that equates the sum of squares for all variables to 1.0. Application of unit vector normalization to the data in Figure 6A produced similar distribution overlap (Table 3) and similar PCA, SIMCA, and frequency plots (data not shown). This is not surprising since equal masses of all Actaea BRMs were prepared and produced very similar FIMS spectra. Normalization did influence spectra for commercial supplements (not shown in Figure 6).
Scaling of each variable by the square root of the variable means or autoscaling (variable division by the standard deviation) decreases the impact of large signals and more evenly weights the contribution of all variables. Both square root and autoscaling produced similar results, as shown in Table 3. Figure 6B presents the frequency plots for spectra (1350 variables) that were normalized (N) by unit vector, scaled by the square root of the variable mean (SQRT), and then mean centered (N-SQRT-MC). The frequency plot of A. racemosa is still skewed, but the plot for other Actaea species has shifted to the right, significantly reducing the distribution overlap for A. racemosa and the other Actaea species to 5 and 18%, respectively, and sensitivities and specificities of 96% and 89%, respectively (Figure 6B and Table 3). Thus, square root and autoscaling allow greater impact of the less intense variables and significantly improved discrimination between A. racemosa and the other Actaea species.
Figure 6C presents the frequency plots for spectra consisting of the 86 conserved variables identified previously and pre-processed as N-SQRT-MC. As previously noted, the 86 variables provided a more homogeneous PCA scores plot (Figure 5), but, ironically, the frequency plot is more skewed (Table 3) than the plots in Figure 6A and B. Again, sensitivity taken from the frequency plots is excellent (94%), but specificity is only 67%. This suggests that while the use of 86 conserved variables makes the spectra of the A. racemosa BRMs more similar, it also diminishes the difference of A. racemosa and the other Actaea species. That is, use of 86 variables also makes the spectra of A. racemosa and the other Actaea species look more similar.
The opposite approach to using conserved variables to emphasize the similarity of the A. racemosa BRMs is to weight the variables to emphasize the difference between A. racemosa and the other Actaea species. Figure 6D shows the frequency plot obtained when each of the 1350 variables are weighted by the individual F values obtained from mANOVA for a comparison of A. racemosa and the other Actaea species. In the current study, F values (Figure 4C) were computed for each variable based on the between-group and within-group variances for A. racemosa and the other Actaea species. A large F value indicates a variable suitable for distinguishing between the two groups. The F values were used as a pre-processing step by multiplying the values of each variable in the array by the corresponding F value. The weighting results in a skewed frequency plot for A. racemosa with overlaps of 10 and 37% and sensitivities and specificities of 97 and 89%, respectively (Table 3). Thus, weighting by F values enhances discrimination between the two populations.
Validation
Validation is the process of establishing the predictability of a model, i.e., demonstrating that it will provide consistent results within specified statistical limits. This can be done several ways, with cross-validation (7), repeat analysis of a superior and inferior test materials (10), and examination of a frequency distribution plot (Figure 6). In each case, the analyst’s calculations are aimed at determining the overlap of distributions of the authentic population with the test population to determine the sensitivity and specificity of the method. In each case, these values are determined by the selected level of significance.
-
Frequency distribution.—Taken in reverse order, the frequency distribution plot provides the most complete set of information and benefits from the most complete collection of authentic samples. The A. racemosa plots in Figure 6 are based on 68 BRMs run in triplicate, and the other Actaea species plots are made up of 34 assorted Actaea BRMs (Table 1) run in triplicate. These plots show that the lack of homogeneity for A. racemosa seen in Figure 1D appears as a skewness at higher concentrations in Figure 6. The deviation from a normal distribution is clearly visible. Computed values for skewness are shown in Table 3, where a value greater than absolute 0.4 is generally accepted as indicating skewness in the data set. The plots in Figure 6 also indicate the importance of the overlap of the distributions. The confidence limit can be chosen at any level (2σ was used in Figure 6) with resulting changes in sensitivity and specificity as shown by the ROC plot in Figure 7. However, the fundamental aspect of the overlap remains unchanged.
Translation of the frequency plots in Figure 6 to cross-validation sensitivity and specificity is visually difficult. Table 3 shows that the changes in the overlap of the A. racemosa and other Actaea species distribution plots have little impact on the sensitivities of the training set (∼90%) and the test set (∼80%). Deviation from 95% can be attributed to the skewness of the data. Cross-validation of an in silico data set (constructed from an A. racemosa FIMS profile with random noise added to each variable to produce 100 spectra) yielded 96% sensitivity for both the training and test sets.
Comparison of superior and inferior test materials.—The Probability of Identification method introduced by AOAC International in 2012 (2, 9) called for the identification of a superior test material (STM) that translates as the authentic material. The inferior test material (ITM) was produced by adulteration of the STM at the highest acceptable level. Validation of the method required 30 analyses of the STMs and ITMs without a misidentification. Successful completion of the analyses would establish that the distribution of the two populations lacked little overlap and proved the accuracy of the method at the 95% confidence level. Care had to be taken to include chemical variability. It can be seen that the distributions and overlaps in Figure 6 would be marginal in achieving performance at the 95% confidence level.
Cross-validation.—Cross-validation constructs a model based on a random selection of a large fraction (60%–90%) of the A. racemosa BRMs as training samples and then classifies the remainder of the samples (test samples) as either authentic (inside the model) or not authentic (outside the model). This process is repeated many times. In the current study, 80% of the samples (162 of the 204 samples) served as the training samples and 20% (42 of the 204 samples) served as test samples, and the process was repeated 20 times. Technical repeats were grouped together.
Figure 7.
ROC (receiver operating curve) plot for Figure 6B: a plot of true positives versus false positives.
The skewness of the A. racemosa BRMs becomes apparent with cross-validation. A homogeneous set of authentic samples would be expected to give sensitivities of 95%. In silico modeling showed cross-validation sensitivities of 96% for both the training and test data (Table 3). In the current study, the sensitivity of the training and test sets in Figure 6B was 88 and 74%, respectively (Table 3), even with square root scaling and reduced overlap of the distributions for A. racemosa and the other Actaea species. The poor sensitivity is not surprising given the distribution of the authentic A. racemosa samples in Figure 1D–F. Random selection of the training samples can produce a highly unbalanced data set. This may either improve or degrade the sensitivity and specificity.
Stratified cross-validation (weighted selection from each source) would undoubtedly produce better sensitivities. Discriminating between the A. racemosa BRMs based on source (AHP, NCA, NIST, or SS) is reasonable as it is the only metadata available and clearly classifies their differences. However, all the raw roots were identified as authentic A. racemosa (G of the GxExMxP), and the only potential differences are environment (E) and management (M) with no processing (P) involved. Almost all BRMs were collected from the eastern half of the United States, but exact locations for each sample are not known, and each group was collected and stored at different institutions under different conditions (Table 1). PCA score plots for the NCA BRMs collected at 22 sites suggest compositional differences between sites (1). It can be speculated that differences arise from genetic drift, from soil and weather differences, and from endophytic bacteria and fungi. These same factors are equally applicable to all the BRMs regardless of their source. However, there is insufficient metadata to justify subgrouping the A. racemosa BRMs.
There is little doubt that the skewness of the A. racemosa BRMs impacts the results of cross-validation. For example, if the random selection of samples to be modeled included the 162 samples furthest to the left in Figure 6A, then the mean and 2σ level would be shifted to the left, and the 42 samples furthest to the right (in the skewed tail) would largely be judged as not A. racemosa and would provide low sensitivity. The shift the 2σ threshold to the left would also reduce the overlap with the other Actaea species and provide a greater specificity. Thus, skewness can produce a statistical bias. Hence, the frequency plot for Figure 6B, which provides the least overlap between the two distributions, provides a sensitivity and specificity of 88 and 74%, respectively (Table 3).
Conclusions
PCA and mANOVA easily distinguished between genera and was generally successful for most Actaea species BRMs. Discrimination between sources of A. racemosa BRMs was successful due to differences in chemical composition. Unexpected sources of variance were found within the technical repeats and the selection of variables. Distribution profiles, profile overlap, cross-validation, and sensitivity and specificity for the combined A. racemosa sources, and other Actaea species were dependent on the selection of technical repeats, selection of variables, and pre-processing methods. Frequency distribution plots allowed a more intuitive understanding of the impact of each of these factors. The sensitivity of all the pre-processing protocols fell between 94 and 97%. The best specificity of 89% was found with square root, autoscaling, or F test weighting of the variables. Use of only the quantitatively conserved variables produced better agreement between the A. racemosa BRM sources and yielded 94% sensitivity but only 67% specificity. Thus, the BRMs were more similar for A. racemosa but also for the other Actaea species. Optimum use of chemometric methods requires an understanding of the many factors that affect the sample variance.
CRediT Author Statement
James Harnly (Conceptualization-Lead, Data curation-Lead, Formal analysis-Lead, Funding acquisition-Lead, Investigation-Lead, Methodology-Lead, Project administration-Lead, Resources-Lead, Software-Lead, Supervision-Lead, Validation-Lead, Visualization-Lead, Writing—original draft-Lead, Writing—review & editing-Lead) and Roy Upton (Conceptualization-Supporting, Validation-Supporting, Writing—original draft-Supporting, Writing—review & editing-Supporting).
Contributor Information
James Harnly, Methods and Applications Food Composition Lab, Beltsville Human Nutrition Research Center, Agricultural Research Service, U.S. Department of Agricultural, Building 307C, BARC-East, Beltsville, MD, USA.
Roy Upton, American Herbal Pharmacopoeia, 3051 Brown’s Lane, Soquel, CA 95073, USA.
Funding
This research was supported by the Agricultural Research Service of the U.S. Department of Agriculture and an Interagency Agreement (AOD1906-001-00004) from the Office of Dietary Supplements of the National Institutes of Health, Health and Human Services.
Conflict of Interest
None declared.
References
- 1. Harnly J., Chen P., Sun J., Huang H., Colson K.L., Yuk J., McCoy J.-A.H., Harbaugh-Reynaud D., Harrington P.B., Fletcher E.J. (2016) Planta Med. 82, 250–262 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Harnly J., Chen P., Harrington P.D.B. (2013) J. AOAC Int. 96, 1258–1265 [DOI] [PubMed] [Google Scholar]
- 3. Harnly J.M., Luthria D.L., Chen P. (2012) J. AOAC Int. 95, 1579–1587 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Zhang M., Wang Y., Moore R., Upton R., Harrington P.B., Chen P. (2022) J. Agric. Food Chem. 70, 5450–5457 [DOI] [PubMed] [Google Scholar]
- 5. Hatfield J.L., Walthall C.L. (2015) Agron. J. 107, 1215–1226 [Google Scholar]
- 6. Chen P., Harnly J.M., Lester G.E. (2010) J. Agric. Food Chem. 58, 4545–4553 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Lester G.E., Manthey J., A., Buslig B.S. (2007) J. Agric. Food Chem. 55, 4474–4480 [DOI] [PubMed] [Google Scholar]
- 8. Brereton R.G. (2009) Chemometrics for Pattern Recognition, John Wiley and Sons. doi: 10.1002/9780470746462 [DOI] [Google Scholar]
- 9. Harnly J.M., Lu Y., Sun J., Chen P. (2017) J Food Comp Anal. 64, 85–92. doi: 10.1016/j.jfca.2017.06.010 [DOI] [Google Scholar]
- 10. Harnly J.M., LaBudde R. (2012) J. AOAC Int. 95, 273–285 [DOI] [PMC free article] [PubMed] [Google Scholar]