Assessment of Respirable Crystalline Silica Analysis Using Proficiency Analytical Testing Results from 2003–2013

Abstract

Analysis of Proficiency Analytical Testing (PAT) results between 2003 and 2013 suggest that the variation in respirable crystalline silica analysis is much smaller today than it was in the period 1990–1998, partly because of a change in sample production procedure and because the colorimetric method has been phased out, although quality improvements in the x-ray diffraction (XRD) or infrared (IR) methods may have also played a role. There is no practical difference between laboratories using XRD or IR methods or between laboratories which are accredited or those which are not. Reference laboratory means (assigned values) are not different from the means of all participants across the current range of mass loading, although there is a small difference in variance in the ratios of all participants to reference laboratory means based on method because the reference laboratories are much more likely to use XRD than are the others. Matrix interference does not lead to biases or substantially larger variances for either XRD or IR methods. Data from proficiency test sample analyses that include results from poorly performing laboratories should not be used to determine the validity of a method. PAT samples are not produced below 40 μg and variance may increase with lower masses, although this is not particularly predictable. PAT data from lower mass loadings will be required to evaluate analytical performance if exposure limits are lowered without change in sampling method. Task-specific exposure measurements for periods shorter than a full shift typically result in lower mass loadings and the quality of these analyses would also be better assured from being within the range of PAT mass loadings. High flow rate cyclones, whose performance has been validated, can be used to obtain higher mass loadings in environments of lower concentrations or where shorter sampling times are desired.

INTRODUCTION

Inhalation of respirable crystalline silica (RCS) is known to be associated with adverse health outcomes. Exposure to RCS is assessed by pulling a known amount of air through a personal size-selective sampler and filter, and then measuring the silica collected on the filter. Industrial hygiene laboratories use one of three analytical techniques (X-ray diffraction spectrometry (XRD), infrared absorption spectrometry (IR), and colorimetric spectrophotometry) for the quantitative determination of RCS. Interlaboratory variability historically has been high for these analyses. Agreement between laboratories, as measured through analyses reported to the American Industrial Hygiene Association (AIHA) Proficiency Analytical Testing (PAT) program over the period April 1990 through April 1998, has been studied and the results published.⁽¹⁾ In that study, colorimetric analysis was more commonly used than today and the main conclusion was that it showed relatively poor recovery at low loadings and overall poor precision compared to XRD and IR methods. Since that time, there have been several factors that may have further affected the variability of analyses, including a trend in the reduction in the number of laboratories using the colorimetric method.

The American Conference of Governmental Industrial Hygienists (ACGIH^®) proposed reducing their Threshold Limit Value (TLV^®) for RCS in 2004 and adopted 0.025 mgm⁻³ as an 8-hour time-weighted average (TWA) in 2006.⁽²⁾ This is an advisory limit; legal limits in the United States are set by the U.S. Occupational Safety and Health Administration (OSHA) and are known as Permissible Exposure Limits (PELs). OSHA is engaged in rulemaking with respect to exposure and control of respirable crystalline silica to better prevent the onset of disease.⁽³⁾ The proposed rule includes a lower PEL for airborne RCS, and actions triggered by measurements below this limit (an “action level”). Limits under scrutiny include a potential PEL of 0.05 mgm⁻³ and an action level of 0.025 mgm⁻³.

The high variability of RCS analyses leads to questions regarding analytical capabilities to support lowered limits, which can be investigated through a new assessment of data from the AIHA PAT program. The AIHA moved the PAT program in 2009 into a separate Limited Liability Company (AIHA PAT, LLC) and the National Institute for Occupational Safety and Health (NIOSH) and AIHA PAT, LLC signed a Letter of Agreement in 2013 to allow data-sharing. NIOSH received the results reported for RCS proficiency sample analysis for all participating laboratories from Round 152 in January 2003 through Round 194 in July 2013 (with accreditation status of the participants from Round 171 in October 2007, onwards). These have been studied to determine whether there has been any improvement in variability since the previous Eller et al. publication.⁽¹⁾

Contributing to the variability in the overall PAT program results is the variability inherent in the sample production. Prior to Round 162 in July, 2005, the production process involved sampling an aerosol of RCS generated in a large chamber through individual size-selective cyclones with filters. Thus any spatial inhomogeneity of the aerosol in the chamber, inter-unit variability in cyclone separation performance, and variability in the calibration and maintenance of pump flow rate contributed to the overall sample variance. In 2004 (reported 2005), a study of PAT Samples and field samples showed that particles collected in cassettes attached to cyclones could deposit on the internal non-filter surfaces of the cassettes.⁽⁴⁾ Variability in the fraction of aspiration depositing on the filter versus other surfaces likely contributed to the variability in PAT Sample results. In Round 162 production was switched to a liquid-suspension (slurry) and deposition process that could provide lower sample variability. In the last 15 years there have also been several initiatives that may have led to further improvements in the inter-laboratory variability of RCS analyses. A comprehensive survey was carried out by the AIHA in collaboration with NIOSH on the practices being used by laboratories engaged in these analyses, which uncovered issues that needed to be addressed, including the calibration material and the levels and frequency of calibration, as well as deviations in procedures from the published methods. A series of recommendations was published⁽⁵⁾ and on-site auditors from the AIHA laboratory accreditation program were given support through checklists and training on issues critical to good quality RCS analysis.

METHODS

Each laboratory had been assigned a unique serial number in the data provided by the AIHA PAT, LLC so that the identities of laboratories are not known to NIOSH. Rounds of PAT Samples are issued on a quarterly basis, with four Samples at different target mass loadings in each Round. NIOSH was provided with the reported results for each Sample in each Round for each participating laboratory, together with the analytical method (XRD, IR, or colorimetric) used by the laboratory, and the identity of the background matrix added to the samples for that Round (coal mine dust (CMD), talc, CMD plus talc, or calcite). AIHA PAT LLC was also able to provide the accreditation status of those laboratories from Round 171. About 80% of all laboratories participating in the PAT scheme are accredited for the analysis of silica, with about two-thirds of these laboratories using XRD, which is a higher proportion than for all laboratories generally.

There is no “true value” of the amount of RCS mass on each Sample although there is a target value and a subset of the samples are checked via quality control analyses by the contractor preparing the samples. The target value must be achieved within set limits (+/− 10%) before the samples are released to participants. The assigned value for each Sample is defined by a sub-group of “reference laboratories” (AIHA PAT, LLC terminology). The average of the values reported by the reference laboratories is compared to the target level required of the Sample provider and to the provider’s quality control data, and if these values are considered under control, the mean value of the reference laboratories after Windsorization (5%–95%) is used as the assigned value. The criteria used for determining a reference laboratory are arguable (an accredited laboratory participating in at least three of the four of the Industrial Hygiene proficiency testing analyses offered by AIHA PAT, LLC and considered proficient for the prior two rounds) and a study had been undertaken by NIOSH to determine the effect of switching to the mean of all laboratories, which suggested that this value could also be used as the assigned value. Hence in the Eller et al. publication⁽¹⁾ there is a statement that the mean of all laboratories will be used in the future. However, when this change was made for all industrial hygiene program analytes there was considerable negative feedback from the asbestos analytical laboratories because the mean of all laboratories in the analysis of chrysotile was sufficiently different from the mean of reference laboratories for reference laboratories to become non-proficient. The change therefore was reversed and reference laboratories have again been used to derive the assigned values since then. Thus, the methodology for determining the assigned value that was used during the period covered by the previous study was also in use during the entire period covered by this study.

Statistical Approach

We have addressed two figures of merit, one being the mean and the other being the relative standard deviation (RSD) of results broken down as below. Less than 5% of results were removed from the data set before calculation of means and RSDs. Eighty-eight of these were 44 pairs of initial and re-test results in the data set provided. Since the “official” result from each pair could not be determined, all the results were removed. Also removed were any results where the method of analysis was not provided, and any results which were more than three standard deviations from the mean, on a Round or per Sample basis. This resulted in a final tally of 9449 observations for analysis. The RSDs of the full data set were compared to those from the Eller et al. study,⁽¹⁾ then broken down by analytical method and compared between XRD and IR (insufficient results from the colorimetric method made this comparison unreasonable), and then broken down by accreditation status and compared for those Rounds where this was known. The means were also compared for the same groups between each other and with the assigned values. Comparisons ranked according to the assigned values highlight any effect of Sample loading.

RESULTS AND DISCUSSION

The 2003–2013 data set is only slightly smaller than the 1990–1998 data set and the RSD for each Sample is plotted in Figure 1 against the mean sample mass for both data sets. There is a clear difference between the older and newer data, with an improvement in RSD from about 29% to 21% fairly consistently across the whole set. The lowest mass loadings are about the same in both data sets and both data sets show an increase in RSD at lower mass loadings that can be expected to increase further at masses below the lowest values plotted. There needs to be a strong justification for any model applied to extrapolate outside of the data range as the selection of model will have the strongest effect on the outcome. In particular, non-linear models may not be an accurate reflection of information regarding the process under consideration. Therefore, we have elected not to attempt such an extrapolation here. If the 2003–2013 data are split between Rounds prior to 162 and from 162 onwards (Figure 2), it can be seen that the change in Sample production process has contributed some of the reduction in variability, but not all. The remainder may be from the reduction of laboratories using the colorimetric technique or it may be from a general improvement resulting from the use of more appropriate calibration materials, more frequent calibration, use of a lower mass calibration standard, and stricter adherence to the published method procedures, or a combination of any or all of these factors (there is also the possibility that there have been improvements in the analytical equipment). When data sets of this size are analyzed even small differences can be statistically significant, although the difference itself may be practically meaningless. Thus we present the results without determining statistically significant differences.

FIGURE 1.

Comparing old data and new data. (Relative standard deviation (RSD) of RCS PAT Samples from Rounds 152–194, 2003–2013 (New data), compared to Rounds 102–132, 1990–1998 (Old data), according to all-laboratory mean mass loading.

FIGURE 2.

Comparing data round number >161 and data round number < = 161. (Improvement in relative standard deviation (RSD) of RCS PAT Samples with change in Sample production from Round 162 onwards (all laboratories).

In Figure 3 we compare XRD and IR analyses in total and in Figure 4 split according to matrix interference. Both XRD and IR methods show higher variability when the matrix is calcite; this was also observed in the previous study, and was not affected by the change in Sample production in the current study period. No explanation has been suggested but it might be due to greater variability in grain size or mass loading of the calcite, or it may be due to the sample preparation process (e.g., reaction between calcite and silica). The lowest variability is encountered for IR when the matrix is a combination of CMD and talc and this may be because both materials are in smaller quantity than when the individual materials are used. The number of participants using the colorimetric method is now so low that there are Rounds without representation, including all Rounds in the past 3 years and because of these small numbers performance has not been evaluated.

FIGURE 3.

Box–plots of infrared (IR) and x-ray diffraction (XRD) relative standard deviations (RSD) for all RCS PAT Rounds and Samples (all laboratories) (2% difference in average RSDs is statistically significant but not important).(color figure avaibale online)

FIGURE 4.

Box-plots for relative standard deviation (RSD) of RCS PAT Samples by infrared (IR) and x-ray diffraction (XRD) methods divided by matrix interference type (all laboratories, Rounds 152–194).(color figure avaibale online)

Accredited and non-accredited laboratories showed similar variance. Comparison of means between the reference laboratories and all laboratories needs to be considered carefully, since the reference laboratories themselves are a subset of all laboratories. Nevertheless, a consistent bias on the part of non-reference laboratories would be apparent in Figure 5, and would be more obvious in a plot of reference laboratory means against the means of non-accredited laboratories (since all reference laboratories are by definition accredited) as in Figure 6. Any bias between these two groups could be a result of a difference in analytical method since reference laboratories are predominantly users of XRD and non-accredited laboratories are predominantly users of IR, but none can be seen.

FIGURE 5.

Plot of reference laboratory RCS PAT Sample mean (assigned value) against all-laboratory RCS PAT Sample mean (Rounds 152–194).

FIGURE 6.

Plot of reference laboratory RCS PAT Sample mean (assigned value) against non-accredited laboratory RCS PAT Sample mean (Rounds 171–194).

Figure 7 shows the ratio of reference laboratory mean to the mean of all laboratories plotted against reference laboratory mean. Figure 8 shows the ratio of reference laboratory mean to the mean of all laboratories separated by method (XRD or IR) plotted against reference laboratory mean. Since the proportion of XRD users is higher than IR users in the subset of reference laboratories it can be expected that the spread of data ratios around unity is tighter for the XRD plot than for the IR plot. No trend in bias by reference laboratory mean mass loading is discernable in either plot. A plot of reference mean divided by all-laboratory mean against reference mean value is presented according to interfering matrix in Figure 9. No difference is observable by matrix and again no trend as a result of loading can be seen. Figures 5–9 also indicate there is no difference between the reference laboratories and the others, suggesting that the mean of all laboratories could be used as the assigned value for this analyte as previously suggested.

FIGURE 7.

Ratio of reference laboratory RCS PAT Sample mean (assigned value) to all-laboratory Sample mean plotted against assigned value (Rounds 152–194).

FIGURE 8.

Ratio of reference laboratory RCS PAT Sample mean (assigned value) to all-laboratory RCS PAT Sample mean plotted against assigned value (Rounds 152–194) divided according to method used in all laboratories.

FIGURE 9.

Ratio of reference laboratory Sample mean (assigned value) to all-laboratory Sample means plotted against assigned value (Rounds 152–194) divided according to matrix interference.

It is clear that the variability in RCS analyses as expressed through the PAT results has improved dramatically since the last assessment 15 years ago. There is a contribution to the reduced variance from reduction in variability in the Sample production and the reduction in use of the colorimetric method, but the remainder is a result of factors arising from the manner in which laboratories perform the analyses. The reduction in RSD is consistent across the range of loadings produced in the PAT program. While there is a trend of increasing variance with decreased mass, the magnitude of the effect appears to have decreased in the recent data set. Increasing variance with decreasing mass loading is not a function of major method (XRD vs. IR), nor is it affected by the type of matrix interference. Variance also is not dependent on the reported accreditation status of the laboratory and this should not be surprising since accreditation is an assurance of an effective quality system in place and not necessarily a guarantee of good performance, while, conversely, the absence of accreditation does not necessarily imply poor practices and poor results. Accreditation is a valuable assurance to clients of commercial laboratories performing Industrial Hygiene analyses for fee, but may be less useful for laboratories serving only “in-house” clients. Many of the latter may be laboratories using IR, while the accredited laboratories, which, by definition, include the references laboratories, are more likely to use XRD. The excellent agreement between non-accredited and reference laboratories is further evidence that there is no difference between these two methods in their ability to produce quality results in the covered mass range.

CONCLUSION

The average RSD of PAT samples since Round 162 is 19.9%, which is a substantial improvement since the last evaluation between 1990 and 1998. The decline in RSD noted here is a result of changes in sample production and the decline of the colorimetric method, but it may also have been driven by improved attention to method implementation by laboratories and improved quality assurance parameters, including the use of new reference materials. The 10-mm nylon Dorr-Oliver cyclone operated at 1.7 liters per min (LPM) for a full 8-hr shift will provide a mass loading of 41 μg at a concentration 0.05 mgm⁻³, which is just within the lowest range of current PAT Samples. If there is further increase in variance below 40–50 micrograms loading, it would have implications for the range of the analytical methods and, if the variance is excessive, it may be difficult to address for either method. To demonstrate that laboratories are capable of determining samples collected at lower concentrations, or for less than full-shift samples, the AIHA PAT, LLC should consider the production of Samples with lower mass loadings, at least down to 20 μg, although it may be technically more difficult to produce Samples with low variability at lower masses. A currently available option to ensure analytical accuracy around lower exposure limits or for shorter sample times to address task-specific measurements is to increase the collected mass through the use of a cyclone that operates at a higher flow rate, where performance has already been evaluated, ^(6–8) and which have already been used in the field.⁽⁹⁾ For example, a GK2.69 cyclone (BGI, Inc.) operated at ≈4 LPM will provide ≈100 μg under the conditions given above. However, to achieve a sample loading within the PAT range for a 4-hr sample at 0.0125 mg/m³ would require a cyclone similar to the FSP-10 (GSA, GmBH) which operates at >11 LPM. Cyclones of this type are larger and heavier than ones typically used today and require correspondingly larger and heavier pumps; moreover, there is a consequent higher level of complaint from workers regarding the additional burden.⁽¹⁰⁾ Therefore, it is recommended that AIHA PAT, LLC should still investigate the possibility of providing PAT Samples for RCS in the 20–50 μg range to support the analysis of lower target concentration levels.