The National Institute for Occupational Safety and Health B Reader Certification Program—An Update Report (1987 to 2018) and Future Directions

Abstract

Objective:

The National Institute for Occupational Safety and Health (NIOSH) B Reader Program provides the opportunity for physicians to demonstrate proficiency in the International Labour Office (ILO) system for classifying radiographs of pneumoconioses. We summarize trends in participation and examinee attributes and performance during 1987 to 2018.

Methods:

Since 1987, NIOSH has maintained details of examinees and examinations. Attributes of examinees and their examination performance were summarized. Simple linear regression was used in trend analysis of passing rates over time.

Results:

The mean passing rate for certification and recertification for the study period was 40.4% and 82.6%, respectively. Since the mid-1990s, the number of B Readers has declined and the mean age and years certified have increased.

Conclusions:

To address the declining B Reader population, NIOSH is currently taking steps to modernize the program and offer more opportunities for training and testing.

In 1950, the International Labour Office (ILO) first published Guidelines for the use of the ILO International Classification of Radiographs of Pneumoconioses in an attempt to improve and standardize the recognition and recording of radiographic abnormalities caused by the inhalation of dusts.¹ Following several modifications and revisions,² the ILO Classification System has become a well-accepted scientific tool that facilitates international comparability of chest radiographic data used in pneumoconiosis research, occupational health surveillance, and compensation systems.³ In applying the ILO Classification System, readers use a systematic process of comparing the examinees’ chest radiograph to a set of standard (prototype) radiographs that illustrate the various types and severities of dust-caused changes. The ILO provides these standard radiographs, as well as a guide to procedures for applying the System to classify chest radiographs and record the results.²

In 1969, the US Congress passed the Federal Coal Mine Health and Safety Act, which required that coal miners be offered periodic chest radiographic screening. One objective was to detect early radiographic evidence of coal workers’ pneumoconiosis (CWP) so that steps could be taken to prevent disease progression in individual miners. In addition, radiographic screening results have been analyzed to track temporal and geographic trends in CWP. Since 1970, the National Institute for Occupational Safety and Health (NIOSH), Centers for Disease Control and Prevention (CDC) has operated the Coal Workers’ Health Surveillance Program (CWHSP)⁴ to administer the congressionally-mandated medical examination surveillance program for monitoring the health of coal miners using chest radiography. From the beginning, the CWHSP has used the ILO Classification System to standardize reporting of abnormalities on chest radiographs.

The “ABC system” of multiple readings was initially used for the first few years of the CWHSP, where the first interpretation of a miner’s chest radiograph was completed by an “A Reader” at the facility (local clinic or hospital) where the radiograph was obtained. A Readers were physicians who had taken a 2-day seminar from NIOSH or the American College of Radiology, or had submitted films to NIOSH with ILO classifications which were judged by NIOSH to be interpreted correctly. The second classification on a miner’s radiograph was completed by a B Reader. At the time B Readers were among a pool of 24 radiologists with “long experience with pneumoconiosis” from one of three radiology departments in the United States. If the A Reader and B Reader’s classifications agreed, the final determination of pneumoconiosis was made. If they did not agree, the radiograph was sent to a “C Reader” for a final determination. C Readers were seven staff radiologists experienced in determining pneumoconiosis at one of the three radiology departments as previously described.⁵

Soon after the establishment of the CWHSP, it became apparent that there was substantial variability between physicians in chest radiograph classification results even with the “ABC System” of multiple independent classifications.⁵ To reduce this variability, and to establish a pool of physicians with demonstrated competence in classifying chest radiographs using the ILO system of chest radiographs, NIOSH developed the B Reader Program. The Program endeavors to train physicians and identify those with competence in the use of the ILO Classification System. An initial proficiency examination to certify new B Readers was developed between 1974 and 1976 by Johns Hopkins University under contract with NIOSH, and validated by the American College of Radiology Task Force on Pneumoconiosis.⁶ This examination focused largely on radiographic findings associated with lung disease caused by coal mine dust. The first B Reader certification examinations were given in 1976 using the 1971 version of the ILO Classification System. Subsequently, a recertification examination was instituted in 1984 for periodic re-examination of B Readers. The recertification examination more prominently featured findings associated with asbestos-induced lung disease in comparison to the certification examination.

The examination is divided into six components, three of which address small opacity profusion: (a) agreement on the presence or absence of small opacities on each film, (b) assessment of the examinee’s over/underreading tendency or the degree an examinee’s small opacity profusion classifications systematically diverge from that of the expert panel, and (c) an inconsistency index which is based upon the standard deviation of differences between the examinee and expert panel classifications. The last three examinations sections address (d) the presence or absence of large opacities, (e) presence or absence of pleural abnormalities, and finally (f) the classification of the other symbols. Further details on the training and certification procedures for B Readers have been previously described⁶ and Wagner et al⁷ provided updated information on the B Reader Program, and an overview of the candidates’ attributes and scores for the years 1987 to 1990.

TRANSITIONING THE CWHSP TO DIGITAL RADIOGRAPHY

Until the most recent version of the ILO Classification System was published in 2011,² the system could only be applied to analog film-based chest radiographs. However, long before that, digital radiography had become the dominant modality used in US radiographic facilities. Thus, it was critical for the CWHSP to enable the use of digital chest radiographic images for ILO classification. To facilitate this, NIOSH (i) conducted research studies demonstrating that digital chest radiographs acquired on systems widely used in clinical practice could provide similar results to traditional film-screen radiographs when used for classification of pneumoconioses using the ILO system^8–12; (ii) provided detailed guidance on appropriate methods for image acquisition and viewing when using digital technology for ILO classification¹³; and (iii) developed the BViewer© Software (a freeware computer program that enables standardized viewing of digitally-acquired chest images for classification purposes, alongside digital versions of the ILO standards, and includes software for electronically recording classification results).¹⁴

With this foundation in place, NIOSH worked with the ILO in updating its classification system to enable use of digital chest radiographs² and amended NIOSH regulations to enable use of digital chest radiography in addition to traditional film radiographs in the CWHSP.⁴ By 2018, more than 98% of chest radiographs submitted to the CWHSP were digitally acquired. NIOSH is now focusing on transitioning the B Reader Program’s educational and testing materials to digital format by developing an entirely new examination using modern, digitally-acquired chest radiographs. As an interim measure, the B Reader examinations are currently available in digital format using digitized versions of the existing film-based certification and recertification examinations.

Pre-test Preparation

NIOSH strongly recommends pre-test preparation for examination (pre-test preparation improves the likelihood of success for new candidates; if candidates fail the initial examination NIOSH requires a waiting period of 90 days before re-testing on the initial examination) to assure familiarity with the ILO Classification System and associated Chest Radiograph Classification Form. In 1983, a Home Study Syllabus on Pneumoconiosis was developed by the American College of Radiology under NIOSH contract. The syllabus, revised in 2002, includes 80 radiographs, instructions, and answer keys. In 2011, the film images used in the teaching syllabus were digitized into Digital Imaging and Communications in Medicine (DICOM) format and made available for free download from the NIOSH website.¹⁵ Over the years, physicians have also prepared for examinations by attending American College of Radiology Symposia on Radiology of the Pneumoconioses, which were offered intermittently, or domestic or international courses taught on occasion by NIOSH.

Initial Certification and Recertification Processes, Examinations, Format, and Grading System

The initial B Reader certification examination consists of 125 film chest radiographs to be classified in 6 hours. The recertification examination consists of 50 radiographs to be classified in 3 hours. US licensed physicians who achieve a score of 50% or more on the initial examination are certified as NIOSH B Readers. B Readers are certified for 4 years from the date of approval, after which passing the recertification examination is required to maintain the B Reader certification.

Since the program’s inception, images used in the examinations have consisted of film chest radiographs demonstrating various types and severities of pneumoconiosis, as well as several films without pneumoconiosis. These were displayed on lighted view boxes, with ILO classification results manually entered onto paper forms. Recently, examinations have also been offered using high quality scanned and digitized versions of the current films, displayed on medical grade monitors using BViewer© Software (NIOSH, Morgantown, WV)¹⁴ for viewing and data entry. Candidates may thus take the examination in a completely computerized format. Physicians currently have the choice to take either the film or digitized examinations. Grading and scoring of the certification and recertification examinations have not changed since the 1992 paper by Wagner et al⁷ The objectives of this report are to summarize B Reader examination results from 1987 to 2018 and to describe future directions of the B Reader Program.

MATERIALS AND METHODS

Forms used to record examinee attributes have varied throughout the program. Since January 1, 1987, NIOSH has maintained a database with details of examination attempts, scores, and demographics of physicians taking the B Reader examination. Prior to that date, only candidate age was recorded consistently. The Interpreting Physician Certification Document, used to record an examinee’s contact information and demographics, was modified in 1992 to include optional items asking (i) the average number of chest radiographs the examinee classified monthly using the ILO system; and (ii) how the candidate anticipated using their B Reader certification (response options: individual patient care, industry programs, medical–legal activities, government programs, other).¹⁶ Using this NIOSH examination database, examinee numbers and demographics, overall scores, pass/fail rates, and false positive and false negative answers were summarized using SAS 9.4 (Cary, NC). Simple linear regression was used in trend analysis of initial and recertification passing rates over time, slopes (β) and the corresponding significance levels (P values) were calculated.

RESULTS

Examination Participation and Candidate Characteristics

Including the initial testing in 1976, 2684 US licensed physicians have taken the examination. From January 1, 1987 through December 31, 2018, 5454 examinations (2857 certification and 2597 recertification) have been administered. During this time, 1202 physicians (53.0%) have taken an examination once, 378 (16.7%) twice, 220 (9.7%) three times, and the remaining 468 examinees (20.6%) have taken a certification or recertification more than 3 times.

Beginning in 1992 information was collected from B Readers estimating the number of ILO classifications they complete per month over the previous year. For their most recent examination, 82.2% of certified B Readers responded to this question and on average reported classifying 70 (range, 1–500) chest radiographs per month using the ILO system. Among the responders, individual patient care (53.4%) and medical–legal activities (47.3%) were the most common anticipated usages of B Reader certification, followed by industry programs (38.5%), government programs (29.7%), and other uses (5.4%).

The number of B Readers in the United States on December 31 each year since 1970 is shown on Fig. 1. The number of B Readers increased from about 100, after the initial examination in 1976 (during 1970 to 1976, a non-examination process for appointing B Readers existed), to a peak of about 750 in 1993, and has steadily declined since then. In December 2018, 165 US physicians were NIOSH certified B Readers. The distribution of clinical specialties is shown in Table 1. The mean age of the certified B Reader population has increased since the mid-1970s, with a mean age of 62.4 in 2018 (Fig. 2; black circles corresponding to the left y-axis). The mean number of years individual B Readers have been certified also continues to increase (Fig. 2; grey circles corresponding to the right y-axis) with 22 years as the mean number of years certified in 2018, further demonstrating that the current population of B Readers is generally an aging survivor population.

FIGURE 1.

Number of certified B Readers on registry on December 31 each year.

TABLE 1.

Primary Medical Specialty of Currently Certified B Readers in 2018

	N	%
Radiology	109	66.1
Pulmonology	27	16.4
Internal Medicine	13	7.9
Other	9	5.5
Occupational Medicine	7	4.2

FIGURE 2.

Mean age of certified B Readers by year, 1970 to 2018, represented by black circles corresponding to the left y-axis. Mean number of years individual B Readers have been certified, by year, 1970 to 2018, represented by grey circles corresponding to the right y-axis.

Pass/Fail Rates and Distribution of Scores

The passing rates for the initial certification and recertification examinations have fluctuated since 1987 (Fig. 3). The mean passing rate for certification for the period 1987 to 2018 was 40.4% and has significantly decreased over this time period (β = −0.4, P = 0.002). In contrast, the mean passing rate for recertification examinations in this period was 82.6% and has significantly increased over time (β = 1.0, P < 0.0001).

FIGURE 3.

B Reader passing rates for certification and recertification examinations 1987 to 2018.

The distributions of total scores for the certification and recertification examinations taken between 1987 and 2018 are displayed in Fig. 4. For the certification examination, scores were normally distributed around the median (44.3). Recertification scores were significantly higher (median 59.2) and the overall distribution was skewed toward higher scores. Component scores are presented in Table 2 and the grading system has been described in detail previously.⁷ Candidates who passed scored about twice as many points as those who failed and this was generally consistent across all examination components.

FIGURE 4.

B Reader final scores, 1987 to 2018. Percent distribution, certification versus recertification examination.

TABLE 2.

Mean and Standard Deviation for Component Scores by Pass/Fail, Certification Versus Recertification Examination, B Readers 1987 to 2018

	Certification Examination					Recertification Examination
		Pass Observed		Fail Observed			Pass Observed		Fail Observed
	Maximum	Mean	(SD)	Mean	(SD)	Maximum	Mean	(SD)	Mean	(SD)
Small opacity
Agreement	20	10.8	(2.6)	5.4	(3.5)	20	9.6	(3.5)	4.5	(3.4)
Over/under reading	10	7.5	(1.8)	4.4	(3.1)	10	6.8	(2.4)	3.0	(2.9)
Inconsistency index	30	19.2	(4.2)	10.0	(5.7)	30	21.7	(4.6)	13.3	(5.7)
Large opacity	20	9.6	(3.8)	5.3	(4.4)	20	13.4	(4.5)	9.4	(5.9)
Pleural disease	10	2.4	(2.0)	1.8	(1.9)	20	11.6	(3.2)	8.4	(4.0)
Other symbols	10	7.4	(1.6)	5.8	(2.2)	0	—	—	—	—
Total	100	56.9	5.9	33.2	10.9	100	63.1	8.8	38.5	8.2

Small Opacity False Positives and False Negatives

The mean percentage of small opacity false positives and false negatives are displayed in Fig. 5. False positives are more commonly found by examinees than false negatives for those passing the examination (~2.3 fold difference) and among those failing the examination (~ 4.5 fold difference). The percentage of false negatives reported for both certification and recertification examinations have remained consistent throughout the years at around 7%. Taken together, those who pass and those who fail misclassify an equivalent amount as false negative. However, candidates who fail tend to have about twice the number of false positives as those who pass.

FIGURE 5.

Mean percentage of false positive and false negatives, small opacities by pass/fail for B Reader certification and recertification examinations, 1987 to 2018.

DISCUSSION

Chest radiographic imaging is a widely applied and important tool for assessing lung health in research investigations and evaluations of workers exposed to dusts capable of producing pneumoconiosis. Monitoring of respiratory health using chest radiography remains one of the essential tools in the recognition and secondary prevention of occupational lung diseases.

In the early 1970s, shortly after the initiation of the Congressionally-mandated radiographic screening program for underground coal miners,¹⁷ it became clear that results of ILO classifications of miner chest radiographs showed excessive inter-reader variability. Felson et al⁵ described that early classification results indicated readers had difficulty in distinguishing between healthy lungs and the threshold for classifying small opacities as consistent with pneumoconiosis. In general, readers with less training tended to identify more pneumoconiosis than experienced readers,⁵ a trend that has been observed across several dust exposed worker populations in multiple countries over the last three decades.^10,18–22 This finding is consistent with the high rate of false positives among examinees who fail the examination observed in the present study (Fig. 5).

Felson et al⁵ also found that a number of physicians performing classifications were not adequately familiar with the various radiographic manifestations of pneumoconiosis, the Guidelines for the use of the ILO Classification System, nor the impact of radiograph quality in assigning a pneumoconiosis classification. To address these concerns, NIOSH instituted the B Reader Program, to provide appropriate training, and examinations to identify readers who were proficient in using the ILO Classification System to interpret chest radiographs for pneumoconiosis. Passing the examination conferred B Reader certification.⁶ Over the past 40 years, B Readers have played an essential role in public health surveillance, secondary disease prevention, medical–legal proceedings, and compensation determinations in the United States, with other countries adopting similar requirements following demonstration of similar concerns.²³

Since the establishment of B Reader training and examination in the 1970s, the Program has undergone a limited number of changes, including introduction of a recertification examination (1984) and the release of digitized versions of the film-based training syllabus (2011) and examinations (2015), allowing for completely digital study and testing environments. Having tools to train physicians and certify that they are able to classify chest radiographic images for presence and severity of changes consistent with pneumoconiosis using the ILO system still remains important for research, surveillance, and compensation. In addition to past needs, we anticipate increasing demand for B Readers able to perform ILO classifications under the US Occupational Safety and Health Administration’s recent Silica Standard.²⁴

Modern radiology is predominantly digital and contemporary physicians no longer work with older film-based radiographs. Thus, in addition to transitioning NIOSH’s surveillance services to digital radiography,⁴ NIOSH is also moving to modernize physician training and certification testing. To have an adequate supply of modern, digitally-acquired chest radiographs for this purpose, NIOSH developed a digital image repository able to collect anonymized digitally-acquired radiographs and computed tomography (CT) scans depicting different types of abnormality associated with the pneumoconioses. The repository contains images derived from NIOSH activities and from those supplied by external partners, both domestic and international. In partnership with the American College of Radiology, a modern B Reader certification examination is being developed using this large repository of images as source material. This effort is identifying key knowledge and competencies to be taught and tested, developing and validating new teaching and testing materials that address the key knowledge and competencies, and establishing plans for ongoing quality assessment and improvement of the teaching and testing program. We anticipate that the new examination currently under development, based entirely on modern digitally-acquired images, will be administered solely in electronic format.

Recent research continues to confirm that readers who are trained and demonstrate ongoing competence in the ILO Classification System (NIOSH B Readers), and who are subjected to regular quality control, are desirable to provide reliable pneumoconiosis classifications, as other readers are more likely to misclassify aspects of pneumoconiosis.⁹ Although chest computerized tomography provides more detailed images and is often the diagnostic procedure of choice for patients with known or suspected diseases of the chest in clinical settings, plain chest radiography’s low radiation dose, economy, and ready availability continue to make it a very important screening tool for pneumoconiosis worldwide.

NIOSH is working to address the precipitous decline in B Readers (Fig. 5) and the yearly increase in mean age of those Readers (Fig. 1). While a systematic study or investigation into why the B Reader population has decreased dramatically has not been conducted, the cause for this decline is likely multifactorial. Those in the field have observed a shift in technology, training, and demand. CT became widely available in the 1980s and gradually dedicated training in most medical training programs placed less focus on chest radiography. B Readers were most commonly used for screening among workers exposed to asbestos and coal mine dust and at the same time medical technology and training was changing, industries with exposed workers were contracting, reducing the demand for readers. By modernizing the syllabus and examination and partnering with academic institutions and professional organizations, NIOSH is working to provide face-to-face training opportunities (including the opportunity to sit for the examination) at sites across the country with efforts to target residents and fellows in training as well as specific specialties to address state-by-state needs. NIOSH is also formally partnering with the American College of Radiology to ensure at least two B Reader Courses and Examinations will be offered at their training center in Reston, VA starting in 2020. Additionally, NIOSH has recently expanded the recertification period from every 4 years to every 5 years.

CONCLUSION

Despite widely available and substantial scientific knowledge regarding the hazards of respirable dust exposure, occupational dust-induced lung diseases remain substantial threats to health and life for many 21st century workers, in both established and emerging economies.^25–32 Programs for monitoring respiratory health that include chest radiography have demonstrated public health utility in the identification, tracking, and secondary prevention of occupational pneumoconioses, but require rigorous implementation.^2,4,13,33 This current report describes recent trends, refinements, and ongoing modernization of the B Reader Program, with the goal of informing NIOSH partners in occupational medicine, industrial hygiene, and public health and, importantly, enhancing preventive actions addressing these dust-induced lung diseases.

Clinical Significance:

Respiratory health monitoring programs that include chest radiography have demonstrated public health utility in the identification, tracking, and secondary prevention of occupational pneumoconioses, but require rigorous implementation including the use of readers trained in the ILO system. NIOSH is in the process of modernizing training and proficiency testing, moving to a fully digital syllabus and examination.