Comparison of HSCORE Assessment of Endometrial β3 Integrin Subunit Expression with Digital HSCORE Using Computerized Image Analysis (ImageJ)

Abstract

OBJECTIVE

To compare intraobserver and interobserver variation between traditional histological score (HSCORE) and digital HSCORE (D-HSCORE) performed by expert and naive researchers.

STUDY DESIGN

Immunohistochemical analysis of β3 integrin subunit of 100 endometrial biopsies obtained from the midluteal phase of the menstrual cycle were reanalyzed using ImageJ software (D-HSCORE). Mean intensity of 3,3′-diaminobenzidine on endometrial glands was read by an expert (HSCORE) versus inexperienced observer using HSCORE and D-HSCORE.

RESULTS

The mean correlation [r_s(95% CI)] between both methods was 0.86 (0.79–0.90) and highly significant (p < 0.0001) for the experienced individual. The naive researcher overestimated immunostaining, resulting (HSCORE) in negative samples. No discrepancies were seen with D-HSCORE. Interobserver variation for the inexperienced reader was 50% using HSCORE (cut-off 0.7) but 0% with D-HSCORE. Intraobserver variation using ImageJ was 0%.

CONCLUSION

The D-HSCORE performed by an inexperienced researcher has high correlation to traditional HSCORE performed by an expert.

Biomarkers of endometrial receptivity have been an intense field of research for over 20 years.^1–3 Integrins are one of the best characterized endometrial biomarkers, first described in 1992 by Lessey et al⁴ and further characterized in cycling endometrium in 1994.⁵ The αvβ3 integrin appears at the time of implantation on cycle day 20 and is expressed into pregnancy in normal women but is lacking in some women with certain conditions including endometriosis,⁶ hydrosalpinges,^7,8 polycystic ovary syndrome,⁹ and unexplained infertility.^10–12 Unfortunately, not all researchers have been able to reproduce these findings,^13–16 suggesting that methodological aspects¹⁷ and/or subjective differences in scoring immunohistochemistry may influence interpretation and results.

Immunohistochemistry has been the predominant method to measure the β3 integrin subunit expression,^4,5,18 but others have use flow cytometry¹⁹ and polymerase chain reaction.^20,21 While a significant quantitative relationship was shown between histological score (HSCORE) and the biochemical analysis in tissue homogenates,²² this semiquantitative assessment of immunohistochemical results is potentially inaccurate due to the high interobserver variation and the potential for subjective misinterpretation. Budwit-Novotny proposed the formula for HSCORE as equal to ΣPi (i + 1), where i is the intensity of the staining, ranging from 0–3 (none, weak, mild, strong), and P is the percentage of the intensity, and the addition of 1 was included as a correction for optical density. This formula has 2 major sources of bias, including the percentage of the area examined and the perceived intensity of the staining. Therefore, the skill and experience of the observer could influence the results obtained. Based on receiver operating characteristic (ROC) curve analysis, an HSCORE of 0.7 was established as the cut-off for a negative test for purposes of studying endometriosis.⁶ However, results using biomarkers such as the β3 integrin subunit have not been consistently reported.

ImageJ is an image analysis software program developed and provided for free by the National Institutes of Health (http://rsbweb.nih.gov/ij/). It has specific built-in features including an algorithm to evaluate staining using hematoxylin and DAB staining (HDAB) called color deconvolution plug-in.²³ The method of immunohistochemistry scoring requires a single subjective action related to marking the area to be analyzed and can be saved in a file, which makes reproducibility identical. In a recent PubMed search using the terms integrin, beta 3, and ImageJ, no results were retrieved (search performed on June 30, 2012, no limits). While ImageJ has been used for immunohistochemistry analysis in other systems,^24–27 its use for endometrial receptivity research is long overdue. In the present study we used ImageJ software to analyze the expression of β3 integrin subunit in a population of women with infertility. The ImageJ analysis, performed by a new researcher without experience at HSCORE assessment, was compared to the HSCORE performed by an expert in the field with 20+ years of experience in reading immunohistochemical analyses. In addition, we evaluated the reproducibility of this method and report these data here.

Materials and Methods

One hundred sections of endometrium obtained from the laboratory archives of the Reproductive Endocrinology and Infertility unit from Greenville Hospital System, Greenville, South Carolina, were used for study analysis. Endometrial biopsies were obtained after informed consent. Each slide had been immunostained for the β3 integrin subunit as previously described⁴ and evaluated by HSCORE by a single expert observer (B.A.L.) in a blinded fashion according to the formula proposed by Budwit-Novotny and colleagues.²² Slides were selected to include 50 positive and 50 negative results, from a larger group of over 400 available samples. All endometrial biopsies were performed during the midluteal phase of the menstrual cycle 7–10 days after the luteinizing hormone surge using commercially available urinary luteining hormone surge prediction kits; no other criteria were used in the selection process. One of the researchers (B.A.L.) had previously assigned the HSCORE to the slides. The same slides were then subjected to β3 integrin subunit expression in a blinded fashion using the traditional HSCORE and the digital HSCORE (D-HSCORE) by a naive researcher (D.G.F., a 3rd year medical student). This researcher had previously received instructions on HSCORE for 2 hours by the experienced reader (B.A.L.). To assess the intraobserver variability of HSCORE, 12 slides (6 positives and 6 negatives) were reanalyzed by the naive researcher 1 week apart. Intervariability was performed comparing the HSCORE readings between the naive and the expert researcher.

Digitalization of Slides in the Hardware

Each slide was digitalized using an Olympus BX51 microscope (Olympus Optical Co., Tokyo, Japan) connected to a digital color camera (Q-Color 5, Olympus). If necessary, multiple digital pictures were taken until the digitalization of the whole specimen was achieved. Images were obtained with a UPLFLN 20× objective (numerical aperture, 0.5; field number, 26.5) at a size of 2,560 × 1,920 pixels (resolution, 1 mm = 2,950 pixels) under standard lighting conditions and saved as a .tif image with 16 bits. Each slide was coded for ImageJ analysis.

ImageJ Analysis

Slides were blindly analyzed with image analysis software (ImageJ 1.43j, National Institutes of Health, Bethesda, Maryland, U.S.A.). Initial settings of the software were applied to measure area (mm²), median and mean intensity. Global scale of the image analysis was set as 2,950 pixels = 1 mm, in a pixel ratio of 1. The following 10 steps were used for image analysis. (1) An empty area from the slide was marked and analyzed with a color histogram for the RGB channels. The RGB values should be similar and close to 255. (2) If they were not similar, the image was adjusted by subtracting the background using the command “Process → subtract background” from the file menu. (3) Selection of the area (region of interest [ROI]), in this case, endometrial glands, was performed using the “Brush” tool, adjusting the size of the brush according to the thickness of the endometrial gland (e.g., 35 pixels) as shown in Figure 1A (scale bar = 100 μm). (4) The ROI file was then saved and (5) stored in a file (ROI Manager) in the computer hard disk. (6) The digitalized area was submitted to the plug-in “color deconvolution” using the built-in vector HDAB, where the staining of hematoxylin and diaminobenzidine (DAB) was separated into 3 different panels with hematoxylin (Figure 1B), with DAB only image (Figure 1C) and background. (7) Panel 2 was selected and (8) the ROI file was overlaid on the image (Figure 1D). (9) From this image the software calculated the area in mm², the mean and the median intensity of DAB, ranging from 0 (black) to 255 (total white). (10) The final DAB intensity was calculated according to the formula f = 255 − i, where f = final DAB intensity, i = mean DAB intensity obtained from the software; i ranges from 0 (zero = deep brown, highest expression), to 255 (total white). When multiple pictures were taken from the same slide, the mean D-HSCORE was calculated on each. A maximal of 3 pictures from the main representative areas was used; e.g., slide #1 yielded 3 pictures. Each picture had their subtotal f = 54.2; 47.47; 50.13, in this case the final f = 50.63.

Figure 1.

Representative pictures from an endometrial gland positively stained for β3 subunit integrin and selected as a ROI. (A) Color deconvolution plug-in built-in vector applied over the picture, separating the hematoxylin (B) and DAB (C) staining from the original picture. Overlay of the saved ROI file over the DAB staining for measuring the DAB intensity (D). Scale bar 100 μm.

Ethics, Sample Size Calculation and Statistical Analysis

All endometrial biopsies were obtained under an approved study consent submitted through the IRB at Greenville Hospital System. The use of these sections was approved by the ethics committee of Hospital de Clínicas de Porto Alegre under the number 11-0651. Sample size was calculated according to the literature²⁸ using the following parameters: an alpha error = 0.01, power = 0.95, and an estimated correlation coefficient (r) of 0.8. These figures yielded a sample size of at least 18 cases. The strength of the correlation coefficient was considered according to the literature²⁹; an r of 0 = no correlation, +0.2 = weak, +0.5 = moderate, +0.8 = strong, and 1 = perfect. Statistical analysis was performed using Spearman’s rank for correlation, Mann-Whitney test to compare the expression of β3 integrin subunit (final DAB intensity) in glandular endometrium, and a ROC curve to identify the cut-off between positive and negative in D-HSCORE. Intervariability and intravariability of methods using HSCORE was assessed with the Bland-Altman method. To assess the intraobserver and interobserver variability of D-HSCORE, analyses were performed using the same digital pictures and saved ROI files described above. D-HSCORE intraobserver variability was performed by one of the researchers (R.F.S.), and interobserver variability was performed between two researchers (D.G.F. and R.F.S.). GraphPad Prism version 5 for Macintosh (GraphPad Software, Inc., San Diego, California, U.S.A.) was used for statistical analysis.

Results

One hundred slides, 50 positive cases (HSCORE > 0.7) and 50 negative cases (HSCORE ≤ 0.7), previously scored by an expert were used for these comparisons. A pilot study with 33 slides revealed that correlation between the whole tissue sample and 3 major representative areas of the same slide had a correlation coefficient of 97.9% (95% CI 95.7–98.9), which was highly significant (p < 0.0001), (data not shown). In addition, the average area of the sample analyzed in these 33 slides was 221 mm². In this study the average of number of pictures analyzed per sample was 3.

HSCORE Performed by Naive Researcher and Expert (Intervariability and Intravariability)

The intraobserver variability of naive researcher had a bias (the difference between the means) of 0.008 and a standard deviation (SD) of 0.3 (95% limits of agreement −0.7 to 0.7) using HSCORE. However the intervariability between naive and expert researcher yielded a bias of 0.9 and a SD = 0.8 (95% limits of agreement −0.7 to 2.5), as shown in Figures 2A and 2B. The inexperienced researcher assigned a higher HSCORE (all 6 negative slides were scored as positive, 1.05 or higher) compared to the expert.

Figure 2.

The graphs depict the intraobserver (A) and interobserver (B) variability (n = 12), according to the Bland-Altman test, using HSCORE as the method that was analyzed (range of the HSCORE scale: 0–4). Note the low intraobserver variation (bias = 0.008; 95% limit of agreement −0.7 to 0.7; shown by dotted lines) in the y axis (A) and the wide variation in the interobserver analysis (bias = 0.9; 95% limit of agreement −0.7 to 2.5; shown by dotted lines) in the y axis (B). Spearman r correlation (C) between manual HSCORE and D-HSCORE (numbers in brackets are 95% CI). ROC analysis showing the area under the curve as 0.9888; p < 0.0001; 95% CI 0.97–1) (D). Expression of β3 subunit based on DAB intensity in negative and positive cases using D-HSCORE (E) and traditional HSCORE (F).

Correlation Between HSCORE and D-HSCORE and ROC Curve

The correlation between D-HSCORE and HSCORE was highly significant (p < 0.0001) and had an r_s = 0.86 (95% CI 0.79–0.90), as shown in Figure 2C. The ROC analysis yielded an area under the curve of 98.8% (95% CI 97–1), with a p < 0.0001 (Figure 2D). A cut-off of 37.2 has a sensitivity of 0.96 and a specificity of 0.98, yielding positive and negative likelihood ratios of 48 and 0.04, respectively.

Comparison between findings of the expert using HSCORE and the naive using D-HSCORE are very similar and are presented in Figures 2E and 2F. In 50 cases that were called “negative” based on the cutoff of ≤ 0.7, 49 were determined to be negative by the naive reader using ImageJ. Likewise, in the 50 cases that were called “positive” based on the same cut-off, 48 were determined to be positive using ImageJ. Therefore, using the 37.2 cut-off in ImageJ, only 3 samples were discordant from the HSCORE.

D-HSCORE Performed by Naive Researcher (Intervariability and Intravariability)

Intravariability and intervariability using ImageJ was 0% (95% CI 0–0.03) as expected since the same slides and ROI files were used (data not shown).

Discussion

Immunohistochemical staining is a commonly used method for assessment of protein expression in situ but is highly subjective and prone to variation based on both experience and training and possibly due to unconscious subjectivity. This study was carried out to compare the semiquantitative HSCORE assessment with a truly quantitative assessment of staining results using ImageJ: D-HSCORE. The HSCORE has been reported to have a low intraobserver variation.²² However, this result is based on correlation analysis instead of the Bland-Altman analysis, which is more appropriate.³⁰ Recent evaluation of HSCORE assessment of the β3 integrin subunit in repeat biopsies on the same woman demonstrated that HSCORE results tended to be reproducible with regards to being positive or negative.³¹ This low intraobserver variability can be seen in Figure 2A: the naive researcher gave almost identical scores in both assessments. Nevertheless, compared to the experienced reader, the inexperienced researcher, using the traditional HSCORE methods, was unable to correctly assign samples 50% of the time due to an overestimate of background staining. It is noteworthy that the naive researcher gave to all samples an HSCORE ≥ 1.05, which is above the previously defined cut-off for a negative score.¹⁰ The interobserver 95% CI limits of agreement reflect this wide variation, from −0.7 to 2.5. Thus, the naive researcher may be −0.7 below or 2.5 above the experienced researcher. This variation is more evident when the comparison of the mean values in both groups is examined. The naive researcher perceived much significantly higher median levels (1.7 [1.1–2.05]) as compared to the experienced researcher (0 [0–0.5]) (Mann-Whitney test p = 0.004, figures are given as (median [range]).

As part of the comparison between traditional HSCORE and D-HSCORE, we were able to show that ImageJ on digitalized images from previously stained and scored sections gave highly comparable results to the experienced researcher. The correlation coefficient between HSCORE and D-HSCORE, r_s = 0.86, is considered strong according to the literature.²⁹ In contrast to a low interobserver correlation when a naive reader used the HSCORE methods, the interobserver agreement was perfect using the D-HSCORE methodology. To our knowledge this is the first published report regarding the use of ImageJ to analyze endometrial β3 integrin subunit immunohistochemical results. Given its ease of use and reduced subjectivity, the cut-off presented here for positive and negative β3 integrin subunit staining will likely increase the consistency of results obtained among researchers.

A cut-off of ≤ 0.7 was developed specifically for the use of HSCORE in predicting the presence of endometriosis⁶ with a high specificity (91%) but lower sensitivity (38%). This reflects the fact that not all women with endometriosis lack integrin expression. As a result, this topic has become controversial since the usefulness of this biomarker for identification of endometriosis and infertility has not been replicated by all researchers.⁶ As a bio-marker of endometrial receptivity, however, β3 has recently been shown to predict IVF failure and therefore may detect those women with endometriosis who have implantation deficits.³¹ Methods to minimize variation and subjectivity will therefore be important going forward, including the use of image analysis software such as ImageJ. We report that the equivalent cut-off of < 0.7 in HSCORE is 37.2 using D-HSCORE, though further studies are needed to apply this cut-off to real-life situations such as endometriosis or fertility outcomes.

One limitation of this methodology is that it has 10 steps to get the final analysis, while the traditional HSCORE assessment has just 1. Each slide must be digitalized and each gland should be manually identified. One possible use for this software might be for training purposes or as a quality control technique for immunohistochemistry results. This is an issue that needs further research.

A separate limitation of this study is our choice of a gold standard. We used slides previously scored with traditional HSCORE by one individual instead of a separate technique such as western blot or real-time polymerase chain reaction from extracted glands. Ongoing studies using such techniques are currently being studied by our group.

In conclusion, this study found that D-HSCORE performed by an inexperienced researcher was comparable to the HSCORE performed by an expert. Further, due to overestimation of DAB staining, none of the negative samples were correctly identified by the naive reader in this blinded comparison. However, that same individual, using ImageJ, was able to establish a high correlation with HSCORE to the experienced researcher. The methodology presented using ImageJ and color deconvolution may be used by anyone and is a more reliable and less subjective method for quantifying β3 integrin subunit in endometrial glands. These methods can be applied to any immunohistochemical result that uses DAB, without the need of expert training in the use of the subjective HSCORE methods.