Abstract
The Japan Pharmaceutical Manufacturers Association (JPMA) has instituted a task force (TF) for the “development of image analysis technology for histopathological changes” as part of the collaboration for realizing cutting-edge drug development since 2016. In recent years, there has been progress in the digital pathology technology; however, few applications in nonclinical drug development studies have been observed. Therefore, TF performed a questionnaire survey to investigate the current status, needs, possibility, and development of image analysis. The subjects were 35 member companies of the JPMA. The questionnaire was set to assess the efficacy and/or safety of researchers engaged in pathological evaluations for each company. The questions focused on the experiences, implementation, and issues regarding histopathological examinations; the need for image analysis software; and future views. Valid responses were obtained from 26 companies. Most companies assumed that the beneficial aspect of image analysis is to gain objectivity and persuasiveness; however, challenges in the analysis conditions with regard to accuracy and without subjectivity persist. Additionally, there seems to be a need for image analysis software with advanced digital pathology technology, with most companies believing that, in the future, pathological evaluations will be partly performed by computers. In conclusion, in this questionnaire survey, TF extracted the current status of image analysis in nonclinical studies performed by pharmaceutical companies and collected opinions on future prospects regarding the development of image analysis software with advanced digital pathology technology.
Keywords: image analysis, questionnaire, pharmaceutical company, digital pathology technology
We set up a task force (TF) as one of the collaborative activities among industries, toward realizing cutting-edge drug development at the Research and Development (R&D) Committee of the Japan Pharmaceutical Manufacturers Association (JPMA). This TF has been working on developing image analysis technology for histopathological changes since 2016.
Pathological evaluations using glass slides are a qualitative or semiquantitative morphological observation performed by pathologists using hematoxylin and eosin (H&E) staining and an optical microscope; however, for several decades, there have been attempts to quantify lesions by morphometry using image analysis software1. In recent years, there has been rapid progress in morphological measurement technology owing to the dissemination of digital technology, such as whole slide imaging, for digitizing the morphological information of glass slides. Furthermore, it is possible to utilize commercially available digital pathology applications (Visiopharma, Definiens’ Tissue Studio, and Indica Labs’ HALO) or open-source software programs such as QuPath or Image J2. In clinics, computerized pathology diagnosis support software has been developed and used for some types of cancer3, 4. In addition, artificial intelligence (AI), which has rapidly advanced in recent years, has excellent image recognition (e.g., human faces and objects). Accordingly, these applications can be employed in image recognition and quantification of histopathological changes in the evaluation of nonclinical efficacy or safety studies during drug development and are expected to provide useful information.
Meanwhile, there has been no tangible implementation of a computerized automated image analysis software for a wide variety of histopathological changes observed in nonclinical drug development studies, with the exception of a few studies5, 6 with a detailed understanding of this situation still unknown. Therefore, the authors conducted a questionnaire survey on the R&D Committee members of the JPMA to understand the issues, possibilities, and the necessity for the development of image analysis of histopathological changes in nonclinical studies.
The questionnaire was prepared by a TF member, and the questionnaire survey was performed among 35 member companies belonging to the R&D Committee of the JPMA. There were differences in the purpose and research contents in nonclinical efficacy or safety studies; therefore, the questionnaire was established for researchers/technicians who were engaged in efficacy and/or safety pathological evaluations. One response was obtained from one company, including answers on efficacy or safety or both. Questionnaires including experiences, implementations, histopathological evaluations, issues, development, and future visions for image analysis software were prepared. The questionnaires consisted of questions numbered Q1 to Q26. Furthermore, the option “Other” with a free description was collected in case of some questions. The questionnaires and the results are shown in charts Q1–Q26. Valid responses were received from 26 out of 35 member companies. Six companies were not able to respond, and three companies did not answer.
The questionnaires were distributed on February 13, 2018, through the office of the JPMA, and collected by March 13, 2018. The answered questionnaires were anonymized through the JPMA secretariat and returned to the TF, and then the TF gathered and analyzed them.
The TF questioned, “Which industry does your company belong to?” Almost all valid responses were from the Japanese pharmaceutical companies (Table 1). Furthermore, among the companies that provided valid responses, responses were obtained from almost all efficacy and safety research divisions.
Table 1. Companies that Responded on the Development of Image Analysis for Histopathological Changes in Nonclinical Studiesa.
I. Current status of experiences and implementations on image analysis (Q1–Q11)
Experience (Q1, Q2):
Q1.
Do you conduct image analysis using histopathological glass slides?
Q2.
What is the frequency of image analysis using histopathological glass slides?
All efficacy research divisions that provided responses had experience in performing image analysis (Q1). Overall, morphometry was performed more frequently in efficacy research divisions than in safety research divisions (Q2). Image analysis was performed several times a year in both divisions. More than two-thirds of the efficacy research divisions or half of the safety research divisions conducted image analysis several times a year or routinely.
Target tissue and endpoint (Q3, Q4):
Q3.
Which organs or tissues are analyzed by the software? (Multiple answers allowed: M/A)
Q4.
What is quantified by image analysis? (M/A)
Notably, the tissue targets for image analysis were the liver, kidney, lung, brain, heart, intestine, spleen, and pancreas (in descending order) (Q3). The endpoints, quantified by image analysis, were the area, the number of cells or any objects, length/thickness, and staining intensity in immunohistochemistry (in descending order) (Q4). The image analysis of the spleen, kidney, brain, and intestinal tract tended to be performed more often in efficacy studies than in safety studies. All these items tended to be more frequent in efficacy studies than in safety studies. In particular, staining intensity in immunohistochemistry was mostly performed in efficacy studies.
Software (Q5):
Q5.
What kind of image analysis software do you use? (M/A)
In descending order, the types of image analysis software used included Image J (free software), Image-Pro (Media Cybernetics), Leica Biosystems software, Definiens Tissue Studio (Definiens AG), and HALO (Indica Labs). In terms of free description, WinROOF (Mitani Corporation) was highly mentioned in both the efficacy and safety research divisions (six and three companies, respectively).
Staining method (Q6):
Q6.
What is the staining method used in image analysis? (M/A)
In image analysis, H&E staining, immunohistochemistry, and special staining methods were equally utilized. Furthermore, all staining methods were more frequently adopted in efficacy studies than in safety studies. A description of special staining in descending order is presented as follows: Masson Trichrome (six companies) and Sirius Red (six companies) in case of efficacy studies, and Sirius Red (four companies) and Masson Trichrome (three companies) in safety studies. Staining methods for collagen fibers were well adopted in both divisions.
Animal species (Q7):
Q7.
What animal species do you use for image analysis? (M/A) This question was asked only for safety.
Q7 was requested only from safety research divisions. The animal species subjected to image analysis in safety studies were rats, mice, dogs, and monkeys in descending order. Rats are the most commonly used experimental animals in nonclinical safety studies.
Target lesion (Q8):
Q8.
What kind of lesion is quantified by image analysis? (M/A) This question was asked only for safety.
Q8 was requested only from the safety research divisions. The most image analyzed lesion was fibrosis, followed by an increase/decrease in cells, atrophy/hypertrophy, degeneration/necrosis, and tumor/hyperplasia. The frequency of measuring inflammation morphology was relatively low.
Apparatus (Q9, Q10):
Q9.
How do you obtain the digital images of histopathological glass slides?
Q10.
To those who answered “virtual slide scanner” in Q9, which company’s digital slide scanners do you use? (M/A)
Digital imaging performed with digital cameras was the most used method, followed by digital imaging with virtual slide scanners (Q9). Among companies that used virtual slide scanners (i.e., whole slide imaging [WSI]) in Q9, Leica Biosystems or Hamamatsu Photonics slide scanners were widely employed models in Japanese pharmaceutical companies (Q10). Most companies use either of these two digitizing methods, employed more frequently in efficacy research divisions than in safety research divisions. Reportedly, WSI represents a paradigm shift in pathology, playing a pivotal role in image analysis7.
Image analysis in drug application study (Q11):
Q11.
Which type of study did you conduct for image analysis?
Image analysis for application studies was performed more frequently in efficacy research divisions than in safety research divisions. Although seven companies responded with “for application,” they comprised less than one-fourth of the nonapplication studies. This is partly due to the fact that key issues that require consensus for the adoption of digital pathology and image analysis have not been established. Furthermore, guidelines covering the use of digital pathology or image analysis in nonclinical studies, as in clinical trials, are lacking2.
II. Histopathological examinations, and issues and merits on image analysis (Q12–Q18)
Histopathological examination and peer review (Q12, Q13, Q14):
Q12.
Do qualified pathologists conduct histopathological examinations in safety studies at your company? This question was asked only for safety.
Q13.
(For Efficacy) Does bias pose a problem when conducting image analysis? (For Safety) Have you ever experienced bias among pathologists or studies regarding findings, terms, and grades obtained by pathologists?
Q14.
What measures need to be employed to resolve bias among pathologists? (M/A) This question was asked only for safety.
The histopathological examination of nonclinical safety studies is routinely conducted by qualified pathologists in almost all companies (Q12). However, approximately two-thirds of the safety researchers opined that bias was present in findings, terminologies, and grading criteria of the qualified pathologists. Similarly, among efficacy researchers, two-thirds of the companies assumed the presence of bias when conducting image analysis (Q13). Pathology peer review is commonly performed to improve the accuracy and quality of pathological diagnosis in nonclinical safety studies8. Therefore, the TF questioned the safety researchers regarding peer review. The results demonstrated that two-thirds of companies utilized pathologists from within the same company (Q14). The conventional, subjective, histopathological scores by experienced pathologists are the gold standard9. Even for each qualified pathologist, it was a formidable dilemma to be consistent in his/her findings among studies.
Bias (Q15, Q16):
Q15.
Do you think that the bias in pathological findings will be eliminated by image analysis and quantification? This question was asked only for safety.
Q16.
How do you reduce bias during image analysis? (M/A)
Most safety researchers provided the reply “partly eliminated” (Q15) and consider that the quantification of histopathological changes will reduce bias. Quantifying morphological changes is considered to lead to improvements in objectivity, reliability, and robustness5.
The caveat of reducing bias, when conducting morphometric analysis, is that the alignment of the staining conditions for specimens is the most common answer for both efficacy and safety research divisions, followed by skilled persons/same persons performing blind analysis (Q16). Bias can affect scientific investigations and even damage a study’s validity depending upon its degree9, 10. To avoid preanalytical variables, the preparation of a suitable specimen to be used for the measurement of tissue morphology should be emphasized. Furthermore, as well known, morphometric analysis requires trained and skilled histotechnologists to produce pristine specimens11.
Merit of image analysis (Q17):
Q17.
What is the merit of quantifying lesions by image analysis? (M/A)
The efficacy and safety researchers of several companies believed that the advantage of quantifying lesions by morphometry was to obtain objectivity and persuasiveness. The term “objectivity” is the opposite of subjectivity. Objectivity is defined as a method, process, statement, or decision that is based on facts and is not influenced by opinions, perceptions, or emotions12. On the contrary, “persuasiveness” is the quality of being able to convince a person to take a course of action or believe a particular idea. Software is vital for digital pathological applications and plays an important role in providing objectivity and persuasiveness in the interpretation of tissue structures and analysis.
Issues in conducting image analysis (Q18):
Q18.
What is the rationale for difficulties in image analysis?
The issues were subdivided into four categories: specimen preparation, bias, software, and evaluation. Interestingly, there were no opinions regarding the captured digital images. WSI is widely utilized in research. Therefore, it is important to collect data that demonstrate a valid equivalent performance of digital imaging against glass slides13.
III. Future visions and development of image analysis software (Q19–Q26)
Future visions (Q19, Q20):
Q19.
(For Efficacy) Do you want to use image analysis software that can automatically analyze lesions quantitatively for efficacy evaluation? (For Safety) Do you want to use image analysis software that has the same morphological recognition capability and ability to accurately obtain findings as a pathologist?
Q20.
What kind of purpose do you have for software as described in Q19? (M/A)
If an image analysis software that could automatically quantify or accurately detect lesions was developed, most efficacy researchers positively agree to use this software (Q19). Furthermore, most safety researchers agree to use software capable of morphometric recognition similar to the pathologists (Q19). The comparable number of the answer was collected in each purpose: add objectivity to findings, quickly conduct image analysis, process a large amount of data in efficacy division, and support use in grading for safety division (Q20).
Useful target tissue and lesion analyzed automatically (Q21, Q22):
Q21.
What kind of organ lesion would be useful to automatically analyze and quantify? (M/A)
Q22.
What kind of lesion would be useful to automatically analyze and quantify? (M/A)
The useful organs or tissues analyzed and quantified automatically by the software were the liver, kidney, lung, spleen (including the thymus and lymph node), brain, heart, and intestine (in descending order) (Q21). Both researchers considered that it would be most beneficial to possess the capability to automatically analyze the liver. Comparing the results of Q3 and Q21 (current status and future, respectively), the safety researchers tended to select all choices compared with Q3.
The useful lesions analyzed and quantified automatically by the software were a decrease/increase in cells, atrophy/hypertrophy, fibrosis, inflammation, degeneration/necrosis, tumor/hyperplasia, and regeneration (in descending order) (Q22). Researchers in both groups opined that it would be highly advantageous to be able to automatically analyze the changes in the number of cells, and it was considered useful if automatic measurement could be performed employing a computer. Comparing the results of Q8 and Q22 (current status and future, respectively), a tendency to select all choices was observed compared to Q8 (not asked for efficacy).
Software as a support tool (Q23, Q24, Q25):
Q23.
Do you think that automatic image analysis of pathological tissue slides can be a supportive tool for efficacy evaluation? This question was asked only for efficacy.
Q24.
(For Efficacy) What do you think about the possibility of future efficacy evaluations being replaced by computers with the advancement of morphological recognition technology? (For Safety) What do you think about the possibility that, in the future, the ability to recognize morphology will utilize computers with the advancement of morphological recognition technology?
Q25.
Regarding the previous question, tell us why you think so. Freely describe your reason.
All companies opined that automated image analysis software could be used as a support tool for efficacy evaluation (Q23). Interestingly, most of the responding pharmaceutical companies believed that morphometric recognition will be partly replaced by computers (Q24). Q25 indicated the rationale behind the Q24 answers. Approximately three reasons were indicated for the answers: affirmative, mutual, and negative. In case of the most selected, “partly replaced,” the respondents had a mixture of the above three positions. For “replaced,” the answers were apparently affirmative and included future forecasts. In contrast, for “not replaced,” the respondents replied that morphological recognition will not be replaced by computers owing to the differences in the roles between humans and computers. Automated image analysis has already been developed and partly implemented in human surgical pathology3, 4. Computer technology with AI and machine learning will increasingly become integrated into the evaluation workflow of nonclinical studies when objectivity and persuasiveness are established.
Development of software with AI technology (Q26):
Q26.
If pathology diagnosis software with an advanced morphological recognition function that features AI technology was developed, what is your highest expectation?
If image analysis software with highly advanced AI technology are developed, efficacy researchers primarily expect an increased data processing capacity followed by labor-saving, accuracy, reproducibility, and innovativeness. The safety researchers selected objectivity the most, with comparable numbers of other items, such as process large amount of data, labor-saving, accuracy, and ability to detect lesions.
Currently, the primary diagnosis by WSI has already been approved overseas14, 15, and the next step is to implement software with a highly advanced image analysis algorithm in nonclinical studies on both efficacy and safety. Nonetheless, the nonclinical safety study in compliance with GLP requires the development of regulation, and the bars seem to be difficult at this time.
In conclusion, in this questionnaire survey, the TF extracted the current situation of image analysis in nonclinical studies performed by pharmaceutical companies and collected opinions on future prospects regarding the development of image analysis software with advanced digital pathology technology.
Disclosure of Potential Conflicts of Interest
The authors declare that they have no conflicts of interests.
Acknowledgments
In conducting this questionnaire survey, TF would like to express great appreciation to the JPMA R&D subcommittee for their valuable advice and contribution to the distribution and collection of questionnaires.
References
- 1.Hamilton PW, Bankhead P, Wang Y, Hutchinson R, Kieran D, McArt DG, James J, and Salto-Tellez M. Digital pathology and image analysis in tissue biomarker research. Methods. 70: 59–73. 2014. [DOI] [PubMed] [Google Scholar]
- 2.Pell R, Oien K, Robinson M, Pitman H, Rajpoot N, Rittscher J, Snead D, Verrill C. UK National Cancer Research Institute (NCRI) Cellular-Molecular Pathology (CM-Path) quality assurance working group. The use of digital pathology and image analysis in clinical trials. J Pathol Clin Res. 5: 81–90. 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Yoshida H, Yamashita Y, Shimazu T, Cosatto E, Kiyuna T, Taniguchi H, Sekine S, and Ochiai A. Automated histological classification of whole slide images of colorectal biopsy specimens. Oncotarget. 8: 90719–90729. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Yoshida H, Shimazu T, Kiyuna T, Marugame A, Yamashita Y, Cosatto E, Taniguchi H, Sekine S, and Ochiai A. Automated histological classification of whole-slide images of gastric biopsy specimens. Gastric Cancer. 21: 249–257. 2018. [DOI] [PubMed] [Google Scholar]
- 5.Horai Y, Kakimoto T, Takemoto K, and Tanaka M. Quantitative analysis of histopathological findings using image processing software. J Toxicol Pathol. 30: 351–358. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Asaoka Y, Togashi Y, Mutsuga M, Imura N, Miyoshi T, and Miyamoto Y. Histopathological image analysis of chemical-induced hepatocellular hypertrophy in mice. Exp Toxicol Pathol. 68: 233–239. 2016. [DOI] [PubMed] [Google Scholar]
- 7.Zarella MD, Bowman D, Aeffner F, Farahani N, Xthona A, Absar SF, Parwani A, Bui M, and Hartman DJ. A practical guide to whole slide imaging: A white paper from the digital pathology association. Arch Pathol Lab Med. 143: 222–234. 2019. [DOI] [PubMed] [Google Scholar]
- 8.Morton D, Sellers RS, Barale-Thomas E, Bolon B, George C, Hardisty JF, Irizarry A, McKay JS, Odin M, and Teranishi M. Recommendations for pathology peer review. Toxicol Pathol. 38: 1118–1127. 2010. [DOI] [PubMed] [Google Scholar]
- 9.Aeffner F, Wilson K, Martin NT, Black JC, Hendriks CLL, Bolon B, Rudmann DG, Gianani R, Koegler SR, Krueger J, and Young GD. The gold standard paradox in digital image analysis: manual versus automated scoring as ground truth. Arch Pathol Lab Med. 141: 1267–1275. 2017. [DOI] [PubMed] [Google Scholar]
- 10.Gibson-Corley KN, Olivier AK, and Meyerholz DK. Principles for valid histopathologic scoring in research. Vet Pathol. 50: 1007–1015. 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Brown DL. Bias in image analysis and its solution: unbiased stereology. J Toxicol Pathol. 30: 183–191. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Tadrous PJ. On the concept of objectivity in digital image analysis in pathology. Pathology. 42: 207–211. 2010. [DOI] [PubMed] [Google Scholar]
- 13.Snead DR, Tsang YW, Meskiri A, Kimani PK, Crossman R, Rajpoot NM, Blessing E, Chen K, Gopalakrishnan K, Matthews P, Momtahan N, Read-Jones S, Sah S, Simmons E, Sinha B, Suortamo S, Yeo Y, El Daly H, and Cree IA. Validation of digital pathology imaging for primary histopathological diagnosis. Histopathology. 68: 1063–1072. 2016. [DOI] [PubMed] [Google Scholar]
- 14.Abels E, and Pantanowitz L. Current state of the regulatory trajectory for the whole slide imaging devices in the USA. J Pathol Inform. 8: 23 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Boyce BF. An update on the validation of whole slide imaging systems following FDA approval of a system for a routine pathology diagnostic service in the United States. Biotech Histochem. 92: 381–389. 2017. [DOI] [PubMed] [Google Scholar]