Interview about Prostate tissue histology with computational methods

Prostate tissue histology with computational methods

Updated in April 2022 by:

Fanny Dobrenova and Elisa Opriessnig

Prostate tissue histology is a growing field of research where the need for new automated image analysis methods is rapidly increasing. State-of-the-art image segmentation methods assist researchers in detecting pathological prostate conditions, grading/staging cancer and assessing the effectiveness of new therapeutic agents based on changes in prostate tissue morphology. 

In this article we offer an extensive overview of existing segmentation methods in prostate histology research. We talked to experts in prostate histology research at the Medical University of Vienna to provide you valuable insights into how automated prostatic tissue analysis is put into practice.        

Download full interview in the PDF version

No email required.

*The following PDF interview document contains screenshots of the IKOSA platform from 2020. The actual interface of IKOSA may look different due to numerous enhancements to the platform.

Normal prostate tissue histology

To better understand pathological changes characteristic of prostate diseases, we need to first take a look at the anatomy of healthy prostate tissue. As a part of the male reproductive system, the prostate is a gland that plays an important role in the generation of seminal fluid. The prostate is constituted of three histological zones: peripheral, transition and central zone. Structures adjacent to the prostate are the bladder, the prostatic urethra, the prostatic ducts and the seminal vesicles.   

Prostate gland tissue is composed of four components: nuclei, lumen, stroma and cytoplasm. Layers of epithelial cells constitute the gland boundaries and are involved in the generation of seminal fluid. Those cells are columnar in shape and have round nuclei positioned near the cell base. Lumen objects are described as empty white spaces within the glandular structure. The prostate basically is a set of tubulo-alveolar glands with lumina lined by epithelium of variable height (Singh et al. 2017). 

The glandular tissue of the prostate includes secretory cells on the glandular facing side and basal cells on the basal side.  The epithelial cells line a central lumen, which is filled with fluid produced by the epithelial cells (Denmeade & Isaacs 2003).

A healthy prostate gland does not have a fixed size or shape: it can be smaller or larger, oval, round or branchy. Benign glands are characterized by large lumina and epithelial cells with prominent nuclei. The nuclei of benign prostatic gland tissue are uniformly dark or uniformly light throughout the areas and don’t show prominent  nucleoli (Nguyen et al. 2012b).           

In the normal prostate, prostatic stroma components vary in each zone. Stromal components include myofibroblasts, fibroblasts, collagen fibers and smooth muscle cells (Zhang et al. 2003). However, there are local differences in stroma morphology and function (e.g., gene expression) in the different zones of the prostate. Such differences may explain why cancer originates more commonly in the peripheral zone of the prostate and not in the transition zone where benign prostatic hyperplasia (BPH) usually develops. The transition zone accounts for only 10 % of prostate glandular tissue (Hägglöf & Bergh 2012).

prostate tissue histology normal tissue

Healthy prostate tissue histology slide

In cases of cancer, changes to lumen properties occur in the gland, which can be used as a feature in digital image analysis. However, prostate cancer can develop in benign basal and luminal stem cells. The aberrant proliferation of basal cells in the prostate ranges from hyperplasia to carcinoma (but carcinoma from basal cells of the prostate is rare).

Focusing on the tissue architecture in prostate cancer histology

Several pathological conditions like prostatitis, prostatic hyperplasia, nodular hyperplasia, prostatic intraepithelial neoplasia and prostatic adenocarcinoma can occur in prostate tissue. All those conditions are characterized by specific changes in tissue morphology. We take a look at how prostate glandular structures affected appear during histological analysis.  


Gleason Score



6 (3+3)

Individual, discrete, well-formed glands or uniform glands


7 (3+4)

Well-formed glands with small component of poorly formed, fused, cribriform or glomeruloid glands (more stroma between glands)


7 (4+3)

Predominantly poorly formed, fused, cribriform or glomeruloid glands with small component of well-formed glands (distinctly infiltrative margins)


8 (4+4, 3+5, 5+3)

Predominantly poorly formed, fused, cribriform or glomeruloid glands with small component of well-formed glands (distinctly infiltrative margins)


9, 10 (4+5, 5+4, 5+5)

Lack of gland formation (+/-necrosis) with or without poorly formed, fused or cribriform glands + sheets of cells

Table 1: The characteristics of prostate tissue with regards to Gleason score (adapted from: Pudasaini & Subedi 2019)  

Pathologists use the presence of atypical gland patterns as hallmarks through which they can distinguish cancerous regions from benign ones. The most common system pathologists use to grade prostate cancer is the Gleason score (Gleason, 1966). 

The assigned cancer grade shows to what extent the appearance of cancer cells deviates from that of normal healthy cells. To assign a Gleason score or a Grade group to a given sample, pathologists look at biopsies taken from the prostate and grade each sample on a scale from 3 to 5. The two dominating Gleason grades are added together to calculate the overall Gleason score, which ranges from 6 to 10 (Chen & Zhou 2016). 

To detect prostate cancer with the help of digital histological images the characteristics of certain nuclear, cytoplasmic and intraluminal features of the glandular region need to be assessed.

Prostate Cancer Stages

Spread of cancer from the prostate to adjacent organs and tissues. Image created with BioRender. 

Changes to nuclear features

Compared to normal prostate epithelial cells, the nuclei in cancerous prostate cancer cells are colored  light blue and have prominent nucleoli appearing as small dark spots within the nucleus.   

Alterations of nuclear features such as size, shape and texture can be observed in cancerous prostate tissue. Nuclear enlargement and the presence of prominent nucleoli are characteristic of cancer affected specimens. 

Another feature prominent in prostatic tissue affected by cancer is nucleus shape irregularities or nuclear dysmorphia. These changes are caused by alterations in the nuclear lamina resulting in lobes and herniations.

Textural changes in the structure of nuclei are mostly due to alterations on the DNA level, including  gene fusions, mutations, copy number variations and translocations. These can be detected with specific staining methods such as fluorescence in situ hybridization during histological analysis (Carleton et al. 2018).       

Further, glands in the cancer-affected region display a smaller nuclei count on the boundary of the gland as compared to non-cancerous regions. Cancerous glands tend to have only one nuclear layer on the boundary, while a normal gland contains multiple layers. Those nuclei are also characterized by a lighter blue color than those in healthy glands (Nguyen et al. 2012).    

Also, cancerous prostate tissues might often display a cribriform pattern, where a number of glands are fused into one which results in the formation of nuclei clusters (Singh et al. 2017). The term "cribriform" refers within this context to a neoplastic epithelial proliferation in the form of large nests perforated by different-sized quite rounded spaces (Branca et al. 2017).  

This is important to note, because the presence of a cribriform growth pattern in radical prostatectomy specimens has been previously associated with distant metastasis and disease-specific death from prostate cancer in patients with Gleason score 7 or higher (Kweldam et al. 2018). Consequently, the presence of a cribriform pattern is now recognized as a clinically important, independent adverse prognostic indicator of prostate cancer (Branca et al. 2017). 

prostate tissue histology cancerous tissue

Cancerous prostate tissue histology slide. Nuclei are prominently marked in dark blue.

Changes to lumen properties      

Lumen objects in atypical glands tend to be smaller in size and more circular than in normal glands. Blue stained mucin invading lumen objects might result in a distinct coloring on histology slides (Nguyen et al. 2012b). 

Existing studies suggest that in cases of higher Gleason grade cancer a lesser density of lumen objects and a decreasing volume of lumen space can be observed in the affected tissue (Chatterjee et al. 2015; McGarry et al. 2018).    

Changes to prostatic stroma

Cancerous prostatic tissue is composed of malignant epithelial cells and supportive stroma whose changes are important for the development of the tumor (Krušlin et al. 2015).  Recent research suggests that a decreasing volume of stroma can be observed in cancer-affected prostate glands while the number of tumor cells increases (Chatterjee et al. 2015).

Cancerous stroma consists of fibroblasts, myofibroblasts, endothelial cells and immune cells, but fibroblasts and myofibroblasts are the predominant cellular entities. They play a significant role in the synthesis, deposition and remodeling of the extracellular matrix and are in constant interaction with tumorous epithelial cells (Krušlin et al. 2015). 

With the help of various molecules of the extracellular matrix (ECM) a microenvironment suitable for cancer cell proliferation, movement and differentiation is created  promoting tumor growth. A complex interaction between cancer cells and various cells in the stroma plays a central role when it comes to the enhancement of tumor progression. This process is a key factor in stimulating angiogenesis and preserving cancer cell survival, proliferation and invasion (Krušlin et al. 2015).  

Download full interview in the PDF version

No email required.

*The following PDF interview document contains screenshots of the IKOSA platform from 2020. The actual interface of IKOSA may look different due to numerous enhancements to the platform.

How does the automated histologic analysis of prostate tissue work?

Segmentation methodologies involve detecting and separating objects and structures of interest in prostate tissue images. With the help of specialized software applications researchers are able to conduct an accurate segmentation of prostate tissue. These automated applications rely on state-of-the-art deep learning technology and facilitate a significantly faster and more efficient data collection than conventional manual methods.    

Various prostate segmentation models have been suggested in existing literature. Some of these methods rely on MR imaging data and are applied for tasks such as localizing prostate boundaries and zones, obtaining volume-related metrics and tracking disease progression. Prostate zone segmentation is used to determine cancer lesion invasion towards adjacent structures such as the urethra and the seminal vesicles (Litjens et al. 2014; Zhu et al. 2017).   

Other methods relying on histology slide data and automated histopathological image analysis allow researchers to obtain valuable quantitative information on the structural features of prostate tissue. Different deep learning techniques for epithelium segmentation, nucleus segmentation, stroma and gland segmentation and lumen objects segmentation have been discussed in literature (Nguyen et al. 2012; Carleton et al. 2018; Bulten et al. 2019). 

AI-backed methods also enable researchers to reliably classify specimens into the different stages of prostate cancer (Nguyen et al. 2012b). Yet, the varying shapes and sizes of prostatic glands often pose a major challenge to common segmentation techniques (Singh et al. 2017).      

However, with the help of advanced computational methods quantitative data related to prostatic tissue alterations can be collected. Such parameters include measures of count, size, shape and texture of tissue components as well as measures of the spatial information about the cellular microenvironment (Bhargava & Madabhushi 2016).  


Morphological structures


nuclei count, epithelial cells count, lumen objects count, stromal cells count


nuclei area, lumen area, epithelial cell area, stromal area


nucleus density, lumen density, epithelial cell density, stromal cell density


nucleus circularity, lumen objects circularity, epithelial cell circularity, stromal cell circularity


nucleus size, lumen object size, epithelial cell size, stromal cell size  


nuclear volume, lumen space volume, epithelium space volume, stroma space volume   

Table 2: Parameters used when assessing the morphological features of prostate gland components with computational methods 

Experts in prostate tissue histology share their experience with the IKOSA software

We contacted Prof. Johannes Schmid and Dr. Bernhard Hochreiter, researchers at the Institute of Vascular Biology and Thrombosis Research at the Medical University of Vienna, and asked them to share their experience with the IKOSA platform regarding the automated histologic analysis of prostate tissue. Here is what the research team reported about their recent study on prostate cancer with the help of advanced computational methods.    

prostate tissue histology staining

Fluorescence antibody staining of mouse prostate tissue: Nuclei(blue), IKK1 (green) and c-Myc(red). Source: Moser et al. (2021)

On the benefits of the IKOSA software

When asked about the benefits of the IKOSA software, Bernhard Hochreiter noted that he was particularly pleased with the many useful capabilities included in the IKOSA software. Especially, the option to view and process image files of different sizes including particularly large images had proven to be very helpful. Transforming images taken with different microscope modalities into an uniform size was no longer necessary. 

“I was pleasantly surprised that images of various sizes can be easily managed on the web platform”, Prof. Johannes Schmid explains. 

The researcher also agrees that the entire analysis workflow has been faster and smoother since they implemented the IKOSA software in their lab. Being able to work remotely and perform the analysis of histologic data on an online platform has been an invaluable asset, especially during the COVID-19 pandemic.

The uses of IKOSA in applied histopathological prostate research

The researchers report how the use of the IKOSA platform helped them conduct the large-scale research project “FFG-BRIDGE Precision Histology.” The dataset used for the project consisted of complex microscopy data acquired with different imaging modalities. Especially prostate tissue images taken with multichannel fluorescence microscopes constituted the larger part of the dataset. This required an image analysis tool which supported these image formats and was flexible enough to adjust the different channels.    

prostate tissue histology multichannel imaging

Multichannel image of histologically stained mouse prostate tissue

Yet, as Dr. Hochreiter explains, the biggest asset of the IKOSA platform turned out to be its AI-backed analysis capability. It has helped the team avoid many mistakes, which might have occurred during manual analysis tasks. Image analysis automation has proven invaluable during the segmentation of prostate cell nuclei.  

“The significant advantage is the support for image analysis using artificial intelligence and machine learning. The use of IKOSA can help avoid mistakes that could affect the research results,” shares Dr. Hochreiter.

The “FFG-BRIDGE Precision Histology” study

Due to the scarcity of human prostate tissue biopsy samples and ethical considerations using patient data a large body of pre-clinical research has been conducted on animal prostate tissue samples. Making use of the similarities in mammalian prostate anatomy, a significant number of articles on prostate histology relies on mouse tissue samples (Fagerland et al. 2020; Ding et al. 2021).      

Similarly, the research team at the Medical University of Vienna used mouse prostate tissue histological images acquired with different microscope modalities like brightfield and fluorescence microscopy. The project resulted in a publication in the Molecular Cancer Journal on the effects of inflammatory kinase IKKα complexes on inflammatory transcription in prostate cancer. 

The study involved analyzing the effects IKK enzymes on the expression of the oncogene c-Myc protein in epithelial prostate cells. To obtain this information cell nuclei have been stained using fluorescent dyes and automatically segmented with the help of image analysis software. Thus, reliable quantitative information on c-Myc protein expression was collected.

The authors propose a model on how IKK enzymes interact c-Myc within prostate cell nuclei. The article suggests a positive correlation between c-Myc and IKKα levels in mouse prostate epithelial cells. The article provides evidence that c-Myc phosphorylates IKKs and regulates gene expression in mouse prostate nuclei. This leads to increased transcriptional activity, higher proliferation and decreased apoptosis (Moser et al. 2021).      

Download full interview in the PDF version

No email required.

*The following PDF interview document contains screenshots of the IKOSA platform from 2020. The actual interface of IKOSA may look different due to numerous enhancements to the platform.


Bhargava R, Madabhushi A. (2016). Emerging themes in image informatics and molecular analysis for digital pathology. Annual review of biomedical engineering, 18, 387–412.    

Branca G, Ieni A, Barresi V, Tuccari G, Caruso RA. An Updated Review of Cribriform Carcinomas with Emphasis on Histopathological Diagnosis and Prognostic Significance. Oncol Rev. 2017;11(1):317. 

Bulten, W., Bándi, P., Hoven, J., Loo, R. V. D., Lotz, J., Weiss, N., ... & Litjens, G. (2019). Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard. Scientific reports, 9(1), 1-10. 

Carleton, N. M., Lee, G., Madabhushi, A., & Veltri, R. W. (2018). Advances in the computational and molecular understanding of the prostate cancer cell nucleus. Journal of cellular biochemistry, 119(9), 7127-7142. 

Chatterjee, A., Watson, G., Myint, E., Sved, P., McEntee, M., & Bourne, R. (2015). Changes in epithelium, stroma, and lumen space correlate more strongly with Gleason pattern and are stronger predictors of prostate ADC changes than cellularity metrics. Radiology, 277(3), 751-762.

Chen, N., & Zhou, Q. (2016). The evolving Gleason grading system. Chinese journal of cancer research = Chung-kuo yen cheng yen chiu, 28(1), 58–64. 

Denmeade SR, Isaacs JT. (2003). Cellular Organization of the Normal Prostate. In: Kufe DW, Pollock RE, Weichselbaum RR, et al., editors. Holland-Frei Cancer Medicine. 6th edition. Hamilton (ON): BC Decker; 2003.

Ding, Y., Lee, M., Gao, Y., Bu, P., Coarfa, C., Miles, B., ... & Ayala, G. (2021). Neuropeptide Y nerve paracrine regulation of prostate cancer oncogenesis and therapy resistance. The Prostate, 81(1), 58-71.  

Fagerland, S. M. T., Hill, D. K., van Wamel, A., de Lange Davies, C., & Kim, J. (2020). Ultrasound and magnetic resonance imaging for group stratification and treatment monitoring in the transgenic adenocarcinoma of the mouse prostate model. The Prostate, 80(2), 186-197.  

Gleason, HF. (1966). Classification of Prostatic Carcinoma. Cancer Chemother Rep 50, 125-128 

Hägglöf, C., & Bergh, A. (2012). The stroma-a key regulator in prostate function and malignancy. Cancers, 4(2), 531–548. 

Krušlin, B., Ulamec, M., & Tomas, D. (2015). Prostate cancer stroma: an important factor in cancer growth and progression. Bosnian journal of basic medical sciences, 15(2), 1–8. 

Kweldam CF, van der Kwast T, van Leenders GJ. On cribriform prostate cancer. Transl Androl Urol. 2018;7(1):145-154. 

Litjens, G., Toth, R., van de Ven, W., Hoeks, C., Kerkstra, S., van Ginneken, B., ... & Madabhushi, A. (2014). Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge. Medical image analysis, 18(2), 359-373.

McGarry, S. D., Hurrell, S. L., Iczkowski, K. A., Hall, W., Kaczmarowski, A. L., Banerjee, A., ... & LaViolette, P. S. (2018). Radio-pathomic maps of epithelium and lumen density predict the location of high-grade prostate cancer. International Journal of Radiation Oncology* Biology* Physics, 101(5), 1179-1187. 

Moser, B., Hochreiter, B., Basílio, J., Gleitsmann, V., Panhuber, A., Pardo-Garcia, A., ... & Schmid, J. A. (2021). The inflammatory kinase IKKα phosphorylates and stabilizes c-Myc and enhances its activity. Molecular cancer, 20(1), 1-17.  

Nguyen, K., Sarkar, A., & Jain, A. K. (2012). Structure and context in prostatic gland segmentation and classification. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 115-123). Springer, Berlin, Heidelberg.  

Nguyen, K., Sabata, B., & Jain, A. K. (2012b). Prostate cancer grading: Gland segmentation and structural features. Pattern Recognition Letters, 33(7), 951-961.     

Pudasaini, S., & Subedi, N. (2019). Understanding the gleason grading system and its changes. Journal of Pathology of Nepal.

Shibuya, T., Takahashi, G., & Kan, T. (2019). Basal cell carcinoma of the prostate: A case report and review of the literature. Molecular and clinical oncology, 10(1), 101–104. 

Singh, M., Kalaw, E. M., Giron, D. M., Chong, K. T., Tan, C. L., & Lee, H. K. (2017). Gland segmentation in prostate histopathological images. Journal of medical imaging, 4(2), 027501. 

Zhang Y, Nojima S, Nakayama H, Jin Y, Enza H. Characteristics of normal stromal components and their correlation with cancer occurrence in human prostate. Oncol Rep. 2003 Jan-Feb;10(1):207-11. PMID: 12469170.

Zhu, Q., Du, B., Turkbey, B., Choyke, P. L., & Yan, P. (2017, May). Deeply-supervised CNN for prostate segmentation. In: 2017 international joint conference on neural networks (IJCNN) (pp. 178-184). IEEE.

  • Prostate Tissue Histology - Automated Analysis Methods

    […] Normal prostate tissue histology […]

Comments are closed.