Immune phenotype data, specifically the description of densities and spatial distribution of immune cells are now frequently included in the clinical pathology report as these features of the cells in the tumor microenvironment (TME) have shown to be associated with prognosis. In addition, immune-therapeutics, which aim at manipulating the patients’ immune system to kill cancer cells, have recently been approved for treatment of triple-negative breast cancers (TNBCs). Thus, quantifying the immune phenotype of the cancer could be important both for prognostication, and for prediction of therapy response. We have studied the immune phenotype of 42 breast cancers using immunofluorescence protein multiplexing and quantitative image analysis. After sectioning, formalin-fixed paraffin-embedded tissues were sequentially stained with a panel of fluorescently-labelled antibodies and imaged with the multiplexer (Cell DIVE, Leica Biosystems). Composite images of antibody-stained sections were then analysed using specialized digital pathology software (HALO, Indica Labs). Binary thresholding was conducted to identify and quantify densities of various immune lineage subsets (T lymphocytes and macrophages). Their cellular localisation was mapped and the spatial features of cellular arrangement were evaluated using a k-nearest neighbor graph (KNNG) method and Louvain community-proximity clustering. The spatial relationship of various immune and cancer cell types was quantified to assess whether cellular arrangements and structures differed among breast cancer subtypes. Our work demonstrates the use of molecular and cellular imaging in quantifying features of the tumor microenvironment in breast cancer classification, and the application of KNNG in studying spatial biology.
ER, PR (estrogen, progesterone receptor), and HER2 (human epidermal growth factor receptor 2) status are assessed using immunohistochemistry and reported in standard clinical workflows as they provide valuable information to help treatment planning. The protein Ki67 has also been suggested as a prognostic biomarker but is not routinely evaluated clinically due to insufficient quality assurance. The routine pathological practice usually relies on small biopsies, such that the reduction in consumption is necessary to save materials for special assays. For this purpose, we developed and validated an automatic system for segmenting and identifying the (ER, PR, HER2, Ki67) positive cells from hæmatoxylin and eosin (H&E) stained tissue sections using multiplexed immunofluorescence (MxIF) images at cellular level as a reference standard. In this study, we used 100 tissue-microarray cores sampled from 56 cases of invasive breast cancer. For ER, we extracted cell nucleus images (HoverNet) from the H&E images and assigned each cell nucleus as ER positive vs. negative based on the corresponding MxIF signals (whole cell segmentation with DeepCSeg) upon H&E to MxIF image registration. We trained a Res-Net 18 and validated the model on a separate test-set for classifying the cells as positive vs. negative for ER, and performed the same experiment for the other three markers. We obtained area-under-the- receiver-operating-characteristic-curves (AUCs) of 0.82 (ER), 0.85 (PR), 0.75 (HER2), 0.82 (Ki67) respectively. Our study demonstrates the feasibility of using machine learning to identify molecular status at cellular level directly from the H&E slides.
The tumor microenvironment (TME) plays an important role in driving cancer progression and affecting treatment efficacy. Cellular components of the TME include various immune subsets (tumor infiltrating lymphocytes (TILs) and macrophages), cancer-associated fibroblasts (CAFs) and vascular cells. While immune lineage has been a main focus of intensive research on the TME, CAFs have also been shown to be highly heterogeneous in their molecular phenotype and function. Using a protein marker immunofluorescence multiplexing system (Cell DIVE, Leica Microsystems) and quantitative imaging tools, we investigated the identity of various CAF clusters based on the expression of α-Smooth Muscle Actin (αSMA) and Fibroblast Activation Protein (FAP), and compared their distributions across breast cancer subtypes. We determined the cell counts of various CAF subsets using binary counting and identified the heterogeneous presentations of clusters using K-means clustering and Uniform Manifold Approximation and Projection (UMAP). We found that the abundance of CAF clusters varied among breast cancer subtypes. An integrated analysis of CAF cluster composition in each cancer and the transcriptomic data of CAF-associated genes such as CD29, IL6 and PDGFRβ was performed. We observed increased densities of proliferative, αSMA-positive CAFs in basal-like breast cancers that exhibited a co-expression signature of CAF-associated genes. Finally, an association analysis of CAF cluster composition and gene expression with previously identified radiomic phenotype was performed, but significant correlation was not detected.
Cytometry plays essential roles in immunology and oncology. Recent advancements in cellular imaging allow more detailed characterization of cells by labeling each cell with multiple protein markers. The increase of dimensionality makes manual analysis challenging. Clustering algorithms provide a means for phenotyping high-dimensional cell populations in an unsupervised manner for downstream analysis. The choice and usability of the methods are critical in practice. Literature provided comprehensive studies on those topics using publicly available flow cytometry data, which validated cell phenotypes by those methods against manual gated cell populations. In order to extend the knowledge for identification of cell phenotypes including unknown cell populations in our dataset, we conducted an exploratory study using clinical relevant tissue types as reference standard. Using our in-house database of multiplexed immunofluorescence images of breast cancer tissue microarrays (TMAs), we experimented with two commonly used algorithms (PhenoGraph and FlowSOM). Our pipeline includes: 1) cell phenotyping using Phenograph/FlowSOM; 2) clustering TMA cores into four groups using the percentage of each cell phenotypes with the algorithms (PhenoGraph/Spectral/K-means); 3) comparing the tissue groups to clinically relevant subtypes that were manually assigned based on the immunohistochemistry scores of serial sections. We experimented with different hyperparameter settings and input markers. Cell phenotypes using Phenograph with 10 markers and tissue clustering using Spectral yielded the highest mean F-measure (average over four tissue subtypes) of 0.71. In general, our results showed that cell phenotypes by Phenograph yielded better performance with larger variations than FlowSOM, which gives very consistent results.
Single cell phenotyping using molecular or protein multiplexing techniques is gaining momentum, especially in the characterization of cancer and the tumor microenvironment. It has proven to be particularly useful in studying the extent of heterogeneity in cancer, and in the profiling of the immune environment to assess whether certain cell subsets could be predictive of treatment response. Using a sequential protein marker labelling system called Multiplex Immunofluorescence (MxIF, GE Research), we have developed quantitative image analysis and computational tools for phenotyping individual immune and cancer cells for various cancer types. The expressions of T cell markers CD3, CD8, macrophage markers CD68, immune checkpoint proteins PD-1 and PD-L1, together with proliferative marker (Ki67) and cancer-specific marker PCK (pan-Cytokeratin) were studied on single 4um sections of formalin-fixed, paraffinembedded (FFPE) ovarian cancer tissue sections. We explored the composition of immune phenotype using t-SNE and quantified cell densities and marker co-expression patterns using binary cell counting. In addition to phenotyping immune cell types, their spatial localizations were analyzed. Neighborhood analysis was conducted using co-occurrence matrices to determine the number of times that a particular cell type is proximal to one another. Cell-to-cell spatial relationship was assessed by quantifying the Euclidean distances between individual cell types. These tools are being applied to specimens from an immunotherapy clinical trial to evaluate the dynamic changes in immune phenotype during the course of immune blockade therapy.
Manual annotation of Hematoxylin and Eosin (H&E) stained tissue images for deep learning classification is difficult, time consuming, and error-prone particularly for multi-class and rare-class problems. Chemical probes in immunohistochemistry (IHC) or immunofluorescence (IF) can automatically tag cellular structures; however, chemical labeling is difficult to use in training a deep classifier for H&E images (e.g. through serial sectioning and registration). In this work, we leverage the novel Multiplexed Immuno-Fluorescencent (MxIF) microscopy method developed by General Electric Global Research Center (GE GRC) which allows sequential, stain-image-bleach (SSB) application of protein markers on formalin-fixed, paraffin-embedded(FFPE) samples followed by traditional H&E staining to build chemically-annotated tissue maps of nuclei, cytoplasm, and cell membranes. This allows us to automate the creation of ground truth class-label maps for training an H&E-based tissue classifier. In this study, a tissue microarray consisting of 149 breast cancer and normal tissue cores were stained using MxIF for our three analytes, followed by traditional H&E staining. The MxIF stains for each TMA core were combined to create a “Virtual H&E” image, which is registered with the corresponding real H&E images. Each MxIF stained spot was segmented to obtain a class-label map for each analyte, which was then applied to the real H&E image to build a dataset consisting of the three analytes. A convolutional neural network (CNN) was then trained to classify this dataset. This system achieved an overall accuracy of 70%, suggesting that the MxIF system can provide useful labels for identifying hard to distinguish structures. A U-net was also trained to generate pseudo-IF stains from H&E and resulted in similar results.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.