Document resource
Background Spatial omics technologies offer novel insights into the molecular organization of tissues. 1 Accurate identification of tumor regions in Haematoxylin and Eosin (H&E)-stained whole-slide images (WSIs) in particular is essential in spatial immunology but such interpretation is still largely dependent on pathologist’s manual annotation, which is time consuming, subjective and often infeasible for large data sets.2 3 Deep learning methods have shown promising results in tissue compartmentalization via leveraging spatial transcriptomics data.4–7 We compare three approaches (Methods A: Leiden clustering of neighborhood graph constructed from raw image patches, Methods B: clustering embeddings from variational graph autoencoder (VGAE) trained on neighborhood graphs and Methods C: clustering embeddings inferred from CONCH, a large pre-trained model) to assess whether clustering based on raw pixel or learned embeddings can support effective unsupervised tissue categorization.8–10 Methods Each WSI is padded and tiled into 16×16 pixel patches, flattened into 1×768-dim vectors and used to build a 15-nearest-neighbor graph. For Methods A, Leiden clustering is performed on the adjacency matrix of this graph. For Methods B, a lightweight VGAE is trained in an unsupervised manner on 40 of such graphs obtained from The Cancer Genome Atlas (TCGA) WSIs. 11 During inference we retrieve the hidden representations of individual patches from the encoder and cluster them using k-means (k set to the number of pathologist annotations). For Methods C, the embeddings are directly retrieved from CONCH for each patch and clustered in the same manner as the VGAE embeddings (figure 1). For all methods cluster labels are mapped back to individual pixels to obtain a segmentation mask. The segmentation is then annotated via visual comparison to the pathologist’s annotation.Results All three approaches were evaluated on TCGA WSIs from different tumor stages and an in-house Visium data set ( figure 2).Conclusions In the task of identifying tumor regions, simply clustering a neighborhood graph of image patches via Leiden can compete with clustering embeddings of (large) pre-trained models while being two orders of magnitude faster than the CONCH approach. While neither Methods can correctly identify the entire tumor region the results can serve both as a baseline or as a fast pre-clustering step, i.e. to define superpixels. Alternatively results can be combined with gene expression or proteomics data to build a more complex multimodal model. This shows how graph clustering and RGB features can be utilized in spatial omics to identify tumor regions without the need for computationally expensive models.Acknowledgements Our work is supported by the Bioinformatics Institute and Singapore Immunology Network, Agency for Science, Technology and Research (A*STAR). Our work is funded in part by the following grants: NMRC OFYIRG23jan-0049 (awarded to Dr. Mai Chan Lau), and IAF-PP T-MoVac Programme (H22J1a0043)References Palla G, Fischer DS, Regev A, et al. Spatial components of molecular tissue biology. Nat Biotechnol. 2022;40:308–318. https://doi.org/10.1038/s41587-021-01182-1Fu T, Dai LJ, Wu SY, et al. Spatial architecture of the immune microenvironment orchestrates tumor immunity and therapeutic response. J Hematol Oncol. 2021;14:98. https://doi.org/10.1186/s13045-021-01103-4Juha P Väyrynen, Mai Chan Lau, et al. Prognostic significance of immune cell populations identified by machine learning in colorectal cancer using routine hematoxylin and eosin-stained sections. Clin Cancer Res. 2020;26(16). https://doi.org/10.1158/1078-0432.CCR-20-0071Zhuohan Yu, Yuning Yang, Xingjian Chen, et al. Accurate spatial heterogeneity dissection and gene regulation interpretation for spatial transcriptomics using dual graph contrastive learning. Advanced Science. 2024. https://doi.org/10.1002/advs.202410081Jianing Yao, Jinglun Yu, Brian Caffo, et al. Spatial domain detection using contrastive self-supervised learning for spatial multi-omics technologies. Genome Research. 2025. https://doi.org/10.1101/gr.279380.124Long Y, Ang KS, Li M, et al. Spatially informed clustering, integration, and deconvolution of spatial transcriptomics with GraphST. Nat Commun. 2023;14:1155. https://doi.org/10.1038/s41467-023-36796-3 Zhang D, Schroeder A, Yan H, et al. Inferring super-resolution tissue architecture by integrating spatial transcriptomics with histology. Nat Biotechnol. 2024;42:1372–1377. https://doi.org/10.1038/s41587-023-02019-9Traag VA, Waltman L, van Eck NJ. From louvain to leiden: guaranteeing well-connected communities. Sci Rep. 2019;9:5233. https://doi.org/10.1038/s41598-019-41695-zThomas N Kipf, Max Welling. Variational graph auto-encoders. Bayesian Deep Learning Workshop (NIPS 2016). https://doi.org/10.48550/arXiv.1611.07308Lu MY, Chen B, Williamson DFK, et al. A visual-language foundation model for computational pathology. Nat Med. 2024;30:863–874. https://doi.org/10.1038/s41591-024-02856-4The Cancer Genome Atlas Research Network, Weinstein J, Collisson E, et al. The cancer genome atlas pan-cancer analysis project. Nat Genet. 2013;45;1113–1120. https://doi.org/10.1038/ng.2764Ethics Approval The study was approved by the Agency for Science, Technology and Research (A*STAR) Human Biomedical Research Office (A*STAR IRB: 2021-188).Consent De-identified patient data was used in our work. In-house samples were collected with consent from patients.Abstract 1102 Figure 1Overview of Experimental workflow. (a) Clustering adjacency matrix of the neighborhood graph via Leiden algorithm (b) Training and inference using VGAE (c) Inference using pre-trained CONCHAbstract 1102 Figure 2Performance comparison using pathologist’s annotations. (a-c) colorectal cancer samples (a) AJCC stage I samples, (b) AJCC stage II sample (c) AJCC stage III sample (d) in-house hepatocellular carcinoma Visium sample (spot annotations)