Document resource
Background High-resolution spatial profiling of the tumor microenvironment (TME) is essential for advancing immunotherapy in solid tumors. While spatial transcriptomics provides detailed insights into the TME it remains expensive, low-throughput, and difficult to integrate into routine pathology workflows. In contrast, hematoxylin and eosin (H&E) slides are universally available and routinely generated during diagnostic evaluation. Here we developed Path2SpaceHD, a deep learning framework that predicts both spatial gene expression and fine-grained cell-type composition directly from standard H&E images.Methods Path2SpaceHD leverages Virchow2, a ViT-huge transformer pre-trained on histopathology, to extract fine-grained features from 60-pixel patches approximating single-cell resolution. We trained two separate models: one for predicting spatial gene expression (~6,000 genes, regression) and one for predicting cell types (9-class classification), using lightweight classifier heads.Model training and validation were performed on publicly available VisiumHD data.1 The training set included three VisiumHD slides from three patients (two colorectal carcinoma [CRC], one normal adjacent tissue [NAT]), and the validation set consisted of two slides from a fourth patient (one CRC, one NAT). Cell-type ground truths were derived using the VisiumHD annotation protocol described in the same study.Each model was trained via leave-one-slide-out cross-validation (LOOCV), and ensemble predictions were generated by mean pooling across the three resulting models. To benchmark performance, we compared Path2SpaceHD against CellViT, a state-of-the-art vision transformer for morphology-based cell-type inference.Results Path2SpaceHD robustly predicted the expression of 546 genes in external validation (Pearson correlation > 0.4 between measured and predicted values across the measured spots), including clinically relevant biomarkers such as CEACAM5, CEACAM6, and EPCAM (correlations > 0.8). For cell-type classification, Path2SpaceHD achieved 77% overall accuracy and 89% top-2 accuracy across nine biologically relevant cell types, including T cells, B cells, and myeloid subsets. Notably, the model could resolve morphologically similar immune subtypes such as B cells vs. T cells at near single-cell resolution ( figure 1). A simplified 4-class version of the model, trained for head-to-head comparison with CellViT, demonstrated consistent performance advantages across epithelial, neoplastic, inflammatory, and connective tissue classes.Conclusions Path2SpaceHD delivers robust, high-resolution spatial insights directly from H&E, enabling immune profiling and biomarker inference without transcriptomic assays. It can be adapted across cancers and immune contexts, offering a powerful tool for digital pathology-driven immuno-oncology at a single cell resolution.Reference Oliveira MFd, Romero JP, Chung M, et al. High-definition spatial transcriptomic profiling of immune cell populations in colorectal cancer. Nat Genet. 2025;57:1512–1523. https://doi.org/10.1038/s41588-025-02193-3Abstract 1109 Figure 1Path2Cell example. Comparison of true versus predicted cell types overlaid on histology in a zoomed-in, immune-enriched test set region. Predictions were made at 16 µm VisiumHD resolution; ground truth cell-type annotations were generated based on true gene expression