Spatial Gene Expression Prediction from Histology Images with STco

In recent years, the rapid development of spatial transcriptome technology has fundamentally transformed our understanding of gene expression regulation within complex biological systems. However, the widespread application of spatial transcriptome technology in large-scale studies is hindered by its high cost and complexity. An economical alternative involves utilizing artificial intelligence to predict gene expression information from entire slices of histological images stained with hematoxylin and eosin (H&E). Nevertheless, existing methods fall short in extracting profound information from pathological images. In this paper, we propose STco, a multi-modal contrastive learning framework which comprehensively integrates multi-modal information, including histological images, gene expression features of spots, positional information of spots, and methods for aggregating gene expression. We utilized spatial transcriptomics data from two different tumors generated by the 10 times Genomics platform: human HER2 positive breast cancer (HER2+) and human cutaneous squamous cell carcinoma (cSCC) datasets. The experimental results demonstrate the superiority of STco compared to other methods in predicting gene expression profiles from histological images. Additionally, STco has proven its capability to interpret cancer-specific highly expressed genes. Our code is available at https://github.com/shizhiceng/STco.