Generative pre-trained models are becoming more common by the day, and such methods have now been applied to single cell sequencing, and multi-omits in general.
Two main players are “scBERT” and “scGPT”:
scGPT: Towards Building a Foundation Model for Single-Cell Multi-omics Using Generative AI
Haotian Cui, Chloe Wang, Hassaan Maan, Bo Wang – Nature Methods doi: 10.1038/s41592-024-02201-0
CODE: https://github.com/bowang-lab/scGPT
scBERT as a large-scale pretrained deep language model for cell type annotation of single-cell RNA-seq data Fan Yang, Wenchuan Wang, Fang Wang, Yuan Fang, Duyu Tang, Junzhou Huang, Hui Lu & Jianhua Yao Nature Machine Intelligence 4(10):1-15 – doi:10.1038/s42256-022-00534-z
CODE: https://github.com/TencentAILabHealthcare/scBERT
A YouTube description of use: