Background: Single-cell transcriptome analysis has fundamentally changed biological research by allowing higher-resolution computational analysis of individual cells and subsets of cell types. However, few methods have met the need to recognize and quantify the underlying cellular programs that determine the specialization and differentiation of the cell types. Methods: In this study, we present scGEM, a nested tree-structured nonparametric Bayesian model, to reveal the gene co-expression modules (GEMs) reflecting transcriptome processes in single cells. Results: We show that scGEM can discover shared and specialized transcriptome signals across different cell types using peripheral blood mononuclear single cells and early brain development single cells. scGEM outperformed other methods in perplexity and topic coherence (p< 0.001) on our simulation data. Larger datasets, deeper trees and pre-trained models are shown to be positively associated with better scGEM performance. The GEMs obtained from triple-negative breast cancer single cells exhibited better correlations with lymphocyte infiltration (p= 0.009) and the cell cycle (p< 0.001) than other methods in additional validation on the bulk RNAseq dataset. Conclusions: Altogether, we demonstrate that scGEM can be used to model the hidden cellular functions of single cells, thereby unveiling the specialization and generalization of transcriptomic programs across different types of cells.
背景：单细胞转录组分析通过实现对单个细胞及细胞亚型进行更高分辨率的计算分析，从根本上改变了生物学研究。然而，目前能够识别并量化决定细胞类型特化与分化的潜在细胞程序的方法仍较为有限。方法：本研究提出一种嵌套树状结构的非参数贝叶斯模型——scGEM，用于揭示反映单细胞转录组过程的基因共表达模块。结果：我们通过外周血单个核细胞及早期脑发育单细胞数据证明，scGEM能够发现不同细胞类型间共享及特异的转录组信号。在模拟数据上，scGEM在困惑度与主题一致性指标上均显著优于其他方法（p<0.001）。更大规模数据集、更深层树结构及预训练模型均被证实与scGEM性能提升呈正相关。在三阴性乳腺癌单细胞数据中获得的基因共表达模块，在批量RNA测序数据集的外部验证中，较其他方法展现出与淋巴细胞浸润（p=0.009）及细胞周期（p<0.001）更显著的相关性。结论：本研究证明scGEM能够有效建模单细胞的潜在细胞功能，从而揭示不同类型细胞间转录组程序的特化与泛化规律。
肿瘤(癌症)患者之家