cirpublications.com-nDzu-1790883433-b943680546 1790883433 Clinical Intelligence Research Press info@cirpublications.com Clinical Intelligence Research Press Journal of Health Informatics and Digital Systems J. Health Inform. Digit. Syst. 3149-8973 2026 6 2 Yuki Yamamoto Kenji Ito 2026 10.68159/b943680546 https://cirpublications.com/pub/journal/2/article/b943680546 Li Y, Rao S, Solares JR, Hassaine A, Ramakrishnan R, Canoy D, et al. BEHRT: transformer for electronic health records. Sci Rep. 2020;10(1):7155. Rasmy L, Xiang Y, Xie Z, Tao C, Zhi D. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. NPJ Digit Med. 2021;4(1):86. Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med. 2018;1(1):18. Zerveas G, Jayaraman S, Patel D, Bhamidipaty A, Eickhoff C. A transformer-based framework for multivariate time series representation learning. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining; 2021. p. 2114-24. Franceschi JY, Dieuleveut A, Jaggi M. Unsupervised scalable representation learning for multivariate time series. Adv Neural Inf Process Syst. 2019;32:4650-61. Devlin J, Chang MW, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; 2019. p. 4171-86. Pang C, Jiang X, Kalluri KS, Spotnitz M, Chen R, Perotte A, et al. CEHR-BERT: incorporating temporal information from structured EHR data to improve prediction tasks. In: Proceedings of Machine Learning for Health. PMLR; 2021. p. 239-60. Zeng X, Linwood SL, Liu C. Pretrained transformer framework on pediatric claims data for population specific tasks. Sci Rep. 2022;12(1):3651. Yang Z, Mitra A, Liu W, Berlowitz D, Yu H. TransformEHR: transformer-based encoder-decoder generative model to enhance prediction of disease outcomes using electronic health records. Nat Commun. 2023;14(1):7857. Guo LL, Steinberg E, Fleming SL, Posada J, Lemmon J, Pfohl SR, et al. EHR foundation models improve robustness in the presence of temporal distribution shift. Sci Rep. 2023;13(1):3767. Kraljevic Z, Bean D, Shek A, Bendayan R, Hemingway H, Yeung JA, et al. Foresight—a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study. Lancet Digit Health. 2024;6(4):e281-90. Guo LL, Fries J, Steinberg E, Fleming SL, Morse K, Aftandilian C, et al. A multi-center study on the adaptability of a shared foundation model for electronic health records. NPJ Digit Med. 2024;7(1):171. Moor M, Banerjee O, Abad ZS, Krumholz HM, Leskovec J, Topol EJ, et al. Foundation models for generalist medical artificial intelligence. Nature. 2023;616(7956):259-65. Chen H, Lundberg SM, Erion G, Kim JH, Lee SI. Forecasting adverse surgical events using self-supervised transfer learning for physiological signals. NPJ Digit Med. 2021;4(1):167. Rao S, Li Y, Ramakrishnan R, Hassaine A, Canoy D, Cleland J, et al. An explainable transformer-based deep learning model for the prediction of incident heart failure. IEEE J Biomed Health Inform. 2022;26(7):3362-72. Zang C, Wang F. SceHR: supervised contrastive learning for clinical risk prediction using electronic health records. In: Proceedings of the IEEE International Conference on Data Mining (ICDM); 2021. p. 857-62. Ma F, Chitta R, Zhou J, You Q, Sun T, Gao J. Dipole: diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2017. p. 1903-11. Choi E, Bahadori MT, Song L, Stewart WF, Sun J. GRAM: graph-based attention model for healthcare representation learning. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2017. p. 787-95. Tonekaboni S, Eytan D, Goldenberg A. Unsupervised representation learning for time series with temporal neighborhood coding. arXiv. 2021;arXiv:2106.00750. Renc P, Jia Y, Samir AE, Was J, Li Q, Bates DW, et al. Zero-shot health trajectory prediction using transformer. NPJ Digit Med. 2024;7(1):256. Makarov N, Bordukova M, Quengdaeng P, Garger D, Rodriguez-Esteban R, Schmich F, et al. Large language models forecast patient health trajectories enabling digital twins. NPJ Digit Med. 2025;8(1):588. Wornow M, Xu Y, Thapa R, Patel B, Steinberg E, Fleming S, et al. The shaky foundations of large language models and foundation models for electronic health records. NPJ Digit Med. 2023;6(1):135. Redekop E, Wang Z, Kulkarni R, Pleasure M, Chin A, Hassanzadeh HR, et al. Zero-shot medical event prediction using a generative pretrained transformer on electronic health records. J Am Med Inform Assoc. 2025;32(12):1833-42. Xian S, Grabowska ME, Kullo IJ, Luo Y, Smoller JW, Walunas TL, et al. Transformer patient embedding using electronic health records enables patient stratification and progression analysis. NPJ Digit Med. 2025;8(1):521. Liu Y, Zhang Z, Mi J, Pan S, Chen T, Guo Y, et al. GatorCLR: personalized predictions of patient outcomes on electronic health records using self-supervised contrastive graph representation. J Biomed Inform. 2025;168:104851. Liu T, Liang L, Che C, Liu Y, Jin B. A transformer-based framework for temporal health event prediction with graph-enhanced representations. J Biomed Inform. 2025;166:104826. Feng W, Wu H, Ma H, Tao Z, Xu M, Zhang X, et al. Applying contrastive pre-training for depression and anxiety risk prediction in type 2 diabetes patients based on heterogeneous electronic health records: a primary healthcare case study. J Am Med Inform Assoc. 2024;31(2):445-55. Wanyan T, Honarvar H, Jaladanki SK, Zang C, Naik N, Somani S, et al. Contrastive learning improves critical event prediction in COVID-19 patients. Patterns. 2021;2(12):100389.