cirpublications.com-DHbk-1790880683-y530488644 1790880683 Clinical Intelligence Research Press info@cirpublications.com Clinical Intelligence Research Press Journal of Artificial Intelligence for Healthcare Systems J. Artif. Intell. Healthc. Syst. 3149-8981 2024 3 1 Paolo Ricci Marco De Luca Giulia Ferraro Antonio Russo 2024 10.68159/y530488644 https://cirpublications.com/pub/journal/1/article/y530488644 Huang K, Altosaar J, Ranganath R. ClinicalBERT: modeling clinical notes and predicting hospital readmission. arXiv [Preprint]. 2019;arXiv:1904.05342. Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2020;36(4):1234-40. Alsentzer E, Murphy J, Boag W, Weng WH, Jindi D, Naumann T, et al. Publicly available clinical BERT embeddings. In: Proc 2nd Clin Nat Lang Process Workshop; 2019; Minneapolis, USA. Association for Computational Linguistics; 2019. p. 72-8. Gu Y, Tinn R, Cheng H, Lucas M, Usuyama N, Liu X, et al. Domain-specific language model pretraining for biomedical natural language processing. ACM Trans Comput Healthc. 2021;3(1):1-23. 10.1145/3458754 Sun C, Yang Z, Wang L, Zhang Y, Lin H, Wang J. Biomedical named entity recognition using BERT in the machine reading comprehension framework. J Biomed Inform. 2021;118:103799. 10.1016/j.jbi.2021.103799 Lehman E, Jain S, Pichotta K, Goldberg Y, Wallace BC. Does BERT pretrained on clinical notes reveal sensitive data? In: Proc 2021 Conf North Am Chapter Assoc Comput Linguist Hum Lang Technol; 2021 Jun; Online. Association for Computational Linguistics; 2021. p. 946-59. Amrollahi F, Shashikumar SP, Razmi F, Nemati S. Contextual embeddings from clinical notes improves prediction of sepsis. AMIA Annu Symp Proc. 2021;2020:197-206. Lyu W, Dong X, Wong R, Zheng S, Abell-Hart K, Wang F, et al. A multimodal transformer: fusing clinical notes with structured EHR data for interpretable in-hospital mortality prediction. AMIA Annu Symp Proc. 2023;2022:719-28. Tinn R, Cheng H, Gu Y, Usuyama N, Liu X, Naumann T, et al. Fine-tuning large neural language models for biomedical natural language processing. Patterns. 2023;4(4):100729. 10.1016/j.patter.2023.100729 Tian S, Erdengasileng A, Yang X, Guo Y, Wu Y, Zhang J, et al. Transformer-based named entity recognition for parsing clinical trial eligibility criteria. In: Proc 12th ACM Int Conf Bioinformatics Comput Biol Health Inform; 2021; Virtual Event. ACM; 2021. p. 1-6. 10.1145/3459930.3469501 Wu H, Wang M, Wu J, Francis F, Chang YH, Shavick A, et al. A survey on clinical natural language processing in the United Kingdom from 2007 to 2022. NPJ Digit Med. 2022;5(1):186. 10.1038/s41746-022-00730-1 Abadeer M. Assessment of DistilBERT performance on named entity recognition task for the detection of protected health information and medical concepts. In: Proc 3rd Clin Nat Lang Process Workshop; 2020; Online. Association for Computational Linguistics; 2020. p. 158-67. Oh SH, Kang M, Lee Y. Protected health information recognition by fine-tuning a pre-training transformer model. Healthc Inform Res. 2022;28(1):16-24. 10.4258/hir.2022.28.1.16 Lin J. De-identification of free-text clinical notes [dissertation]. Cambridge (MA): Massachusetts Institute of Technology; 2021. Consens ME, Dufault C, Wainberg M, Forster D, Karimzadeh M, Goodarzi H, et al. To transformers and beyond: large language models for the genome. arXiv [Preprint]. 2023;arXiv:2311.07621. Shome D, Kar T, Mohanty SN, Tiwari P, Muhammad K, AlTameem A, et al. COVID-Transformer: interpretable COVID-19 detection using vision transformer for healthcare. Int J Environ Res Public Health. 2021;18(21):11086. 10.3390/ijerph182111086 Rukhsar S, Tiwari AK. Lightweight convolution transformer for cross-patient seizure detection in multi-channel EEG signals. Comput Methods Programs Biomed. 2023;242:107856. 10.1016/j.cmpb.2023.107856 Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. 2023;29(8):1930-40. 10.1038/s41591-023-02448-8 Chen P, Wang J, Lin H, Zhao D, Yang Z. Few-shot biomedical named entity recognition via knowledge-guided instance generation and prompt contrastive learning. Bioinformatics. 2023;39(8):btad496. Tian Y, Shen W, Song Y, Xia F, He M, Li K. Improving biomedical named entity recognition with syntactic information. BMC Bioinformatics. 2020;21(1):539. 10.1186/s12859-020-03899-7 Perera N, Dehmer M, Emmert-Streib F. Named entity recognition and relation detection for biomedical information extraction. Front Cell Dev Biol. 2020;8:673. 10.3389/fcell.2020.00673 Zhang X, Tian C, Yang X, Chen L, Li Z, Petzold LR. AlpaCare: instruction-tuned large language models for medical application. arXiv [Preprint]. 2023;arXiv:2310.14558. Pham T, Tran T, Phung D, Venkatesh S. Predicting healthcare trajectories from medical records: a deep learning approach. J Biomed Inform. 2017;69:218-29. 10.1016/j.jbi.2017.04.001 Subahi AF. BERT-based approach for greening software requirements engineering through non-functional requirements. IEEE Access. 2023;11:103001-13. 10.1109/ACCESS.2023.3317058 Sharaf S, Anoop VS. An analysis on large language models in healthcare: a case study of BioBERT. arXiv [Preprint]. 2023;arXiv:2309.02537. Amin-Nejad A, Ive J, Velupillai S. Transformer models trained on MIMIC-III to generate synthetic patient notes. PhysioNet. 2020. 10.13026/9fyw-2q82 Lin C, Bethard S, Savova G, Miller T, Dligach D. EntityBERT: BERT-based models pretrained on MIMIC-III with or without entity-centric masking strategy for the clinical domain. PhysioNet. 2020. Su Q, Cheng G, Huang J. A review of research on eligibility criteria for clinical trials. Clin Exp Med. 2023;23(6):1867-79. 10.1007/s10238-022-00935-8 Joshy A, Sundar S. Analyzing the performance of sentiment analysis using BERT, DistilBERT, and RoBERTa. In: 2022 IEEE Int Power Renew Energy Conf (IPRECON); 2022 Dec 16; Kollam, India. IEEE; 2022. p. 1-6. 10.1109/IPRECON55716.2022.10059541 Névéol A, Dalianis H, Velupillai S, Savova G, Zweigenbaum P. Clinical natural language processing in languages other than English: opportunities and challenges. J Biomed Semantics. 2018;9(1):12. 10.1186/s13326-018-0179-8 Tay Y, Dehghani M, Rao J, Fedus W, Abnar S, Chung HW, et al. Scale efficiently: insights from pre-training and fine-tuning transformers. arXiv [Preprint]. 2021;arXiv:2109.10686. Norgeot B, Muenzen K, Peterson TA, Fan X, Glicksberg BS, Schenk G, et al. Protected Health Information filter (Philter): accurately and securely de-identifying free-text clinical notes. NPJ Digit Med. 2020;3(1):57. 10.1038/s41746-020-0258-y