Multimodal Transformer Architecture for ARDS Detection: A Framework Integrating Chest X-Ray, Clinical Notes, and Laboratory Values

cirpublications.com-vMM6-1790880680-z613962324 1790880680 Clinical Intelligence Research Press info@cirpublications.com Clinical Intelligence Research Press Journal of Artificial Intelligence for Healthcare Systems J. Artif. Intell. Healthc. Syst. 3149-8981 2022 1 1 Multimodal Transformer Architecture for ARDS Detection: A Framework Integrating Chest X-Ray, Clinical Notes, and Laboratory Values Ravi Kumar Neha Sharma Aniket Deshmukh Arjun Nair Meera Pillai 2022 10.68159/z613962324 https://cirpublications.com/pub/journal/1/article/z613962324 Sjoding MW, Hofer TP, Co I, Courey A, Cooke CR, Iwashyna TJ. Interobserver reliability of the Berlin ARDS definition and strategies to improve the reliability of ARDS diagnosis. Chest. 2018;153(2):361-7. 10.1016/j.chest.2017.11.037 Sayed M, Riaño D, Villar J. Novel criteria to classify ARDS severity using a machine learning approach. Crit Care. 2021;25(1):150. 10.1186/s13054-021-03563-w Li H, Odeyemi YE, Weister TJ, Liu C, Chalmers SJ, Lal A, et al. Rule-based cohort definitions for acute respiratory distress syndrome: a computable phenotyping strategy based on the Berlin definition. Crit Care Explor. 2021;3(6):e0451. 10.1097/CCE.0000000000000451 Baltruschat IM, Nickisch H, Grass M, Knopp T, Saalbach A. Comparison of deep learning approaches for multi-label chest X-ray classification. Sci Rep. 2019;9(1):6381. 10.1038/s41598-019-42294-8 Alsentzer E, Murphy J, Boag W, Weng WH, Jindi D, Naumann T, et al. Publicly available clinical BERT embeddings. In: Proc 2nd Clin Nat Lang Process Workshop. Stroudsburg (PA): Association for Computational Linguistics; 2019. p. 72-78. Hung CY, Lin CH, Chang CS, Li JL, Lee CC. Predicting gastrointestinal bleeding events from multimodal in-hospital electronic health records using deep fusion networks. In: 2019 41st Annu Int Conf IEEE Eng Med Biol Soc (EMBC). Piscataway (NJ): IEEE; 2019. p. 2447-50. 10.1109/EMBC.2019.8857402 Mohsen F, Ali H, El Hajj N, Shah Z. Artificial intelligence-based methods for fusion of electronic health records and imaging data. Sci Rep. 2022;12(1):17981. 10.1038/s41598-022-22514-4 Tang W, He F, Liu Y, Duan Y. MATR: multimodal medical image fusion via multiscale adaptive transformer. IEEE Trans Image Process. 2022;31:5134-49. 10.1109/TIP.2022.3195278 Song X, Chao H, Xu X, Guo H, Xu S, Turkbey B, et al. Cross-modal attention for multi-modal image registration. Med Image Anal. 2022;82:102612. 10.1016/j.media.2022.102612 Jana S, Dasgupta T, Dey L. Predicting medical events and ICU requirements using a multimodal multiobjective transformer network. Exp Biol Med (Maywood). 2022;247(22):1988-2002. 10.1177/15353702221121775 Majdi MS, Salman KN, Morris MF, Merchant NC, Rodriguez JJ. Deep learning classification of chest X-ray images. In: 2020 IEEE Southwest Symp Image Anal Interpr (SSIAI). Piscataway (NJ): IEEE; 2020. p. 116-9. 10.1109/SSIAI49293.2020.9094615 Khan E, Rehman MZ, Ahmed F, Alfouzan FA, Alzahrani NM, Ahmad J. Chest X-ray classification for the detection of COVID-19 using deep learning techniques. Sensors (Basel). 2022;22(3):1211. 10.3390/s22031211 Shelke A, Inamdar M, Shah V, Tiwari A, Hussain A, Chafekar T, et al. Chest X-ray classification using deep learning for automated COVID-19 screening. SN Comput Sci. 2021;2(4):300. 10.1007/s42979-021-00695-8 Hussain E, Hasan M, Rahman MA, Lee I, Tamanna T, Parvez MZ. CoroDet: a deep learning based classification for COVID-19 detection using chest X-ray images. Chaos Solitons Fractals. 2021;142:110495. 10.1016/j.chaos.2020.110495 Pawar Y, Henriksson A, Hedberg P, Naucler P. Leveraging ClinicalBERT in multimodal mortality prediction models for COVID-19. In: 2022 IEEE 35th Int Symp Comput Based Med Syst (CBMS). Piscataway (NJ): IEEE; 2022. p. 199-204. 10.1109/CBMS55023.2022.00045 Kalusivalingam AK, Sharma A, Patel N, Singh V. Leveraging BERT and LSTM for enhanced natural language processing in clinical data analysis. Int J AI ML. 2021;2(3):1-9. Roy A, Pan S. Incorporating medical knowledge in BERT for clinical relation extraction. In: Proc 2021 Conf Empir Methods Nat Lang Process. Stroudsburg (PA): Association for Computational Linguistics; 2021. p. 5357-66. Lamproudis A, Henriksson A, Dalianis H. Developing a clinical language model for Swedish: continued pretraining of generic BERT with in-domain data. In: Proc Int Conf Recent Adv Nat Lang Process (RANLP 2021). Stroudsburg (PA): Association for Computational Linguistics; 2021. p. 790-7. 10.26615/978-954-452-072-4_089 Zhang C, Chu X, Ma L, Zhu Y, Wang Y, Wang J, et al. M3Care: learning with missing modalities in multimodal healthcare data. In: Proc 28th ACM SIGKDD Conf Knowl Discov Data Min. New York (NY): ACM; 2022. p. 2418-28. 10.1145/3534678.3539397 Lopez K, Fodeh SJ, Allam A, Brandt CA, Krauthammer M. Reducing annotation burden through multimodal learning. Front Big Data. 2020;3:19. 10.3389/fdata.2020.00019 Sun Q, Fang N, Liu Z, Zhao L, Wen Y, Lin H. HybridCTrm: bridging CNN and transformer for multimodal brain image segmentation. J Healthc Eng. 2021;2021:7467261. Zhang Y, Deng Y, Zhou Z, Zhang X, Jiao P, Zhao Z. Multimodal learning for fetal distress diagnosis using a multimodal medical information fusion framework. Front Physiol. 2022;13:1021400. 10.3389/fphys.2022.1021400 Zhang Y, Ou W, Shi Y, Deng J, You X, Wang A. Deep medical cross-modal attention hashing. World Wide Web. 2022;25(4):1519-36. 10.1007/s11280-021-00973-0 Shi T, Jiang H, Zheng B. C2MA-Net: cross-modal cross-attention network for acute ischemic stroke lesion segmentation based on CT perfusion scans. IEEE Trans Biomed Eng. 2022;69(1):108-18. 10.1109/TBME.2021.3086210 Song X, Zhang X, Ji J, Liu Y, Wei P. Cross-modal contrastive attention model for medical report generation. In: Proc 29th Int Conf Comput Linguistics. Stroudsburg (PA): Association for Computational Linguistics; 2022. p. 2388-97. Zhou Z, Guo X, Yang W, Shi Y, Zhou L, Wang L, et al. Cross-modal attention-guided convolutional network for multi-modal cardiac segmentation. In: Int Workshop Mach Learn Med Imaging. Cham: Springer; 2019. p. 601-10. 10.1007/978-3-030-32692-0_69 Yan S, Wang C, Chen W, Lyu J. Swin transformer-based GAN for multi-modal medical image translation. Front Oncol. 2022;12:942511. 10.3389/fonc.2022.942511 Koivunen M, Saranto K. Nursing professionals' experiences of the facilitators and barriers to the use of telehealth applications: a systematic review of qualitative studies. Scand J Caring Sci. 2018;32(1):24-44. 10.1111/scs.12445 Markello RD, Shafiei G, Tremblay C, Postuma RB, Dagher A, Misic B. Multimodal phenotypic axes of Parkinson’s disease. npj Parkinsons Dis. 2021;7(1):6. 10.1038/s41531-020-00144-6