{"id":51,"date":"2017-08-12T12:01:15","date_gmt":"2017-08-12T12:01:15","guid":{"rendered":"http:\/\/cem.denizlab.org\/?page_id=51"},"modified":"2019-02-27T06:23:04","modified_gmt":"2019-02-27T06:23:04","slug":"proceedings","status":"publish","type":"page","link":"https:\/\/www.cemdeniz.me\/index.php\/publications\/proceedings\/","title":{"rendered":"Conference Proceedings"},"content":{"rendered":"
<\/a><\/form>
85 entries<\/span> «<\/a> ‹<\/a> 1 of 9 ›<\/a> »<\/a> <\/div><\/div>

2024<\/h3>

Huang, Haoxu; Rawlekar, Samyak; Chopra, Sumit; Deniz, Cem M<\/p>

Radiology Reports Improve Visual Representations Learned from Radiographs<\/a> Proceedings Article<\/span> <\/p>

In: <\/span> Oguz, Ipek; Noble, Jack; Li, Xiaoxiao; Styner, Martin; Baumgartner, Christian; Rusu, Mirabela; Heinmann, Tobias; Kontos, Despina; Landman, Bennett; Dawant, Benoit (Ed.): Medical Imaging with Deep Learning, <\/span>pp. 1385\u20131405, <\/span>PMLR, <\/span>2024<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@inproceedings{pmlr-v227-huang24a,
\r\ntitle = {Radiology Reports Improve Visual Representations Learned from Radiographs},
\r\nauthor = {Haoxu Huang and Samyak Rawlekar and Sumit Chopra and Cem M Deniz},
\r\neditor = {Ipek Oguz and Jack Noble and Xiaoxiao Li and Martin Styner and Christian Baumgartner and Mirabela Rusu and Tobias Heinmann and Despina Kontos and Bennett Landman and Benoit Dawant},
\r\nurl = {https:\/\/proceedings.mlr.press\/v227\/huang24a.html},
\r\nyear  = {2024},
\r\ndate = {2024-07-01},
\r\nbooktitle = {Medical Imaging with Deep Learning},
\r\nvolume = {227},
\r\npages = {1385\u20131405},
\r\npublisher = {PMLR},
\r\nseries = {Proceedings of Machine Learning Research},
\r\nabstract = {Although human\u2019s ability to visually understand the structure of the World plays a crucial role in perceiving the World and making appropriate decisions, human perception does not solely rely on vision but amalgamates the information from acoustic, verbal, and visual stimuli. An active area of research has been revolving around designing an efficient framework that adapts to multiple modalities and ideally improves the performance of existing tasks. While numerous frameworks have proved effective on natural datasets like ImageNet, a limited number of studies have been carried out in the biomedical domain. In this work, we extend the available frameworks for natural data to biomedical data by leveraging the abundant, unstructured multi-modal data available as radiology images and reports. We attempt to answer the question, \u201dFor multi-modal learning, self-supervised learning and joint learning using both learning strategies, which one improves the visual representation for downstream chest radiographs classification tasks the most?\u201d. Our experiments indicated that in limited labeled data settings with 1% and 10% labeled data, the joint learning with multi-modal and self-supervised models outperforms self-supervised learning and is at par with multi-modal learning. Additionally, we found that multi-modal learning is generally more robust on out-of-distribution datasets.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {inproceedings}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>
Although human\u2019s ability to visually understand the structure of the World plays a crucial role in perceiving the World and making appropriate decisions, human perception does not solely rely on vision but amalgamates the information from acoustic, verbal, and visual stimuli. An active area of research has been revolving around designing an efficient framework that adapts to multiple modalities and ideally improves the performance of existing tasks. While numerous frameworks have proved effective on natural datasets like ImageNet, a limited number of studies have been carried out in the biomedical domain. In this work, we extend the available frameworks for natural data to biomedical data by leveraging the abundant, unstructured multi-modal data available as radiology images and reports. We attempt to answer the question, \u201dFor multi-modal learning, self-supervised learning and joint learning using both learning strategies, which one improves the visual representation for downstream chest radiographs classification tasks the most?\u201d. Our experiments indicated that in limited labeled data settings with 1% and 10% labeled data, the joint learning with multi-modal and self-supervised models outperforms self-supervised learning and is at par with multi-modal learning. Additionally, we found that multi-modal learning is generally more robust on out-of-distribution datasets.<\/div>
Close<\/a><\/p><\/div>