Skip to content

Commit

Permalink
Merge pull request #79 from bartoszkochanski/master
Browse files Browse the repository at this point in the history
Add 04.11.2024 presentation
  • Loading branch information
sobieskibj authored Nov 28, 2024
2 parents 0f20cc1 + e7c3a78 commit 2ed63a7
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 1 deletion.
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training

## Abstract

We will discuss pre-training multimodal vision-language models for applications in computer-aided radiology. The multimodal models we will examine are trained jointly on raw medical images and corresponding free-text radiology reports. Radiology reports, generated abundantly within typical clinical workflows, serve as a valuable source of medical image annotations but have yet to be fully leveraged in modeling efforts.

I will present a [recent ICML 2024 conference paper](https://icml.cc/virtual/2024/poster/34857) that addresses this issue. I will begin with examples to illustrate the rationale for developing multimodal models in radiology and provide an overview of recent work and public dataset that form the basis of this research. Then, I will detail the paper’s main contributions: (1) extending the multimodal framework to account for multiple representations of anatomy in chest radiographs, and (2) advancing temporal modeling of longitudinal data.

## Source paper

[Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training](https://arxiv.org/abs/2405.19654)
Binary file not shown.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Join us at https://meet.drwhy.ai.
* 14.10 - Do Not Explain Vision Models without Context - Paulina Tomaszewska
* 21.10 - [Positional Label for Self-Supervised Vision Transformer](https://github.com/MI2DataLab/MI2DataLab_Seminarium/tree/master/2024/2024_10_21_Positional_Label_for_Self-Supervised_Vision_Transformer) - Filip Kołodziejczyk
* 28.10 - [Adversarial examples vs. context consistency defense for object detection](https://github.com/MI2DataLab/MI2DataLab_Seminarium/tree/master/2024/2024_10_28_Adversarial_attacks_against_object_detection.md) - Hubert Baniecki
* 04.11 - Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training - Bartosz Kochański
* 04.11 - [Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training](https://github.com/MI2DataLab/MI2DataLab_Seminarium/tree/master/2024/2024_11_04_Unlocking_the_Power_of_Spatial_and_Temporal_Information_in_Medical_Multimodal_Pre-training) - Bartosz Kochański
* 18.11 - User study: Visual Counterfactual Explanations for Improved Model Understanding - Bartek Sobieski
* 25.11 - Vision Transformers provably learn spatial structure - Vladimir Zaigrajew
* 02.12 - Null-text Inversion for Editing Real Images using Guided Diffusion Models - Dawid Płudowski
Expand Down

0 comments on commit 2ed63a7

Please sign in to comment.