Skip to content

Commit

Permalink
Update index.html
Browse files Browse the repository at this point in the history
  • Loading branch information
dnwjddl authored Jun 4, 2024
1 parent f7b8afb commit 5407fb3
Showing 1 changed file with 8 additions and 6 deletions.
14 changes: 8 additions & 6 deletions cxrl2024/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -195,19 +195,21 @@ <h4 class="title is-4 has-text-centered">Pipeline</h4>
<br>
<p>
The pipeline of <i>CXRL</i>.
Our model employs policy gradient optimization utilizing multi-reward feedbacks, fine-tuning image generator and ACE to produce realistic and accurate CXR that corresponds closely to the input report.
Our model employs policy gradient optimization utilizing multi-reward feedback, fine-tuning image generator, and ACE to produce realistic and accurate CXR that corresponds closely to the input report.
</p>
</div>

<br>
<h4 class="title is-4 has-text-centered">Eigen Aggregation Module</h4>
<h4 class="title is-4 has-text-centered">Reward Feedback Models</h4>
<div class="content has-text-justified">
<img src="./static/images/feedback.png" alt="Main figure">
<p>
An illustration of the EiCue generation. From the input image, both color affinity
matrix \( \mathrm{A_{color}} \) and semantic similarity matrix \( \mathrm{A_{seg}} \) are derived,
which are combined to form the Laplacian \( \mathrm{L_{sym}} \). An eigenvector
subset \( \mathrm{\hat{V}} \) of \( \mathrm{L_{sym}} \) are clustered to produce EiCue.
A detailed illustration of our reward feedback models. We incorporate three different feedbacks for report-to-CXR generation model to generate goal-oriented CXRs.
<ul>
<li><b>Posture Alignment Feedback</b>: Generated CXRs often face scaling issues, like excessive zooming or rotation, obscuring essential details. To counter these undesirable effects, we introduce a reward signal to align the CXR's posture with a canonical orientation to preserve essential parts.</li>
<li><b>Diagnostic Condition Feedback</b>: To accurately reflect generated CXRs with referenced pathologies, we classify them using a parsed report label, rewarding its accuracy.</li>
<li><b>Multimodal Consistency Feedback</b>: We enforce the generated CXRs to better match their reports. We leverage a multimodal latent representation pretrained with CXR-report pairs for semantic agreement assessment.</li>
</ul>
</p>
</div>

Expand Down

0 comments on commit 5407fb3

Please sign in to comment.