Skip to content

Commit

Permalink
TYPO : wasserstein
Browse files Browse the repository at this point in the history
  • Loading branch information
YannDubs authored Nov 24, 2019
1 parent b60a60d commit 535bbd2
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ All the previous losses are special cases of the following loss:

![Loss Overview](doc/imgs/loss.png)

1. **Index-code mutual information**: the mutual information between the latent variables **z** and the data variable **x**. There is contention in the literature regarding the correct way to treat this term. From the [information bottleneck perspective](https://arxiv.org/abs/1706.01350) this should be penalized. [InfoGAN](https://arxiv.org/abs/1606.03657) get good results by increasing the mutual information (negative α). Finally, [Wassertein Auto-Encoders](https://arxiv.org/abs/1711.01558) drops this term.
1. **Index-code mutual information**: the mutual information between the latent variables **z** and the data variable **x**. There is contention in the literature regarding the correct way to treat this term. From the [information bottleneck perspective](https://arxiv.org/abs/1706.01350) this should be penalized. [InfoGAN](https://arxiv.org/abs/1606.03657) get good results by increasing the mutual information (negative α). Finally, [Wasserstein Auto-Encoders](https://arxiv.org/abs/1711.01558) drops this term.

2. **Total Correlation (TC)**: the KL divergence between the joint and the product of the marginals of the latent variable. *I.e.** a measure of dependence between the latent dimensions. Increasing β forces the model to find statistically independent factors of variation in the data distribution.

Expand Down

0 comments on commit 535bbd2

Please sign in to comment.