Hypersphere Loss, Stability Improvements and Updated Documentation

Hypersphere Loss, Stability Improvements and Updated Documentation

Hypersphere Loss (@EelcoHoogendoorn)

Implemented the loss function described here, which achieves competitive results with more cited ones (symmetric negative cosine similarity & contrastive loss) while providing better interpretability.

You can use the loss in combination with all other losses supported by lightly:

# initialize loss function
loss_fn = HypersphereLoss()

# generate two random transforms of images
t0 = transforms(images)
t1 = transforms(images)

# feed through (e.g. SimSiam) model
out0, out1 = model(t0, t1)

# calculate loss
loss = loss_fn(out0, out1)

Thank you, @EelcoHoogendoorn, for your contribution

Minor Updates and Fixes

Updated documentation and docstrings to make working with lightly simpler.
Minor bug fixes and improvements.

Models

• Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021
• SimSiam: Exploring Simple Siamese Representation Learning, 2020
• MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019
• SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020