A curated list of awesome Capsule Network works
- Capsule Networks – A survey, Menash et al., JKSU 2022 | bibtex
- Capsule networks for image classification: A review, Pawan et al., Neurocomputing 2022 | bibtex
- What happened with Capsule Neural Networks?, Endless Origins 2022.
- Google’s AI Wizard Unveils a New Twist on Neural Networks, Wired 2017.
- Capsule networks: The next generation of deep learning architecture, Dmitry Ermakov, 2023.
Papers by Hinton et al.
- Matrix capsules with EM routing, Hiton et al., ICLR 2018 | bibtex
- Dynamic routing between capsules, Sabour et al., NIPS 2017 | bibtex
- Transforming Auto-encoders, Hinton et al., ICANN 2011 | bibtex
- Learning to Parse Images, Hinton et al., NIPS 1999 | bibtex
- TRAFFIC: Recognizing objects using hierarchical reference frame transformations, Richard et al., NIPS 1989 | bibtex
- Stacked Capsule Autoencoders, Adam Kosiorek et al., NIPS 2019 | bibtex
- Detecting and diagnosing adversarial images with class-conditional capsule reconstructions, Yao Qin et al., ICLR 2020 | bibtex
- Canonical capsules: Self-supervised capsules in canonical pose, Weiwei Sun et al., NIPS 2021 | bibtex
- Unsupervised part representation by flow capsules, Sabour et al., PMLR 2021 | bibtex
- Darccc: Detecting adversaries by reconstruction from class conditional capsules, Nicholas Frosst et al., | bibtex
Architecture
- PT-CapsNet: A novel prediction-tuning capsule network suitable for deeper architectures, Pan et al., ICCV 2021 | bibtex
- MS-CapsNet: A Novel Multi-Scale Capsule Network, Xiang et al., IEEE 2018 | bibtex
- DeepCaps: Going Deeper with Capsule Networks, Rajasegaran et al., CVPR 2019 | bibtex
- Two-phase Dynamic Routing for Micro and Macro-level Equivariance in Multi-Column Capsule Networks, Bodhisatwa et al., Pattern Recognition 2021 | bibtex
Routing
- Matrix capsules with EM routing, Hiton et al., ICLR 2018 | bibtex
- Dynamic routing between capsules, Sabour et al., NIPS 2017 | bibtex
- Capsule network with shortcut routing, Vu et al., IEICE 2021 | bibtex
- Path capsule networks, Mohammed Amer et al., Neural Processing Letters 2020 | bibtex
- Fast dynamic routing based on weighted kernel density estimation, Zhang et al., International symposium on artificial intelligence and robotics 2018 | bibtex
- Self-routing capsule networks, Hahn et al., NIPS 2019 | bibtex
- Capsules with Inverted Dot-Product Attention Routing, Tsai et al., ICLR 2019 | bibtex
- Hitnet: a neural network with capsules embedded in a hit-or-miss layer, extended with hybrid data augmentation and ghost capsules, Adrien et al., 2018 | bibtex
- Capsule Networks with Max-Min Normalization, Zhao et al., 2019 | bibtex
- Generalized capsule networks with trainable routing procedure, Chen et al., 2018 | bibtex
- Neural network encapsulation, Li et al., ECCV 2018 | bibtex
- Capsule routing via variational bayes, Ribeiro et al., AAAI 2020 | bibtex
- CapProNet: Deep Feature Learning via Orthogonal Projections onto Capsule Subspaces, Zhang et al., NIPS 2018 | bibtex
- Group Equivariant Capsule Networks, Lenssen et al., NIPS 2018 | bibtex
- Star-caps: Capsule networks with straight-through attentive routing, Ahmed et al., NIPS 2019
- Efficient-CapsNet: capsule network with self-attention routing, Vittorio et al., Scientific Reports 2021 | bibtex
Notable
- An Optimization View on Dynamic Routing Between Capsules , Wang et al., ICLR 2018 Workshop | bibtex
- Interpretable graph capsule networks for object recognition, Gu et al., AAAI 2021 | bibtex
- Sparse Unsupervised Capsules Generalize Better, David et al., Preprint 2018 | bibtex
- Improved explainability of capsule networks: Relevance path by agreement, Atefeh et al., IEEE 2018 | bibtex
- Introducing routing uncertainty in capsule networks, Ribeiro et al., NIPS 2020 | bibtex
- Conditional variational capsule network for open set recognition, ICCV 2021 | bibtex
- Deep convolutional inverse graphics network, Kulkarni et al., NIPS 2015 | bibtex
Blogs
- Capsule neural network, Wikipedia.
- Understanding Hinton’s Capsule Networks series, Max Pechyonkin, Medium 2017.
- Uncovering the Intuition behind Capsule Networks and Inverse Graphic, Tanay Kothari, Hackernoon 2017.
- A Visual Representation of Capsule Connections in Dynamic Routing Between Capsules, Mike Ross, Medium 2017.
- What is a CapsNet or Capsule Network? - Debarko De, Hackernoon 2017.
- Capsule Networks Are Shaking up AI — Here’s How to Use Them, Nick Bourdakos, Hackernoon 2017.
- Understanding Capsule Networks — AI’s Alluring New Architecture - Nick Bourdakos, Medium 2018.
- Capsule Networks Explained, Kendrick Tan.
- Understanding Dynamic Routing between Capsules (Capsule Networks), Jonathan Hui, Blog 2017.
- Matrix capsules with EM routing, Adrian Colyer, Blog 2017.
- Capsule Networks: A Quick Primer, Vihar Kurama, Paperspace 2020.
- Capsule Networks: The New Deep Learning Network, Aryan Misra, Towardsdatascience 2019.
Lectures
- Capsule Networks (CapsNets) – Tutorial, Aurélien Géron, Youtube 2017.
- How to implement CapsNets using TensorFlow, Aurélien Géron, Youtube 2017.
- Capsule network explained, Count From Zero, Youtube 2021
- Geoffrey Hinton Capsule theory, 2017.
- Geoffrey Hinton – Capsule Networks, 2018.
- Capsule Networks for Computer Vision, CVPR 2019.
- Does the Brain do Inverse Graphics?, Hinton, Brain and Cognitive Sciences Fall Colloquium 2012.
| Reseach | Main results | Innovation |
|---|---|---|
| PathCapsNet | Mnist: 99.65 | fan-in routing technique, deep parallel multi-path |
| Fast Dynamic Routing | smallNORB: 97.4 MNIST: 99.58 FMnist: 94 CIFAR10: 84.6 |
weighted kernel density estimation |
| Spiking CapsNet | MNIST: 99.17 FMnist: 91.07 |
Spiking Neural Networks |
| Self-attention CapsNet | CIFAR-10: 92.14 SVHN: 96.88 SmallNORB: 92.38 |
Self-attention routing |
| CVAECapOSR | TinyImageNet: 71.5 CIFAR10: 83.5 Mnist: 99.2 SVHN: 95.6 |
Conditional Variational |
| HitNet | Mnist: 99.68 FMnist: 92.3 CIFAR10: 73.3 SVHN: 94.5 affNIST: 83.03 |
Hit-Miss layer, Ghost Capsule |
| Max-Min routing | Mnist: 99.55 FMnist: 92.07 CIFAR10: 75.92 |
replace Softmax with Max-Min Normalization |
| Two-phase Dynamic Routing | SVHN: 90.19 FMnist: 90.96 CIFAR10: 75.82 |
Micro and Macro-level routing |
| Inverted dot-product | CIFAR10: 85.17 CIFAR100: 57.32 |
inverted dot-product attention routing |
| G-CapsNet | Mnist: 99.34 | trainable routing procedure |
| Efficient-CapsNet | Mnist: 99.74 smallNORB: 97.66 MultiMNIST: 88.75 |
Self-attention routing |
| Res-CapsNet | Mnist: 99.4 FMnist: 89.2 SVHN: 92.4 SmallNORB: 90.3 |
residual connections |
| DCNet | Mnist: 99.75 SVHN: 96.90 CIFAR10: 89.32 SmallNORB: 95.27 |
Dense connections |
| DeepCaps | CIFAR10: 92.74 SVHN: 97.56 FMnist: 94.73 |
3D Capsule convolutions |
| NASCaps | CIFAR10: 76.46 Mnist: 99.7 FMnist: 93.87 SVHN: 96.59 |
Neural Architecture Search for CapsNet |
| EncapNet | CIFAR10: 95.45 CIFAR100: 73.33 SVHN: 97.99 h-ImageNet: 59.95 |
master branch for primary information and an aide branch for pattern |
| PT-CapsNet | CIFAR10: 95.71 CIFAR100: 78.36 FMnist: 95.99 ISIC2018: 83.12 VOC: 78.2 |
more difficult vision tasks |
| DeeperCaps | CIFAR10: 81.29 smallNORB: 91.75 Mnist: 99.84 |
capsule pool |
| MT-CapsNet | CIFAR10: 92.96 FMNIST: 94.25 |
The Multi-Lane Capsule Network |
| MS-CapsNet | FMNist: 92.7 CIFAR10: 75.7 |
multi-scale feature extraction |
| DE-CapsNet | CIFAR10: 92.96 FMNIST: 94.25 |
Spatial Group-wise Enhance mechanism |
| VB-Caps | smallNORB: 98.4 FMNIST: 94.8 SVHN: 96.1 CIFAR10: 89.8 |
Variational Bayes |
| Capsule-VAE | smallNORB: 96.3 affNIST: 94.08 Mnist: 99.02 SVHN: 94.02 |
Spatial Group-wise Enhance mechanism |
| SparseCaps | affNist: 90.12 Mnist: 99 |
Unsupervised sparsening of latent capsule layer |
| GraCapsNets | Mnist: 99.50 FMnist: 93.1 CIFAR10: 82.21 |
Multi-head attention-based Graph Pooling approach incorporates built-in explanation |
| STAR-Caps | MNIST: 99.49 SmallNORB: 95.72 CIFAR10: 91.23 CIFAR100: 67.66 |
straight-through attentive routing, differentiable binary routers |
| Group-Caps | Mnist: 98.42 AffNist: 89.1 |
group equivariant capsule network |
| Em routing | smallNORB: 98.2 Mnist: 99.54 CIFAR10: 88.1 |
EM-based routing mixture coefficients |
| Dynamic routing | Mnist: 99.75 MultiMNIST: 94.8 CIFAR10: 89.4 smallNORB: 97.3 SVHN: 95.7 |
Cosine based routing coefficient |
| shortcut routing | Mnist: 99.57 affNist: 89.02 smallNorb: 94.77 FNist: 92.18 |
shortcut connection and fuzzy routing |
- naturomics/CapsNet-Tensorflow
- bourdakos1/capsule-networks
- JunYeopLee/capsule-networks
- jaesik817/adv_attack_capsnet
- thibo73800/capsnet-traffic-sign-classifier
- XifengGuo/CapsNet-Keras
- gusgad/capsule-GAN
- gyang274/capsulesEM
- www0wwwjs1/Matrix-Capsules-EM-Tensorflow
- gram-ai/capsule-networks
- higgsfield/Capsule-Network-Tutorial
- danielhavir/capsule-network
- Ka0Ri/Capsule-Network
- shzygmyx/Matrix-Capsules-pytorch
MIT