Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

Code for our paper Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures.

We study the applicability of Direct Feedback Alignment (DFA) to neural view synthesis, recommender systems, geometric learning, and natural language processing. At variance with common beliefs, we show that challenging tasks can be tackled in the absence of weight transport.

Reproducing our results

• Instructions for reproduction are given within each task folder, in the associated README.md file.

Requirements

• A requirements.txt file is available at the root of this repository, specifying the required packages for all of our experiments;
• Our DFA implementation, TinyDFA, is pip-installable: from the TinyDFA folder, run pip install .;
• tsnecuda may require installation from source: see the tsne-cuda repository for details;
• Neural rendering datasets can be found on the NeRF website—other datasets will be automatically fetched.

Citation

If you found this code and findings useful in your research, please consider citing:

@article{launay2020dfascaling,
  title={Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures},
  author={Launay, Julien and Iacopo, Poli and Francois, Boniface and Krzakala, Florent},
  journal={arXiv preprint arXiv:2006.12878},
  year={2020}
}

About LightOn/LightOn Cloud

LightOn develops a light-based technology required in large scale artificial intelligence computations. Our ambition is to significantly reduce the time and energy required to make sense of the world around us.

Please visit https://cloud.lighton.ai/ for more information on accessing our technology.