Machine Learning Papers Summary

Docsify Book: https://gitycc.github.io/machine-learning-papers-summary
Github Repo: https://github.com/GitYCC/machine-learning-papers-summary

Paper Survey

• terryum/awesome-deep-learning-papers
• floodsung/Deep-Learning-Papers-Reading-Roadmap
• mhagiwara/100-nlp-papers
• thunlp/GNNPapers

Content

Understanding / Generalization / Transfer

• Rethinking Pre-training and Self-training (2020), Barret Zoph et al. [➤ summary]
• Distilling the knowledge in a neural network (2015), G. Hinton et al. [➤ summary]
• Deep neural networks are easily fooled: High confidence predictions for unrecognizable images (2015), A. Nguyen et al. [➤ summary]
• How transferable are features in deep neural networks? (2014), J. Yosinski et al. [➤ summary]

Optimization / Training Techniques

• On the Variance of the Adaptive Learning Rate and Beyond (2020), Michael Liyuan Liu et al. [➤ summary]
• Lookahead Optimizer: k steps forward, 1 step back (2019), Michael R. Zhang et al. [➤ summary]
• Decoupled Weight Decay Regularization (2019), Ilya Loshchilov et al. [➤ summary]
• On the Convergence of Adam and Beyond (2018), Sashank J. Reddi et al. [➤ summary]
• Large Batch Training of Convolutional Networks (2017), Yang You. [➤ summary]
• An overview of gradient descent optimization algorithms (2017), S. Ruder. [➤ summary]

Unsupervised

Generative Models

• Glow: Generative Flow with Invertible 1×1 Convolutions (2018), D. P. Kingma et al. [➤ summary]
• Density Estimation Using Real NVP (2017), J. Sohl-Dickstein et al. [➤ summary]

Computer Vision

• Masked Autoencoders Are Scalable Vision Learners (2021), Kaiming He et al. [➤ summary]
• LAMBERT: Layout-Aware Language Modeling for Information Extraction (2021), Lukasz Garncarek et al. [➤ summary]
• Pay Attention to MLPs (2021), Hanxiao Liu et al. [➤ summary]
• MLP-Mixer: An all-MLP Architecture for Vision (2021), Ilya Tolstikhin et al. [➤ summary]
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021), Ze Liu et al. [➤ summary]
• Training data-efficient image transformers & distillation through attention (2020), Hugo Touvron et al. [➤ summary]
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2020), Alexey Dosovitskiy et al. [➤ summary]
• YOLOv4: Optimal Speed and Accuracy of Object Detection (2020), A. Bochkovskiy et al. [➤ summary]
• EfficientDet: Scalable and Efficient Object Detection (2020), Mingxing Tan et al. [➤ summary]
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (2020), Mingxing Tan et al. [➤ summary]
• ArcFace: Additive Angular Margin Loss for Deep Face Recognition (2019), Jiankang Deng et al. [➤ summary]
• MnasNet: Platform-Aware Neural Architecture Search for Mobile (2019), Mingxing Tan et al. [➤ summary]
• What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis (2019), Jeonghun Baek et al. [➤ summary]
• Character Region Awareness for Text Detection (2019), Youngmin Baek et al. [➤ summary]
• Mask R-CNN (2018), Kaiming He et al. [➤ summary]
• YOLOv3: An Incremental Improvement (2018), Joseph Redmon et al. [➤ summary]
• YOLO9000: Better, Faster, Stronger (2016), Joseph Redmon et al. [➤ summary]
• You Only Look Once: Unified, Real-Time Object Detection (2016), Joseph Redmon et al. [➤ summary]
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2016), Shaoqing Ren et al. [➤ summary]
• Fast R-CNN (2015), Ross Girshick. [➤ summary]
• An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition (2015), Baoguang Shi et al. [➤ summary]
• Rich feature hierarchies for accurate object detection and semantic segmentation (2014), Ross Girshick et al. [➤ summary]

Advertising / Commerce

• DeepFM: A Factorization-Machine based Neural Network for CTR Prediction (2017), H. Guo et al. [➤ summary]

Natural Language Processing

• Efficient Transformers: A Survey (2020), Yi Tay et al. [➤ summary]
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2020), Colin Raffel et al. [➤ summary]
• Language Models are Few-Shot Learners (2020), Tom B. Brown et al. [➤ summary]
• ALBERT: A Lite BERT for Self-Supervised Learning of Language Representations (2020), Zhenzhong Lan et al. [➤ summary]
• XLNet: Generalized Autoregressive Pretraining for Language Understanding (2020), Zhilin Yang et al. [➤ summary]
• RoBERTa: A Robustly Optimized BERT Pretraining Approach (2019), Yinhan Liu et al. [➤ summary]
• Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context (2019), Zihang Dai et al. [➤ summary]
• Language Models are Unsupervised Multitask Learners (2019), Alec Radford et al. [➤ summary]
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019), Jacob Devlin et al. [➤ summary]
• Improving Language Understanding by Generative Pre-Training (2018), Alec Radford et al. [➤ summary]
• Deep contextualized word representations (2018), C. Clark et al. [➤ summary]
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension (2018), Adams Wei Yu et al. [➤ summary]
• Bi-Directional Attention Flow for Machine Comprehension (2017), Minjoon Seo et al. [➤ summary]
• Attention Is All You Need (2017), A. Vaswani et al. [➤ summary]
• Reading Wikipedia to Answer Open-Domain Questions (2017), Danqi Chen et al. [➤ summary]
• Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling (2016), Bing Liu et al. [➤ summary]
• Neural Machine Translation by Jointly Learning to Align and Translate (2016), D. Bahdanau et al. [➤ summary]
• Convolutional Neural Networks for Sentence Classification (2014), Yoon Kim. [➤ summary]
• Sequence to Sequence Learning with Neural Networks (2014), I. Sutskever et al. [➤ summary]
• A Fast and Accurate Dependency Parser using Neural Networks (2014), Chen and Manning. [➤ summary]
• Glove: Global vectors for word representation (2014), J. Pennington et al. [➤ summary]
• Distributed Representations of Sentences and Documents (2014), Q. Le and T. Mikolov et al. [➤ summary]
• Distributed representations of words and phrases and their compositionality (2013), T. Mikolov et al. [➤ summary]
• Efficient estimation of word representations in vector space (2013), T. Mikolov et al. [➤ summary]

Graph Neural Network

• SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check (2020), Xingyi Cheng et al. [➤ summary]
• Handling Missing Data with Graph Representation Learning (2020), Jiaxuan You et al. [➤ summary]
• Graph Neural Networks: A Review of Methods and Applications (2018), Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Maosong Sun. [➤ summary]
• How Powerful Are Graph Neural Networks? (2018), K. Xu et al. [➤ summary]
• Inductive Representation Learning on Large Graphs (2018), William L. Hamilton et al. [➤ summary]
• Semi-supervised Classification with Graph Convolutional Networks (2017), Thomas N. Kipf et al. [➤ summary]
• DeepWalk: Online Learning of Social Representations (2014), B. Perozzi et al. [➤ summary]

Speech

• WaveGlow: A Flow-based Generative Network for Speech Synthesis (2018), R. Prenger et al. [➤ summary]
• Natural TTS Synthesis by Conditioning WaveNet On Mel Spectrogram Predictions (2018), J. Shen et al. [➤ summary]
• Tacotron: Towards End-to-end Speech Synthesis (2017), Y. Wang et al. [➤ summary]
• WaveNet: A Generative Model For Raw Audio (2016), van den Oord, et al. [➤ summary]
• Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks (2006), Alex Graves et al. [➤ summary]

Reinforcement Learning / Robotics

Time Series

• Neural Ordinary Differential Equations (2019), Ricky T. Q. Chen et al. [➤ summary]

Contributors

• @GitYCC [website: www.ycc.idv.tw, email: [email protected]]

Welcome to contribute this repo. together.

Contribution Rules

• Please use pull-request to merge new changes in feature/xxx branch into master branch
• Add new article
  • In README.md, choose or create a category, add new paper item (sorted by years decreasing) and add link [➤ summary] to your new article.
  • In summary.md, add path of your article into sidebar.
  • Please follow this file structure: (ref: https://docsify.now.sh)

    README.md
    summary.md
    some-category ---- assets
                   |     |--- some-image-1
                   |     |--- your-image-2
                   |-- your-article.md
    

  • In your-article.md, add contributor, paper or code at head of the article.