This repository is to collect BERT related resources.

Papers:

1. arXiv:1810.04805, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , Authors: Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova

2. arXiv:1812.06705, Conditional BERT Contextual Augmentation, Authors: Xing Wu, Shangwen Lv, Liangjun Zang, Jizhong Han, Songlin Hu

3. arXiv:1812.03593, SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering, Authors: Chenguang Zhu, Michael Zeng, Xuedong Huang

4. arXiv:1901.02860, Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context, Authors: Zihang Dai, Zhilin Yang, Yiming Yang, William W. Cohen, Jaime Carbonell, Quoc V. Le and Ruslan Salakhutdinov.

5. arXiv:1901.04085, Passage Re-ranking with BERT, Authors: Rodrigo Nogueira, Kyunghyun Cho

6. arXiv:1902.02671, BERT and PALs: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning, Authors: Asa Cooper Stickland, Iain Murray

Github Repositories:

official implement:

1. google-research/bert, officical TensorFlow code and pre-trained models for BERT , [10053 stars]

implement of BERT besides tensorflow:

1. codertimo/BERT-pytorch, Google AI 2018 BERT pytorch implementation

2. huggingface/pytorch-pretrained-BERT, A PyTorch implementation of Google AI's BERT model with script to load Google's pre-trained models , [2422 stars]

3. Separius/BERT-keras, Keras implementation of BERT with pre-trained weights, [325 stars]

4. soskek/bert-chainer, Chainer implementation of "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", [130 stars]

5. innodatalabs/tbert, PyTorch port of BERT ML model

6. guotong1988/BERT-tensorflow, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

7. dreamgonfly/BERT-pytorch, PyTorch implementation of BERT in "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"

8. CyberZHG/keras-bert, Implementation of BERT that could load official pre-trained models for feature extraction and prediction

9. soskek/bert-chainer, Chainer implementation of "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"

10. MaZhiyuanBUAA/bert-tf1.4.0, bert-tf1.4.0

11. dhlee347/pytorchic-bert, Pytorch Implementation of Google BERT, [138 stars]

12. kpot/keras-transformer, Keras library for building (Universal) Transformers, facilitating BERT and GPT models, [17 stars]

13. miroozyx/BERT_with_keras, A Keras version of Google's BERT model, [5 stars]

14. conda-forge/pytorch-pretrained-bert-feedstock, A conda-smithy repository for pytorch-pretrained-bert. , [0 stars]

15. Rshcaroline/BERT_Pytorch_fastNLP, A PyTorch & fastNLP implementation of Google AI's BERT model.

16. PaddlePaddle/LARK, LAnguage Representations Kit, PaddlePaddle implementation of BERT. It also contains an improved version of BERT, ERNIE, for chinese NLP tasks. BERT 的中文改进版 ERNIE , [716 stars]

other resources for BERT:

1. brightmart/bert_language_understanding, Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN, [503 stars]

2. Y1ran/NLP-BERT--ChineseVersion, 谷歌自然语言处理模型BERT：论文解析与python代码 , [83 stars]

3. yangbisheng2009/cn-bert, BERT在中文NLP的应用, 语法检查

4. JayYip/bert-multiple-gpu, A multiple GPU support version of BERT, [16 stars]

5. HighCWu/keras-bert-tpu, Implementation of BERT that could load official pre-trained models for feature extraction and prediction on TPU, [6 stars]

6. Willyoung2017/Bert_Attempt, PyTorch Pretrained Bert, [0 stars]

7. Pydataman/bert_examples, some examples of bert, run_classifier.py 是基于谷歌bert实现了Quora Insincere Questions Classification二分类比赛。run_ner.py是基于瑞金医院AI大赛 第一赛季数据和bert写的一个命名实体识别。

8. guotong1988/BERT-chinese, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 中文 汉语

9. zhongyunuestc/bert_multitask, 多任务task

10. Microsoft/AzureML-BERT, End-to-end walk through for fine-tuning BERT using Azure Machine Learning , [14 stars]

11. bigboNed3/bert_serving, export bert model for serving, [10 stars]

12. yoheikikuta/bert-japanese, BERT with SentencePiece for Japanese text. [135 stars]

13. whqwill/seq2seq-keyphrase-bert, add BERT to encoder part for https://github.com/memray/seq2seq-keyphrase-pytorch, [19 stars]

14. algteam/bert-examples, bert-demo, [9 stars]

15. cedrickchee/awesome-bert-nlp, A curated list of NLP resources focused on BERT, attention mechanism, Transformer networks, and transfer learning. [9 stars]

16. cnfive/cnbert, 中文注释一下bert代码功能, [5 stars]

17. brightmart/bert_customized, bert with customized features, [20 stars]

18. yuanxiaosc/BERT_Paper_Chinese_Translation, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 论文的中文翻译。Chinese Translation! , [5 stars]

19. JayYip/bert-multitask-learning, BERT for Multitask Learning, [29 stars]

20. yuanxiaosc/BERT_Paper_Chinese_Translation, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 论文的中文翻译。Chinese Translation! https://yuanxiaosc.github.io/2018/12/… , [16 stars]

21. yaserkl/BERTvsULMFIT, Comparing Text Classification results using BERT embedding and ULMFIT embedding, [32 stars]

22. kpot/keras-transformer, Keras library for building (Universal) Transformers, facilitating BERT and GPT models, [86 stars]

23. 1234560o/Bert-model-code-interpretation, 解读tensorflow版本Bert中modeling.py数据流

24. cdathuraliya/bert-inference, A helper class for Google BERT (Devlin et al., 2018) to support online prediction and model pipelining.

25. policeme/chinese_bert_similarity, bert chinese similarity

26. zhihu/cuBERT, Fast implementation of BERT inference directly on NVIDIA (CUDA, CUBLAS) and Intel MKL

BERT QA task:

1. benywon/ChineseBert, This is a chinese Bert model specific for question answering, [10 stars]

2. vliu15/BERT, Tensorflow implementation of BERT for QA

3. matthew-z/R-net, R-net in PyTorch, with BERT and ELMo, [77 stars]

4. nyu-dl/dl4marco-bert, Passage Re-ranking with BERT, [92 stars]

5. xzp27/BERT-for-Chinese-Question-Answering, [12 stars]

6. chiayewken/bert-qa, BERT for question answering starting with HotpotQA, [2 stars]

7. krishna-sharma19/SBU-QA, This repository uses pretrain BERT embeddings for transfer learning in QA domain, [7 stars]

8. ankit-ai/BertQA-Attention-on-Steroids, BertQA - Attention on Steroids

BERT classification task:

1. zhpmatrix/Kaggle-Quora-Insincere-Questions-Classification, Kaggle新赛(baseline)-基于BERT的fine-tuning方案+基于tensor2tensor的Transformer Encoder方案

2. maksna/bert-fine-tuning-for-chinese-multiclass-classification, use google pre-training model bert to fine-tuning for the chinese multiclass classification

3. NLPScott/bert-Chinese-classification-task, bert中文分类实践, [51 stars]

4. Socialbird-AILab/BERT-Classification-Tutorial, [151 stars]

5. fooSynaptic/BERT_classifer_trial, BERT trial for chinese corpus classfication

6. xiaopingzhong/bert-finetune-for-classfier, 微调BERT模型,同时构建自己的数据集实现分类

7. brightmart/sentiment_analysis_fine_grain, Multi-label Classification with BERT; Fine Grained Sentiment Analysis from AI challenger, [268 stars]

8. pengming617/bert_classification, 利用bert预训练的中文模型进行文本分类, [15 stars]

9. xieyufei1993/Bert-Pytorch-Chinese-TextClassification, Pytorch Bert Finetune in Chinese Text Classification, [7 stars]

10. liyibo/text-classification-demos, Neural models for Text Classification in Tensorflow, such as cnn, dpcnn, fasttext, bert ..., [6 stars]

11. circlePi/BERT_Chinese_Text_Class_By_pytorch, A Pytorch implements of Chinese text class based on BERT_Pretrained_Model, [3 stars]

12. kaushaltrivedi/bert-toxic-comments-multilabel, Multilabel classification for Toxic comments challenge using Bert, [45 stars]

13. lonePatient/BERT-chinese-text-classification-pytorch, This repo contains a PyTorch implementation of a pretrained BERT model for text classification., [6 stars]

14. lynnna-xu/bert_sa, bert sentiment analysis tensorflow serving with RESTful API

BERT NER task:

1. JamesGu14/BERT-NER-CLI, Bert NER command line tester with step by step setup guide, [20 stars]

2. zhpmatrix/bert-sequence-tagging, 基于BERT的中文序列标注

3. kyzhouhzau/BERT-NER, Use google BERT to do CoNLL-2003 NER ! , [285 stars]

4. king-menin/ner-bert, NER task solution (bert-Bi-LSTM-CRF) with google bert https://github.com/google-research.

5. macanv/BERT-BiLSMT-CRF-NER, Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning , [349 stars]

6. FuYanzhe2/Name-Entity-Recognition, Lstm-crf,Lattice-CRF,bert-ner及近年ner相关论文follow, [64 stars]

7. mhcao916/NER_Based_on_BERT, this project is based on google bert model, which is a Chinese NER

8. ProHiryu/bert-chinese-ner, 使用预训练语言模型BERT做中文NER, [177 stars]

9. sberbank-ai/ner-bert, BERT-NER (nert-bert) with google bert, [116 stars]

10. kyzhouhzau/Bert-BiLSTM-CRF, This model base on bert-as-service. Model structure : bert-embedding bilstm crf. , [3 stars]

11. Hoiy/berserker, Berserker - BERt chineSE woRd toKenizER, Berserker (BERt chineSE woRd toKenizER) is a Chinese tokenizer built on top of Google's BERT model. , [2 stars]

12. Kyubyong/bert_ner, Ner with Bert, [95 stars]

13. jiangpinglei/BERT_ChineseWordSegment, A Chinese word segment model based on BERT, F1-Score 97%, [7 stars]

14. yanwii/ChineseNER, 基于Bi-GRU + CRF 的中文机构名、人名识别 中文实体识别, 支持google bert模型

15. lemonhu/NER-BERT-pytorch, PyTorch solution of NER task Using Google AI's pre-trained BERT model.

BERT Text Generation Task:

1. asyml/texar, Toolkit for Text Generation and Beyond https://texar.io, Texar is a general-purpose text generation toolkit, has also implemented BERT here for classification, and text generation applications by combining with Texar's other modules. [892 stars]

BERT Knowledge Graph Task :

1. lvjianxin/Knowledge-extraction, 基于中文的知识抽取，BaseLine：Bi-LSTM+CRF 升级版：Bert预训练

2. sakuranew/BERT-AttributeExtraction, USING BERT FOR Attribute Extraction in KnowledgeGraph. fine-tuning and feature extraction. 使用基于bert的微调和特征提取方法来进行知识图谱百度百科人物词条属性抽取。 [31 stars]

3. aditya-AI/Information-Retrieval-System-using-BERT, [7 stars]

4. jkszw2014/bert-kbqa-NLPCC2017, A trial of kbqa based on bert for NLPCC2016/2017 Task 5 (基于BERT的中文知识库问答实践）

BERT visualization toolkit:

1. jessevig/bertviz, Tool for visualizing BERT's attention, [339 stars]

BERT chatbot :

1. GaoQ1/rasa_nlu_gq, turn natural language into structured data(支持中文，自定义了N种模型，支持不同的场景和任务), [48 stars]

2. GaoQ1/rasa_chatbot_cn, 基于rasa-nlu和rasa-core 搭建的对话系统demo, [91 stars]

3. GaoQ1/rasa-bert-finetune, 支持rasa-nlu 的bert finetune, [5 stars]

4. akikaaa/wxbot, chatbot on wechat. qa ranking model is based on google bert

5. geodge831012/bert_robot, 用于智能助手回答问题的训练，基于BERT模型进行训练改造

BERT language model and embedding:

1. hanxiao/bert-as-service, Mapping a variable-length sentence to a fixed-length vector using pretrained BERT model, [1941 stars]

2. YC-wind/embedding_study, 中文预训练模型生成字向量学习，测试BERT，ELMO的中文效果, [17 stars]

3. Kyubyong/bert-token-embeddings, Bert Pretrained Token Embeddings, [61 stars]

4. xu-song/bert_as_language_model, bert as language model, fork from https://github.com/google-research/bert, [27 stars]

5. yuanxiaosc/Deep_dynamic_word_representation, TensorFlow code and pre-trained models for deep dynamic word representation (DDWR). It combines the BERT model and ELMo's deep context word representation., [7 stars]

6. imgarylai/bert-embedding, Token level embeddings from BERT model on mxnet and gluonnlp http://bert-embedding.readthedocs.io/, [133 stars]

7. terrifyzhao/bert-utils, BERT生成句向量，BERT做文本分类、文本相似度计算, [39 stars]

8. fennuDetudou/BERT_implement, 使用BERT模型进行文本分类，相似句子判断，以及词性标注, [9 stars]

9. whqwill/seq2seq-keyphrase-bert, add BERT to encoder part for https://github.com/memray/seq2seq-keyphrase-pytorch, [30 stars]

10. charles9n/bert-sklearn, a sklearn wrapper for Google's BERT model, [14 stars]

11. allenai/allennlp-bert-qa-wrapper, This is a simple wrapper on top of pretrained BERT based QA models from pytorch-pretrained-bert to make AllenNLP model archives, so that you can serve demos from AllenNLP.

BERT tutorials:

1. graykode/nlp-tutorial, Natural Language Processing Tutorial for Deep Learning Researchers https://www.reddit.com/r/MachineLearn…, [1779 stars]