LSTM-CRF in PyTorch

A minimal PyTorch implementation of bidirectional LSTM-CRF for sequence labelling.

Supported features:

• Character and/or word embeddings in the input layer
• A PyTorch implementation of conditional random field (CRF)
• Vectorized computation of CRF loss
• Vectorized Viterbi decoding
• Mini-batch training with CUDA

Usage

Training data should be formatted as below:

token/tag token/tag token/tag ...
token/tag token/tag token/tag ...
...

To prepare data:

python prepare.py training_data

To train:

python train.py model char_to_idx word_to_idx tag_to_idx training_data.csv num_epoch

To predict:

python predict.py model.epochN word_to_idx tag_to_idx test_data

References

Zhiheng Huang, Wei Xu, Kai Yu. 2015. Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv:1508.01991.

Xuezhe Ma, Eduard Hovy. 2016. End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF. arXiv:1603.01354.

Shotaro Misawa, Motoki Taniguchi, Yasuhide Miura, Tomoko Ohkuma. 2017. Character-based Bidirectional LSTM-CRF with Words and Characters for Japanese Named Entity Recognition. In Proceedings of the First Workshop on Subword and Character Level Models in NLP.

Yan Shao, Christian Hardmeier, Jörg Tiedemann, Joakim Nivre. 2017. Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF. arXiv:1704.01314.

Slav Petrov, Dipanjan Das, Ryan McDonald. 2011. A Universal Part-of-Speech Tagset. arXiv:1104.2086.

Nils Reimers, Iryna Gurevych. 2017. Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks. arXiv:1707.06799.

Zenan Zhai, Dat Quoc Nguyen, Karin Verspoor. 2018. Comparing CNN and LSTM Character-level Embeddings in BiLSTM-CRF Models for Chemical and Disease Named Entity Recognition. arXiv:1808.08450.