widis-lstm-tools v0.4

Various Tools for working with Long Short-Term Memory (LSTM) networks and sequences in Pytorch, aimed at getting your LSTM networks under control and providing a flexible template for your LSTM-needs.

LSTM paper (not earliest): https://www.mitpressjournals.org/doi/abs/10.1162/neco.1997.9.8.1735

Quickstart

You can directly install the package from GitHub using the command below:

pip install git+https://github.com/widmi/widis-lstm-tools

To update your installation with dependencies, you can use:

pip install --upgrade git+https://github.com/widmi/widis-lstm-tools

To update your installation without dependencies, you can use:

pip install --no-dependencies --upgrade git+https://github.com/widmi/widis-lstm-tools

Run the simple example via

python3 widis_lstm_tools/examples/basic/main.py config.json

Run the more advanced example via

python3 widis_lstm_tools/examples/model_selection/main.py config.json

Includes

• Flexible LSTMCell and LSTMLayer implementation including
  • Easy access to individual forward and recurrent LSTM connections and biases, with options to cut/modify individual connections to gates or cell input (see e.g. https://arxiv.org/abs/1503.04069 for some useful modifications)
  • Plotting function for LSTM internal states
  • Support for Ticker/Tinker-Steps at the end of the sequence (https://arxiv.org/abs/1603.08983)
  • Automatic optimizations via TorchScript (use classes LSTMCellTorchScript or LSTMLayerTorchScript for TorchScript optimization. Loop optimization might cause issues with gradient - use only for inference.)
• Preprocessing tools
  • Padding of different sequence lengths
  • Encoding for nicely feeding float or integer numbers as inputs to the LSTM
• Other utilities
  • Printing to log-file and console
  • Splitting PyTorch datasets, e.g. into training-, validation-, testset
  • One-Hot encoding of n-dimensional arrays
  • SaverLoader class for saving/loading the most recent models to files or RAM objects
• Documented examples
  • Basic LSTM example with variable sequence lenghts
  • Advanced LSTM example with input encoding and model selection

Requirements

• Python3.6 or higher
• Python packages:
  • Pytorch (tested with version 1.1.0)
  • numpy (tested with version 1.16.2)
  • matplotlib (tested with version 3.0.3)

I wish you the best for your work with LSTM!

-- Michael W. (widi)