We use the code provided by the original paper authors.
In our experiments, we use bert-base-cased instead of bert-base-uncased. We use the same set of hyperparameters as the original work. We replace the default BERT encoder with our RuleTaker pretrained encoder.