Skip to content

LiBai v0.2.0 Release
Compare
Choose a tag to compare
@CPFLAME CPFLAME released this 08 Jul 03:06
· 134 commits to main since this release
v0.2.0
8fd79ee
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

v0.2.0 (07/07/2022)

New Features:

  • Support evaluation enabled and set eval_iter
  • Support customized sampler in config.py
  • Support rdma for pipeline-model-parallel
  • Support multi fused kernel
    • fused_scale_mask_softmax_dropout
    • fused_scale_tril_softmax_mask_scale
    • fused_self_attention in branch libai_bench
  • User Experience Optimization
  • Optimization for training throughput, see benchmark for more details

New Supported Models:

  • Support 3D parallel Roberta model
  • Support 2D parallel (data parallel + tensor model parallel) SimCSE model
  • Support Data parallel MAE model
  • Support Data parallel MOCOV3 model
Assets 2
Loading