v0.4.9

What's Changed

• Update LitServe version and tests by @rasbt in #1654
• Support for using large models in the Python API via sequential generation by @rasbt in #1637
• Add a PyTorch Lightning example by @rasbt in #1656
• Refactor Python API to introduce new distribute method (part of a larger refactor for PTL support) by @rasbt in #1657
• Fix some issues with circular and relative imports by @rasbt in #1658
• Optionally return benchmark info in Python API by @rasbt in #1660
• Bumb version for 0.4.9 release by @rasbt in #1664

Full Changelog: v0.4.8...v0.4.9

v0.4.8

What's Changed

• Adds unit test to test for parity between streaming and non-streaming API by @rasbt in #1650
• Add Gemma 2 2B by @rasbt in #1651
• Pin litserve version by @rasbt in #1652
• Version bumb for Gemma 2 2B release by @rasbt in #1653

Full Changelog: v0.4.7...v0.4.8

v0.4.7

What's Changed

• Apply prompt style for tp.py and sequentially.py by @Andrei-Aksionov in #1629
• Fix prompt docstring in Python API by @rasbt in #1635
• Update windows cpu-tests.yml by @rasbt in #1630
• Remove NumPy < 2.0 pin by @rasbt in #1631
• Fix kv-cache issue in Python API streaming mode by @rasbt in #1633
• Updates installation requirements to install minimal required packages for basic use by @rasbt in #1634
• Faster safetensors conversion when downloading model by @awaelchli in #1624
• Add Sebastian as code owner by @awaelchli in #1641
• Add missing super() call in data modules by @awaelchli in #1639
• Update Lightning version to 2.4.0 pre by @awaelchli in #1640
• Add tunable kvcache with error handling for nonsense inputs. by @apaz-cli in #1636
• Use Python API in serve code by @rasbt in #1644
• Fix autodownload + conversion issue by @rasbt in #1645
• Properly clear kv-cache by @rasbt in #1647
• Fix error raising where max_returned_tokens > max_seq_length_setting by @rasbt in #1648
• Add quantization support to litgpt serve by @rasbt in #1646
• Bump for 0.4.7 release by @rasbt in #1649

Full Changelog: v0.4.6...v0.4.7

v0.4.6

What's Changed

• Change default top_k to 50 everywhere for consistency by @rasbt in #1592
• Fix kv-cache clearing in Python API and Serve by @rasbt in #1596
• dynamic KV Cache batching by @aniketmaurya in #1600
• Remove non-used eos_id in Python API by @rasbt in #1594
• Add quantization test and revert lightning version by @rasbt in #1605
• Dynamically set kv-cache size in serve by @rasbt in #1602
• Update LitData version and restore previous LitData assertions in tests by @awaelchli in #1609
• Gemma 2: 9b and 27b versions by @Andrei-Aksionov in #1545
• Update config hub table qlora sections by @rasbt in #1611
• max_returned_tokens -> max_new_tokens by @rasbt in #1612
• Add warning about pretrain preprocessing by @rasbt in #1618
• Print warning about unsupported repo_ids by @rasbt in #1617
• Restore capability to load alternative weights by @rasbt in #1620
• Enable unbalanced number of layers in sequential generation by @awaelchli in #1623
• Llama 3.1 8B and 70B checkpoints by @rasbt in #1619
• Add Llama 3.1 405B config by @awaelchli in #1622
• Bumb version to 0.4.6 for next release (Gemma 2 and Llama 3.1) by @rasbt in #1626

Full Changelog: v0.4.5...v0.4.6

v0.4.5

What's Changed

• Update LitData to latest version 0.2.16 by @awaelchli in #1573
• Fixed warning during model download and conversion. by @bhargavyagnik in #1574
• Bump lightning version by @awaelchli in #1579
• chat/base.py: extend checkpoint_dir before accessing it by @Andrei-Aksionov in #1575
• Chat: dynamically set kv-cache size by @Andrei-Aksionov in #1583
• Add slow interconnect warning by @rasbt in #1577
• Mathstral checkpoints by @rasbt in #1587
• Make sure quantization is used correctly by @rasbt in #1585
• Improve mathstral tests by @rasbt in #1589
• Fixes an issue where the LitGPT Python API was consuming too much memory by @rasbt in #1590
• Increase version number for 0.4.5 release by @rasbt in #1591

New Contributors

• @bhargavyagnik made their first contribution in #1574

Full Changelog: v0.4.4...v0.4.5

v0.4.4

What's Changed

• Tokenizer: fix issue when decoding a single token at a time by @Andrei-Aksionov in #1559
• Filter out false positive PyTorch scheduler warning by @rasbt in #1562
• Add an explanation on how to evaluate on a custom test set by @rasbt in #1564
• Remove duplicated bos_token for CodeLlama by @alealv in #1566
• Fix SFTDataset sample generation by @alealv in #1568
• Add Python API section to 0 to LitGPT docs by @rasbt in #1570
• Fix multi-turn prompting error handling and extra newline by @apaz-cli in #1569
• Add automatic downloading to CLI by @rasbt in #1571
• Bumb version for 0.4.4 release by @rasbt in #1572

New Contributors

• @alealv made their first contribution in #1566
• @apaz-cli made their first contribution in #1569

Full Changelog: v0.4.3...v0.4.4

v0.4.3

What's Changed

• Fix Twine Build Issue by @rasbt in #1530
• Fix Mixtral prompt style by @janEbert in #1531
• Raise warning when loading a large model on a CPU device by @rasbt in #1532
• Tutorial convert_from_litgpt doc fix to remove output_dir command argument by @jentfoo in #1533
• Test against eager attention on GPTNeoX by @rasbt in #1537
• Bump to 0.4.3.dev0 version by @rasbt in #1536
• Fix for LM Eval harness 0.4.3 by @rasbt in #1542
• Add Phi-3-mini-4k-instruct checkpoint by @rasbt in #1341
• Update phi-3 prompt template by @rasbt in #1544
• Add progress bar for weight loading by @rasbt in #1534
• Remove redundant code in checkpoint conversion script by @rasbt in #1547
• Update how evaluate tasks are listed by @rasbt in #1550
• Fix generate issue in finetuning script when finetuning with small datasets by @rasbt in #1551
• Tokenizer: check that bos and eos tokens are applied w.r.t. to a provided config/args by @Andrei-Aksionov in #1527
• Add num_nodes argument by @janEbert in #1546
• Add Phi-3 Configs by @rasbt in #1553
• Update to version 0.4.3 for new release by @rasbt in #1554

New Contributors

• @janEbert made their first contribution in #1531
• @jentfoo made their first contribution in #1533

Full Changelog: v0.4.2...v0.4.3

v0.4.2

What's Changed

• Disable "validating ..." comment if initial_validation is False by @rasbt in #1499
• Add simple issue templates by @rasbt in #1501
• Add note about dependencies by @rasbt in #1502
• Update test_readme.py by @Andrei-Aksionov in #1505
• Print message when unnecessary can't be deleted by @rasbt in #1507
• Bumb version by @rasbt in #1509
• LitGPT Python API v1 by @rasbt in #1463
• Fix pip upgrade command on windows by @rasbt in #1513
• add LLM.generate with streaming by @aniketmaurya in #1511
• Fix and update Python API docs by @rasbt in #1514
• Update LLM import and make stream/non-stream option consistent by @rasbt in #1515
• Extend Python API to initialize model with random weights by @rasbt in #1512
• Add Python API download option by @rasbt in #1516
• Add Rope group size as constexpr to fix thunder tests by @rasbt in #1520
• Explain how to access pretraining dataset options by @rasbt in #1510
• Bump thunder by @rasbt in #1519
• Add automatic checkpoint downloading in Python API by @rasbt in #1521
• Tokenizer: prefer HF Tokenizer by @Andrei-Aksionov in #1518
• iterate response from streaming server by @aniketmaurya in #1523
• Add new python API to readme by @williamFalcon in #1525
• Remove init="local" option by @rasbt in #1524
• Readme improvements by @rasbt in #1526
• Add evaluate to Readme by @rasbt in #1528

Full Changelog: v0.4.1...v0.4.2

v0.4.1

What's Changed

• Fix typos in Download Model Weights documentation by @rasbt in #1477
• 'Add models' docs by @rasbt in #1478
• Exclude prompt from generated response by @rasbt in #1485
• Make litgpt serve consistent with LitServe by @rasbt in #1466
• add multi turn prompt by @aniketmaurya in #1487
• Add docs explaining the streaming mode by @rasbt in #1484
• Fix Mistral 7B v0.3 weight download by @davmacario in #1486
• Fixes for latest lm_eval by @rasbt in #1489
• bugfix: LLM generated text with containing {KEY} would result in error by @aniketmaurya in #1490
• Add SAMBA to project gallery by @rasbt in #1493
• Add resume="auto" by @awaelchli in #1492
• Fix imports by @rasbt in #1491
• Pin NumPy version by @rasbt in #1495
• Version bump for new release to fix NumPy issue by @rasbt in #1496

Full Changelog: v0.4.0...v0.4.1

v0.4.0

What's Changed

• Set litdata < 0.2.6 by @carmocca in #1400
• Remove per-file CLIs by @carmocca in #1397
• Simillar -> Similar by @rasbt in #1405
• LitData: set iterate_over_all to False for CombinedStreamingDataset by @Andrei-Aksionov in #1404
• Allow multiline prompts by @rasbt in #1279
• Explain dataset options by @rasbt in #1407
• Support no_sync with Thunder FSDP by @carmocca in #1414
• Minimal Python example by @rasbt in #1410
• Fix bug where LitData doesn't use seed by @bradfordlynch in #1425
• Add prompt style mapping for llama3 by @davmacario in #1406
• Simplify code by @rasbt in #1429
• OptimizerArgs by @rasbt in #1409
• Fix optimizer init with fused=True by @carmocca in #1434
• Fix learning rate calculation in pretrain by @rasbt in #1435
• Align readme by @rasbt in #1438
• Pin litdata by @rasbt in #1440
• Fix README.md alignment by @rasbt in #1439
• Update README.md for one last time by @rasbt in #1442
• A more centered look by @rasbt in #1449
• New CLI by @rasbt in #1437
• Update error message by @rasbt in #1453
• Explain how to list all available models by @rasbt in #1455
• Detect tensor cores by @rasbt in #1456
• Check checkpoint_dir and add checkpoints to path by @rasbt in #1454
• Add MicroLlama training support by @keeeeenw in #1457
• Streaming for serving with chat's generate function by @rasbt in #1426
• Fix sequence length bug by @rasbt in #1462
• Add lr_warmup_steps, max_steps values validation by @shenxiangzhuang in #1460
• Fix issue where path in merge_lora is overwritten by @rasbt in #1465
• Option to skip expensive final validation by @rasbt in #1372
• Allow batch size "auto" setting in evaluate by @rasbt in #1469
• Warn users when there is a bnb mismatch by @rasbt in #1468
• Allow batch argument with batch recomputation by @rasbt in #1470
• LitGPT Python API draft by @rasbt in #1459
• Bump version for PyPI release by @rasbt in #1476

New Contributors

• @bradfordlynch made their first contribution in #1425
• @davmacario made their first contribution in #1406
• @keeeeenw made their first contribution in #1457

Full Changelog: v0.3.1...v0.4.0