Add comments Argilla - David Berenstein #119
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
davidberenstein1957
changed the title
strickvl
merged commit caf0dca
into
zenml-io:feature/finetune-embeddings
2 of 3 checks passed
strickvl
added a commit
that referenced
this pull request
Aug 8, 2024
* add embeddings finetuning quick and dirty code * formatting * add embeddings finetuning pipeline * refactor * works for multi-gpu setup * update gitignore * scratch test in notebook * pipeline is working again * no need for custom class * visualize performance comparison * compare apples to apples * fix viz bug and strip out code / logs examples * update print statement * mini updates * add dummy embedding pipeline to compare against * updates to collate * add dummy pipeline and refactor * new pipelines * run locally, fix attribute naming error * full run * next experiment * Add ollama ignores to .gitignore * add retries to question generation * generate 3 sets of questions * handle multiple question generation * add retries for question generation function * more robustness around failures * formatting * fix dataloader splitting (on newline) * inspect test dataloader construction WIP * split chunks by headers * strip out boilerplate at top of markdown docs * refactor & fix chunking logic * add artifact metadata logging * fix annotations and artifact metadata logging * update metadata * update logging statements * add time estimates to logging * log estimated completion time * update log statements * add rate for logging * Update generation rate message * fix eval * limited run + limited eval * full eval * distilroberta as base model * smaller batch size for a larger base model * fix eval mismatch and chunking * chore: Update .gitignore to exclude .flashrank_cache file * chore: Add notebook for argilla * chore: Add .gitignore entry for bge-base-financial-matryoshka * Update sentence-transformers to version 3 and add transformers dependency * add data and get finetuning working * chore: Add .gitignore entry for embeddings * Update zenml dependency to version 0.58.2 with server support * chore: Update dependencies and add .gitignore entry for embeddings * Add comments Argilla - David Berenstein (#119) * Add introduction to argilla_embeddings * Update context for data and synthetic query generation * Added argilla data section * embeddings updates * update gitignore * format * add push to HF imports correctly * remove extra connection cell * add final step to pipeline as active * fix typo * updates * Feature/finetune embeddings (#120) * Add introduction to argilla_embeddings * Update context for data and synthetic query generation * Added argilla data section * Update Argilla code * Remove cell outputs * Remove secrets * Remove duplicate Argilla section * Update dataset name * Update phrasing Argilla * Updatge merge conflicts * fixes for dataset generation * Update vector naming (#121) * Add introduction to argilla_embeddings * Update context for data and synthetic query generation * Added argilla data section * Update Argilla code * Remove cell outputs * Remove secrets * Remove duplicate Argilla section * Update dataset name * Update phrasing Argilla * Updatge merge conflicts * Update vector naming * working argilla upload * fix argilla upload * add wandb requirement * add finetuning step to pipeline * ignore local model files * working finetune step * finetune step pushes to the hub now * update gitignore * update pipeline * remove wandb tracking * broken version * remove automatic wandb tracking * return results * try with larger base model * add new model to gitignore * use snowflake model * add model registration * remove constant * add model throughout both pipelines * finetuning steps in logical order now * add new constant * update model version * use constant for model name * various improvements :) * add argilla flywheel functionality * refactor out a constant for the dataset name * add license * more small changes * Various improvements (#123) * Resolve argilla client default workspace warning * Add constant for other pipeline steps * Add constant for finetune_embeddings * Remove step embeddings constants * import contants in pipeline directly * Remove constants from distilabel_generation * Add workflow for basic dataset filtering * Add updated plots --------- Co-authored-by: Alex Strick van Linschoten <[email protected]> * format and remove unused imports * fix push step * fixed finetuning step * use github repo to install for latest changes * update README * Delete llm-complete-guide/__init__.py * Update llm-complete-guide/requirements.txt * update README * format and add results visualization * remove extra comma * add visualization step * visualization corrections * make distilabel imports clearer * update README docs * address TODO comment * credit phil * fix typo * add actual values to chart * add link to LLMOps guide * make visualization a bit nicer * update main README --------- Co-authored-by: David Berenstein <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.