bug(dspy): Move ensemble import to teleprompt #542
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CyrusNuevoDia
pushed a commit
that referenced
this pull request
Mar 22, 2024
* Update setup.py * Update pyproject.toml * OpenAI: No retry on ServiceUnavailableError, APIError * Some refinements to signature, allowing instantiation and fixing lint issues. * Avoid shadowing by lint * Update skycamp2023.ipynb * Support for n=... and type-strings * Linting of dummy * Rewrote type parsing to be 3.9 compatible * Python 3.9 compatability * Validate main signature fields * Fixes to completions * Fixed #520 * Changed max retries back to 3 * ci(dspy): Add Ruff linting workflow (#538) * Add Ruff linting workflow * Update GitHub Actions workflows * Make ruff apply but not fail * Style fixes by Ruff * Combine Workflows * Add caching * Add import related rules * Automatic Style fixes * Try and fix caching * Fix import * Use natve python dep caching * Update actions/checkout to v4 * Use autofix in tests * Try to cache "install poetry" * Empty-Commit to test caching * Remove workflow deps * Rename action * Update workflow name * Fix Docker stop command and subprocess run check * Fix functional imports --------- Co-authored-by: isaacbmiller <[email protected]> * ci(dspy): Fix tests failing - remove --no-root flag (#541) * Add Ruff linting workflow * Update GitHub Actions workflows * Make ruff apply but not fail * Style fixes by Ruff * Combine Workflows * Add caching * Add import related rules * Automatic Style fixes * Try and fix caching * Fix import * Use natve python dep caching * Update actions/checkout to v4 * Use autofix in tests * Try to cache "install poetry" * Empty-Commit to test caching * Remove workflow deps * Rename action * Update workflow name * Fix Docker stop command and subprocess run check * Fix functional imports * Remove --no-root flag --------- Co-authored-by: isaacbmiller <[email protected]> * Batched Generation, SIgnature Based Generation * Automatic Style fixes * fix: printing signature fields in verbose mode for signature_opt * Update README.md A few more typed examples * import fix * Automatic Style fixes * First try at a new optimizer/teleprompt * Add support for neo4j vector index * switch to text * Add docs & some improvements * Fixes * Update README.md Added Franck SN's medium post on chess with typed dspy * Update README.md Added credit * docs(dspy): add toy invocation of model to minimal-example * Move ensemble import to teleprompt (#542) * Fix return_all_scores in evaluate * add 'k' as argument to FaissRM.forward() * Add search to docs * remove extra space * Update minimal-example.mdx * Update code snippets in cheatsheet * Fix applymap deprecation warning * Add Claude wrapper * project dependencies * Adds support to call Claude from dsp modules. * Removed (deactiviated) key * Pull from original fork repo * make CI fail on fix * Add error handling and comment for formatting issues * Add write permissions for pull requests in run_tests workflow * Fail on any error * separate comment * separate comment remove continue * Allow PR comment write * try echo syntax * fix echo syntax * Move to separate file to run with elevated perms * Give name to workflow * Recombine into one file, but separate by action type * change target branch * Purposely create ruff errors * Add console.log * Split into separate workflow * Revert "Fix return_all_scores in evaluate" This reverts commit 0f39a43. * add ruff as dev dependency * Add verbose to ruff check * Make comment action follow github best practices * 2 commits: 1 bad, 1 good (#569) * bad commit, should fail CI * Good commit, should pass CI * Empty-Commit * Empty-Commit * Bad commit, should fail lint * Good commit, should pass lint * Rename workflow * Fix the "no inputs" case * Added a new method to Module * Fix a bug with generics * Formatting * Created typed signature optimizer * Updated notebook * Automatic Style fixes * Removed submodules * Fixed tests for python 3.9 * Automatic Style fixes * Prevent import reorder * Automatic Style fixes * Prevent import reorder * removed sub module * Update ProgramOfThought.md: Example Uses the extended signature syntax by utilizing a class wrapping dspy.Signature, because .attach() seems deprecated * Richer output from signature optimizer * Force the optimizer to not repeat signatures * Automatic Style fixes * Remove unused kwargs passed to requests sent through `AzureOpenAI` This commit is to remove unused kwargs that we include in the requests sent through `AzureOpenAI` class. This seems to fix this issue: #543 * Adding metric_threshold to BootstrapFewShotWithRandomSearch * Update cheatsheet.md Added typed signature optimizer * refactoring optimizers, updates to optimizer notebook * Changed MIPRO and COPRO fields * ensure deprecated optimizers return expected output * FIX: BootstrapFewShotWithRandomSearch.metric_threshold is not set in __init__ * will result in AttributeError: 'BootstrapFewShotWithRandomSearch' object has no attribute 'metric_threshold' when execute `compile` * can be reproduced by executing the following colab's `Compilation With Assertions` part. ** https://colab.research.google.com/github/stanfordnlp/dspy/blob/main/examples/longformqa/longformqa_assertions.ipynb#scrollTo=544FmRbtuzgS * additonal updates to notebook, merging in refactor changes * updates to notebook * notebook name change * minor changes to notebook+copro code (better print statements) * Automatic Style fixes * Add local language model folder * Fixed multi-module typed signature optimizer * ci(dspy): Add main push test run back (#600) * Add main push action back and fix comment bot * Try to fix comment bot * Remove comment bot workflow - to be added later * Fix code issues in signature-optimizer.mdx 1. use the HotPotQA dataset instead of gsm8k 2. result.rationale instead of result.reason 3. process devset to include input_keys * a few optimizer updates * adding in an import that was accidentally removed * updating mipro tests * fixing warning statement * fixing bug when requires_permission_to_run=False, and doing ruff cleanups * fixing some variable names in test * fixing variable name * adding api references and docs files for functional * Update provider name to Claude or Bedrock * Inspect history for Claude models * updating tests for new optimizers * renaming tests * small cleanup of breakpoints * minor updates * removing accidentally added warning from copro * make num_trials a required variable, plus minor changes to notebook * adding notebook cache_dir in front of dspy import * removing re import from notebook * feature(dspy): added Metadata for ChromaDB retrieval * Update setup.py * Update pyproject.toml * Update setup.py * Update pyproject.toml * fix to signature optimizer - setting num_candidates * Update setup.py * Update pyproject.toml * adding in colab badge * Adding typed predictor docs * Update 8-typed_predictors.md * 📝 (ChromadbRM.md): Update the return type description in the documentation The return type description for the search function in the ChromadbRM documentation was updated to provide more detailed information about the structure of the returned object. This change will help developers understand the structure of the returned object and how to use it in their code. * hf models can use auth tokens now (#611) * Update pyproject.toml * Update pyproject.toml (Whoops left a comma) * Match version * update linter * run ruff linter * update * Revert "run ruff linter" This reverts commit 829e573. * Update pyproject.toml * Update databricks_rm.py -added support for doc_ids * Update databricks_rm.py * Add ruff rule back * Make anthropic import optional for tests * Apply ruff fix * Update poetry.lock * Revert "Update poetry.lock" This reverts commit e7443d6. * Update poetry.lock * update with self.history * higher poetry openai req * Update README.md Updated table of contents * run ruff check --fix-only dspy/retrieve/faiss_rm.py * Fix return_all_scores in evaluate again to return all (#625) * fix neo4j docs * folder structure change, example driven optimization, mix of lm generation, teacher based output gen. * typo fixes * lint fixes * Update 8-typed_predictors.md A few fixes to the typed docs * fix dotdict in marqo * feedback driven generation * fix(dspy): use DataFrame.applymap in older pandas * Automated feedback incorporations * lint fixes * print removal * fix(dspy): remove debug print statement * lint(dspy): refactor * Remove kwargs print call in Claude LM * Revert "Remove kwargs print call in Claude LM" This reverts commit 9242b01. * Fixed but with prefixes and annotated types * Automatically copy over "description" from pydantic field to dspy field * Fixed issue with boolean outputs * Formatting * formatting * fix: updated predict tests with both lm and backend functionality * fix: updated test_program for experimental flag * fix: updated test_chain_of_thought for experimental flag * chore: update formatting for functional.py * fix: update backend arguments to ensure, that parameter clashes between lm and input arguments are accomodated for * fix: update functional for completion/prediction changes * chore: update formatting for copro optimizer * fix: aligned copro_optimizer with new prediction and completions api * chore: format mipro_optimizer * fix: passing all tests * chore: ruff fixes for backends * chore: ruff check and fix * fix: remove typing.Self * chore: clean up tests/predict * chore: clean up tests/backends * chore: clean up tests/evaluate * chore: clean up tests/functional * chore: clean up tests/primitives * chore: clean up tests/signatures * chore: clean up tests/teleprompt * fix: remove clean_up_lm_test * fix: remove commented out code * fix: moved BaseLM to output strings versus dictionary of content --------- Co-authored-by: Omar Khattab <[email protected]> Co-authored-by: Omar Khattab <[email protected]> Co-authored-by: Thomas D Ahle <[email protected]> Co-authored-by: Arian Askari <[email protected]> Co-authored-by: Isaac Miller <[email protected]> Co-authored-by: isaacbmiller <[email protected]> Co-authored-by: Herumb Shandilya <[email protected]> Co-authored-by: Herumb Shandilya <[email protected]> Co-authored-by: krypticmouse <[email protected]> Co-authored-by: ragul-kachiappan-dev <[email protected]> Co-authored-by: Tomaz Bratanic <[email protected]> Co-authored-by: Nat Taylor <[email protected]> Co-authored-by: quajak <[email protected]> Co-authored-by: Raja Rajendran <[email protected]> Co-authored-by: arnavsinghvi11 <[email protected]> Co-authored-by: Sumedha Kucherlapati <[email protected]> Co-authored-by: Isaac Miller <[email protected]> Co-authored-by: thomasahle <[email protected]> Co-authored-by: SimonB97 <[email protected]> Co-authored-by: mingyi yang <[email protected]> Co-authored-by: Insop Song <[email protected]> Co-authored-by: klopsahlong <[email protected]> Co-authored-by: Michael Ryan <[email protected]> Co-authored-by: chentienan <[email protected]> Co-authored-by: klopsahlong <[email protected]> Co-authored-by: swairshah <[email protected]> Co-authored-by: VivHarsha <[email protected]> Co-authored-by: domci <[email protected]> Co-authored-by: klopsahlong <[email protected]> Co-authored-by: dominik <[email protected]> Co-authored-by: Matthew Billman <[email protected]> Co-authored-by: Connor Shorten <[email protected]> Co-authored-by: Arnav Singhvi <[email protected]> Co-authored-by: Darin <[email protected]> Co-authored-by: Quajak <[email protected]> Co-authored-by: demontego <[email protected]> Co-authored-by: Max Friedrich <[email protected]> Co-authored-by: Rick Battle <[email protected]> Co-authored-by: nbqu <[email protected]> Co-authored-by: Amir Mehr <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds ensemble to the teleprompt imports. Solves #533