Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Introducing vector and text search #9345

Merged
merged 14 commits into from
Nov 6, 2024
Merged

Commits on Nov 5, 2024

  1. Python: Introducing vector search for the new memory connectors (#8298)

    <!-- Thank you for your contribution to the semantic-kernel repo!
    Please help reviewers and future users, providing the following
    information:
      1. Why is this change required?
      2. What problem does it solve?
      3. What scenario does it contribute to?
      4. If it fixes an open issue, please link to the issue here.
    -->
    Initial version contains the generic classes and a implementation for
    Azure AI Search.
    
    TODO: other stores
    TODO: tests
    
    <!-- Describe your changes, the overall approach, the underlying design.
    These notes will help understanding how your code works. Thanks! -->
    
    <!-- Before submitting this PR, please make sure: -->
    
    - [x] The code builds clean without any errors or warnings
    - [x] The PR follows the [SK Contribution
    Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
    and the [pre-submission formatting
    script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
    raises no violations
    - [x] All unit tests pass, and I have added new tests where possible
    - [x] I didn't break anyone 😄
    eavanvalkenburg committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    20d7928 View commit details
    Browse the repository at this point in the history
  2. Python: reordered the data folder (#8910)

    <!-- Thank you for your contribution to the semantic-kernel repo!
    Please help reviewers and future users, providing the following
    information:
      1. Why is this change required?
      2. What problem does it solve?
      3. What scenario does it contribute to?
      4. If it fixes an open issue, please link to the issue here.
    -->
    
    Reordered the data folder
    Removed NOT_EQUAL_TO filter clause
    Changed search functions to search, get_search_result and
    get_text_search_result for TextSearch and vectorizable_text_search and
    vectorized_search for VectorSearch to reflect updates to dotnet.
    Removed other query types for VectorSearch
    Added exceptions and handling of no result error.
    
    <!-- Describe your changes, the overall approach, the underlying design.
    These notes will help understanding how your code works. Thanks! -->
    
    <!-- Before submitting this PR, please make sure: -->
    
    - [x] The code builds clean without any errors or warnings
    - [x] The PR follows the [SK Contribution
    Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
    and the [pre-submission formatting
    script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
    raises no violations
    - [ ] All unit tests pass, and I have added new tests where possible
    - [x] I didn't break anyone 😄
    
    ---------
    
    Co-authored-by: Tao Chen <[email protected]>
    eavanvalkenburg and TaoChenOSU committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    6165d55 View commit details
    Browse the repository at this point in the history
  3. updated paths

    eavanvalkenburg committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    d72db5a View commit details
    Browse the repository at this point in the history
  4. test fixes

    eavanvalkenburg committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    ec3a624 View commit details
    Browse the repository at this point in the history
  5. mistral fix

    eavanvalkenburg committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    8cb0d9f View commit details
    Browse the repository at this point in the history
  6. Python: Search sync (#9418)

    ### Motivation and Context
    
    <!-- Thank you for your contribution to the semantic-kernel repo!
    Please help reviewers and future users, providing the following
    information:
      1. Why is this change required?
      2. What problem does it solve?
      3. What scenario does it contribute to?
      4. If it fixes an open issue, please link to the issue here.
    -->
    Syncing the python implementation with the dotnet work.
    
    Introduces VectorStoreTextSearch
    Methods on KernelPlugin
    etc
    
    Includes:
    
    - Adding additional distance and index definitions
    - Renamed to Top and Offset (instead of Count)
    - restructured base classes for different search types
    - add tests
    
    ### Description
    
    <!-- Describe your changes, the overall approach, the underlying design.
    These notes will help understanding how your code works. Thanks! -->
    
    ### Contribution Checklist
    
    <!-- Before submitting this PR, please make sure: -->
    
    - [x] The code builds clean without any errors or warnings
    - [x] The PR follows the [SK Contribution
    Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
    and the [pre-submission formatting
    script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
    raises no violations
    - [x] All unit tests pass, and I have added new tests where possible
    - [ ] I didn't break anyone 😄
    eavanvalkenburg committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    e536805 View commit details
    Browse the repository at this point in the history
  7. Python: Search integration test fixes (#9497)

    ### Motivation and Context
    
    <!-- Thank you for your contribution to the semantic-kernel repo!
    Please help reviewers and future users, providing the following
    information:
      1. Why is this change required?
      2. What problem does it solve?
      3. What scenario does it contribute to?
      4. If it fixes an open issue, please link to the issue here.
    -->
    couple of small fixes for the integration tests
    
    ### Description
    
    <!-- Describe your changes, the overall approach, the underlying design.
    These notes will help understanding how your code works. Thanks! -->
    
    ### Contribution Checklist
    
    <!-- Before submitting this PR, please make sure: -->
    
    - [x] The code builds clean without any errors or warnings
    - [x] The PR follows the [SK Contribution
    Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
    and the [pre-submission formatting
    script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
    raises no violations
    - [x] All unit tests pass, and I have added new tests where possible
    - [x] I didn't break anyone 😄
    eavanvalkenburg committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    da2faa0 View commit details
    Browse the repository at this point in the history
  8. Python: Search - addressed feedback from reviews (#9517)

    ### Motivation and Context
    
    <!-- Thank you for your contribution to the semantic-kernel repo!
    Please help reviewers and future users, providing the following
    information:
      1. Why is this change required?
      2. What problem does it solve?
      3. What scenario does it contribute to?
      4. If it fixes an open issue, please link to the issue here.
    -->
    Added new exceptions
    Cleaned up the samples
    removed old/commented code
    
    
    ### Description
    
    <!-- Describe your changes, the overall approach, the underlying design.
    These notes will help understanding how your code works. Thanks! -->
    
    ### Contribution Checklist
    
    <!-- Before submitting this PR, please make sure: -->
    
    - [x] The code builds clean without any errors or warnings
    - [x] The PR follows the [SK Contribution
    Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
    and the [pre-submission formatting
    script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
    raises no violations
    - [x] All unit tests pass, and I have added new tests where possible
    - [x] I didn't break anyone 😄
    eavanvalkenburg committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    10abd5a View commit details
    Browse the repository at this point in the history
  9. updated int test

    eavanvalkenburg committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    bceeeaa View commit details
    Browse the repository at this point in the history
  10. update merge tests

    eavanvalkenburg committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    6a4308e View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    5736f1a View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    15c75ef View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    ff5df13 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2024

  1. Configuration menu
    Copy the full SHA
    0c21c5c View commit details
    Browse the repository at this point in the history