Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add SASEvaluator #7428

Merged
merged 8 commits into from
Apr 4, 2024
Merged

feat: Add SASEvaluator #7428

merged 8 commits into from
Apr 4, 2024

Conversation

silvanocerza
Copy link
Contributor

Related Issues

Proposed Changes:

Add a SASEvaluator Component to calculate Semantic Answer Similarity.

How did you test it?

Added unit and integration tests.

Notes for the reviewer

The logic is identical to the SASEvaluator added in #6980 and refactored with in #6998.

Checklist

@silvanocerza silvanocerza self-assigned this Mar 26, 2024
@silvanocerza silvanocerza requested review from a team as code owners March 26, 2024 16:23
@silvanocerza silvanocerza requested review from dfokina, masci, shadeMe and julian-risch and removed request for a team and masci March 26, 2024 16:23
@github-actions github-actions bot added topic:tests 2.x Related to Haystack v2.0 type:documentation Improvements on the docs labels Mar 26, 2024
@coveralls
Copy link
Collaborator

coveralls commented Mar 26, 2024

Pull Request Test Coverage Report for Build 8541962462

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 46 unchanged lines in 6 files lost coverage.
  • Overall coverage decreased (-0.2%) to 89.331%

Files with Coverage Reduction New Missed Lines %
components/embedders/hugging_face_tei_document_embedder.py 1 98.48%
components/embedders/hugging_face_tei_text_embedder.py 1 97.73%
components/websearch/searchapi.py 2 96.36%
core/pipeline/pipeline.py 4 93.85%
components/embedders/azure_text_embedder.py 12 65.12%
components/embedders/azure_document_embedder.py 26 52.24%
Totals Coverage Status
Change from base Build 8439010873: -0.2%
Covered Lines: 5618
Relevant Lines: 6289

💛 - Coveralls

haystack/components/evaluators/sas_evaluator.py Outdated Show resolved Hide resolved
haystack/components/evaluators/sas_evaluator.py Outdated Show resolved Hide resolved
haystack/components/evaluators/sas_evaluator.py Outdated Show resolved Hide resolved
haystack/components/evaluators/sas_evaluator.py Outdated Show resolved Hide resolved
@silvanocerza silvanocerza merged commit 12acb3f into main Apr 4, 2024
23 checks passed
@silvanocerza silvanocerza deleted the sas-evaluator branch April 4, 2024 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x Related to Haystack v2.0 topic:tests type:documentation Improvements on the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement function to calculate Semantic Answer Similarity metric
3 participants