v0.2.14

What's Changed

• feat: add http request-response logging with env flag control by @ganeshrvel in #1903
• Metric with different parameter gives different score by @sahusiddharth in #1908
• Updated the instruction of StatementGeneratorPrompt in faithfulness by @sahusiddharth in #1906
• Fixed simple criteria metric by @sahusiddharth in #1909
• Evaluating Multi-Turn Conversations by @sahusiddharth in #1919
• fix SingleHopQuerySynthesizer::prepare_combinations() by @lryan599 in #1921
• docs: fix semantic similarity description (cross-encoder -> bi-encoder) by @Ayaka-mogumogu in #1910
• docs: list googles model provider by @jjmachan in #1920
• nvidia end-2-end accuracy, relevance and groundedness metrics by @titericz in #1913
• Enable Runtime Checks for ModeMetric Protocol by @sahusiddharth in #1929
• r2r integration by @sahusiddharth in #1918
• Haystack llm and embedding wrapper by @sahusiddharth in #1901
• Easy fix, Groundedness metric, 5 retries early break. by @titericz in #1935
• nvidia docs by @sahusiddharth in #1940
• unnecessary Noise Sensitivity name update by @sahusiddharth in #1943
• When saving kg only save the node id in the relationships by @lryan599 in #1926
• fix: return valid JSON output schema by @alasdairsmith in #1933
• Update default.py - Missing cosine_similarity transform for docs with Token Count 101–500 by @rgrizzo-linksmt in #1934
• Ndarray dtype fix, improve error wording by @zybinmikhail in #1924

New Contributors

• @lryan599 made their first contribution in #1921
• @Ayaka-mogumogu made their first contribution in #1910
• @titericz made their first contribution in #1913
• @alasdairsmith made their first contribution in #1933
• @rgrizzo-linksmt made their first contribution in #1934
• @zybinmikhail made their first contribution in #1924

Full Changelog: v0.2.13...v0.2.14

v0.2.13

What's Changed

• Improved the documentation in the migration docs by @sahusiddharth in #1873
• docs: match default timeout in RunConfig by @neo in #1872
• fix(prompt/mixin): Add name property and add it to saving/loading path. by @tim-hilde in #1853
• Adding missing evaluate import to evals guide by @carlosgsouza in #1876
• Improve error message for missing embeddings in Answer Relevancy Metric by @sahusiddharth in #1877
• Change the validate_samples functionality by @Vidit-Ostwal in #1879
• chore: added openhands for solving some github issues by @jjmachan in #1884
• Fix Broken Test Case Due to Changed Error Message by @sahusiddharth in #1881
• Faithfulness prompt update to avoid having single quotes in response by @michaelromagne in #1874
• Internal logging and tracing improvements. by @ganeshrvel in #1886
• docs: added office hour links by @jjmachan in #1888
• added the end of sequence token for the WatsonX family of models by @malikaltakrori in #1883
• Enhance langgraph integration to preserve metadata by @i-w-a in #1878
• Adding tutorial for Haystack Integration by @sahusiddharth in #1887
• Remove llm argument from ToolCallAccuracy initialization by @sahusiddharth in #1891
• Updated Application Page by @sahusiddharth in #1899
• fix: UnicodeEncode error when save/load knowledge graph by @jjmachan in #1900

New Contributors

• @tim-hilde made their first contribution in #1853
• @carlosgsouza made their first contribution in #1876
• @Vidit-Ostwal made their first contribution in #1879
• @malikaltakrori made their first contribution in #1883

Full Changelog: v0.2.12...v0.2.13

v0.2.12

What's Changed

• Add token parser for Bedrock & fix anthropic typo by @michaelromagne in #1851
• fix tp&fp calc error by @oslijunw in #1848
• docs: added canonical_url by @jjmachan in #1855
• docs: be consistent with the variable naming of evaluator_embeddings by @neo in #1858
• Add optional use_effective_order parameter to BLEU score by @JonasElburgUVA in #1862
• docs: updated broken links in ReadMe and evaluation documentation by @iamarunbrahma in #1847
• docs: fix incorrect reference to LLM in NonLLMContextPrecisionWithReference metric by @sahusiddharth in #1863
• fix: output parser bug by @jjmachan in #1864

New Contributors

• @michaelromagne made their first contribution in #1851
• @neo made their first contribution in #1858
• @JonasElburgUVA made their first contribution in #1862
• @iamarunbrahma made their first contribution in #1847

Full Changelog: v0.2.11...v0.2.12

v0.2.11

What's Changed

• fix: add min requirement for optimisation by @shahules786 in #1820
• fix: rubrics based metrics by @sahusiddharth in #1821
• Enhance ToolCall message to support all argument types by @sahusiddharth in #1823
• Added from_url method to fetch evaluation annotation from the api. by @ganeshrvel in #1795
• docs: fix incorrect imports by @sahusiddharth in #1824
• docs: update some misc doc errors by @jjmachan in #1825
• Feat: Fix for Instance Based Metrics and Updated docs by @sahusiddharth in #1827
• Docs/updating the LangSmith docs by @sahusiddharth in #1828
• fix: remove pysbd and sentence segmenting by @shahules786 in #1826
• docs: Fixed most of the broken links by @sahusiddharth in #1830
• docs: updated the formula for some metrics by @sahusiddharth in #1834
• docs: update _language_adaptation.md by @eltociear in #1839
• feat: added option to provide experiment_name to evaluate() by @jjmachan in #1843
• Change save and load encode to utf-8 by @joaorura in #1836
• fix numpy invert error in metrics by @oslijunw in #1835
• Adding swarm integration and tutorial by @sahusiddharth in #1841
• Adding Swarm integration to the list of integrations by @sahusiddharth in #1845
• Adjustment when calculating hash | Adjustment of the hash calculation… by @joaorura in #1837
• docs: removed SUPPORTED_LANGUAGE by @jjmachan in #1846

New Contributors

• @ganeshrvel made their first contribution in #1795
• @oslijunw made their first contribution in #1835

Full Changelog: v0.2.10...v0.2.11

v0.2.10

What's Changed

• docs: new get started guide by @shahules786 in #1786
• Update README.md by @shahules786 in #1794
• docs: fix typo by @shahules786 in #1801
• fix: llamaindex testset generator by @jjmachan in #1804
• Docs improvements by @EricGustin in #1805
• Docs improvements by @hundredeuk2 in #1808
• docs: add new rag eval tutorial by @shahules786 in #1815
• Fix typo in index.md by @sahusiddharth in #1811
• docs: clean get-started by @shahules786 in #1816
• Improve question generation efficiency In Response Relevancy metrics by @hundredeuk2 in #1810
• docs: langchain v3 integration tutorial by @sahusiddharth in #1818

New Contributors

• @EricGustin made their first contribution in #1805
• @hundredeuk2 made their first contribution in #1808

Full Changelog: v0.2.9...v0.2.10

v0.2.9

What's Changed

• feat: replaced NLTK's implementation of BLEU with sacrebleu's implementation by @trent-sp in #1744
• doc: add note by @shahules786 in #1753
• docs: add blog to readme by @shahules786 in #1755
• fixes: some minor fixes by @shahules786 in #1756
• docs: added ragas-app to getting started by @jjmachan in #1754
• feat: add exact match caching by @ayulockin in #1717
• fix: temperature needs to be added handled effectively by @jjmachan in #1759
• Fix/hhem: divide by zero error by @jjmachan in #1765
• fix: add reference to simple scoring by @shahules786 in #1758
• fix: import train config and add tests by @shahules786 in #1776
• chore: make analytics into threading by @jjmachan in #1777
• fix: change to schema by @shahules786 in #1780
• Fix link to langchain and add pip install by @pamelafox in #1762
• fix: distance measure in non llm CR by @shahules786 in #1787
• docs: caching in ragas by @jjmachan in #1779
• fix: remove reproducibility arg by @shahules786 in #1790

New Contributors

• @trent-sp made their first contribution in #1744
• @pamelafox made their first contribution in #1762

Full Changelog: v0.2.8...v0.2.9

v0.2.8

What's Changed

• fix: Update README.md by @avanco in #1738
• feat: genetic algo based optimizer by @shahules786 in #1724
• Added native support for Gemini models generation completion signals in LangchainLLMWrapper class / is_finished method by @cnemri in #1727
• Added custom is_finished_parser logic to Google Vertex AI customizati… by @cnemri in #1728
• feat: added basics for upload by @jjmachan in #1741
• feat: few shot example optimzier by @jjmachan in #1739
• fix: correct url for upload() by @jjmachan in #1742
• fix: prompt naming related issues by @shahules786 in #1743
• feat: load from annotated testset by @jjmachan in #1747
• docs: add demo example for metric training by @shahules786 in #1746
• docs: demo related fixes by @shahules786 in #1750

New Contributors

• @avanco made their first contribution in #1738
• @cnemri made their first contribution in #1727

Full Changelog: v0.2.7...v0.2.8

v0.2.7

What's Changed

• feat: removed critics and rubrics from examples by @jjmachan in #1691
• feat: test-generation improvements by @shahules786 in #1698
• chore: updated tracking for single scores by @jjmachan in #1716
• feat: remove examples by @jjmachan in #1708
• Bug in documentation rag_testset_generation.md: API changes required arguments by @bmerkle in #1714
• feat: add output type to metrics by @shahules786 in #1722
• feat: dataloader for annotated json by @shahules786 in #1723
• feat: train your own evaluators by @shahules786 in #1701
• fix: attempting a simple fix for #1718 by @ayulockin in #1721
• fix: add missing query qualifier by @shahules786 in #1733
• feat: Test generation for non-english corpus by @shahules786 in #1734

New Contributors

• @bmerkle made their first contribution in #1714
• @ayulockin made their first contribution in #1721

Full Changelog: v0.2.6...v0.2.7

v0.2.6

What's Changed

• Avoid unnecessary decompose_claims call when mode='precision' in _factual_correctness.py by @dfhssilva in #1669
• fixes: instruction format by @shahules786 in #1672
• docs: updated readme by @jjmachan in #1674
• docs: update readme by @jjmachan in #1675
• fixes: add extraction limit to Extractors by @shahules786 in #1673
• fix: upload for testset by @jjmachan in #1676
• removed MetricWithLLM from SemanticSimilarity by @Luka958Pixion in #1678
• feat: improvements in default test generation by @shahules786 in #1661
• Update rag.md by @TimWue in #1686
• fix: missing embeddings argument in testset and some E2E tests by @jjmachan in #1690
• fixes: handle long context extraction by @shahules786 in #1680
• feat: make general purpose metrics more general by @jjmachan in #1666
• Fixes ToolCallAccuracy raises ZeroDivisionError when called without any arguments. by @sahusiddharth in #1685

New Contributors

• @dfhssilva made their first contribution in #1669
• @Luka958Pixion made their first contribution in #1678
• @TimWue made their first contribution in #1686

Full Changelog: v0.2.5...v0.2.6

v0.2.5

What's Changed

• feat: Unify the use of sentence_segmenter by @jasinliu in #1629
• feat: aspect critic with reference by @shahules786 in #1638
• docs: change setting llm by @shahules786 in #1639
• fixed wrong variable names in starter guide using azure openai by @MichaelPerger in #1641
• feat: improvements in test gen by @shahules786 in #1645
• Add tutorial for evaluating LangGraph agents by @sahusiddharth in #1636
• docs: add cost tracker back by @shahules786 in #1653
• docs: tracing and debugging metrics by @shahules786 in #1652
• feat: upload testset generation upload by @jjmachan in #1647
• docs: reorganise files by @shahules786 in #1662
• docs: fix _arize.md by @suekou in #1643
• docs: persona generator by @jjmachan in #1664
• Bug fixing recall in _factual_correctness.py by @GitMarco27 in #1650

New Contributors

• @MichaelPerger made their first contribution in #1641
• @GitMarco27 made their first contribution in #1650

Full Changelog: v0.2.4...v0.2.5