0.5.0

What's Changed

• [MS-632] [API] Update connector retry mechanism with tenacity library. Remove allow_retries by @imda-lionelteo in #122
• [MS-657] Update connector and metrics logging by @imda-lionelteo in #125
• [MS-818][BE] Issue running recipes and cookbooks when using jupyter notebooks by @imda-lionelteo in #136
• [Sprint 19] New features and fixes by @imda-jacksonboey in #137

Full Changelog: 0.4.11...0.5.0

0.4.11

What's Changed

• [MS-493] Support for RAG application testing by @imda-lionelteo in #101
• [MS-669] Update Requirement.txt versions by @imda-normanchia in #127
• [MS-708] Update requirements and pypoetry by @imda-lionelteo in #131
• [Sprint 17] New Features & Fixes by @imda-jacksonboey in #130

Full Changelog: 0.4.10...0.4.11

0.4.10

What's Changed

• [Ms 583] Add CLCC cookbook by @imda-kelvinkok in #111
• [MS-507] Updated sg safety qns dataset by @imda-kelvinkok in #107
• [Sprint 16] New Features by @imda-jacksonboey in #126

Full Changelog: 0.4.9...0.4.10

0.4.9

What's Changed

• [MS-524] Update package requirements incompatibility by @imda-lionelteo in #93
• [MS-459] Add 2 new connector-endpoints/connector for Azure Dall-e and ollama llama3.1 by @imda-lionelteo in #88
• [MS-394] Create Dataset by @imda-normanchia in #84
• Ms 475 extract am req by @imda-kelvinkok in #97
• Ms 477 return metric config by @imda-kelvinkok in #96
• New recipe with cyberseceval dataset by @waynekoo in #92
• Added Facticity APAC Multi lingual benchmark dataset (sample subset) by @shahruj in #98
• Adding new attackmodules by @Mungusbean in #94
• [MS-503] Gemini API Connector by @imda-normanchia in #99
• [MS-477] return metric config by @imda-kelvinkok in #100
• push by @imda-lionelteo in #116
• fix error by @imda-lionelteo in #117
• sync up main by @imda-lionelteo in #113
• updated requirements and pyproject by @imda-kelvinkok in #118
• [Ms 646] [BE] Unable to create new cook book & view list of recipes by @imda-kelvinkok in #120
• [MS 641][BE] utf-8 charmap codec bug by @imda-kelvinkok in #119
• [Sprint 15] New Features & Fixes by @imda-jacksonboey in #121

New Contributors

• @waynekoo made their first contribution in #92
• @shahruj made their first contribution in #98
• @Mungusbean made their first contribution in #94

Full Changelog: 0.4.8...0.4.9

v0.4.8

What's Changed

• Connector for Amazon Bedrock by @athewsey in #91

Full Changelog: 0.4.7...0.4.8

v0.4.7

What's Changed

• [Ms 524][BE] Fix datasets package incompatibility by @imda-kelvinkok in #103
• [Sprint 14] Bug Fixes by @jacksonboey in #104

Full Changelog: 0.4.6...0.4.7

v0.4.6

What's Changed

• Support for text-to-image benchmarks with OpenAI Dall-E connector
  • [MS-378] Integrate I2P for Benchmark by @imda-lionelteo in #80
  • [MS-399] Integrate Gender for Benchmark by @imda-lionelteo in #81
• [MS-432] Fix range bug by @imda-lionelteo in #86
• [Sprint 13] New Features & Fixes by @imda-benedictlee in #90

Full Changelog: 0.4.5...0.4.6

v0.4.5

What's Changed

• [MS-356] Benchmarking score bug by @imda-lionelteo in #78
• Add Smoke Test by @imda-benedictlee in #83
• Add Software Composition Analysis Scan by @imda-benedictlee in #58
• [Sprint 12] New features and fixes by @imda-lionelteo in #82

Full Changelog: 0.4.4...0.4.5

v0.4.4

What's Changed

• [MS-294] Removal of attack modules field in all recipes by @imda-lionelteo in #69
• [MS-292] Removal of generating dynamic benchmark prompts on the fly by @imda-lionelteo in #71
• [MS-348] Update Lionguard in Moonshot by @imda-lionelteo in #72
• FlagEval team tries to add the flagjudge model to help judge the clcc dataset. by @eyuansu62 in #65
• [MS-379] update clcc, update prints to logs, cache by @imda-lionelteo in #74
• update not implmeneted error exception by @imda-lionelteo in #75
• [Sprint 11] New features and fixes by @imda-lionelteo in #73

Full Changelog: 0.4.2...0.4.4

v0.4.2

What's Changed

New Features and Enhancements

• Red team Bookmark feature
  • Added a new Bookmark feature to enhance the Red Team's ability to mark and rerun saved prompts.
• CLI Filtering function
  • Added a new filtering function to the CLI, allowing users to filter and display specific information more efficiently.
• Standardize use of id and name
  • Unified the use of id and name across the tool for consistency and clarity.
• Detect hardcoded secrets
  • Implemented a feature to detect and alert on hardcoded secrets within the codebase, enhancing security.
• Add PR Template
  • Introduced a Pull Request template to streamline the PR process and ensure all necessary information is provided.
• Add Software Composition Analysis Scan
  • Integrated Software Composition Analysis (SCA) scans to identify vulnerabilities in third-party dependencies.
• Add Smoke Test Workflow
  • Established a new Smoke Test Workflow to automatically verify the core functionalities of the tool after every deployment.
• Updated context strategy and number of prompts in CLI
  • Improved the context strategy and adjusted the number of prompts in the Command Line Interface (CLI) for better user experience and efficiency.
• Modify print statements to logging
  • Replaced all print statements with logging to improve the monitoring and debugging processes.

Bug Fixes

• Fixed a bug which disallows a runner name to contain hyphen
  • Resolved an issue where runner names containing hyphens were not allowed, improving naming flexibility.
• Update Storage Return Path
  • Fixed the storage return path to ensure accurate and reliable data retrieval.

Full Changelog: 0.4.0...0.4.2