From f0553e90106b99b0a88182dab6f7e5291ad45210 Mon Sep 17 00:00:00 2001 From: Nicole White Date: Fri, 1 Mar 2024 17:55:34 -0500 Subject: [PATCH] update readmes --- JavaScript/testing-sdk/README.md | 93 ++++++++++++++++++++ JavaScript/testing-sdk/package.json | 2 +- Python/testing-sdk-with-prompt-sdk/README.md | 2 +- Python/testing-sdk/README.md | 2 +- README.md | 1 + 5 files changed, 97 insertions(+), 3 deletions(-) create mode 100644 JavaScript/testing-sdk/README.md diff --git a/JavaScript/testing-sdk/README.md b/JavaScript/testing-sdk/README.md new file mode 100644 index 00000000..cc0bfca2 --- /dev/null +++ b/JavaScript/testing-sdk/README.md @@ -0,0 +1,93 @@ + +

+ +

+ +

+ 📚 + Documentation +   + • +   + 🖥️ + Application +   + • +   + 🏠 + Home +

+ + +## Setup + +### Install Autoblocks CLI + +See [Autoblocks CLI documentation](https://docs.autoblocks.ai/cli/setup) + +### Install dependencies + +``` +npm install +``` + +## Run Autoblocks tests + +### Set your Autoblocks API key + +Retrieve your **local testing API key** from the [settings page](https://app.autoblocks.ai/settings/api-keys) and set it as an environment variable: + +```bash +export AUTOBLOCKS_API_KEY=... +``` + +### Set your OpenAI API key + +```bash +export OPENAI_API_KEY=... +``` + +### Run the tests + +```bash +npx autoblocks testing exec -m "my first run" -- npm run start +``` + +You should see something like: + +Screenshot 2024-03-01 at 5 53 27 PM + +You can click on the links next to each test name to dig into more details. +You can also find all of your tests on the testing homepage in the [Autoblocks application](https://app.autoblocks.ai/testing/local). + +## GitHub Actions setup + +A starter workflow was added in [`.github/workflows/autoblocks-testing.yml`](./.github/workflows/autoblocks-testing.yml). +This workflow runs the tests on every push to the repository and also +on a daily schedule. + +## Repo structure + +``` +src/ + run.ts <-- imports all tests from test-suites/ and runs them + evaluators/ <-- all common evaluators are implemented here + some-shared-evaluator1.ts + some-shared-evaluator2.ts + tasks/ <-- all "tasks" are implemented here + task1.ts + task2.ts + test_suites/ <-- tests for each task + task1/ + index.ts <-- implements the runner for task1 + evaluators.ts <-- evaluators used only for task1 + test-cases.ts <-- contains test cases for task1 + task2/ + index.ts <-- implements the runner for task2 + evaluators.ts <-- evaluators used only for task2 + test-cases.ts <-- contains test cases for task2 +``` + +## Futher Reading + +- [Autoblocks Testing documentation](https://docs.autoblocks.ai/testing/sdks) diff --git a/JavaScript/testing-sdk/package.json b/JavaScript/testing-sdk/package.json index 8d2e6015..09af6166 100644 --- a/JavaScript/testing-sdk/package.json +++ b/JavaScript/testing-sdk/package.json @@ -1,6 +1,6 @@ { "name": "testing-sdk", - "description": "Using the Autoblocks Testing SDK for experimentation and regression testing", + "description": "Using the Autoblocks Testing SDK with the Prompt SDK for experimentation, regression testing, and rapid prototyping.", "version": "0.0.0", "private": true, "engines": { diff --git a/Python/testing-sdk-with-prompt-sdk/README.md b/Python/testing-sdk-with-prompt-sdk/README.md index 1b55a023..799edcb0 100644 --- a/Python/testing-sdk-with-prompt-sdk/README.md +++ b/Python/testing-sdk-with-prompt-sdk/README.md @@ -298,7 +298,7 @@ on a daily schedule. ``` my_project/ - run.py <-- imports all tests from tests/ and runs them + run.py <-- imports all tests from test_suites/ and runs them evaluators/ <-- all common evaluators are implemented here some_shared_evaluator1.py some_shared_evaluator2.py diff --git a/Python/testing-sdk/README.md b/Python/testing-sdk/README.md index 9edd936b..d7116392 100644 --- a/Python/testing-sdk/README.md +++ b/Python/testing-sdk/README.md @@ -76,7 +76,7 @@ on a daily schedule. ``` my_project/ - run.py <-- imports all tests from tests/ and runs them + run.py <-- imports all tests from test_suites/ and runs them evaluators/ <-- all common evaluators are implemented here some_shared_evaluator1.py some_shared_evaluator2.py diff --git a/README.md b/README.md index 049267c6..a4d0a436 100644 --- a/README.md +++ b/README.md @@ -44,6 +44,7 @@ | [prompt-sdk-headless](/JavaScript/prompt-sdk-headless) | Typesafe and backwards-compatability-ensured headless prompt management | | [prompt-sdk-local](/JavaScript/prompt-sdk-local) | Automated prompt versioning and typesafe prompt building with the local prompt SDK | | [spans](/JavaScript/spans) | Establish parent / child relationships between your events with the `spanId` and `parentSpanId` properties | +| [testing-sdk](/JavaScript/testing-sdk) | Using the Autoblocks Testing SDK with the Prompt SDK for experimentation, regression testing, and rapid prototyping. |