Skip to content

Commit

Permalink
update readmes
Browse files Browse the repository at this point in the history
  • Loading branch information
Nicole White committed Mar 1, 2024
1 parent 8002862 commit f0553e9
Show file tree
Hide file tree
Showing 5 changed files with 97 additions and 3 deletions.
93 changes: 93 additions & 0 deletions JavaScript/testing-sdk/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
<!-- banner start -->
<p align="center">
<img src="https://app.autoblocks.ai/images/logo.png" width="300px">
</p>

<p align="center">
📚
<a href="https://docs.autoblocks.ai/">Documentation</a>
&nbsp;
&nbsp;
🖥️
<a href="https://app.autoblocks.ai/">Application</a>
&nbsp;
&nbsp;
🏠
<a href="https://www.autoblocks.ai/">Home</a>
</p>
<!-- banner end -->

## Setup

### Install Autoblocks CLI

See [Autoblocks CLI documentation](https://docs.autoblocks.ai/cli/setup)

### Install dependencies

```
npm install
```

## Run Autoblocks tests

### Set your Autoblocks API key

Retrieve your **local testing API key** from the [settings page](https://app.autoblocks.ai/settings/api-keys) and set it as an environment variable:

```bash
export AUTOBLOCKS_API_KEY=...
```

### Set your OpenAI API key

```bash
export OPENAI_API_KEY=...
```

### Run the tests

```bash
npx autoblocks testing exec -m "my first run" -- npm run start
```

You should see something like:

<img width="1107" alt="Screenshot 2024-03-01 at 5 53 27 PM" src="https://github.com/autoblocksai/autoblocks-examples/assets/7498009/92d50df8-7e9e-43dd-848c-f0711f20ce4b">

You can click on the links next to each test name to dig into more details.
You can also find all of your tests on the testing homepage in the [Autoblocks application](https://app.autoblocks.ai/testing/local).

## GitHub Actions setup

A starter workflow was added in [`.github/workflows/autoblocks-testing.yml`](./.github/workflows/autoblocks-testing.yml).
This workflow runs the tests on every push to the repository and also
on a daily schedule.

## Repo structure

```
src/
run.ts <-- imports all tests from test-suites/ and runs them
evaluators/ <-- all common evaluators are implemented here
some-shared-evaluator1.ts
some-shared-evaluator2.ts
tasks/ <-- all "tasks" are implemented here
task1.ts
task2.ts
test_suites/ <-- tests for each task
task1/
index.ts <-- implements the runner for task1
evaluators.ts <-- evaluators used only for task1
test-cases.ts <-- contains test cases for task1
task2/
index.ts <-- implements the runner for task2
evaluators.ts <-- evaluators used only for task2
test-cases.ts <-- contains test cases for task2
```

## Futher Reading

- [Autoblocks Testing documentation](https://docs.autoblocks.ai/testing/sdks)
2 changes: 1 addition & 1 deletion JavaScript/testing-sdk/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "testing-sdk",
"description": "Using the Autoblocks Testing SDK for experimentation and regression testing",
"description": "Using the Autoblocks Testing SDK with the Prompt SDK for experimentation, regression testing, and rapid prototyping.",
"version": "0.0.0",
"private": true,
"engines": {
Expand Down
2 changes: 1 addition & 1 deletion Python/testing-sdk-with-prompt-sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,7 @@ on a daily schedule.

```
my_project/
run.py <-- imports all tests from tests/ and runs them
run.py <-- imports all tests from test_suites/ and runs them
evaluators/ <-- all common evaluators are implemented here
some_shared_evaluator1.py
some_shared_evaluator2.py
Expand Down
2 changes: 1 addition & 1 deletion Python/testing-sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ on a daily schedule.

```
my_project/
run.py <-- imports all tests from tests/ and runs them
run.py <-- imports all tests from test_suites/ and runs them
evaluators/ <-- all common evaluators are implemented here
some_shared_evaluator1.py
some_shared_evaluator2.py
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
| [prompt-sdk-headless](/JavaScript/prompt-sdk-headless) | Typesafe and backwards-compatability-ensured headless prompt management |
| [prompt-sdk-local](/JavaScript/prompt-sdk-local) | Automated prompt versioning and typesafe prompt building with the local prompt SDK |
| [spans](/JavaScript/spans) | Establish parent / child relationships between your events with the `spanId` and `parentSpanId` properties |
| [testing-sdk](/JavaScript/testing-sdk) | Using the Autoblocks Testing SDK with the Prompt SDK for experimentation, regression testing, and rapid prototyping. |

<!-- JavaScript end -->

Expand Down

0 comments on commit f0553e9

Please sign in to comment.