Skip to content

Commit

Permalink
Merge pull request #52 from AutoResearch/51-add-shared-variables-docu…
Browse files Browse the repository at this point in the history
…mentation

51 add shared variables documentation
  • Loading branch information
younesStrittmatter authored Dec 1, 2024
2 parents 44b6fd3 + f67f629 commit 043761a
Show file tree
Hide file tree
Showing 4 changed files with 90 additions and 4 deletions.
3 changes: 3 additions & 0 deletions docs/User Guide/data_variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,6 @@ experiment = Experiment([train_block])

experiment.to_html("stroop.html")
```

There is one additional variable type, that we can use: The shared variable. It is a special variable that can be shared across different trials and can be updated via side effects:
[Shared Variables And Side Effects](data_variables.md)
6 changes: 2 additions & 4 deletions docs/User Guide/llm_synthetic_participant.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ pipe = transformers.pipeline(


def generate(input):
return pipe(input)[0]["generated_text"]
return pipe(input)[0]["generated_text"][len(input):]
```

Now, we can run the experiment on the model:
Expand All @@ -113,7 +113,5 @@ data = experiment.run_on_language(get_input=generate)
```

**Note**: The `run_on_language` function will return a dictionary with the data from the experiment. Any model
(for example,
using [OpenAI](https://platform.openai.com/docs/overview), [HuggingFace](https://huggingface.co/), [LLama](https://www.llama-api.com/)
or [Google](https://console.cloud.google.com/apis/library) API) can be used as a synthetic participant.
(for example, using [OpenAI](https://platform.openai.com/docs/overview), [HuggingFace](https://huggingface.co/), [LLama](https://www.llama-api.com/), or [Google](https://console.cloud.google.com/apis/library) API) can be used as a synthetic participant.

84 changes: 84 additions & 0 deletions docs/User Guide/shared_variables_and_side_effects.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Shared Variables And Side Effects

Shared Variables are variables that can be shared across different trials. They can be updated via Side Effects. This can be useful, for example, when we want to keep track of the participant's score.

Assume, we have a two-armed bandit task and want to present the score of the participant after each choice.

As in the previous examples, we first define the timeline and the timeline variables:
```python
from sweetbean import Block, Experiment
from sweetbean.stimulus import Bandit, Text
from sweetbean.variable import (
DataVariable,
FunctionVariable,
SharedVariable,
SideEffect,
TimelineVariable,
)

timeline = [
{
"bandit_1": {"color": "orange", "value": 10},
"bandit_2": {"color": "blue", "value": 0},
},
{
"bandit_1": {"color": "orange", "value": 9},
"bandit_2": {"color": "blue", "value": 1},
},
{
"bandit_1": {"color": "orange", "value": 8},
"bandit_2": {"color": "blue", "value": 2},
},
{
"bandit_1": {"color": "orange", "value": 7},
"bandit_2": {"color": "blue", "value": 3},
},
{
"bandit_1": {"color": "orange", "value": 6},
"bandit_2": {"color": "blue", "value": 4},
},
]

bandit_1 = TimelineVariable("bandit_1")
bandit_2 = TimelineVariable("bandit_2")
```

Next, we define the shared variable `score`. The start value will be 0, and we will update it with the `update_score` function by adding the `value` of the chosen bandit to the score. Since we need the `value` of the chosen bandit, we also define a data variable that gets the value with the window 0 (current trial).

```python

score = SharedVariable("score", 0)
value = DataVariable("value", 0)

updated_score = FunctionVariable(
"updated_score", lambda sc, val: sc + val, [score, value]
)
```

We can now create a side effect that updates the score with the `update_score` function. We pass the score and the function to the side effect. A side effects takes in the variable to set (in this case the score) and the variable it will be set to (in this case the function variable `update_score`).

We can pass in as many side effects as we want to a stimulus as a list of side effects. Here, we only have one side effect that updates the score.

```python
update_score_side_effect = SideEffect(score, updated_score)

bandit_task = Bandit(
bandits=[bandit_1, bandit_2, bandit_3],
side_effects=[update_score_side_effect],
)
```

Finally, we can show the score after each choice with a text stimulus. We pass the score to the text stimulus.

```python
show_score = Text(duration=1000, text=score)
```

Then we create the experiment as before and export it as an html file.

```python

trial_sequence = Block([bandit_task, show_score], timeline=timeline)
experiment = Experiment([trial_sequence])
experiment.to_html("bandit.html")
```
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ nav:
- Timeline Variables: 'User Guide/timeline_variables.md'
- Function Variables: 'User Guide/function_variables.md'
- Data Variables: 'User Guide/data_variables.md'
- Shared Variables And Side Effects: 'User Guide/shared_variables_and_side_effects.md'
- LLM Synthetic Participant: 'User Guide/llm_synthetic_participant.md'
- Stimuli: Stimuli/
- Contributor Guide: Contributor Guide/
Expand Down

0 comments on commit 043761a

Please sign in to comment.