Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

51 add shared variables documentation #52

Merged
merged 2 commits into from
Dec 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/User Guide/data_variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,6 @@ experiment = Experiment([train_block])

experiment.to_html("stroop.html")
```

There is one additional variable type, that we can use: The shared variable. It is a special variable that can be shared across different trials and can be updated via side effects:
[Shared Variables And Side Effects](data_variables.md)
6 changes: 2 additions & 4 deletions docs/User Guide/llm_synthetic_participant.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ pipe = transformers.pipeline(


def generate(input):
return pipe(input)[0]["generated_text"]
return pipe(input)[0]["generated_text"][len(input):]
```

Now, we can run the experiment on the model:
Expand All @@ -113,7 +113,5 @@ data = experiment.run_on_language(get_input=generate)
```

**Note**: The `run_on_language` function will return a dictionary with the data from the experiment. Any model
(for example,
using [OpenAI](https://platform.openai.com/docs/overview), [HuggingFace](https://huggingface.co/), [LLama](https://www.llama-api.com/)
or [Google](https://console.cloud.google.com/apis/library) API) can be used as a synthetic participant.
(for example, using [OpenAI](https://platform.openai.com/docs/overview), [HuggingFace](https://huggingface.co/), [LLama](https://www.llama-api.com/), or [Google](https://console.cloud.google.com/apis/library) API) can be used as a synthetic participant.

84 changes: 84 additions & 0 deletions docs/User Guide/shared_variables_and_side_effects.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Shared Variables And Side Effects

Shared Variables are variables that can be shared across different trials. They can be updated via Side Effects. This can be useful, for example, when we want to keep track of the participant's score.

Assume, we have a two-armed bandit task and want to present the score of the participant after each choice.

As in the previous examples, we first define the timeline and the timeline variables:
```python
from sweetbean import Block, Experiment
from sweetbean.stimulus import Bandit, Text
from sweetbean.variable import (
DataVariable,
FunctionVariable,
SharedVariable,
SideEffect,
TimelineVariable,
)

timeline = [
{
"bandit_1": {"color": "orange", "value": 10},
"bandit_2": {"color": "blue", "value": 0},
},
{
"bandit_1": {"color": "orange", "value": 9},
"bandit_2": {"color": "blue", "value": 1},
},
{
"bandit_1": {"color": "orange", "value": 8},
"bandit_2": {"color": "blue", "value": 2},
},
{
"bandit_1": {"color": "orange", "value": 7},
"bandit_2": {"color": "blue", "value": 3},
},
{
"bandit_1": {"color": "orange", "value": 6},
"bandit_2": {"color": "blue", "value": 4},
},
]

bandit_1 = TimelineVariable("bandit_1")
bandit_2 = TimelineVariable("bandit_2")
```

Next, we define the shared variable `score`. The start value will be 0, and we will update it with the `update_score` function by adding the `value` of the chosen bandit to the score. Since we need the `value` of the chosen bandit, we also define a data variable that gets the value with the window 0 (current trial).

```python

score = SharedVariable("score", 0)
value = DataVariable("value", 0)

updated_score = FunctionVariable(
"updated_score", lambda sc, val: sc + val, [score, value]
)
```

We can now create a side effect that updates the score with the `update_score` function. We pass the score and the function to the side effect. A side effects takes in the variable to set (in this case the score) and the variable it will be set to (in this case the function variable `update_score`).

We can pass in as many side effects as we want to a stimulus as a list of side effects. Here, we only have one side effect that updates the score.

```python
update_score_side_effect = SideEffect(score, updated_score)

bandit_task = Bandit(
bandits=[bandit_1, bandit_2, bandit_3],
side_effects=[update_score_side_effect],
)
```

Finally, we can show the score after each choice with a text stimulus. We pass the score to the text stimulus.

```python
show_score = Text(duration=1000, text=score)
```

Then we create the experiment as before and export it as an html file.

```python

trial_sequence = Block([bandit_task, show_score], timeline=timeline)
experiment = Experiment([trial_sequence])
experiment.to_html("bandit.html")
```
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ nav:
- Timeline Variables: 'User Guide/timeline_variables.md'
- Function Variables: 'User Guide/function_variables.md'
- Data Variables: 'User Guide/data_variables.md'
- Shared Variables And Side Effects: 'User Guide/shared_variables_and_side_effects.md'
- LLM Synthetic Participant: 'User Guide/llm_synthetic_participant.md'
- Stimuli: Stimuli/
- Contributor Guide: Contributor Guide/
Expand Down