From d87a4d1b96656c75b2035ffb9c16816e6a014baf Mon Sep 17 00:00:00 2001 From: Younes Strittmatter Date: Sun, 1 Dec 2024 16:59:47 -0500 Subject: [PATCH 1/2] docs: add shared variables and side effects --- docs/User Guide/data_variables.md | 3 + .../shared_variables_and_side_effects.md | 84 +++++++++++++++++++ mkdocs.yml | 1 + 3 files changed, 88 insertions(+) create mode 100644 docs/User Guide/shared_variables_and_side_effects.md diff --git a/docs/User Guide/data_variables.md b/docs/User Guide/data_variables.md index b8a7fb76..673be667 100644 --- a/docs/User Guide/data_variables.md +++ b/docs/User Guide/data_variables.md @@ -72,3 +72,6 @@ experiment = Experiment([train_block]) experiment.to_html("stroop.html") ``` + +There is one additional variable type, that we can use: The shared variable. It is a special variable that can be shared across different trials and can be updated via side effects: +[Shared Variables And Side Effects](data_variables.md) diff --git a/docs/User Guide/shared_variables_and_side_effects.md b/docs/User Guide/shared_variables_and_side_effects.md new file mode 100644 index 00000000..838c4b6a --- /dev/null +++ b/docs/User Guide/shared_variables_and_side_effects.md @@ -0,0 +1,84 @@ +# Shared Variables And Side Effects + +Shared Variables are variables that can be shared across different trials. They can be updated via Side Effects. This can be useful, for example, when we want to keep track of the participant's score. + +Assume, we have a two-armed bandit task and want to present the score of the participant after each choice. + +As in the previous examples, we first define the timeline and the timeline variables: +```python +from sweetbean import Block, Experiment +from sweetbean.stimulus import Bandit, Text +from sweetbean.variable import ( + DataVariable, + FunctionVariable, + SharedVariable, + SideEffect, + TimelineVariable, +) + +timeline = [ + { + "bandit_1": {"color": "orange", "value": 10}, + "bandit_2": {"color": "blue", "value": 0}, + }, + { + "bandit_1": {"color": "orange", "value": 9}, + "bandit_2": {"color": "blue", "value": 1}, + }, + { + "bandit_1": {"color": "orange", "value": 8}, + "bandit_2": {"color": "blue", "value": 2}, + }, + { + "bandit_1": {"color": "orange", "value": 7}, + "bandit_2": {"color": "blue", "value": 3}, + }, + { + "bandit_1": {"color": "orange", "value": 6}, + "bandit_2": {"color": "blue", "value": 4}, + }, +] + +bandit_1 = TimelineVariable("bandit_1") +bandit_2 = TimelineVariable("bandit_2") +``` + +Next, we define the shared variable `score`. The start value will be 0, and we will update it with the `update_score` function by adding the `value` of the chosen bandit to the score. Since we need the `value` of the chosen bandit, we also define a data variable that gets the value with the window 0 (current trial). + +```python + +score = SharedVariable("score", 0) +value = DataVariable("value", 0) + +updated_score = FunctionVariable( + "updated_score", lambda sc, val: sc + val, [score, value] +) +``` + +We can now create a side effect that updates the score with the `update_score` function. We pass the score and the function to the side effect. A side effects takes in the variable to set (in this case the score) and the variable it will be set to (in this case the function variable `update_score`). + +We can pass in as many side effects as we want to a stimulus as a list of side effects. Here, we only have one side effect that updates the score. + +```python +update_score_side_effect = SideEffect(score, updated_score) + +bandit_task = Bandit( + bandits=[bandit_1, bandit_2, bandit_3], + side_effects=[update_score_side_effect], +) +``` + +Finally, we can show the score after each choice with a text stimulus. We pass the score to the text stimulus. + +```python +show_score = Text(duration=1000, text=score) +``` + +Then we create the experiment as before and export it as an html file. + +```python + +trial_sequence = Block([bandit_task, show_score], timeline=timeline) +experiment = Experiment([trial_sequence]) +experiment.to_html("bandit.html") +``` \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index ad880658..caf8ae5d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -87,6 +87,7 @@ nav: - Timeline Variables: 'User Guide/timeline_variables.md' - Function Variables: 'User Guide/function_variables.md' - Data Variables: 'User Guide/data_variables.md' + - Shared Variables And Side Effects: 'User Guide/shared_variables_and_side_effects.md' - LLM Synthetic Participant: 'User Guide/llm_synthetic_participant.md' - Stimuli: Stimuli/ - Contributor Guide: Contributor Guide/ From f67f629b2b35d15e68a338b3fb6b0c1950a4bdc0 Mon Sep 17 00:00:00 2001 From: Younes Strittmatter Date: Sun, 1 Dec 2024 17:01:09 -0500 Subject: [PATCH 2/2] docs: add shared variables and side effects --- docs/User Guide/llm_synthetic_participant.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/User Guide/llm_synthetic_participant.md b/docs/User Guide/llm_synthetic_participant.md index b0d46921..8e5ba57b 100644 --- a/docs/User Guide/llm_synthetic_participant.md +++ b/docs/User Guide/llm_synthetic_participant.md @@ -103,7 +103,7 @@ pipe = transformers.pipeline( def generate(input): - return pipe(input)[0]["generated_text"] + return pipe(input)[0]["generated_text"][len(input):] ``` Now, we can run the experiment on the model: @@ -113,7 +113,5 @@ data = experiment.run_on_language(get_input=generate) ``` **Note**: The `run_on_language` function will return a dictionary with the data from the experiment. Any model -(for example, -using [OpenAI](https://platform.openai.com/docs/overview), [HuggingFace](https://huggingface.co/), [LLama](https://www.llama-api.com/) -or [Google](https://console.cloud.google.com/apis/library) API) can be used as a synthetic participant. +(for example, using [OpenAI](https://platform.openai.com/docs/overview), [HuggingFace](https://huggingface.co/), [LLama](https://www.llama-api.com/), or [Google](https://console.cloud.google.com/apis/library) API) can be used as a synthetic participant.