Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add shared variables and side effects #53

Merged
merged 1 commit into from
Dec 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .github/workflows/joss-paper.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Draft PDF
on:
workflow_dispatch

jobs:
paper:
runs-on: ubuntu-latest
name: Paper Draft
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Build draft PDF
uses: openjournals/openjournals-draft-action@master
with:
journal: joss
# This should be the path to the paper within your repo.
paper-path: paper/paper.md
- name: Upload
uses: actions/upload-artifact@v4
with:
name: paper
# This is the output path where Pandoc will write the compiled
# PDF. Note, this should be the same directory as the input
# paper.md
path: paper/paper.pdf
27 changes: 27 additions & 0 deletions paper/paper.bib
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
@article{de2015jspsych,
title={jsPsych: A JavaScript library for creating behavioral experiments in a Web browser},
author={De Leeuw, Joshua R},
journal={Behavior research methods},
volume={47},
pages={1--12},
year={2015},
publisher={Springer}
}

@inproceedings{
demircan2024evaluating,
title={Evaluating alignment between humans and neural network representations in image-based learning tasks},
author={Can Demircan and Tankred Saanum and Leonardo Pettini and Marcel Binz and Blazej M Baczkowski and Christian F. Doeller and Mona M. Garvert and Eric Schulz},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=8i6px5W1Rf}
}

@article{musslick2020sweetpea,
title={SweetPea: A standard language for factorial experimental design},
author={Musslick, Sebastian and Cherkaev, Anastasia and Draut, Ben and Butt, Ahsan Sajjad and Darragh, Pierce and Srikumar, Vivek and Flatt, Matthew and Cohen, Jonathan D},
journal={Behavior Research Methods},
pages={1--25},
year={2020},
publisher={Springer}
}
43 changes: 43 additions & 0 deletions paper/paper.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
title: 'SweetBean: A declarative language for behavioral experiments with human and artificial participants'
tags:
- Python
- online behavioral experiments
- large-language model experiments
- declarative language
- synthetic participants
authors:
- name: Younes Strittmatter
orcid: 0000-0002-3414-2838
corresponding: true
affiliation: 1
- name: Sebastian Musslick
orcid: 0000-0002-8896-639X
affiliation: "2, 3"
affiliations:
- name: Princeton University, USA
index: 1
- name: Osnabrück University, Germany
index: 2
- name: University of Chicago, USA
index: 3
date: 13 August 2017
bibliography: paper.bib

---

# Summary

`SweetBean` is an open-source, domain-specific declarative programming language built in Python, designed for the synthesis of behavioral experiments. It allows researchers to specify experiments once and seamlessly compile them into a `jsPsych` experiment [@de2015jspsych] for conducting studies with human participants or text-based simulations with synthetic participants using large-language models.

The `SweetBean` package seamlessly integrates with packages aimed at running online behavioral experiments, such as `AutoRA` [@AUTORA-MISSING] for automated research, participant recruitment, and data collection, and `SweetPea` [@musslick2020sweetpea] for experimental design.

# Statement of need

The design and implementation of behavioral experiments often involve complex workflows, particularly when researchers aim to run the same experiments with both human participants and large-language models—a practice that is becoming increasingly common, for example, to train models to simulate human behavior [@demircan2024evaluating] or to support AI alignment research. While platforms like `jsPsych` offer powerful solutions for running online experiments, they require knowledge of *JavaScript*, which poses a significant barrier for researchers without programming expertise. Additionally, integrating synthetic participants, such as large-language models, often necessitates ad hoc solutions, leading to inefficiencies and errors due to the need for replicating the same experiment across different codebases.

`SweetBean` offers a *Python*-based, declarative programming language specifically designed for behavioral experiment design. Python’s accessibility and widespread use in the behavioral sciences make `SweetBean` an ideal tool for researchers and educators. By enabling users to specify experiments in a single, concise format, SweetBean simplifies workflows and ensures compatibility with both human and synthetic participants.

Beyond reducing technical barriers, `SweetBean` enhances reproducibility and collaboration by standardizing experiment specifications within a flexible and intuitive framework. This standardization not only saves time but also fosters more robust and scalable experimental workflows for the research community.

# References