Skip to content

Commit

Permalink
Update baseline_request.yml (#2614)
Browse files Browse the repository at this point in the history
  • Loading branch information
jafermarq authored Nov 17, 2023
1 parent 8b4d7a7 commit 77b715a
Showing 1 changed file with 11 additions and 21 deletions.
32 changes: 11 additions & 21 deletions .github/ISSUE_TEMPLATE/baseline_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,32 +41,22 @@ body:
- [ ] Read the [`first contribution` doc](https://flower.dev/docs/first-time-contributors.html)
- [ ] Complete the Flower tutorial
- [ ] Read the Flower Baselines docs to get an overview:
- [ ] [https://flower.dev/docs/using-baselines.html](https://flower.dev/docs/using-baselines.html)
- [ ] [https://flower.dev/docs/contributing-baselines.html](https://flower.dev/docs/contributing-baselines.html)
- [ ] [How to use Flower Baselines](https://flower.dev/docs/baselines/how-to-use-baselines.html)
- [ ] [How to contribute a Flower Baseline](https://flower.dev/docs/baselines/how-to-contribute-baselines.html)
- type: checkboxes
attributes:
label: Prepare - understand the scope
options:
- label: Read the paper linked above
- label: Create the directory structure in Flower Baselines (just the `__init__.py` files and a `README.md`)
- label: Before starting to write code, write down all of the specs of this experiment in a README (dataset, partitioning, model, number of clients, all hyperparameters, …)
- label: Open a draft PR
- label: Decide which experiments you'd like to reproduce. The more the better!
- label: Follow the steps outlined in [Add a new Flower Baseline](https://flower.dev/docs/baselines/how-to-contribute-baselines.html#add-a-new-flower-baseline).
- label: You can use as reference [other baselines](https://github.com/adap/flower/tree/main/baselines) that the community merged following those steps.
- type: checkboxes
attributes:
label: Implement - make it work
label: Verify your implementation
options:
- label: Implement some form of dataset loading and partitioning in a separate `dataset.py` (doesn’t have to match the paper exactly)
- label: Implement the model in PyTorch
- label: Write a test that shows that the model has the number of parameters mentioned in the paper
- label: Implement the federated learning setup outlined in the paper, maybe starting with fewer clients
- label: Plot accuracy and loss
- label: Run it and check if the model starts to converge
- type: checkboxes
attributes:
label: Align - make it converge
options:
- label: Implement the exact data partitioning outlined in the paper
- label: Use the exact hyperparameters outlined in the paper
- label: Make it converge to roughly the same accuracy that the paper states
- label: Commit the final hyperparameters and plots
- label: Mark the PR as ready
- label: Follow the steps indicated in the `EXTENDED_README.md` that was created in your baseline directory
- label: Ensure your code reproduces the results for the experiments you chose
- label: Ensure your `README.md` is ready to be run by someone that is no familiar with your code. Are all step-by-step instructions clear?
- label: Ensure running the formatting and typing tests for your baseline runs without errors.
- label: Clone your repo on a new directory, follow the guide on your own `README.md` and verify everything runs.

0 comments on commit 77b715a

Please sign in to comment.