- Create instance & install neccessary dependencies
- Clone our own repo with fewshotbench_v2 and comet on instance
- Include comet into fewshotbench and make neccessary changes to run it as a method on tabula_muris
- Implement mapcell as a method (implemented from scratch) and run it on tabula_muris and swissprot
- Run "benchmarks_script.py" to run all benchmarks on original mail.yaml configurations (5 shot, 5 way)
- Change 5 shot to 1 shot in main.yaml
- Run "benchmarks_script.py" to run all benchmarks for 1 shot 5 way
- Run "new_methods_runs.py" script to hypertune parameters for new methods (comet & mapcell) (5 shot, 5 way)
- Run "new_methods_runs.py" script to hypertune parameters for new methods (comet & mapcell) (1 shot, 5 way)
- Choose best models (hyperparameters) for both methods
- For comet: Change euclidean distance to manhatten distance and run again on same hyperparameters
- For comet: Change backbone from EnFCNet to EnFCNet_4 and run again on same hyperparameters
- For mapcell: Change euclidean distance to manhatten distance and run again on same hyperparameters
- For mapcell: Change margin for contrastive loss and run again on same hyperparameters
in /home/timwiebelhaus/CS-502-DL-in-Biomedicine/fewshotbench_v2
Do these changes:
-
conf/dataset/tabula_muris.yaml
- uncomment target: backbones.fcnet.EnFCNet
- comment out target: backbones.fcnet.FCNet
- comment out layer_dim: [ 64, 64 ]
-
conf/dataset/swissprot.yaml
- uncomment target: backbones.fcnet.EnFCNet
- comment out target: backbones.fcnet.FCNet
- comment out layer_dim: [ 64, 64 ]
-
conf/main.yaml
- uncomment model: EnFCNet
- comment out model: FCNet
--> automated in script comet_changes_script.py