Dev (ashleve#125)

* add conference badges * change TestDataset to ImgDataset * update conda env * bump package versions * update README * move optuna config to subfolder
aksl-ow · Apr 15, 2021 · e27c11c · e27c11c
1 parent 296c5bf
commit e27c11c
Show file tree

Hide file tree

Showing 8 changed files with 95 additions and 87 deletions.
diff --git a/README.md b/README.md
@@ -12,7 +12,7 @@
 <a href="https://black.readthedocs.io/en/stable/"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-black.svg?style=for-the-badge"></a>
 
 A clean and scalable template to kickstart your deep learning project 🚀⚡🔥<br>
-Click on [<kbd>Use this template</kbd>](https://github.com/hobogalaxy/lightning-hydra-template/generate) to initialize new repository.
+Click on [<kbd>Use this template</kbd>](https://github.com/ashleve/lightning-hydra-template/generate) to initialize new repository.
 
 *Currently uses dev version of Hydra.<br>Suggestions are always welcome!*
 
@@ -21,7 +21,7 @@ Click on [<kbd>Use this template</kbd>](https://github.com/hobogalaxy/lightning-
 
 <!--
 If you use this template please add <br>
-[![](https://shields.io/badge/-Lightning--Hydra--Template-017F2F?style=flat&logo=github&labelColor=303030)](https://github.com/hobogalaxy/lightning-hydra-template) <br>
+[![](https://shields.io/badge/-Lightning--Hydra--Template-017F2F?style=flat&logo=github&labelColor=303030)](https://github.com/ashleve/lightning-hydra-template) <br>
 to your `README.md`.
 <br><br>
 -->
@@ -69,12 +69,12 @@ The directory structure of new project looks like this:
 │   ├── callbacks               <- Callbacks configs
 │   ├── datamodule              <- Datamodule configs
 │   ├── experiment              <- Experiment configs
+│   ├── hparams_search          <- Hyperparameter search configs
 │   ├── logger                  <- Logger configs
 │   ├── model                   <- Model configs
 │   ├── trainer                 <- Trainer configs
 │   │
-│   ├── config.yaml             <- Main project configuration file
-│   └── config_optuna.yaml      <- Configuration of Optuna hyperparameter search
+│   └── config.yaml             <- Main project configuration file
 │
 ├── data                    <- Project data
 │
@@ -128,7 +128,7 @@ Template contains example with MNIST classification.<br>
 When running `python run.py` you should see something like this:
 <div align="center">
 
-![](https://github.com/hobogalaxy/lightning-hydra-template/blob/resources/terminal.png)
+![](https://github.com/ashleve/lightning-hydra-template/blob/resources/terminal.png)
 
 </div>
 
@@ -205,7 +205,7 @@ python run.py logger=wandb
 
 > Experiment configurations are placed in [configs/experiment/](configs/experiment/).
 ```yaml
-python run.py +experiment=exp_example_simple
+python run.py experiment=exp_example_simple
 ```
 
 </details>
@@ -301,9 +301,9 @@ python run.py -m datamodule.batch_size=32,64,128 optimizer.lr=0.001,0.0005
 
 > Using [Optuna Sweeper](https://hydra.cc/docs/next/plugins/optuna_sweeper) plugin doesn't require you to code any boilerplate into your pipeline, everything is defined in a [single config file](configs/config_optuna.yaml)!
 ```yaml
-# this will run hyperparameter search defined in `configs/config_optuna.yaml`
+# this will run hyperparameter search defined in `configs/hparams_search/mnist_optuna.yaml`
 # over chosen experiment config
-python run.py -m --config-name config_optuna.yaml +experiment=exp_example_simple
+python run.py -m hparams_search=mnist_optuna experiment=exp_example_simple
 ```
 
 </details>
@@ -313,7 +313,7 @@ python run.py -m --config-name config_optuna.yaml +experiment=exp_example_simple
 
 > Hydra provides special syntax for controlling behavior of multiruns. Learn more [here](https://hydra.cc/docs/next/tutorials/basic/running_your_app/multi-run). The command below executes all experiments from folder [configs/experiment/](configs/experiment/).
 ```yaml
-python run.py -m '+experiment=glob(*)'
+python run.py -m 'experiment=glob(*)'
 ```
 
 </details>
@@ -362,7 +362,7 @@ If you want to use some popular official image instead, I recommend the [nvidia
 
 
 ## :heart:&nbsp; Contributions
-Have a question? Found a bug? Missing a specific feature? Ran into a problem? Feel free to file a new issue with respective title and description. If you already found a solution to your problem, don't hesitate to share it. Suggestions for new best practices and tricks are always welcome!
+Have a question? Found a bug? Missing a specific feature? Ran into a problem? Feel free to file a new issue or PR with respective title and description. If you already found a solution to your problem, don't hesitate to share it. Suggestions for new best practices and tricks are always welcome!
 <br><br><br><br>
 
 
@@ -656,12 +656,7 @@ List of extra utilities available in the template:
 - method for logging hyperparameters to loggers
 - (TODO) resuming latest run
 
-You can easily remove all of those by modifying [run.py](run.py) and [src/train.py](src/train.py).
-<br><br>
-
-
-### How To Speed Up Training
-(TODO)
+You can easily remove any of those by modifying [run.py](run.py) and [src/train.py](src/train.py).
 <br><br>
 
 
@@ -824,7 +819,7 @@ setup(
     author="",
     author_email="",
     # replace with your own github project link
-    url="https://github.com/hobogalaxy/lightning-hydra-template",
+    url="https://github.com/ashleve/lightning-hydra-template",
     install_requires=["pytorch-lightning>=1.2.0", "hydra-core>=1.0.6"],
     packages=find_packages(),
 )
@@ -994,7 +989,9 @@ This template was inspired by:
 <a href="https://pytorch.org/get-started/locally/"><img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-ee4c2c?logo=pytorch&logoColor=white"></a>
 <a href="https://pytorchlightning.ai/"><img alt="Lightning" src="https://img.shields.io/badge/-Lightning-792ee5"></a>
 <a href="https://hydra.cc/"><img alt="Config: Hydra" src="https://img.shields.io/badge/Config-Hydra-89b8cd"></a>
-<a href="https://github.com/hobogalaxy/lightning-hydra-template"><img alt="Template" src="https://img.shields.io/badge/-Lightning--Hydra--Template-017F2F?style=flat&logo=github&labelColor=gray"></a>
+<a href="https://github.com/ashleve/lightning-hydra-template"><img alt="Template" src="https://img.shields.io/badge/-Lightning--Hydra--Template-017F2F?style=flat&logo=github&labelColor=gray"></a><br>
+[![Paper](http://img.shields.io/badge/paper-arxiv.1001.2234-B31B1B.svg)](https://www.nature.com/articles/nature14539)
+[![Conference](http://img.shields.io/badge/AnyConference-year-4b44ce.svg)](https://papers.nips.cc/paper/2020)
 
 </div>
 

diff --git a/conda_env_gpu.yaml b/conda_env_gpu.yaml
@@ -3,7 +3,6 @@ name: myenv
 channels:
   - pytorch
   - conda-forge
-  - defaults
 
 dependencies:
   - python=3.8
@@ -13,3 +12,5 @@ dependencies:
   - cudatoolkit
   - pytorch=1.8.1
   - torchvision=0.9.1
+  - pip:
+      - -r requirements.txt
diff --git a/configs/config.yaml b/configs/config.yaml
@@ -8,9 +8,12 @@ defaults:
     - callbacks: default_callbacks.yaml  # set this to null if you don't want to use callbacks
     - logger: null  # set logger here or use command line (e.g. `python run.py logger=wandb`)
 
+    - experiment: null
+    - hparams_search: null
+
     # enable color logging
-    # - override hydra/hydra_logging: colorlog
-    # - override hydra/job_logging: colorlog
+    - override hydra/hydra_logging: colorlog
+    - override hydra/job_logging: colorlog
 
 
 # path to original working directory

diff --git a/configs/config_optuna.yaml b/configs/config_optuna.yaml
diff --git a/configs/hparams_search/mnist_optuna.yaml b/configs/hparams_search/mnist_optuna.yaml
@@ -0,0 +1,63 @@
+# @package _global_
+
+# example hyperparameter optimization of some experiment with Optuna:
+# python run.py -m hparams_search=mnist_optuna experiment=exp_example_simple
+# python run.py -m hparams_search=mnist_optuna experiment=exp_example_simple hydra.sweeper.n_trials=30
+# python run.py -m hparams_search=mnist_optuna experiment=exp_example_simple logger=wandb
+
+defaults:
+    - override /hydra/sweeper: optuna
+
+
+# choose metric which will be optimized by Optuna
+optimized_metric: "val/acc_best"
+
+
+hydra:
+    # here we define Optuna hyperparameter search
+    # it optimizes for value returned from function with @hydra.main decorator
+    # learn more here: https://hydra.cc/docs/next/plugins/optuna_sweeper
+    sweeper:
+        _target_: hydra_plugins.hydra_optuna_sweeper.optuna_sweeper.OptunaSweeper
+        storage: null
+        study_name: null
+        n_jobs: 1
+
+        # 'minimize' or 'maximize' the objective
+        direction: maximize
+
+        # number of experiments that will be executed
+        n_trials: 20
+
+        # choose Optuna hyperparameter sampler
+        # learn more here: https://optuna.readthedocs.io/en/stable/reference/samplers.html
+        sampler:
+            _target_: optuna.samplers.TPESampler
+            seed: 12345
+            consider_prior: true
+            prior_weight: 1.0
+            consider_magic_clip: true
+            consider_endpoints: false
+            n_startup_trials: 10
+            n_ei_candidates: 24
+            multivariate: false
+            warn_independent_sampling: true
+
+        # define range of hyperparameters
+        search_space:
+            datamodule.batch_size:
+                type: categorical
+                choices: [32, 64, 128]
+            model.lr:
+                type: float
+                low: 0.0001
+                high: 0.2
+            model.lin1_size:
+                type: categorical
+                choices: [32, 64, 128, 256, 512]
+            model.lin2_size:
+                type: categorical
+                choices: [32, 64, 128, 256, 512]
+            model.lin3_size:
+                type: categorical
+                choices: [32, 64, 128, 256, 512]
diff --git a/requirements.txt b/requirements.txt
@@ -1,17 +1,17 @@
 # --------- pytorch --------- #
 torch==1.8.1
 torchvision==0.9.1
-pytorch-lightning>=1.2.5
+pytorch-lightning>=1.2.7
 
 # --------- hydra --------- #
 hydra-core==1.1.0.dev5
-hydra-colorlog==1.0.0
-hydra-optuna-sweeper==0.9.0rc2
+hydra-colorlog==1.1.0.dev1
+hydra-optuna-sweeper==1.1.0.dev1
 # hydra-ray-launcher==0.1.2
 # hydra-submitit-launcher==1.1.0
 
 # --------- loggers --------- #
-wandb>=0.10.20
+wandb>=0.10.26
 # neptune-client
 # comet-ml
 # mlflow
@@ -24,6 +24,6 @@ scikit-learn
 pandas
 matplotlib
 seaborn
-pudb
-pytest
+# pudb
+# pytest
 # dvc
diff --git a/src/datamodules/datasets/img_dataset.py b/src/datamodules/datasets/img_dataset.py
@@ -5,7 +5,7 @@
 from torch.utils.data import Dataset
 
 
-class TestDataset(Dataset):
+class ImgDataset(Dataset):
     """Example dataset class for loading images from folder."""
 
     def __init__(self, dir: str, transform: Optional[Callable] = None):

diff --git a/src/datamodules/mnist_datamodule.py b/src/datamodules/mnist_datamodule.py
@@ -9,14 +9,14 @@
 class MNISTDataModule(LightningDataModule):
     """
     Example of LightningDataModule for MNIST dataset.
-    
+
     A DataModule implements 5 key methods:
         - prepare_data (things to do on 1 GPU/TPU, not on every GPU/TPU in distributed mode)
         - setup (things to do on every accelerator in distributed mode)
         - train_dataloader (the training dataloader)
         - val_dataloader (the validation dataloader(s))
         - test_dataloader (the test dataloader(s))
-    
+
     This allows you to share a full dataset without explaining how to download,
     split, transform and process the data
 
@@ -31,7 +31,7 @@ def __init__(
         batch_size: int = 64,
         num_workers: int = 0,
         pin_memory: bool = False,
-        **kwargs
+        **kwargs,
     ):
         super().__init__()