Skip to content

Commit

Permalink
Make .env non-mandatory
Browse files Browse the repository at this point in the history
  • Loading branch information
@Eve-ning
Eve-ning committed Jun 26, 2024
1 parent 965b623 commit 6b49b5e
Show file tree
Hide file tree
Showing 19 changed files with 225 additions and 213 deletions.
5 changes: 5 additions & 0 deletions Writerside/topics/Get-Started-with-Dev-Containers.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,8 @@ steps such as:
- Google Cloud Application Default Credentials
- Weight & Bias API Key
- Label Studio API Key

> You can set the API Keys in the `.env` file in the root of the project.
> Be careful not to commit the `.env` file to the repository, which should
> have been ignored by default.
{style='note'}
260 changes: 134 additions & 126 deletions Writerside/topics/Getting-Started.md
View file
Original file line number Diff line number Diff line change
@@ -1,155 +1,161 @@
# Getting Started

> Want to use a Dev Container? See [Get Started with Dev Containers](Get-Started-with-Dev-Containers.md)
> Want to use a Dev Container?
> See [Get Started with Dev Containers](Get-Started-with-Dev-Containers.md)
<procedure title="Installing the Dev. Environment" id="install">
<step>Ensure that you have the right version of Python.
The required Python version can be seen in <code>pyproject.toml</code>
<code-block lang="ini">
[tool.poetry.dependencies]
python = "..."
</code-block>
</step>
<step>Start by cloning our repository.
<code-block lang="shell">
git clone https://github.com/FR-DC/FRDC-ML.git
</code-block>
</step>
<step>Then, create a Python Virtual Env <code>pyvenv</code>
<tabs>
<tab title="Windows">
<code-block lang="shell">python -m venv venv/</code-block>
</tab>
<tab title="Linux">
<code-block lang="shell">python3 -m venv venv/</code-block>
</tab>
</tabs>
</step>
<step>
<a href="https://python-poetry.org/docs/">Install Poetry</a>
Then check if it's installed with
<code-block lang="shell">poetry --version</code-block>
<warning>
If <code>poetry</code> is not found, it's likely not in the user PATH.
</warning>
</step>
<step>Activate the virtual environment
<tabs>
<tab title="Windows">
<step>Ensure that you have the right version of Python.
The required Python version can be seen in <code>pyproject.toml</code>
<code-block lang="ini">
[tool.poetry.dependencies]
python = "..."
</code-block>
</step>
<step>Start by cloning our repository.
<code-block lang="shell">
git clone https://github.com/FR-DC/FRDC-ML.git
</code-block>
</step>
<step>Then, create a Python Virtual Env <code>pyvenv</code>
<tabs>
<tab title="Windows">
<code-block lang="shell">python -m venv venv/</code-block>
</tab>
<tab title="Linux">
<code-block lang="shell">python3 -m venv venv/</code-block>
</tab>
</tabs>
</step>
<step>
<a href="https://python-poetry.org/docs/">Install Poetry</a>
Then check if it's installed with
<code-block lang="shell">poetry --version</code-block>
<warning>
If <code>poetry</code> is not found, it's likely not in the user PATH.
</warning>
</step>
<step>Activate the virtual environment
<tabs>
<tab title="Windows">
<code-block lang="shell">
cd venv/Scripts
activate
cd ../..
cd venv/Scripts
activate
cd ../..
</code-block>
</tab>
<tab title="Linux">
</tab>
<tab title="Linux">
<code-block lang="shell">
source venv/bin/activate
source venv/bin/activate
</code-block>
</tab>
</tabs>
</step>
<step>Install the dependencies. You should be in the same directory as
<code>pyproject.toml</code>
<code-block lang="shell">
poetry install --with dev
</code-block>
</step>
<step>Install Pre-Commit Hooks
<code-block lang="shell">
pre-commit install
</code-block>
</step>
</tab>
</tabs>
</step>
<step>Install the dependencies. You should be in the same directory as
<code>pyproject.toml</code>
<code-block lang="shell">
poetry install --with dev
</code-block>
</step>
<step>Install Pre-Commit Hooks
<code-block lang="shell">
pre-commit install
</code-block>
</step>
</procedure>


<procedure title="Setting Up Google Cloud" id="gcloud">
<step>
We use Google Cloud to store our datasets. To set up Google Cloud,
<a href="https://cloud.google.com/sdk/docs/install">
install the Google Cloud CLI
</a>
</step>
<step>
Then,
<a href="https://cloud.google.com/sdk/docs/initializing">
authenticate your account
</a>.
<code-block lang="shell">gcloud auth login</code-block>
</step>
<step>
Finally,
<a href="https://cloud.google.com/docs/authentication/provide-credentials-adc">
set up Application Default Credentials (ADC)
</a>.
<code-block lang="shell">gcloud auth application-default login</code-block>
</step>
<step>
To make sure everything is working, <a anchor="tests">run the tests</a>.
</step>
<step>
We use Google Cloud to store our datasets. To set up Google Cloud,
<a href="https://cloud.google.com/sdk/docs/install">
install the Google Cloud CLI
</a>
</step>
<step>
Then,
<a href="https://cloud.google.com/sdk/docs/initializing">
authenticate your account
</a>.
<code-block lang="shell">gcloud auth login</code-block>
</step>
<step>
Finally,
<a href="https://cloud.google.com/docs/authentication/provide-credentials-adc">
set up Application Default Credentials (ADC)
</a>.
<code-block lang="shell">gcloud auth application-default login</code-block>
</step>
<step>
To make sure everything is working, <a anchor="tests">run the tests</a>.
</step>
</procedure>

<procedure title="Setting Up Label Studio" id="ls">
<tip>This is only necessary if any task requires Label Studio annotations</tip>
<step>
We use Label Studio to annotate our datasets.
We won't go through how to install Label Studio, for contributors, it
should be up on <code>localhost:8080</code>.
</step>
<step>
Then, retrieve your own API key from Label Studio.
<a href="http://localhost:8080/user/account"> Go to your account page </a>
and copy the API key. <br/></step>
<step> Set your API key as an environment variable.
<tabs>
<tab title="Windows">
<tip>This is only necessary if any task requires Label Studio annotations</tip>
<step>
We use Label Studio to annotate our datasets.
We won't go through how to install Label Studio, for contributors, it
should be up on <code>localhost:8080</code>.
</step>
<step>
Then, retrieve your own API key from Label Studio.
<a href="http://localhost:8080/user/account"> Go to your account page </a>
and copy the API key. <br/></step>
<step> Set your API key as an environment variable.
<tabs>
<tab title="Windows">
In Windows, go to "Edit environment variables for
your account" and add this as a new environment variable with name
<code>LABEL_STUDIO_API_KEY</code>.
</tab>
<tab title="Linux">
</tab>
<tab title="Linux">
Export it as an environment variable.
<code-block lang="shell">export LABEL_STUDIO_API_KEY=...</code-block>
</tab>
</tabs>
</step>
</tab>
<tab title=".env">
In all cases, you can create a <code>.env</code> file in the root of
the project and add the following line:
<code>LABEL_STUDIO_API_KEY=...</code>
</tab>
</tabs>
</step>
</procedure>

<procedure title="Setting Up Weight and Biases" id="wandb">
<step>
We use W&B to track our experiments. To set up W&B,
<a href="https://docs.wandb.ai/quickstart">
install the W&B CLI
</a>
</step>
<step>
Then,
<a href="https://docs.wandb.ai/quickstart">
authenticate your account
</a>.
<code-block lang="shell">wandb login</code-block>
</step>
<step>
We use W&B to track our experiments. To set up W&B,
<a href="https://docs.wandb.ai/quickstart">
install the W&B CLI
</a>
</step>
<step>
Then,
<a href="https://docs.wandb.ai/quickstart">
authenticate your account
</a>.
<code-block lang="shell">wandb login</code-block>
</step>
</procedure>

<procedure title="Pre-commit Hooks" collapsible="true">
<note>This is optional but recommended.
Pre-commit hooks are a way to ensure that your code is formatted correctly.
This is done by running a series of checks before you commit your code.
</note>
<step>
<code-block lang="shell">
pre-commit install
</code-block>
</step>
<note>This is optional but recommended.
Pre-commit hooks are a way to ensure that your code is formatted correctly.
This is done by running a series of checks before you commit your code.
</note>
<step>
<code-block lang="shell">
pre-commit install
</code-block>
</step>
</procedure>

<procedure title="Running the Tests" id="tests">
<step>
Run the tests to make sure everything is working
<code-block lang="shell">
pytest
</code-block>
</step>
<step>
Run the tests to make sure everything is working
<code-block lang="shell">
pytest
</code-block>
</step>
</procedure>

## Troubleshooting
Expand All @@ -174,13 +180,15 @@ See [Setting Up Google Cloud](#gcloud)
### Couldn't connect to Label Studio

Label Studio must be running locally, exposed on `localhost:8080`. Furthermore,
you need to specify the `LABEL_STUDIO_API_KEY` environment variable. See
you need to specify the `LABEL_STUDIO_API_KEY` environment variable. See
[Setting Up Label Studio](#ls)

### Cannot login to W&B

You need to authenticate your W&B account. See [Setting Up Weight and Biases](#wandb)
If you're facing difficulties, set the `WANDB_MODE` environment variable to `offline`
You need to authenticate your W&B account.
See [Setting Up Weight and Biases](#wandb)
If you're facing difficulties, set the `WANDB_MODE` environment variable
to `offline`
to disable W&B.

## Our Repository Structure
Expand Down
2 changes: 1 addition & 1 deletion docs/HelpTOC.json
View file
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"entities":{"pages":{"Overview":{"id":"Overview","title":"Overview","url":"overview.html","level":0,"tabIndex":0},"ML-Architecture":{"id":"ML-Architecture","title":"ML Architecture","url":"ml-architecture.html","level":0,"tabIndex":1},"Getting-Started":{"id":"Getting-Started","title":"Getting Started","url":"getting-started.html","level":0,"pages":["Get-Started-with-Dev-Containers"],"tabIndex":2},"Get-Started-with-Dev-Containers":{"id":"Get-Started-with-Dev-Containers","title":"Get Started with Dev Containers","url":"get-started-with-dev-containers.html","level":1,"parentId":"Getting-Started","tabIndex":0},"-6vddrq_5799":{"id":"-6vddrq_5799","title":"Tutorials","level":0,"pages":["Retrieve-our-Datasets"],"tabIndex":3},"Retrieve-our-Datasets":{"id":"Retrieve-our-Datasets","title":"Retrieve our Datasets","url":"retrieve-our-datasets.html","level":1,"parentId":"-6vddrq_5799","tabIndex":0},"mix-match":{"id":"mix-match","title":"MixMatch","url":"mix-match.html","level":0,"pages":["mix-match-module","custom-k-aug-dataloaders"],"tabIndex":4},"mix-match-module":{"id":"mix-match-module","title":"MixMatch Module","url":"mix-match-module.html","level":1,"parentId":"mix-match","tabIndex":0},"custom-k-aug-dataloaders":{"id":"custom-k-aug-dataloaders","title":"Custom K-Aug Dataloaders","url":"custom-k-aug-dataloaders.html","level":1,"parentId":"mix-match","tabIndex":1},"-6vddrq_5804":{"id":"-6vddrq_5804","title":"Model Tests","level":0,"pages":["Model-Test-Chestnut-May-Dec"],"tabIndex":5},"Model-Test-Chestnut-May-Dec":{"id":"Model-Test-Chestnut-May-Dec","title":"Model Test Chestnut May-Dec","url":"model-test-chestnut-may-dec.html","level":1,"parentId":"-6vddrq_5804","tabIndex":0},"-6vddrq_5806":{"id":"-6vddrq_5806","title":"API","level":0,"pages":["load.dataset","load.gcs","preprocessing.scale","preprocessing.extract_segments","preprocessing.morphology","preprocessing.glcm_padded"],"tabIndex":6},"load.dataset":{"id":"load.dataset","title":"load.dataset","url":"load-dataset.html","level":1,"parentId":"-6vddrq_5806","tabIndex":0},"load.gcs":{"id":"load.gcs","title":"load.gcs","url":"load-gcs.html","level":1,"parentId":"-6vddrq_5806","tabIndex":1},"preprocessing.scale":{"id":"preprocessing.scale","title":"preprocessing.scale","url":"preprocessing-scale.html","level":1,"parentId":"-6vddrq_5806","tabIndex":2},"preprocessing.extract_segments":{"id":"preprocessing.extract_segments","title":"preprocessing.extract_segments","url":"preprocessing-extract-segments.html","level":1,"parentId":"-6vddrq_5806","tabIndex":3},"preprocessing.morphology":{"id":"preprocessing.morphology","title":"preprocessing.morphology","url":"preprocessing-morphology.html","level":1,"parentId":"-6vddrq_5806","tabIndex":4},"preprocessing.glcm_padded":{"id":"preprocessing.glcm_padded","title":"preprocessing.glcm_padded","url":"preprocessing-glcm-padded.html","level":1,"parentId":"-6vddrq_5806","tabIndex":5}}},"topLevelIds":["Overview","ML-Architecture","Getting-Started","-6vddrq_5799","mix-match","-6vddrq_5804","-6vddrq_5806"]}
{"entities":{"pages":{"Overview":{"id":"Overview","title":"Overview","url":"overview.html","level":0,"tabIndex":0},"ML-Architecture":{"id":"ML-Architecture","title":"ML Architecture","url":"ml-architecture.html","level":0,"tabIndex":1},"Getting-Started":{"id":"Getting-Started","title":"Getting Started","url":"getting-started.html","level":0,"pages":["Get-Started-with-Dev-Containers"],"tabIndex":2},"Get-Started-with-Dev-Containers":{"id":"Get-Started-with-Dev-Containers","title":"Get Started with Dev Containers","url":"get-started-with-dev-containers.html","level":1,"parentId":"Getting-Started","tabIndex":0},"-6vddrq_6549":{"id":"-6vddrq_6549","title":"Tutorials","level":0,"pages":["Retrieve-our-Datasets"],"tabIndex":3},"Retrieve-our-Datasets":{"id":"Retrieve-our-Datasets","title":"Retrieve our Datasets","url":"retrieve-our-datasets.html","level":1,"parentId":"-6vddrq_6549","tabIndex":0},"mix-match":{"id":"mix-match","title":"MixMatch","url":"mix-match.html","level":0,"pages":["mix-match-module","custom-k-aug-dataloaders"],"tabIndex":4},"mix-match-module":{"id":"mix-match-module","title":"MixMatch Module","url":"mix-match-module.html","level":1,"parentId":"mix-match","tabIndex":0},"custom-k-aug-dataloaders":{"id":"custom-k-aug-dataloaders","title":"Custom K-Aug Dataloaders","url":"custom-k-aug-dataloaders.html","level":1,"parentId":"mix-match","tabIndex":1},"-6vddrq_6554":{"id":"-6vddrq_6554","title":"Model Tests","level":0,"pages":["Model-Test-Chestnut-May-Dec"],"tabIndex":5},"Model-Test-Chestnut-May-Dec":{"id":"Model-Test-Chestnut-May-Dec","title":"Model Test Chestnut May-Dec","url":"model-test-chestnut-may-dec.html","level":1,"parentId":"-6vddrq_6554","tabIndex":0},"-6vddrq_6556":{"id":"-6vddrq_6556","title":"API","level":0,"pages":["load.dataset","load.gcs","preprocessing.scale","preprocessing.extract_segments","preprocessing.morphology","preprocessing.glcm_padded"],"tabIndex":6},"load.dataset":{"id":"load.dataset","title":"load.dataset","url":"load-dataset.html","level":1,"parentId":"-6vddrq_6556","tabIndex":0},"load.gcs":{"id":"load.gcs","title":"load.gcs","url":"load-gcs.html","level":1,"parentId":"-6vddrq_6556","tabIndex":1},"preprocessing.scale":{"id":"preprocessing.scale","title":"preprocessing.scale","url":"preprocessing-scale.html","level":1,"parentId":"-6vddrq_6556","tabIndex":2},"preprocessing.extract_segments":{"id":"preprocessing.extract_segments","title":"preprocessing.extract_segments","url":"preprocessing-extract-segments.html","level":1,"parentId":"-6vddrq_6556","tabIndex":3},"preprocessing.morphology":{"id":"preprocessing.morphology","title":"preprocessing.morphology","url":"preprocessing-morphology.html","level":1,"parentId":"-6vddrq_6556","tabIndex":4},"preprocessing.glcm_padded":{"id":"preprocessing.glcm_padded","title":"preprocessing.glcm_padded","url":"preprocessing-glcm-padded.html","level":1,"parentId":"-6vddrq_6556","tabIndex":5}}},"topLevelIds":["Overview","ML-Architecture","Getting-Started","-6vddrq_6549","mix-match","-6vddrq_6554","-6vddrq_6556"]}
Loading

0 comments on commit 6b49b5e

Please sign in to comment.