Skip to content

Commit

Permalink
Add note about configuring lithops logging to send to a file
Browse files Browse the repository at this point in the history
  • Loading branch information
tomwhite committed Feb 22, 2024
1 parent 4a371a9 commit c6b30c8
Show file tree
Hide file tree
Showing 7 changed files with 4 additions and 7 deletions.
3 changes: 2 additions & 1 deletion examples/lithops/aws-lambda/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ pip install -r requirements.txt # use requirements file from same directory as
```

2. Configure Lithops with an [AWS Lambda compute backend](https://lithops-cloud.github.io/docs/source/compute_config/aws_lambda.html), and an [AWS S3 storage backend](https://lithops-cloud.github.io/docs/source/storage_config/aws_s3.html).
- Note: it may be useful to put the configuration in a different place to the default (e.g. `~/.lithops/config.aws`), and then call `export LITHOPS_CONFIG_FILE=~/.lithops/config.aws`)
- Note: it may be useful to put the configuration in a different place to the default (e.g. `~/.lithops/config.aws`), and then call `export LITHOPS_CONFIG_FILE=~/.lithops/config.aws`
- Although optional, it is convenient to [configure Lithops logging](https://lithops-cloud.github.io/docs/source/configuration.html) by setting `log_filename` (to `lithops.log`, for example), so that messages are sent to a file, rather than the console.
3. Create a new S3 bucket (called `cubed-<username>-temp`, for example) in the same region you chose when configuring Lambda and S3 for Lithops. This will be used for intermediate zarr data. Note that this is different to the bucket created when configuring Lithops, which just stores configuration data.
4. Build a Lithops runtime image for Cubed (this will build and upload a Docker image, which can take a while, although it only needs to be done once).
- Note: if you are building on an arm64 machine (e.g. Apple Silicon) then make sure that your Lithops config file contains `architecture: arm64` under the `aws_lambda` section.
Expand Down
1 change: 0 additions & 1 deletion examples/lithops/aws-lambda/lithops-add-random.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
from cubed.extensions.tqdm import TqdmProgressBar
from cubed.runtime.executors.lithops import LithopsDagExecutor

logging.basicConfig(level=logging.INFO)
# suppress harmless connection pool warnings
logging.getLogger("urllib3.connectionpool").setLevel(logging.ERROR)

Expand Down
1 change: 0 additions & 1 deletion examples/lithops/aws-lambda/lithops-matmul-random.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
from cubed.extensions.tqdm import TqdmProgressBar
from cubed.runtime.executors.lithops import LithopsDagExecutor

logging.basicConfig(level=logging.INFO)
# suppress harmless connection pool warnings
logging.getLogger("urllib3.connectionpool").setLevel(logging.ERROR)

Expand Down
3 changes: 2 additions & 1 deletion examples/lithops/gcf/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ pip install -r requirements.txt # use requirements file from same directory as
```

2. Configure Lithops with a [Google Cloud Functions compute backend](https://lithops-cloud.github.io/docs/source/compute_config/gcp_functions.html#configuration), and a [Google Cloud Storage backend](https://lithops-cloud.github.io/docs/source/storage_config/gcp_storage.html#configuration).
- Note: it may be useful to put the configuration in a different place to the default (e.g. `~/.lithops/config.gcf`), and then call `export LITHOPS_CONFIG_FILE=~/.lithops/config.gcf`)
- Note: it may be useful to put the configuration in a different place to the default (e.g. `~/.lithops/config.gcf`), and then call `export LITHOPS_CONFIG_FILE=~/.lithops/config.gcf`
- Although optional, it is convenient to [configure Lithops logging](https://lithops-cloud.github.io/docs/source/configuration.html) by setting `log_filename` (to `lithops.log`, for example), so that messages are sent to a file, rather than the console.
3. Give permissions to access your GCP buckets by calling `export GOOGLE_APPLICATION_CREDENTIALS=<path-to-json>`, where `<path-to-json>` is the full path to the `.json` key file you downloaded when configuring GCP storage for Lithops in the previous step.
4. Create a new GCP bucket (called `cubed-<username>-temp`, for example) in the same region you chose when configuring Google Cloud Functions and Storage for Lithops. This will be used for intermediate zarr data. Note that this is different to the bucket created when configuring Lithops, which just stores configuration data.
5. Build a Lithops runtime image for Cubed (this is similar to building a Docker image like you need for AWS Lambda, but is simpler and faster as it only requires a `requirements.txt` file).
Expand Down
1 change: 0 additions & 1 deletion examples/lithops/gcf/lithops-add-random.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
from cubed.extensions.tqdm import TqdmProgressBar
from cubed.runtime.executors.lithops import LithopsDagExecutor

logging.basicConfig(level=logging.INFO)
# suppress harmless connection pool warnings
logging.getLogger("urllib3.connectionpool").setLevel(logging.ERROR)

Expand Down
1 change: 0 additions & 1 deletion examples/lithops/gcf/lithops-matmul-ones.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
from cubed.extensions.tqdm import TqdmProgressBar
from cubed.runtime.executors.lithops import LithopsDagExecutor

logging.basicConfig(level=logging.INFO)
# suppress harmless connection pool warnings
logging.getLogger("urllib3.connectionpool").setLevel(logging.ERROR)

Expand Down
1 change: 0 additions & 1 deletion examples/lithops/gcf/lithops-matmul-random.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
from cubed.extensions.tqdm import TqdmProgressBar
from cubed.runtime.executors.lithops import LithopsDagExecutor

logging.basicConfig(level=logging.INFO)
# suppress harmless connection pool warnings
logging.getLogger("urllib3.connectionpool").setLevel(logging.ERROR)

Expand Down

0 comments on commit c6b30c8

Please sign in to comment.