Skip to content

Commit

Permalink
find : tifeatures :: replace : tipg
Browse files Browse the repository at this point in the history
  • Loading branch information
jonaraphael committed Jul 28, 2023
1 parent c6e7495 commit 5627c8b
Show file tree
Hide file tree
Showing 14 changed files with 45 additions and 45 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/test_and_deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
python -m pip install -r requirements.txt
python -m pip install -r cerulean_cloud/cloud_run_offset_tiles/requirements.txt
python -m pip install -r cerulean_cloud/cloud_run_orchestrator/requirements.txt
python -m pip install -r cerulean_cloud/cloud_run_tifeatures/requirements.txt
python -m pip install -r cerulean_cloud/cloud_run_tipg/requirements.txt
python -m pip install -r cerulean_cloud/titiler_sentinel/requirements.txt
- name: Run pytest
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ WORKDIR $APP_HOME
COPY cerulean_cloud/ /app/cerulean_cloud/

# Install production dependencies.
RUN pip install -r cerulean_cloud/cloud_run_tifeatures/requirements.txt
RUN pip install -r cerulean_cloud/cloud_run_tipg/requirements.txt

# Run the web service on container startup. Here we use the gunicorn
# webserver, with one worker process and 8 threads.
# For environments with multiple CPU cores, increase the number of workers
# to be equal to the cores available.
# Timeout is set to 0 to disable the timeouts of the workers to allow Cloud Run to handle instance scaling.
CMD exec uvicorn --port $PORT --host 0.0.0.0 cerulean_cloud.cloud_run_tifeatures.handler:app
CMD exec uvicorn --port $PORT --host 0.0.0.0 cerulean_cloud.cloud_run_tipg.handler:app
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ For each of these deployments there exists a configuration directory that includ

__Initial deployment__

If you are deploying a completely new stack, make sure to create matching configuration files in `cerulean-cloud-images` and `cerulean-cloud`, with matching stack names. In addition, specifically for the tifeatures deployment, since the database is empty when a stack is deployed for the first time (alembic migrations occur after the initial migration), if you want to access `tifeatures` after this initial deployment make sure to poll the `/register` endpoint of the resulting URL in order to correctly load the tables (i.e. `curl https://some-tifeatures-url.app/register`). For any deployments after the first one, this is not required.
If you are deploying a completely new stack, make sure to create matching configuration files in `cerulean-cloud-images` and `cerulean-cloud`, with matching stack names. In addition, specifically for the tipg deployment, since the database is empty when a stack is deployed for the first time (alembic migrations occur after the initial migration), if you want to access `tipg` after this initial deployment make sure to poll the `/register` endpoint of the resulting URL in order to correctly load the tables (i.e. `curl https://some-tipg-url.app/register`). For any deployments after the first one, this is not required.

__Decreasing cold starts for Cloud Run__

Expand All @@ -38,7 +38,7 @@ Google Cloud provides nice dashboards for tracking the stability, response time
- [Cloud Run orchestrator metrics](https://console.cloud.google.com/run/detail/europe-west1/cerulean-cloud-production-cloud-run-orchestrator/metrics?project=cerulean-338116)
- [Cloud Run offset tile metrics](https://console.cloud.google.com/run/detail/europe-west1/cerulean-cloud-production-cloud-run-offset-tiles/metrics?project=cerulean-338116)
- [Cloud Tasks queue metrics](https://console.cloud.google.com/cloudtasks/queue/europe-west1/cerulean-cloud-production-queue-cloud-run-orchestrator-d8ed51e/metrics?project=cerulean-338116)
- [Cloud Run tifeatures metrics](https://console.cloud.google.com/run/detail/europe-west1/cerulean-cloud-production-cloud-run-tifeatures/metrics?project=cerulean-338116)
- [Cloud Run tipg metrics](https://console.cloud.google.com/run/detail/europe-west1/cerulean-cloud-production-cloud-run-tipg/metrics?project=cerulean-338116)
- [Cloud Function scene relevancy metrics](https://console.cloud.google.com/functions/details/europe-west1/cerulean-cloud-production-cloud-function-scene-relevancy?env=gen1&project=cerulean-338116)
- [Cloud Function historical run metrics](https://console.cloud.google.com/functions/details/europe-west1/cerulean-cloud-production-cloud-function-historical-run?env=gen1&project=cerulean-338116)

Expand Down Expand Up @@ -92,7 +92,7 @@ pip install -r requirements-test.txt
# Additional requirements files
pip install -r cerulean_cloud/cloud_run_offset_tiles/requirements.txt
pip install -r cerulean_cloud/cloud_run_orchestrator/requirements.txt
pip install -r cerulean_cloud/cloud_run_tifeatures/requirements.txt
pip install -r cerulean_cloud/cloud_run_tipg/requirements.txt
pip install -r cerulean_cloud/titiler_sentinel/requirements.txt
# Setup pre-commit
pre-commit install
Expand Down Expand Up @@ -154,9 +154,9 @@ Previewing update (test):
docker:index:RemoteImage cerulean-cloud-images-test-remote-offset completing deletion from previous update
docker:index:RemoteImage cerulean-cloud-images-test-remote-orchestrator completing deletion from previous update
docker:index:RemoteImage cerulean-cloud-images-test-remote-tifeatures completing deletion from previous update
docker:index:RemoteImage cerulean-cloud-images-test-remote-tipg completing deletion from previous update
- docker:index:RemoteImage cerulean-cloud-images-test-remote-offset delete completing deletion from previous update
- docker:index:RemoteImage cerulean-cloud-images-test-remote-tifeatures delete completing deletion from previous update
- docker:index:RemoteImage cerulean-cloud-images-test-remote-tipg delete completing deletion from previous update
- docker:index:RemoteImage cerulean-cloud-images-test-remote-orchestrator delete completing deletion from previous update
...
pulumi:pulumi:Stack cerulean-cloud-test running Creating lambda package [running in Docker]...
Expand Down Expand Up @@ -261,7 +261,7 @@ orchestrator_result = httpx.post(URL+"/orchestrate",
```

The services deployed by cerulean-cloud that DO NOT require this API key are:
- Tifeatures Cloud Run
- tipg Cloud Run
- Historical run Cloud Function
- Scene relevancy Cloud Function

Expand Down
10 changes: 0 additions & 10 deletions cerulean_cloud/cloud_run_tifeatures/README.md

This file was deleted.

10 changes: 10 additions & 0 deletions cerulean_cloud/cloud_run_tipg/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# M1 MacBooks
set `DOCKER_DEFAULT_PLATFORM=linux/amd64` in M1 macbooks

# Debug image

```
cd stack/
docker build -f ../Dockerfiles/Dockerfile.cloud_run_tipg -t gcr.io/cerulean-338116/cloud-run-tipg ../ --no-cache
PORT=8080 && docker run -p 8080:${PORT} -e PORT=${PORT} -e DATABASE_URL=${DB_URL} gcr.io/cerulean-338116/cloud-run-tipg
```
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
"""Cloud run main for tifeatures
"""Cloud run main for tipg
Run this locally with:
uvicorn --port $PORT --host 0.0.0.0 cerulean_cloud.cloud_run_tifeatures.handler:app
uvicorn --port $PORT --host 0.0.0.0 cerulean_cloud.cloud_run_tipg.handler:app
Make sure to set in your environment:
- TIFEATURES_NAME
- TIFEATURES_TEMPLATES
- tipg_NAME
- tipg_TEMPLATES
- DATABASE_URL
"""
Expand Down Expand Up @@ -35,7 +35,7 @@ class PostgresSettings(pydantic.BaseSettings):
"""Postgres-specific API settings.
Note: We can't use PostgresSettings from TiPG because of the weird GCP DB url
See https://github.com/developmentseed/tifeatures/issues/32
See https://github.com/developmentseed/tipg/issues/32
Attributes:
postgres_user: postgres username.
Expand Down Expand Up @@ -75,7 +75,7 @@ class Config:

templates_location: List[Any] = [
jinja2.FileSystemLoader(
"cerulean_cloud/cloud_run_tifeatures/templates/"
"cerulean_cloud/cloud_run_tipg/templates/"
), # custom template directory
jinja2.PackageLoader("tipg", "templates"), # default template directory
]
Expand Down
2 changes: 1 addition & 1 deletion images/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@
# Export the DNS name of the bucket
pulumi.export("offset_tile", cloud_run_images.cloud_run_offset_tile_image_url)
pulumi.export("orchestrator", cloud_run_images.cloud_run_orchestrator_image_url)
pulumi.export("tifeatures", cloud_run_images.cloud_run_tifeatures_image_url)
pulumi.export("tipg", cloud_run_images.cloud_run_tipg_image_url)
pulumi.export("model", cloud_run_images.weigths_name)
12 changes: 6 additions & 6 deletions images/cloud_run_images.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ def construct_name(resource_name: str) -> str:
cloud_run_orchestrator_image_url = registry_url.apply(
lambda url: f"{url}/{construct_name('cloud-run-orchestrator-image')}"
)
cloud_run_tifeatures_image_url = registry_url.apply(
lambda url: f"{url}/{construct_name('cloud-run-tifeatures-image')}"
cloud_run_tipg_image_url = registry_url.apply(
lambda url: f"{url}/{construct_name('cloud-run-tipg-image')}"
)
registry_info = None # use gcloud for authentication.

Expand Down Expand Up @@ -81,13 +81,13 @@ def construct_name(resource_name: str) -> str:
image_name=cloud_run_orchestrator_image_url,
registry=registry_info,
)
cloud_run_tifeatures_image = docker.Image(
construct_name("cloud-run-tifeatures-image"),
cloud_run_tipg_image = docker.Image(
construct_name("cloud-run-tipg-image"),
build=docker.DockerBuild(
context="../",
dockerfile="../Dockerfiles/Dockerfile.cloud_run_tifeatures",
dockerfile="../Dockerfiles/Dockerfile.cloud_run_tipg",
extra_options=["--no-cache", "--quiet"],
),
image_name=cloud_run_tifeatures_image_url,
image_name=cloud_run_tipg_image_url,
registry=registry_info,
)
4 changes: 2 additions & 2 deletions stack/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import cloud_function_scene_relevancy
import cloud_run_offset_tile
import cloud_run_orchestrator
import cloud_run_tifeatures
import cloud_run_tipg
import database
import pulumi
import sns_subscription
Expand All @@ -18,7 +18,7 @@
pulumi.export(
"cloud_run_orchestrator_url", cloud_run_orchestrator.default.statuses[0].url
)
pulumi.export("cloud_run_tifeatures_url", cloud_run_tifeatures.default.statuses[0].url)
pulumi.export("cloud_run_tipg_url", cloud_run_tipg.default.statuses[0].url)
pulumi.export("database_url", database.sql_instance_url)
pulumi.export("database_instance_name", database.instance.connection_name)
pulumi.export("database_url_alembic", database.sql_instance_url_alembic)
Expand Down
14 changes: 7 additions & 7 deletions stack/cloud_run_images.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@ def construct_name_images(resource_name: str) -> str:
).image_url,
opts=pulumi.ResourceOptions(provider=gcr_docker_provider),
)
cloud_run_tifeatures_registry_image = docker.get_registry_image(
cloud_run_tipg_registry_image = docker.get_registry_image(
name=gcp.container.get_registry_image(
name=construct_name_images("cloud-run-tifeatures-image:latest")
name=construct_name_images("cloud-run-tipg-image:latest")
).image_url,
opts=pulumi.ResourceOptions(provider=gcr_docker_provider),
)
Expand All @@ -51,13 +51,13 @@ def construct_name_images(resource_name: str) -> str:
pull_triggers=[cloud_run_orchestrator_registry_image.sha256_digest],
)

cloud_run_tifeatures_image = docker.RemoteImage(
construct_name_images("remote-tifeatures"),
name=cloud_run_tifeatures_registry_image.name,
pull_triggers=[cloud_run_tifeatures_registry_image.sha256_digest],
cloud_run_tipg_image = docker.RemoteImage(
construct_name_images("remote-tipg"),
name=cloud_run_tipg_registry_image.name,
pull_triggers=[cloud_run_tipg_registry_image.sha256_digest],
)


cloud_run_offset_tile_sha = cloud_run_offset_tile_registry_image.sha256_digest[8:20]
cloud_run_orchestrator_sha = cloud_run_orchestrator_registry_image.sha256_digest[8:20]
cloud_run_tifeatures_sha = cloud_run_tifeatures_registry_image.sha256_digest[8:20]
cloud_run_tipg_sha = cloud_run_tipg_registry_image.sha256_digest[8:20]
8 changes: 4 additions & 4 deletions stack/cloud_run_tifeatures.py → stack/cloud_run_tipg.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

config = pulumi.Config()

service_name = construct_name("cloud-run-tifeatures")
service_name = construct_name("cloud-run-tipg")
default = gcp.cloudrun.Service(
service_name,
name=service_name,
Expand All @@ -19,7 +19,7 @@
spec=gcp.cloudrun.ServiceTemplateSpecArgs(
containers=[
gcp.cloudrun.ServiceTemplateSpecContainerArgs(
image=cloud_run_images.cloud_run_tifeatures_image.name,
image=cloud_run_images.cloud_run_tipg_image.name,
envs=[
gcp.cloudrun.ServiceTemplateSpecContainerEnvArgs(
name="DATABASE_URL",
Expand All @@ -35,7 +35,7 @@
timeout_seconds=420,
),
metadata=dict(
name=service_name + "-" + cloud_run_images.cloud_run_tifeatures_sha,
name=service_name + "-" + cloud_run_images.cloud_run_tipg_sha,
annotations={
"run.googleapis.com/cloudsql-instances": instance.connection_name,
},
Expand All @@ -54,7 +54,7 @@
],
)
noauth_iam_policy = gcp.cloudrun.IamPolicy(
construct_name("cloud-run-noauth-iam-policy-tifeatures"),
construct_name("cloud-run-noauth-iam-policy-tipg"),
location=default.location,
project=default.project,
service=default.name,
Expand Down

0 comments on commit 5627c8b

Please sign in to comment.