Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept profile files #5

Closed
wants to merge 65 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
f2275c3
CHO: Update pre-commit config
cortadocodes Dec 4, 2023
b0fbc59
OPS: Update semantic version checker
cortadocodes Dec 4, 2023
d6bd4c1
OPS: Update release notes generator
cortadocodes Dec 4, 2023
846d2a0
DEP: Update `octue` version
cortadocodes Dec 4, 2023
4dc0a14
ENH: Remove unused sections of `octue.yaml`
cortadocodes Dec 4, 2023
fa53a5a
OPS: Add deployment workflow
cortadocodes Dec 4, 2023
9ea71e6
OPS: Fix namespace field in `octue.yaml`
cortadocodes Dec 4, 2023
5e02a0a
ENH: Enable crash diagnostics
cortadocodes Dec 4, 2023
9bf4b4c
OPS: Run cloud run service using service account
cortadocodes Dec 4, 2023
4273407
FIX: Set output location correctly
cortadocodes Dec 4, 2023
9409548
ENH: Update output locaiton
cortadocodes Dec 4, 2023
4e4f3b0
TST: Update deployment test
cortadocodes Dec 4, 2023
9220d08
DEP: Use `octue` version with fixed manifests
cortadocodes Dec 5, 2023
5d2b984
DEP: Use fixed `octue` version
cortadocodes Dec 6, 2023
0302760
TST: Update test bucket in test
cortadocodes Dec 6, 2023
023c0c2
OPS: Swith to poetry for package management
cortadocodes Dec 6, 2023
8262d5e
TST: Update deployment test
cortadocodes Dec 6, 2023
ff60ce0
OPS: Update namespace to `octue`
cortadocodes Dec 6, 2023
c36c531
MRG: Merge pull request #1 from octue/upgrade-octue-version
cortadocodes Dec 6, 2023
58c6f97
OPS: Run `cd` workflow on `main` branch
cortadocodes Dec 6, 2023
4131a95
OPS: Update reusable workflows location
cortadocodes Apr 15, 2024
ee8ef45
CHO: Add license and attributions
cortadocodes Apr 15, 2024
97d13fa
MRG: Merge pull request #4 from octue/add-license
cortadocodes Apr 15, 2024
7a57e25
MRG: Merge branch 'main' into use-poetry
cortadocodes Apr 26, 2024
8c08e10
FIX: Move app into `turbsim_service` directory so poetry install works
cortadocodes Apr 26, 2024
516a43a
OPS: Update minor version number
cortadocodes Apr 26, 2024
4fb1f7b
TST: Update test with new app location
cortadocodes Apr 26, 2024
8389df0
DEP: Use latest `octue` branch
cortadocodes Apr 26, 2024
cb2c80c
ENH: Update dockerfile with new base and installation instructions
cortadocodes Apr 26, 2024
e07a141
FIX: Fix `diagnostics_cloud_path` key in `octue.yaml`
cortadocodes Apr 26, 2024
e0959e9
OPS: Restrict CodeQL runs to pushes to `main`
cortadocodes Apr 26, 2024
c73cbfe
FIX: Fix app source path in `octue.yaml`
cortadocodes Apr 26, 2024
27c9602
TST: Test service/app configuration in app test
cortadocodes Apr 26, 2024
c3779c5
TST: Factor out SRUID in deployment test
cortadocodes Apr 26, 2024
1f39726
OPS: Update file for semantic version check and use reusable workflow
cortadocodes Apr 26, 2024
fa23cb2
TST: Fix deployment test
cortadocodes Apr 26, 2024
dce073d
CHO: Add `.DS_store` to .gitignore
cortadocodes Apr 29, 2024
c14c25b
TST: Update deployment test
cortadocodes Apr 29, 2024
0638031
MRG: Merge pull request #2 from octue/use-poetry
cortadocodes Apr 29, 2024
eec0b70
OPS: Update release workflow for poetry
cortadocodes Apr 29, 2024
5292525
DEP: Use branch of `octue` with question chaining
cortadocodes Jun 5, 2024
f89cc3c
DEP: Use latest commit in `octue` branch
cortadocodes Jun 5, 2024
07c5d54
DEP: Use latest commit in `octue` branch
cortadocodes Jun 5, 2024
be0fb84
OPS: Use OpenFAST version `3.5.3`
cortadocodes Jun 6, 2024
69daece
DEP: Use latest commit in `octue` branch
cortadocodes Jun 6, 2024
448549d
OPS: Fix dockerfile base casing
cortadocodes Jun 6, 2024
805dd69
FIX: Allow `octue` to manage cleaning up temporary directories
cortadocodes Jun 6, 2024
638230c
FIX: Stop redelivery of questions by checking event store
cortadocodes Jun 25, 2024
52d5cdb
DEP: Use tip of `octue` branch
cortadocodes Jun 26, 2024
ef25403
TST: Remove unused variables and imports from app test
cortadocodes Jun 26, 2024
0c4e2c7
DEP: Use tip of `octue` branch
cortadocodes Jun 28, 2024
48f6aac
DEP: Use tip of `octue` branch
cortadocodes Jun 28, 2024
a27f2e9
OPS: Specify `OCTUE_SERVICE_CONFIGURATION_PATH` envvar
cortadocodes Jul 1, 2024
ca844dc
DEP: Use `tip` of octue branch
cortadocodes Jul 4, 2024
5b122a8
DEP: Use `octue=0.56.0`
cortadocodes Jul 15, 2024
7486b15
OPS: Add openfast version to service version
cortadocodes Jul 15, 2024
234deba
MRG: Merge pull request #5 from octue/use-question-chained-octue
cortadocodes Jul 15, 2024
80e024c
OPS: Remove openfast version from service version
cortadocodes Jul 15, 2024
910e45d
DEP: Use `octue=0.59.0`
cortadocodes Aug 16, 2024
26c4f75
MRG: Merge pull request #6 from octue/update-octue
cortadocodes Aug 16, 2024
65d6d2b
DEP: Use `octue=0.59.1`
cortadocodes Aug 19, 2024
abe20c3
MRG: Merge pull request #7 from octue/fix-output-manifest-dataset-access
cortadocodes Aug 19, 2024
032a669
ENH: Update description in `twine.json`
cortadocodes Dec 9, 2024
d1f71ac
FEA: Allow multiple files in input dataset
cortadocodes Dec 9, 2024
2fe73a7
ENH: Raise error if input dataset has no files or more than 2 files
cortadocodes Dec 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/cd.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: cd

on:
pull_request:
branches:
- main

push:
branches:
- main

jobs:
deploy:
if: "!contains(github.event.head_commit.message, 'skipci')"
uses: octue/workflows/.github/workflows/deploy-cloud-run-service.yml@main
permissions:
id-token: write
contents: read
with:
gcp_project_name: octue-openfast
gcp_project_number: 86611255144
gcp_region: europe-west3
gcp_resource_affix: octue
gcp_service_name: turbsim-service
local_dockerfile: ./Dockerfile
cloud_run_flags: '--ingress=all --allow-unauthenticated --service-account=turbsim-service@octue-openfast.iam.gserviceaccount.com --max-instances=10 --memory=2048Mi'
env_vars: 'OCTUE_SERVICE_CONFIGURATION_PATH=/workspace/octue.yaml'
70 changes: 70 additions & 0 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
push:
branches: ["main"]

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
language: ["python"]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
- name: Checkout repository
uses: actions/checkout@v3

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.

# Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality

# Autobuild attempts to build any compiled languages (C/C++, C#, Go, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2

# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun

# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.

# - run: |
# echo "Run, Build Application using script"
# ./location_of_script_within_repo/buildscript.sh

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{matrix.language}}"
20 changes: 4 additions & 16 deletions .github/workflows/python-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,7 @@ on:
jobs:
check-semantic-version:
if: "!contains(github.event.head_commit.message, 'skipci')"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
# Set fetch-depth to 0 to fetch all tags (necessary for git-mkver to determine the correct semantic version).
fetch-depth: 0
- uses: actions/setup-python@v2
- name: Install git-mkver
run: |
curl -L https://github.com/idc101/git-mkver/releases/download/v1.2.1/git-mkver-linux-amd64-1.2.1.tar.gz \
| tar xvz \
&& sudo mv git-mkver /usr/local/bin
- name: Install semantic version checker
run: pip install git+https://github.com/octue/conventional-commits
- name: Check version
run: check-semantic-version setup.py
uses: octue/workflows/.github/workflows/check-semantic-version.yml@main
with:
path: pyproject.toml
breaking_change_indicated_by: minor
9 changes: 8 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,19 @@ on:

jobs:
release:
if: "github.event.pull_request.merged == true"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Install Poetry
uses: snok/[email protected]

- name: Check pyproject.toml file
run: poetry check

- name: Get package version
run: echo "PACKAGE_VERSION=$(python setup.py --version)" >> $GITHUB_ENV
run: echo "PACKAGE_VERSION=$(poetry version -s)" >> $GITHUB_ENV

- name: Create Release
uses: actions/create-release@v1
Expand Down
39 changes: 15 additions & 24 deletions .github/workflows/update-pull-request.yml
Original file line number Diff line number Diff line change
@@ -1,35 +1,26 @@
# This workflow updates the pull request description with an auto-generated section containing the categorised commit
# message headers of the commits since the last pull request merged into main. The auto generated section is enveloped
# between two comments: "<!--- START AUTOGENERATED NOTES --->" and "<!--- END AUTOGENERATED NOTES --->". Anything
# outside these in the description is left untouched. Auto-generated updates can be skipped for a commit if
# message headers of the pull request's commits. The auto generated section is enveloped between two comments:
# "<!--- START AUTOGENERATED NOTES --->" and "<!--- END AUTOGENERATED NOTES --->". Anything outside these in the
# description is left untouched. Auto-generated updates can be skipped for a commit if
# "<!--- SKIP AUTOGENERATED NOTES --->" is added to the pull request description.

name: update-pull-request

# Only trigger for pull requests into main branch.
on:
pull_request:
branches:
- main
on: pull_request

jobs:
description:
if: "!contains(github.event.pull_request.body, '<!--- SKIP AUTOGENERATED NOTES --->')"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
# Set fetch-depth to 0 to fetch all commit history (necessary for compiling pull request description).
fetch-depth: 0
- name: Install release note compiler
run: pip install git+https://github.com/octue/conventional-commits
- name: Compile new pull request description
run: |
echo 'PULL_REQUEST_NOTES<<EOF' >> $GITHUB_ENV
echo "$(compile-release-notes PULL_REQUEST_START --pull-request-url=${{ github.event.pull_request.url }} --api-token=${{ secrets.GITHUB_TOKEN }})" >> $GITHUB_ENV
echo EOF >> $GITHUB_ENV
- name: Update pull request body
uses: riskledger/update-pr-description@v2
with:
body: ${{ env.PULL_REQUEST_NOTES }}
token: ${{ secrets.GITHUB_TOKEN }}
- uses: octue/[email protected]
id: pr-description
with:
pull_request_url: ${{ github.event.pull_request.url }}
api_token: ${{ secrets.GITHUB_TOKEN }}

- name: Update pull request body
uses: riskledger/update-pr-description@v2
with:
body: ${{ steps.pr-description.outputs.pull_request_description }}
token: ${{ secrets.GITHUB_TOKEN }}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -129,3 +129,4 @@ dmypy.json
.pyre/

.idea
.DS_store
15 changes: 3 additions & 12 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ repos:
- id: black
args: ['--line-length', '120']

- repo: https://gitlab.com/pycqa/flake8
- repo: https://github.com/PyCQA/flake8
rev: 3.9.2
hooks:
- id: flake8
Expand All @@ -35,19 +35,10 @@ repos:
- id: check-branch-name
args:
- '^main$'
- '^development$'
- '^devops/([a-z][a-z0-9]*)(-[a-z0-9]+)*$'
- '^doc/([a-z][a-z0-9]*)(-[a-z0-9]+)*$'
- '^feature/([a-z][a-z0-9]*)(-[a-z0-9]+)*$'
- '^enhancement/([a-z][a-z0-9]*)(-[a-z0-9]+)*$'
- '^fix/([a-z][a-z0-9]*)(-[a-z0-9]+)*$'
- '^hotfix/([a-z][a-z0-9]*)(-[a-z0-9]+)*$'
- '^review/([a-z][a-z0-9]*)(-[a-z0-9]+)*$'
- '^refactor/([a-z][a-z0-9]*)(-[a-z0-9]+)*$'
- '^test/([a-z][a-z0-9]*)(-[a-z0-9]+)*$'
- "^([a-z][a-z0-9]*)(-[a-z0-9]+)*$"

- repo: https://github.com/octue/conventional-commits
rev: 0.5.3
rev: 0.9.0
hooks:
- id: check-commit-message-is-conventional
stages: [commit-msg]
26 changes: 26 additions & 0 deletions ATTRIBUTION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# ATTRIBUTION

In addition to some modules of our own, `octue/turbsim-service` is based on a collection of other projects that we've
drawn together, patched, enhanced and documented.

The nature of OSS is to stand on the shoulders of giants, and we owe them a ton... here are their licenses:

## `aerosense-ai/turbsim-service`

<details>
<summary>Show license</summary>


# turbsim-service

Copyright (c) 2020-2022 The Aerosense Research Partners

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
All advertising materials mentioning features or use of this software must display the following acknowledgement: This product includes software developed by the The Aerosense Research Partners.
Neither the name of the The Aerosense Research Partners nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY The Aerosense Research Partners AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL The Aerosense Research Partners BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

</details>
27 changes: 23 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,15 +1,34 @@
FROM octue/openfast:3.1.0
FROM ghcr.io/openfast/openfast:3.5.3

# Allow statements and log messages to immediately appear in the Knative logs on Google Cloud.
ENV PYTHONUNBUFFERED True

ENV PROJECT_ROOT=/app
ENV PROJECT_ROOT=/workspace
WORKDIR $PROJECT_ROOT

COPY setup.py setup.py
RUN apt-get update -y && apt-get install -y --fix-missing curl python3.11 && rm -rf /var/lib/apt/lists/*

# Install poetry.
ENV POETRY_HOME=/root/.poetry
ENV PATH "$POETRY_HOME/bin:$PATH"
RUN curl -sSL https://install.python-poetry.org | python3.11 - && poetry config virtualenvs.create false;

# Copy in the dependencies files for caching.
COPY pyproject.toml poetry.lock ./

# Install the dependencies only to utilise layer caching for quick rebuilds.
RUN poetry install \
--no-ansi \
--no-interaction \
--no-cache \
--no-root \
--only main

# Copy local code to the application root directory.
COPY . .

RUN pip3 install -e .
# Install local packages.
RUN poetry install --only main

EXPOSE $PORT

Expand Down
26 changes: 18 additions & 8 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,11 +1,21 @@
# turbsim-service
MIT License

Copyright (c) 2020-2022 The Aerosense Research Partners
Copyright (c) 2024 Octue

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
All advertising materials mentioning features or use of this software must display the following acknowledgement: This product includes software developed by the The Aerosense Research Partners.
Neither the name of the The Aerosense Research Partners nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY The Aerosense Research Partners AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL The Aerosense Research Partners BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Octue Turbsim Service
50 changes: 0 additions & 50 deletions app.py

This file was deleted.

2 changes: 1 addition & 1 deletion app_configuration.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{
"output_location": "gs://openfast-data/output"
"output_location": "gs://octue-openfast-data/turbsim/output"
}
Loading
Loading