Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Processed bsm features #2

Closed
wants to merge 64 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
b01d1a8
updates to use gihub artifacts for jpo-ode references
Michael7371 Jul 16, 2024
d9cdb94
env variable rename and documentation
Michael7371 Jul 16, 2024
85061b2
Add release notes and update the pom.xml version
drewjj Sep 6, 2024
7bef71f
Merge pull request #68 from usdot-jpo-ode/Update/release-notes-1.4.0
drewjj Sep 9, 2024
23adfe7
update Jpo-ode submodule to the latest version
SaikrishnaBairamoni Sep 30, 2024
2904a68
Merge pull request #69 from usdot-jpo-ode/release/2024-q3
SaikrishnaBairamoni Sep 30, 2024
e88017c
Update pom.xml
SaikrishnaBairamoni Sep 30, 2024
712b505
Update pom.xml
SaikrishnaBairamoni Sep 30, 2024
e72a111
Update pom.xml
SaikrishnaBairamoni Sep 30, 2024
3fe0077
Merge pull request #70 from usdot-jpo-ode/hotfix_1.4.1
SaikrishnaBairamoni Sep 30, 2024
ed6ffd2
update submodule ref with latest jpo-ode version
SaikrishnaBairamoni Sep 30, 2024
1800d42
Merge pull request #71 from usdot-jpo-ode/hotfix_1.4.2
dan-du-car Sep 30, 2024
7339843
Merge pull request #72 from usdot-jpo-ode/master
dan-du-car Oct 7, 2024
d1cd7e6
Adding zstd compression to GeoJsonConverter
John-Wiens Oct 23, 2024
380672f
updates to test ci cd with updated build pipeline
Michael7371 Oct 23, 2024
38a9a8a
Merge branch 'develop' into testing-github-artifacts
Michael7371 Oct 23, 2024
89b7397
fix environmental vars in docker build pipeline
Michael7371 Oct 23, 2024
77659b1
fixing other ci pipelines with new build process
Michael7371 Oct 23, 2024
e2bac23
Made kafka linger ms configurable
John-Wiens Oct 23, 2024
cd173e6
switching linger ms to 50 ms
John-Wiens Oct 23, 2024
67adda1
fixing ci workflow
Michael7371 Oct 23, 2024
8630476
testing artifact publishing
Michael7371 Oct 23, 2024
ddefcc8
fix failing action
Michael7371 Oct 23, 2024
fe4c8c7
fix syntax error
Michael7371 Oct 23, 2024
8f4f12c
github artifact env variables renaming
Michael7371 Oct 23, 2024
e60d991
testing different config
Michael7371 Oct 23, 2024
c789e3c
testing repackage skip
Michael7371 Oct 23, 2024
532b660
update testing with a token that can read and write packages
Michael7371 Oct 24, 2024
168a194
updates to use the jpo-utils repository
Michael7371 Oct 24, 2024
874254e
removing pull request trigger on artifact publishing
Michael7371 Oct 24, 2024
f0e31a1
updates to sample.env and docs
Michael7371 Oct 24, 2024
1fc3815
updates to the documentation on environmental variables
Michael7371 Oct 25, 2024
2c2cad9
adding docker compose errors to all required environmental variables
Michael7371 Oct 28, 2024
e3ff03c
Merge pull request #73 from usdot-jpo-ode/compression
John-Wiens Oct 28, 2024
a8d8a22
fix docker compose errors
Michael7371 Oct 28, 2024
4867495
fixing dockerfile warning
Michael7371 Oct 28, 2024
6d77cef
adding docs for docker-compose profiles
Michael7371 Oct 28, 2024
0efc72f
updates to address PR comments
Michael7371 Oct 28, 2024
d25ef4d
improved dockerfile caching
Michael7371 Oct 28, 2024
233b9ff
Merge pull request #75 from usdot-jpo-ode/Feature/submodule-refactor-…
Michael7371 Oct 28, 2024
cf47633
Make J2735 optional values optional in the JSON schema
drewjj Nov 18, 2024
bcba8c7
Upgrade ProcessedMap schema version to 2020 since GCP doesn't support…
drewjj Nov 18, 2024
ed7dcf1
Update schema versions to 2020
drewjj Nov 18, 2024
c7e854e
Added Logic to Generate timestamp from odeReceivedAt
John-Wiens Nov 20, 2024
7b3042f
Switching adopt builder to temurin
John-Wiens Nov 26, 2024
6f99eb1
Merge pull request #77 from usdot-jpo-ode/timestamp-modification
drewjj Nov 27, 2024
488b5b1
Merge pull request #78 from usdot-jpo-ode/adopt-workflow-fix
Michael7371 Dec 5, 2024
e8b0e41
Switch the Position3D back to a reference
drewjj Dec 10, 2024
0daa80f
Merge pull request #76 from usdot-jpo-ode/processedmap-schema-update
drewjj Dec 10, 2024
fe441eb
adding caching to github actions
Michael7371 Dec 24, 2024
02ed49d
Merge pull request #80 from usdot-jpo-ode/github-actions-caching
John-Wiens Dec 24, 2024
7143df9
Change ProcessedBsm to a Feature. Move metadata properties from Proc…
iyourshaw Dec 31, 2024
65daf7d
Update related classes and tests
iyourshaw Jan 1, 2025
63a4a57
Update sample bsm for test
iyourshaw Jan 6, 2025
2d280dd
Add more conditions to deserializer test
iyourshaw Jan 6, 2025
11cb4c3
Add serializer test for bsm
iyourshaw Jan 6, 2025
fa856a2
Update BsmFeatureTest
iyourshaw Jan 6, 2025
773053e
Fix tests
iyourshaw Jan 6, 2025
741a5df
Add more conditions to converter test
iyourshaw Jan 6, 2025
ee37a39
Fix converter; include metadata
iyourshaw Jan 6, 2025
e8e1dee
Make ProcessedBsmCollection generic
iyourshaw Jan 6, 2025
d80403c
Test artifact publish
iyourshaw Jan 6, 2025
ab14d59
Update imports
iyourshaw Jan 6, 2025
f385694
Test publish on pr edited
iyourshaw Jan 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
env variable rename and documentation
  • Loading branch information
Michael7371 committed Jul 16, 2024
commit d9cdb94f5c7bfc449e7af722c82269bb92124d64
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ FROM maven:3.8-eclipse-temurin-21-alpine as builder

WORKDIR /home

ARG GITHUB_USERNAME
ARG GITHUB_TOKEN_NAME
ARG GITHUB_TOKEN
ARG GITHUB_ORG

ENV GITHUB_USERNAME=$GITHUB_USERNAME
ENV GITHUB_TOKEN_NAME=$GITHUB_TOKEN_NAME
ENV GITHUB_TOKEN=$GITHUB_TOKEN
ENV GITHUB_ORG=$GITHUB_ORG

Expand Down
21 changes: 19 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ The jpo-geojsonconverter software system consists of the following modules hoste
You may download the stable, default branch for ALL of these dependencies by using the following recursive git clone command:

```bash
git clone --recurse-submodules https://github.com/usdot-jpo-ode/jpo-geojsonconverter.git
git clone https://github.com/usdot-jpo-ode/jpo-geojsonconverter.git
```

Once you have these repositories obtained, you are ready to build and deploy the application.
Expand All @@ -231,7 +231,21 @@ docker-compose ps

Verify the jpo-ode, kafka, zookeeper, asn1-decoder and asn1-encoder are running before performing step 3.

#### Step 3 - Build and run jpo-geojsonconverter application
#### Step 3 - Generate GitHub Token

A GitHub token is required to pull artifacts from GitHub repositories. This is required to obtain the jpo-ode jars and must be done before attempting to build this repository.

1. Log into GitHub.
2. Navigate to Settings -> Developer settings -> Personal access tokens.
3. Click "New personal access token (classic)".
1. As of now, GitHub does not support `Fine-grained tokens` for obtaining packages.
4. Provide a name and expiration for the token.
5. Select the read:packages scope.
6. Click "Generate token" and copy the token.

Copy the token name, token value, and target jpo-ode organization into your `.env` file. If using your local IDE set the GitHub environmental variable to your local system as well.

#### Step 4 - Build and run jpo-geojsonconverter application

**Notes:**
- Docker builds may fail if you are on a corporate network due to DNS resolution errors.
Expand Down Expand Up @@ -272,6 +286,9 @@ docker-compose ps
### Purpose & Usage

- The DOCKER_HOST_IP environment variable is used to communicate with the bootstrap server that the instance of Kafka is running on.
- The GITHUB_TOKEN_NAME environment variable is the name of the generated token from xxx used for pulling the jpo-ode java image.
- The GITHUB_TOKEN environment variable is the value of the generated token from xxx used for pulling the jpo-ode java image.
- The GITHUB_ORG environment variable is the name of the GitHub organization to use for the jpo-ode repository.

### Values
In order to utilize Confluent Cloud:
Expand Down
2 changes: 1 addition & 1 deletion docker-compose-standalone.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ services:
context: .
dockerfile: Dockerfile
args:
GITHUB_USERNAME: ${GITHUB_USERNAME}
GITHUB_TOKEN_NAME: ${GITHUB_TOKEN_NAME}
GITHUB_TOKEN: ${GITHUB_TOKEN}
GITHUB_ORG: ${GITHUB_ORG}
restart: always
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.confluent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ services:
context: .
dockerfile: Dockerfile
args:
GITHUB_USERNAME: ${GITHUB_USERNAME}
GITHUB_TOKEN_NAME: ${GITHUB_TOKEN_NAME}
GITHUB_TOKEN: ${GITHUB_TOKEN}
GITHUB_ORG: ${GITHUB_ORG}
image: jpo_geojsonconverter:latest
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ services:
context: .
dockerfile: Dockerfile
args:
GITHUB_USERNAME: ${GITHUB_USERNAME}
GITHUB_TOKEN_NAME: ${GITHUB_TOKEN_NAME}
GITHUB_TOKEN: ${GITHUB_TOKEN}
GITHUB_ORG: ${GITHUB_ORG}
image: jpo_geojsonconverter:latest
Expand Down
4 changes: 2 additions & 2 deletions jpo-geojsonconverter/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,14 @@
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<sonar.coverage.jacoco.xmlReportPaths>${project.basedir}/target/site/jacoco/jacoco.xml</sonar.coverage.jacoco.xmlReportPaths>
<sonar.language>java</sonar.language>
<github.username>${env.GITHUB_USERNAME}</github.username>
<github.username>${env.GITHUB_TOKEN_NAME}</github.username>
<github.token>${env.GITHUB_TOKEN}</github.token>
</properties>
<repositories>
<repository>
<id>github</id>
<name>GitHub usdot-jpo-ode Apache Maven Packages</name>
<url>https://maven.pkg.github.com/CDOT-CV/jpo-ode</url>
<url>https://maven.pkg.github.com/${env.GITHUB_ORG}/jpo-ode</url>
<releases><enabled>true</enabled></releases>
</repository>
</repositories>
Expand Down
2 changes: 1 addition & 1 deletion jpo-geojsonconverter/settings.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<servers>
<server>
<id>github</id>
<username>${env.GITHUB_USERNAME}</username>
<username>${env.GITHUB_TOKEN_NAME}</username>
<password>${env.GITHUB_TOKEN}</password>
</server>
</servers>
Expand Down
2 changes: 1 addition & 1 deletion sample.env
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
DOCKER_HOST_IP=

# GitHub properties for pulling the latest version of the JPO-ODE
GITHUB_USERNAME=
GITHUB_TOKEN_NAME=
GITHUB_TOKEN=
GITHUB_ORG=usdot-jpo-ode

Expand Down