Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decoder #83

Merged
merged 33 commits into from
May 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
79d96b8
Adding Mock Decoder for ASN.1
John-Wiens May 7, 2024
fb51985
Adding Asn.1 Decoder Class
John-Wiens May 7, 2024
7a7f7d4
Assing Basic BSM Message Decoder
John-Wiens May 8, 2024
6a81a48
Adding asn1_codec dependency
John-Wiens May 8, 2024
d6c4c24
Updating Dockerfile to build asn1c_decoder
John-Wiens May 8, 2024
23d8103
Added classes for decoded message types.
John-Wiens May 8, 2024
fc2103f
Updating Bsm Reference
John-Wiens May 8, 2024
0665895
Merge branch 'develop' of github.com:usdot-jpo-ode/jpo-conflictvisual…
John-Wiens May 8, 2024
f802cf7
Created Mock Decoder Data Sources
John-Wiens May 9, 2024
838bf36
Added decoder logic for various message types
John-Wiens May 9, 2024
333fd8a
Fixed message serializers to include parent information.
John-Wiens May 10, 2024
4b907f1
Added conversion for ProcessedMap and ProcessedSpat
John-Wiens May 10, 2024
b65f327
Fixed Tim Decoding
John-Wiens May 10, 2024
34cef13
Updating Dockerfile to use Amazon Corretto
John-Wiens May 10, 2024
c697dc8
Adding Submodule References
John-Wiens May 13, 2024
b5cfbc3
Removing Asn1_Codec
John-Wiens May 13, 2024
fc04a4b
Adding asn1_codec as submodule
John-Wiens May 13, 2024
bdbecf7
Adding Schemas to Classpath
John-Wiens May 14, 2024
7886405
Adding Decoder Unit tests
John-Wiens May 15, 2024
f7ba0ec
Updated Pom to exclude redundant dependency
John-Wiens May 16, 2024
d984522
manipulating decoded messages
jacob6838 May 24, 2024
254113a
Merging dev changes
jacob6838 May 24, 2024
a6420e2
rendering decoder data on map
jacob6838 May 24, 2024
be6c302
Debugging SPAT rendering error
jacob6838 May 26, 2024
d2bbbec
Embedding decoder map in page
jacob6838 May 29, 2024
b325c97
Final decoder visual updates
jacob6838 May 29, 2024
5e9da36
Filtering out unselected BSMs
jacob6838 May 29, 2024
e7e78e0
Merge branch 'develop' of github.com:usdot-jpo-ode/jpo-conflictvisual…
John-Wiens May 30, 2024
af7bfd4
Fixed Broken Dockerfile after merge
John-Wiens May 30, 2024
e78e82d
Updating Conflict Monitor Submodule
John-Wiens May 30, 2024
891fae4
Fixed Broken Unit Tests from Updated GeoJson Converter
John-Wiens May 31, 2024
815ca73
Removed Version label in Conflict Visualizer API
John-Wiens May 31, 2024
8c2bd5e
Merge branch 'develop' of github.com:usdot-jpo-ode/jpo-conflictvisual…
John-Wiens May 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
[submodule "api/jpo-conflictmonitor"]
path = api/jpo-conflictmonitor
url = https://github.com/usdot-jpo-ode/jpo-conflictmonitor
branch = develop
branch = develop

[submodule "api/asn1_codec"]
path = api/asn1_codec
url = https://github.com/iyourshaw/asn1_codec.git
2 changes: 1 addition & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"java.jdt.ls.vmargs": "-XX:+UseParallelGC -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xmx2G -Xms100m -Xlog:disable",
"cSpell.words": ["Bsms"]
"cSpell.words": ["uuidv", "Bsms"]
}
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ The below section provides additional instruction on how to setup the conflict v

The conflict visualizer API requires the following dependencies be installed to run locally

- Java 11
- Java 21
- Maven

Additionally there are other dependencies installed through maven.
Expand Down
102 changes: 91 additions & 11 deletions api/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,54 @@
FROM maven:3.8-eclipse-temurin-21-alpine as builder
# === BUILDER IMAGE for ACM ===
# FROM alpine:3.12 as builder
FROM amazonlinux:2023 as builder
USER root
WORKDIR /asn1_codec
## add build dependencies
RUN yum install -y cmake g++ make bash automake libtool autoconf flex bison

# Install librdkafka from Confluent repo
RUN rpm --import https://packages.confluent.io/rpm/7.6/archive.key
COPY ./confluent.repo /etc/yum.repos.d
RUN yum clean all
RUN yum install -y librdkafka-devel

# Install pugixml
ADD ./asn1_codec/pugixml /asn1_codec/pugixml
RUN cd /asn1_codec/pugixml && mkdir -p build && cd build && cmake .. && make && make install

# Build and install asn1c submodule
ADD ./asn1_codec/asn1c /asn1_codec/asn1c
RUN cd asn1c && test -f configure || autoreconf -iv && ./configure && make && make install

# Make generated files available to the build & compile example
RUN export LD_LIBRARY_PATH=/usr/local/lib
ADD ./asn1_codec/asn1c_combined /asn1_codec/asn1c_combined
RUN cd /asn1_codec/asn1c_combined && bash doIt.sh

# Remove any lingering .asn files
RUN rm -rf /asn1c_codec/asn1c_combined/j2735-asn-files
RUN rm -rf /asn1c_codec/asn1c_combined/semi-asn-files

# Remove duplicate files
RUN rm -rf /asn1c_codec/asn1c_combined/generated-files

# add the source and build files
ADD ./asn1_codec/CMakeLists.txt /asn1_codec
ADD ./asn1_codec/config /asn1_codec/config
ADD ./asn1_codec/include /asn1_codec/include
ADD ./asn1_codec/src /asn1_codec/src
ADD ./asn1_codec/kafka-test /asn1_codec/kafka-test
ADD ./asn1_codec/unit-test-data /asn1_codec/unit-test-data
ADD ./asn1_codec/data /asn1_codec/data
ADD ./asn1_codec/run_acm.sh /asn1_codec
ADD ./asn1_codec/data /asn1_codec/data

# Build acm.
RUN mkdir -p /build && cd /build && cmake /asn1_codec && make


# # === Build image for Java ===
FROM maven:3.8-eclipse-temurin-21-alpine as jbuilder

WORKDIR /home

Expand Down Expand Up @@ -37,17 +87,49 @@ RUN mvn clean install -DskipTests
WORKDIR /home/jpo-conflictvisualizer-api

RUN mvn clean package -DskipTests
# ENTRYPOINT ["tail", "-f", "/dev/null"]
FROM eclipse-temurin:21-jre-alpine
ENTRYPOINT ["tail", "-f", "/dev/null"]


# # === RUNTIME IMAGE for Java and ACM ===
# # Use Amazon Corretto Java on Amazon Linum 2023 to match the codec build env
# # FROM alpine:3.12
FROM amazoncorretto:21-al2023

WORKDIR /home

COPY --from=builder /home/jpo-conflictvisualizer-api/src/main/resources/application.yaml /home
COPY --from=builder /home/jpo-conflictvisualizer-api/src/main/resources/logback.xml /home
COPY --from=builder /home/jpo-conflictvisualizer-api/target/jpo-conflictvisualizer-api-1.3.0-SNAPSHOT.jar /home
COPY --from=jbuilder /home/jpo-conflictvisualizer-api/src/main/resources/application.yaml /home
COPY --from=jbuilder /home/jpo-conflictvisualizer-api/src/main/resources/logback.xml /home
COPY --from=jbuilder /home/jpo-conflictvisualizer-api/target/jpo-conflictvisualizer-api.jar /home
# # Copy asn1_codec executable and test files
USER root
WORKDIR /asn1_codec



# # add runtime dependencies
RUN yum install -y bash


# Install librdkafka from Confluent repo
RUN rpm --import https://packages.confluent.io/rpm/7.6/archive.key
COPY ./confluent.repo /etc/yum.repos.d
RUN yum clean all
RUN yum install -y librdkafka-devel

#COPY cert.crt /home/cert.crt
#RUN keytool -import -trustcacerts -keystore /opt/java/openjdk/lib/security/cacerts -storepass changeit -noprompt -alias mycert -file cert.crt
# # copy the built files from the builder
COPY --from=builder /asn1_codec /asn1_codec
COPY --from=builder /build /build

# # Add test data. This changes frequently so keep it low in the file.
ADD ./asn1_codec/docker-test /asn1_codec/docker-test

# # Put workdir back to Java home
WORKDIR /home


# # ENTRYPOINT ["java", "-jar", "/home/asn1-codec-java.jar"]
# # #COPY cert.crt /home/cert.crt
# # #RUN keytool -import -trustcacerts -keystore /opt/java/openjdk/lib/security/cacerts -storepass changeit -noprompt -alias mycert -file cert.crt

ENTRYPOINT ["java", \
"-Djava.rmi.server.hostname=$DOCKER_HOST_IP", \
Expand All @@ -59,6 +141,4 @@ ENTRYPOINT ["java", \
"-Dcom.sun.management.jmxremote.ssl=false", \
"-Dlogback.configurationFile=/home/logback.xml", \
"-jar", \
"/home/jpo-conflictvisualizer-api-0.0.1-SNAPSHOT.jar"]

# ENTRYPOINT ["tail", "-f", "/dev/null"]
"/home/jpo-conflictvisualizer-api.jar"]
1 change: 1 addition & 0 deletions api/asn1_codec
Submodule asn1_codec added at 422bca
13 changes: 13 additions & 0 deletions api/confluent.repo
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[Confluent]
name=Confluent repository
baseurl=https://packages.confluent.io/rpm/7.6
gpgcheck=1
gpgkey=https://packages.confluent.io/rpm/7.6/archive.key
enabled=1

[Confluent-Clients]
name=Confluent Clients repository
baseurl=https://packages.confluent.io/clients/rpm/centos/9/x86_64
gpgcheck=1
gpgkey=https://packages.confluent.io/clients/rpm/archive.key
enabled=1
13 changes: 13 additions & 0 deletions api/jpo-conflictvisualizer-api/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
<groupId>usdot.jpo.ode</groupId>
<artifactId>jpo-conflictvisualizer-api</artifactId>
<version>1.3.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>jpo-conflictvisualizer-api</name>
<description>Conflict Visualizer</description>
<properties>
Expand Down Expand Up @@ -77,6 +78,12 @@
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>com.vaadin.external.google</groupId>
<artifactId>android-json</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
Expand Down Expand Up @@ -105,6 +112,11 @@
<artifactId>jpo-ode-plugins</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>usdot.jpo.ode</groupId>
<artifactId>jpo-ode-svcs</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>usdot.jpo.ode</groupId>
<artifactId>jpo-geojsonconverter</artifactId>
Expand Down Expand Up @@ -211,6 +223,7 @@
</dependency>
</dependencies>
<build>
<finalName>${project.artifactId}</finalName>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,17 +1,24 @@
package us.dot.its.jpo.ode.api;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;

import us.dot.its.jpo.ode.api.asn1.DecoderManager;
import org.springframework.boot.web.servlet.support.SpringBootServletInitializer;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.scheduling.annotation.EnableScheduling;

@EnableWebMvc
@SpringBootApplication
@EnableScheduling
@ComponentScan(basePackages = {"us.dot.its.jpo.ode.api", "us.dot.its.jpo.geojsonconverter.validator"})
public class ConflictApiApplication extends SpringBootServletInitializer {

@Autowired DecoderManager manager;

@Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder builder) {
return builder.sources(ConflictApiApplication.class);
Expand All @@ -21,9 +28,7 @@ public static void main(String[] args) {
SpringApplication.run(ConflictApiApplication.class, args);
System.out.println("Started Conflict Monitor API");
System.out.println("Conflict Monitor API docs page found here: http://localhost:8081/swagger-ui/index.html");
System.out.println("Startup Complete");


System.out.println("Startup Complete");
}

// @Bean
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@
import us.dot.its.jpo.ode.api.models.IntersectionReferenceData;

public interface ProcessedMapRepository extends DataLoader<ProcessedMap<LineString>>{
Query getQuery(Integer intersectionID, Long startTime, Long endTime,boolean latest, boolean compact);
Query getQuery(Integer intersectionID, Long startTime, Long endTime, boolean latest, boolean compact);

long getQueryResultCount(Query query);

List<ProcessedMap> findProcessedMaps(Query query);
List<ProcessedMap<LineString>> findProcessedMaps(Query query);

List<IntersectionReferenceData> getIntersectionIDs();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.stereotype.Component;

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.mongodb.MongoException;
import com.mongodb.client.DistinctIterable;
Expand Down Expand Up @@ -50,6 +51,8 @@ public class ProcessedMapRepositoryImpl implements ProcessedMapRepository {
@Autowired
ConflictMonitorApiProperties props;

TypeReference<ProcessedMap<LineString>> processedMapTypeReference = new TypeReference<>(){};

private String collectionName = "ProcessedMap";
private ObjectMapper mapper = DateJsonMapper.getInstance();
private Logger logger = LoggerFactory.getLogger(ProcessedMapRepositoryImpl.class);
Expand Down Expand Up @@ -93,12 +96,12 @@ public long getQueryResultCount(Query query) {
return mongoTemplate.count(query, ProcessedMap.class, collectionName);
}

public List<ProcessedMap> findProcessedMaps(Query query) {
public List<ProcessedMap<LineString>> findProcessedMaps(Query query) {
List<Map> documents = mongoTemplate.find(query, Map.class, collectionName);
List<ProcessedMap> convertedList = new ArrayList<>();
List<ProcessedMap<LineString>> convertedList = new ArrayList<>();
for (Map document : documents) {
document.remove("_id");
ProcessedMap bsm = mapper.convertValue(document, ProcessedMap.class);
ProcessedMap<LineString> bsm = mapper.convertValue(document, processedMapTypeReference);
convertedList.add(bsm);
}
return convertedList;
Expand Down
Loading