MTC deploy with updates from dev branch #617

binh-dam-ibigroup · 2025-03-28T16:52:20Z

Checklist

Appropriate branch selected => Review and test changes from dev branches into the mtc branch.
Any modified or new methods or classes have helpful JavaDoc and code is thoroughly commented => Comments added where non-trivial conflicts were found.
[na] The description lists all applicable issues this PR seeks to resolve
[na] The description lists any configuration setting(s) that differ from the default settings
All tests and CI builds passing

Description

This PR brings updates from dev into the MTC branch.
Original merge conflicts can be reviewed at #590.

Mtc optimize feedupdater to dev branch

…ests and restructuring of mongo

…usive between pinned and latest

Add MobilityData GTFS Validator to validation step

Bumps [gson](https://github.com/google/gson) from 2.8.6 to 2.8.9. - [Release notes](https://github.com/google/gson/releases) - [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md) - [Commits](google/gson@gson-parent-2.8.6...gson-parent-2.8.9) --- updated-dependencies: - dependency-name: com.google.code.gson:gson dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]>

chore(deps): Upgrade sparkjava to 2.9.4.

…tement

Deployed feed version summaries

… has a pinned deployment and re

Bump GTFS lib version (Dev)

…ailable via a new end point

MTC Delete publishedVersion (dev branch)

Update gtfs-lib May 2023

…nges.

deps: update AWS sdk to latest version

Fix Append Transformation Edge Cases

Support R7A Deployment

MobilityData Validator v6

remove project-wide deployment settings

Jan 25 gtfs-lib update

ci(maven): Use GitHub cache action v4.

chore(deps): Update snakeyaml to v2.4 and jackson* to 2.18.3.

…notify-errors Notify subscribers of auto publish failure

br648

I've done a cursory check and highlighted some minor items. Are MTC aware this is taking place and are aware of the benifits e.g.

Mobility data validation
Pattern name persisting on export/import
Quicker feed source summaries retrival
New transformations?
Others

br648 · 2025-03-31T11:31:11Z

README.md

This redirects to: https://data-tools-docs.ibi-transit.com/en/dev/. Might as well use this instead.

I am going to leave any branding changes out of this PR.

br648 · 2025-03-31T12:53:37Z

src/main/java/com/conveyal/datatools/manager/jobs/CreateFeedVersionFromSnapshotJob.java

@@ -35,7 +37,7 @@ public void jobLogic() {
        // Add the jobs to handle this operation in order.
        addNextJob(
            // First export the snapshot to GTFS.
-            new ExportSnapshotToGTFSJob(owner, snapshot, feedVersion),
+            new ExportSnapshotToGTFSJob(owner, snapshot, feedVersion, publishProprietaryFiles),


publishProprietaryFiles is coming up a lot in the diff. MTC, if they don't already, should know that this allows patterns to be included in the snapshot. Patterns are then preserved on import.

src/main/java/com/conveyal/datatools/manager/models/OtpBuildConfig.java

br648 · 2025-03-31T13:20:15Z

src/main/java/com/conveyal/datatools/manager/models/OtpRouterConfig.java

This doesn't appear to be used. Can it be removed?

This file is still in the dev branch, and there is a TODO for removing it safely from that branch.

br648 · 2025-03-31T13:21:07Z

src/main/java/com/conveyal/datatools/manager/models/Project.java

-    public OtpBuildConfig buildConfig;
-
-    public OtpRouterConfig routerConfig;
-


Backs up previous comments on these two classes.

binh-dam-ibigroup and others added 30 commits March 3, 2023 09:57

Merge pull request #517 from ibi-group/mtc-optimize-feedupdater

17d9469

Mtc optimize feedupdater to dev branch

refactor: begin MobilityData validator cleanup

842df9d

improve mobility data thread wait

3d07275

Merge branch 'dev' into mobility-data-validator

a20146b

refactor: address pr feedback

a7830b9

refactor(Update to include latest deployment feed version. Matching t…

5e822fb

…ests and restructuring of mongo

refactor(FeedSource.java): Added missing comment

9ea575b

refactor(FeedSource.java): deployed feed version is now mutually excl…

f4479ad

…usive between pinned and latest

Merge pull request #512 from ibi-group/mobility-data-validator

3def660

Add MobilityData GTFS Validator to validation step

Merge branch 'dev' into deployed-feed-version-summaries

63497d5

chore(deps): Upgrade sparkjava to 2.9.4.

1815299

Merge pull request #522 from ibi-group/sparkjava-2.9.4-dev

a2cc128

chore(deps): Upgrade sparkjava to 2.9.4.

refactor(FeedSource.java): Updated comment and refactored if/else sta…

b55c40b

…tement

Merge pull request #516 from ibi-group/deployed-feed-version-summaries

f1167b7

Deployed feed version summaries

fix(MobilityDataValidation): slow on large gtfs files

e958960

refactor(Check for pinned deployment in mongo db): Check if a project…

cd0d0f8

… has a pinned deployment and re

chore(gtfs lib): bump GTFS lib version

b16eee7

ci: attempt adjusting java version

90e20d1

Merge pull request #531 from ibi-group/update-gtfs-lib-april2023

d61c757

Bump GTFS lib version (Dev)

adjust OtpRouterConfig to be otp2-compatible

cafd7f9

add more otp2 fields

128a180

refactor(Added feed source summary): All feed source summaries are av…

438a3c0

…ailable via a new end point

Merge branch 'dev' into feature/DT-315-Speed-up-feed-version-retrieval

211e4d4

support merging gui and textbox router configs

ea9e41b

Merge pull request #534 from ibi-group/mtc-delete-publishedversion

66ecdd0

MTC Delete publishedVersion (dev branch)

chore(GTFS-lib): update gtfs-lib

27aed32

Merge pull request #535 from ibi-group/update-gtfs-lib-may-2023

512bc34

Update gtfs-lib May 2023

feat(transformations): add preserve fields transformation

ae40ddd

refactor(transformations): update gtfs-lib to include Table class cha…

e831764

…nges.

miles-grant-ibigroup and others added 22 commits May 14, 2024 11:18

correctly handle newlines while appending

a4deb2c

Merge pull request #594 from ibi-group/update-aws-sdk

ef7670f

deps: update AWS sdk to latest version

refactor: address pr feedback

bb5b2da

Merge pull request #595 from ibi-group/fix-append-transform-edge-cases

18dbbc3

Fix Append Transformation Edge Cases

disable tokens requirement

6a11f0e

update snapshots

e9753de

Merge pull request #597 from ibi-group/deploy-r7a

8eda625

Support R7A Deployment

chore(deps): update mobilitydata validator

d09319a

Merge pull request #603 from ibi-group/md-validator-v6

43178b5

MobilityData Validator v6

remove project-wide deployment settings

91f5b18

clean up

47bfc27

Merge pull request #604 from ibi-group/remove-project-otp-configs

d7b98db

remove project-wide deployment settings

ci(maven): Use GitHub cache action v4.

3853e33

update gtfs-lib

3c827f6

Merge pull request #606 from ibi-group/gtfs-lib-update-jan-25

de2090d

Jan 25 gtfs-lib update

Merge pull request #605 from ibi-group/ci-upgrade-maven-cache

57c9943

ci(maven): Use GitHub cache action v4.

feat(AutoPublishJob.java): Notify subscribers of auto publish failure

fd97cfb

chore(deps): Update snakeyaml to v2.4 and jackson* to 2.18.3.

628f3ab

Merge pull request #613 from ibi-group/dependabot-alerts

e709fd4

chore(deps): Update snakeyaml to v2.4 and jackson* to 2.18.3.

Merge pull request #607 from ibi-group/feature/DT-491-auto-pub-email-…

7b456c6

…notify-errors Notify subscribers of auto publish failure

Merge branch 'dev' into mtc-deploy-with-dev2

25bb024

style(ReplaceFileFromStringTransformation): Remove extra blank space.

da621e2

binh-dam-ibigroup requested a review from br648 March 28, 2025 17:55

binh-dam-ibigroup assigned br648 Mar 28, 2025

br648 approved these changes Mar 31, 2025

View reviewed changes

br648 assigned binh-dam-ibigroup and unassigned br648 Mar 31, 2025

binh-dam-ibigroup requested a review from daniel-heppner-ibigroup April 7, 2025 17:44

binh-dam-ibigroup assigned daniel-heppner-ibigroup and unassigned binh-dam-ibigroup Apr 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MTC deploy with updates from dev branch #617

MTC deploy with updates from dev branch #617

binh-dam-ibigroup commented Mar 28, 2025 •

edited

Loading

br648 left a comment

br648 Mar 31, 2025

binh-dam-ibigroup Apr 1, 2025

br648 Mar 31, 2025

br648 Mar 31, 2025

binh-dam-ibigroup Apr 1, 2025

br648 Mar 31, 2025

		public OtpBuildConfig buildConfig;

		public OtpRouterConfig routerConfig;

MTC deploy with updates from dev branch #617

Are you sure you want to change the base?

MTC deploy with updates from dev branch #617

Conversation

binh-dam-ibigroup commented Mar 28, 2025 • edited Loading

Checklist

Description

br648 left a comment

Choose a reason for hiding this comment

br648 Mar 31, 2025

Choose a reason for hiding this comment

binh-dam-ibigroup Apr 1, 2025

Choose a reason for hiding this comment

br648 Mar 31, 2025

Choose a reason for hiding this comment

br648 Mar 31, 2025

Choose a reason for hiding this comment

binh-dam-ibigroup Apr 1, 2025

Choose a reason for hiding this comment

br648 Mar 31, 2025

Choose a reason for hiding this comment

binh-dam-ibigroup commented Mar 28, 2025 •

edited

Loading