Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Postgres support #116

Closed
wants to merge 2 commits into from
Closed

Postgres support #116

wants to merge 2 commits into from

Conversation

LuiggiTenorioK
Copy link
Member

In GitLab by @LuiggiTenorioK on May 2, 2024, 09:35

  • Build pytest fixture to test with sqlite and pg
    • tempdir fixture with and without sqlite files
    • .autosubmitrc builder fixture
    • fixture to copy SQLite db files content to a Postgres instance
    • mock session
  • Centralize all SQL operations in one module with "repositories" (interfaces following DDD principles)
    • autosubmit.db
    • as_times.db
    • structures_xxxx.db (skipping it for now, using autosubmit.database.db_structure instead)
    • job_data_xxxx.db
    • job_packages.db
    • graph_xxxx.db
  • Refactor classes that directly call sqlite3 or sqlalchemy
  • Support changes on https://earth.bsc.es/gitlab/es/autosubmit/-/merge_requests/426
    • Support autosubmit.database.tables module
    • Refactor session to handle pooling in multithreading workers of Gunicorn

@LuiggiTenorioK
postgress support
e7e9a6f 
add engine switch

abstract experiment_graph_drawing table adapter

fix engine

use autosubmit db_structure

use autosubmit db_manager in graph draw

add status adapter

fix bg tasks

remove legacy code

add db adapter layer for main DDBBs

refactor db_common

limit SQLAlchemy usage

fix graph_draw table doesn't exist

use tmpdir for testing

refactor jobdata module

move table_manager from autosubmit

history refactor

replace _mapping to _asdict()

fix fixtures

migrate SQLAlchemy table def to Core

normalize adapters responses

move session module to API

use APIBasicConfig in session

update name from adapters to repository

replace db_structure handler
@LuiggiTenorioK
delete default cipher
9916e0a
@LuiggiTenorioK LuiggiTenorioK self-assigned this Nov 12, 2024
@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 2, 2024, 09:36

added 1 commit

  • 31d9b8eb - add engine switch

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 3, 2024, 16:31

added 1 commit

  • f92c956b - abstract experiment_graph_drawing table adapter

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 7, 2024, 13:07

added 1 commit

  • 6dac0145 - fix engine

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 14, 2024, 12:48

deleted the v4.0.0b7 branch. This merge request now targets the main branch

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 15, 2024, 14:38

added 1 commit

  • b0234af5 - use autosubmit db_structure

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 15, 2024, 15:17

added 1 commit

  • 46e2a521 - use autosubmit db_manager in graph draw

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 16, 2024, 14:24

added 1 commit

  • e1295c83 - add status adapter

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 16, 2024, 14:33

added 16 commits

  • e1295c83...1db51f2f - 9 commits from branch main
  • f28bdfd - extend Basic Config to handle postgres (should be reverted later)
  • ba4821c - add engine switch
  • 535aa9c - abstract experiment_graph_drawing table adapter
  • a0d6996 - fix engine
  • 591927a - use autosubmit db_structure
  • fd77bfd - use autosubmit db_manager in graph draw
  • a845be0 - add status adapter

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 16, 2024, 14:56

added 1 commit

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 16, 2024, 15:00

added 1 commit

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 16, 2024, 17:09

added 1 commit

  • 07b83c1 - add db adapter layer for main DDBBs

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 17, 2024, 14:45

added 1 commit

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 17, 2024, 16:40

added 1 commit

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 21, 2024, 10:18

added 1 commit

  • fc34795 - fix graph_draw table doesn't exist

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 24, 2024, 10:25

added 2 commits

  • ce19e580 - use tmpdir for testing
  • d0f906b3 - Merge branch 'pg-testing' into 'postgres-support'

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 24, 2024, 12:39

added 1 commit

  • 65ea8206 - refactor jobdata module

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 27, 2024, 14:32

added 1 commit

  • 198abcfa - move table_manager from autosubmit

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 27, 2024, 15:02

added 18 commits

  • 198abcfa...94f0510b - 8 earlier commits
  • b90a7db9 - add status adapter
  • 0c691c44 - fix bg tasks
  • b1622f1b - remove legacy code
  • ee380ec3 - add db adapter layer for main DDBBs
  • 72ef48e1 - refactor db_common
  • a0c0938b - limit SQLAlchemy usage
  • 0886864e - fix graph_draw table doesn't exist
  • 8f16c177 - use tmpdir for testing
  • b844a154 - refactor jobdata module
  • 5ee991a7 - move table_manager from autosubmit

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 27, 2024, 17:17

added 1 commit

  • 05c52ea2 - history refactor

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 28, 2024, 09:51

added 1 commit

  • 18182537 - replace _mapping to _asdict()

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 28, 2024, 11:08

added 1 commit

  • 46de6b94 - fix fixtures

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 28, 2024, 13:29

added 1 commit

  • dde49d8f - migrate SQLAlchemy table def to Core

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 28, 2024, 14:31

added 1 commit

  • 25578a41 - normalize adapters responses

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 28, 2024, 16:22

added 24 commits

  • 25578a41...e9955ae2 - 3 commits from branch main
  • e9955ae2...700910af - 11 earlier commits
  • fa847305 - limit SQLAlchemy usage
  • d2cac39b - fix graph_draw table doesn't exist
  • 9131d401 - use tmpdir for testing
  • 9457b1d5 - refactor jobdata module
  • b2c3cb77 - move table_manager from autosubmit
  • 221aa432 - history refactor
  • 42f38dcb - replace _mapping to _asdict()
  • 8ff9cfac - fix fixtures
  • 06699632 - migrate SQLAlchemy table def to Core
  • bd7b2e45 - normalize adapters responses

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 28, 2024, 16:26

added 1 commit

  • a4d6acc9 - Revert "extend Basic Config to handle postgres (should be reverted later)"

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 29, 2024, 15:27

added 1 commit

  • 9e8f4c5d - move session module to API

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 29, 2024, 15:29

added 1 commit

  • edbbf3aa - use APIBasicConfig in session

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 29, 2024, 15:31

added 1 commit

  • c2551ff8 - update name from adapters to repository

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 30, 2024, 10:11

marked the checklist item mock session as completed

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on May 30, 2024, 10:11

marked the checklist item Refactor session to handle pooling in multithreading workers of Gunicorn as completed

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Jun 3, 2024, 13:40

marked the checklist item Support changes on https://earth.bsc.es/gitlab/es/autosubmit/-/merge_requests/426 as completed

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Jun 3, 2024, 13:40

marked the checklist item Support changes on https://earth.bsc.es/gitlab/es/autosubmit/-/merge_requests/426 as incomplete

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Jun 7, 2024, 13:39

added 1 commit

  • 7d95d215 - update parser version

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Jul 5, 2024, 16:44

added 31 commits

  • 7d95d215...0d617cb2 - 6 commits from branch main
  • 0d617cb2...c540ba6f - 15 earlier commits
  • b3919d6e - move table_manager from autosubmit
  • 2ecb53ed - history refactor
  • c1048751 - replace _mapping to _asdict()
  • 567e35f1 - fix fixtures
  • c7884a82 - migrate SQLAlchemy table def to Core
  • 6e4fb023 - normalize adapters responses
  • 0ae16e74 - Revert "extend Basic Config to handle postgres (should be reverted later)"
  • 441f2d5e - move session module to API
  • 02af032d - use APIBasicConfig in session
  • 03989a39 - update name from adapters to repository

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Jul 8, 2024, 17:35

added 1 commit

  • cfe8e94e - replace db_structure handler

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Jul 8, 2024, 17:35

marked the checklist item structures_xxxx.db (skipping it for now, using autosubmit.database.db_structure instead) as completed

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Jul 8, 2024, 17:35

marked the checklist item Support changes on https://earth.bsc.es/gitlab/es/autosubmit/-/merge_requests/426 as completed

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Jul 16, 2024, 16:37

added 35 commits

  • cfe8e94e...5848d788 - 9 commits from branch main
  • 5848d788...68fe7a9c - 16 earlier commits
  • 7ab1b908 - history refactor
  • ef37327a - replace _mapping to _asdict()
  • 34763f81 - fix fixtures
  • 9e073ca4 - migrate SQLAlchemy table def to Core
  • 16f736d4 - normalize adapters responses
  • 7d7d38a5 - Revert "extend Basic Config to handle postgres (should be reverted later)"
  • 2329fb0f - move session module to API
  • f4e54dd9 - use APIBasicConfig in session
  • 6842dde7 - update name from adapters to repository
  • 34ad9284 - replace db_structure handler

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Sep 3, 2024, 10:36

added 33 commits

  • 34ad9284...e5000d7a - 9 commits from branch main
  • e5000d7a...96084ff3 - 14 earlier commits
  • 9be6df7b - move table_manager from autosubmit
  • 0fc86d17 - history refactor
  • f8bec011 - replace _mapping to _asdict()
  • c0b6d262 - fix fixtures
  • 1ae429ae - migrate SQLAlchemy table def to Core
  • 4bca7f09 - normalize adapters responses
  • 1ecc1ad5 - move session module to API
  • f01257af - use APIBasicConfig in session
  • 29b22036 - update name from adapters to repository
  • b6532e3c - replace db_structure handler

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Sep 3, 2024, 10:36

added 1 commit

  • a0a02074 - replace db_structure handler

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Sep 3, 2024, 10:45

added 3 commits

  • 792acafc - update typing hints
  • 72f80075 - fix import issues (F401 & F811)
  • 1afeb6dd - postgress support

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Sep 3, 2024, 10:47

added 5 commits

  • 1afeb6dd...e5000d7a - 4 commits from branch main
  • 6b537c18 - postgress support

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Sep 6, 2024, 16:07

added 12 commits

  • 6b537c18...7a43e2a3 - 11 commits from branch main
  • 3f20a357 - postgress support

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Sep 17, 2024, 11:39

added 8 commits

  • 3f20a357...3af8e5f2 - 7 commits from branch main
  • e7e9a6f - postgress support

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Sep 19, 2024, 15:45

added 1 commit

  • dd1d1e96 - delete default cipher

Compare with previous version

@LuiggiTenorioK
Copy link
Member Author

In GitLab by @LuiggiTenorioK on Sep 19, 2024, 15:49

added 1 commit

Compare with previous version

@LuiggiTenorioK LuiggiTenorioK marked this pull request as draft December 10, 2024 11:33
@kinow kinow deleted the postgres-support branch January 29, 2025 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@LuiggiTenorioK LuiggiTenorioK

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@LuiggiTenorioK