Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SCHEMATIC-30, SCHEMATIC-200] Add version to click cli / use pathlib.Path module for checking cache size #1542

Merged
merged 31 commits into from
Nov 23, 2024

Conversation

thomasyu888
Copy link
Member

Problem

  • It was not easy to surface the version of the cli or library used
  • we were using du -h system to check for cache size which isn't accurate AND may fail on windows

solution

Add version to cli

schematic -h
Usage: schematic [OPTIONS] COMMAND [ARGS]...

  Command line interface to the `schematic` backend services.

Options:
  -v, --verbosity LVL  Either CRITICAL, ERROR, WARNING, INFO or DEBUG
  --version            Show the version and exit.
  -h, --help           Show this message and exit.

Commands:
  manifest  Sub-commands with Manifest Generation utilities/methods.
  model     Sub-commands for Metadata Model related utilities/methods.
  schema    Sub-commands for Schema related utilities/methods.
  viz       Sub-commands for Visualization methods.

schematic --version
schematic, version 24.12.1
import schematic
schematic.__version__
Out[5]: '24.12.1'

This confirms that it will pull from the pyproject.toml, so that needs to be updated

docker build . -t test
docker run test schematic --version
schematic, version 24.12.1

@thomasyu888 thomasyu888 marked this pull request as ready for review November 19, 2024 02:49
@thomasyu888 thomasyu888 requested a review from a team as a code owner November 19, 2024 02:49
Copy link
Collaborator

@BryanFauble BryanFauble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm!

Assuming the version works fine id like to look into attaching it to the telemetry data created. That way we can know the version that produced the telemetry data.

Copy link
Contributor

@GiaJordan GiaJordan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I think we can change the default value of the cache in another PR

@thomasyu888 thomasyu888 merged commit 998e295 into develop Nov 23, 2024
7 checks passed
@thomasyu888 thomasyu888 deleted the schematic-30-add-version-to-cli branch November 23, 2024 01:33
andrewelamb added a commit that referenced this pull request Dec 16, 2024
* add new tests

* add unit tests

* ran black

* Update schematic/models/validate_attribute.py

Co-authored-by: BryanFauble <[email protected]>

* added tests

* Update README.md

* Update README.md

* add unit tests

* run black

* Update README.md

* temp commit

* remove old tests

* [FDS-2386] Synapse entity tracking and code concurrency updates (#1505)

* [FDS-2386] Synapse entity tracking and code concurrency updates

* ran black

* Update CODEOWNERS

* updated data model type rules to include error param

* fix validate type attribute to use msg level param

* added error handling

* run black

* create Node class

* sat up Node class so that nodes with no displayName fields cause an error on creation

* ran black

* ran mypy

* added new configs for CLI tests

* added new manifests for testing CLI commands

* automate manual CLI tests

* ran black

* Update CODEOWNERS

* Update scan_repo.yml

* Update .github/CODEOWNERS

* Update .github/workflows/scan_repo.yml

* Attach additional telemetry data to OTEL traces (#1519)

* Attach additional telemetry data to OTEL traces

* feat: added tracing for cross manifest validation and file name validation  (#1509)

* add tracing for GX validation

* temp commit

* Updating contribution doc to expect squash and merge (#1534)

* [FDS-2491] Integration tests for Schematic API Test plan (#1512)

Integration tests for Schematic API Test plan

* [FDS-2500] Add Integration Tests for: Manifest Validation (#1516)

* Add Integration Tests for: Manifest Validation

* [FDS-2449] Lock `sphinx` version and update `poetry.lock` (#1530)

Also install `typing-extensions` in the build

* manual test files now being saved in manifests folder

* manual test files now being saved in manifests folder

* remove lines to delete json files that were under git control

* ran black

* add try finally blocks to remove created files

* ran black

* add lines to remove created json files

* Update file annotation store process to require filename be present in order to annotate file

* add lines to remove created json files

* Revert "Update file annotation store process to require filename be present in order to annotate file"

This reverts commit f57c718.

* Don't attempt to annotate the table

* add code in finally blocks to reset config to default values, when tests change them

* complete submit manifest command test

* ran black

* add test for bug case

* update test for table tidyness

* remove unused import

* remove etag column if already present when building temp file view

* catch all exceptions to switch to sequential mode

* update test for updated data

* Revert "update test for updated data"

This reverts commit 255e3c0.

* Revert "catch all exceptions to switch to sequential mode"

This reverts commit 68b0b24.

* catch ValueErrors as well

* Updates for integration test failures (#1537)

* Updates for integration test failures, Config file reset and scope changes

* add todos for removing config resets

* [FDS-2525] Authenticated export of telemetry data (#1527)

* Authenticated export of telemetry data, updating to HTTP otel library

* temp reduce tests

* restore tests

* uncomment tests

* redid how files are deleted, manual tests values are set

* ran black

* [SCHEMATIC-157] Make some dependencies required to avoid `schematic CLI` commands from potentially erroring when doing a pip install (#1540)

* Make otel flash non-optional

* Add dependencies as non-optional

* Include schematic_api for now (#1547)

* update toml version to 24.11.1 (#1548)

* [SCHEMATIC-193] Support exporting telemetry data from GH integration test runs (#1550)

* Support exporting telemetry data from GH run via access token retrieved via oauth2

* [SCHEMATIC-30, SCHEMATIC-200] Add version to click cli / use pathlib.Path module for checking cache size (#1542)

* Add version to click cli

* Add version

* Run black

* Reformat

* Fix

* Update schematic/schemas/data_model_parser.py

* Add test for check_synapse_cache_size

* Reformat

* Fix tests

* Remove unused parameter

* Install all-extras for now

* Make otel flash non-optional

* Update dockerfile

* Add dependencies as non-optional

* Update pyproject toml

* Fix trivy issue

* Add service version

* Run black

* Move all utils.general tests into separate folder

* Use pre-commit

* Add updates to contribution doc

* Fix

* Add service version to log provider

---------

Co-authored-by: BryanFauble <[email protected]>

* [SCHEMATIC-212] Prevent traces from being combined (#1552)

* Set instance id in github CI run, uninstrument flask auto during integration test run

* [SCHEMATIC-163] Catch error when manifest is generated and existing one doesn't have `entityId` (#1551)

* adds error handling

* adds unit tests for _get_file_entityIds

* updates error message

* adds entityid check to parent func

* updates docstring

* [SCHEMATIC-183] Use paths from file view for manifest generation (#1529)

source manifest file paths from synapse fileviews at generation

* [SCHEMATIC-214] Wrap pandas functions to support not including `None` with the NA values argument (#1553)

* Wrap pandas functions to support not including `None` with the NA values argument

* Ignore types

* pylint issues

* ordering of ignore

* Add to integration test to cover none in a manifest

* Add additional test for manifest

* [SCHEMATIC-210] Add attribute to nones data model (#1555)

Update example_test_nones.model.csv component and add new invalid manifest with nones

* first commit

* ran black

* add test for validateModelManifest

* [SCHEMATIC-214] change data model and component (#1556)

* add valid values to Patient attributes

* update data model

* add test manifests

* update test for new model

* update test for new valid value

* change test to use new manifests

* remove uneeded test file

* revert file

* revert file

* change tests to use new manifests

* remove uneeded manifests

* ran black

* add tests back in

* ran black

* revert manifest

* Split up valid and errored test as separate testing functions

* Remove unused import

---------

Co-authored-by: Gianna Jordan <[email protected]>
Co-authored-by: Andrew Lamb <[email protected]>
Co-authored-by: Thomas Yu <[email protected]>

* incremented packge version number

* Update publish.yml

* Update test.yml

* Update api_test.yml

* Update pdoc.yml

* Update version.py

* updates publish.yml (#1558) (#1561)

Co-authored-by: Brad Macdonald <[email protected]>

---------

Co-authored-by: BryanFauble <[email protected]>
Co-authored-by: Jenny V Medina <[email protected]>
Co-authored-by: Thomas Yu <[email protected]>
Co-authored-by: Lingling <[email protected]>
Co-authored-by: GiaJordan <[email protected]>
Co-authored-by: Brad Macdonald <[email protected]>
Co-authored-by: Gianna Jordan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants