Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added initial support for OCP-TV output format #20

Merged
merged 5 commits into from
Aug 21, 2024
Merged

Conversation

sksekar
Copy link
Collaborator

@sksekar sksekar commented Aug 19, 2024

This is to add initial support. Yet to add support for validators, pass/fail etc.

Sample OCP-TV output:

    {"schemaVersion": {"major": 2, "minor": 0}, "sequenceNumber": 0, "timestamp": "2024-08-19T00:30:31.094657Z"}
    {"testRunArtifact": {"testRunStart": {"name": "pci_lmt", "version": "1.3.1", "commandLine": "-o ocp -c lmt.cfg auto", "parameters": {}, "dutInfo": {"dutInfoId": "319706019", "name": "hostname", "platformInfos": [], "softwareInfos": [], "hardwareInfos": []}}}, "sequenceNumber": 1, "timestamp": "2024-08-19T00:30:31.094946Z"}
    {"testStepArtifact": {"testStepId": "0", "testStepStart": {"name": "Rcvr:6 Step:5 Ann:CPU Tx -> Retimer Rx (Timing left)"}}, "sequenceNumber": 2, "timestamp": "2024-08-19T00:30:31.095273Z"}
    {"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:0", "value": 0.0, "validators": []}}, "sequenceNumber": 3, "timestamp": "2024-08-19T00:30:36.691321Z"}
    {"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:1", "value": 0.0, "validators": []}}, "sequenceNumber": 4, "timestamp": "2024-08-19T00:30:36.691972Z"}
    {"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:2", "value": 0.0, "validators": []}}, "sequenceNumber": 5, "timestamp": "2024-08-19T00:30:36.692493Z"}
    {"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:3", "value": 0.0, "validators": []}}, "sequenceNumber": 6, "timestamp": "2024-08-19T00:30:36.692845Z"}
    {"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:4", "value": 0.0, "validators": []}}, "sequenceNumber": 7, "timestamp": "2024-08-19T00:30:36.693209Z"}
    {"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:5", "value": 0.0, "validators": []}}, "sequenceNumber": 8, "timestamp": "2024-08-19T00:30:36.693568Z"}
    {"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:6", "value": 0.0, "validators": []}}, "sequenceNumber": 9, "timestamp": "2024-08-19T00:30:36.693947Z"}
    {"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:7", "value": 0.0, "validators": []}}, "sequenceNumber": 10, "timestamp": "2024-08-19T00:30:36.694338Z"}
    {"testStepArtifact": {"testStepId": "0", "testStepEnd": {"status": "COMPLETE"}}, "sequenceNumber": 11, "timestamp": "2024-08-19T00:30:36.694586Z"}
    {"testStepArtifact": {"testStepId": "1", "testStepStart": {"name": "Rcvr:6 Step:6 Ann:CPU Tx -> Retimer Rx (Timing left)"}}, "sequenceNumber": 12, "timestamp": "2024-08-19T00:30:36.694755Z"}
    {"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:0", "value": 0.0, "validators": []}}, "sequenceNumber": 13, "timestamp": "2024-08-19T00:30:42.283159Z"}
    {"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:1", "value": 0.0, "validators": []}}, "sequenceNumber": 14, "timestamp": "2024-08-19T00:30:42.283600Z"}
    {"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:2", "value": 0.0, "validators": []}}, "sequenceNumber": 15, "timestamp": "2024-08-19T00:30:42.283974Z"}
    {"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:3", "value": 0.0, "validators": []}}, "sequenceNumber": 16, "timestamp": "2024-08-19T00:30:42.284312Z"}
    {"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:4", "value": 0.0, "validators": []}}, "sequenceNumber": 17, "timestamp": "2024-08-19T00:30:42.284669Z"}
    {"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:5", "value": 0.0, "validators": []}}, "sequenceNumber": 18, "timestamp": "2024-08-19T00:30:42.285002Z"}
    {"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:6", "value": 0.0, "validators": []}}, "sequenceNumber": 19, "timestamp": "2024-08-19T00:30:42.285358Z"}
    {"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:7", "value": 0.0, "validators": []}}, "sequenceNumber": 20, "timestamp": "2024-08-19T00:30:42.287100Z"}
    {"testStepArtifact": {"testStepId": "1", "testStepEnd": {"status": "COMPLETE"}}, "sequenceNumber": 21, "timestamp": "2024-08-19T00:30:42.287347Z"}
    {"testRunArtifact": {"testRunEnd": {"status": "COMPLETE", "result": "PASS"}}, "sequenceNumber": 22, "timestamp": "2024-08-19T00:30:42.287470Z"}

Sample output:

{"schemaVersion": {"major": 2, "minor": 0}, "sequenceNumber": 0, "timestamp": "2024-08-19T00:30:31.094657Z"}
{"testRunArtifact": {"testRunStart": {"name": "pci_lmt", "version": "1.3.1", "commandLine": "-o ocp -c lmt.cfg auto", "parameters": {}, "dutInfo": {"dutInfoId": "319706019", "name": "hostname", "platformInfos": [], "softwareInfos": [], "hardwareInfos": []}}}, "sequenceNumber": 1, "timestamp": "2024-08-19T00:30:31.094946Z"}
{"testStepArtifact": {"testStepId": "0", "testStepStart": {"name": "Rcvr:6 Step:5 Ann:CPU Tx -> Retimer Rx (Timing left)"}}, "sequenceNumber": 2, "timestamp": "2024-08-19T00:30:31.095273Z"}
{"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:0", "value": 0.0, "validators": []}}, "sequenceNumber": 3, "timestamp": "2024-08-19T00:30:36.691321Z"}
{"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:1", "value": 0.0, "validators": []}}, "sequenceNumber": 4, "timestamp": "2024-08-19T00:30:36.691972Z"}
{"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:2", "value": 0.0, "validators": []}}, "sequenceNumber": 5, "timestamp": "2024-08-19T00:30:36.692493Z"}
{"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:3", "value": 0.0, "validators": []}}, "sequenceNumber": 6, "timestamp": "2024-08-19T00:30:36.692845Z"}
{"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:4", "value": 0.0, "validators": []}}, "sequenceNumber": 7, "timestamp": "2024-08-19T00:30:36.693209Z"}
{"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:5", "value": 0.0, "validators": []}}, "sequenceNumber": 8, "timestamp": "2024-08-19T00:30:36.693568Z"}
{"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:6", "value": 0.0, "validators": []}}, "sequenceNumber": 9, "timestamp": "2024-08-19T00:30:36.693947Z"}
{"testStepArtifact": {"testStepId": "0", "measurement": {"name": "BDF:0000:1a:00.0 Lane:7", "value": 0.0, "validators": []}}, "sequenceNumber": 10, "timestamp": "2024-08-19T00:30:36.694338Z"}
{"testStepArtifact": {"testStepId": "0", "testStepEnd": {"status": "COMPLETE"}}, "sequenceNumber": 11, "timestamp": "2024-08-19T00:30:36.694586Z"}
{"testStepArtifact": {"testStepId": "1", "testStepStart": {"name": "Rcvr:6 Step:6 Ann:CPU Tx -> Retimer Rx (Timing left)"}}, "sequenceNumber": 12, "timestamp": "2024-08-19T00:30:36.694755Z"}
{"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:0", "value": 0.0, "validators": []}}, "sequenceNumber": 13, "timestamp": "2024-08-19T00:30:42.283159Z"}
{"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:1", "value": 0.0, "validators": []}}, "sequenceNumber": 14, "timestamp": "2024-08-19T00:30:42.283600Z"}
{"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:2", "value": 0.0, "validators": []}}, "sequenceNumber": 15, "timestamp": "2024-08-19T00:30:42.283974Z"}
{"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:3", "value": 0.0, "validators": []}}, "sequenceNumber": 16, "timestamp": "2024-08-19T00:30:42.284312Z"}
{"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:4", "value": 0.0, "validators": []}}, "sequenceNumber": 17, "timestamp": "2024-08-19T00:30:42.284669Z"}
{"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:5", "value": 0.0, "validators": []}}, "sequenceNumber": 18, "timestamp": "2024-08-19T00:30:42.285002Z"}
{"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:6", "value": 0.0, "validators": []}}, "sequenceNumber": 19, "timestamp": "2024-08-19T00:30:42.285358Z"}
{"testStepArtifact": {"testStepId": "1", "measurement": {"name": "BDF:0000:1a:00.0 Lane:7", "value": 0.0, "validators": []}}, "sequenceNumber": 20, "timestamp": "2024-08-19T00:30:42.287100Z"}
{"testStepArtifact": {"testStepId": "1", "testStepEnd": {"status": "COMPLETE"}}, "sequenceNumber": 21, "timestamp": "2024-08-19T00:30:42.287347Z"}
{"testRunArtifact": {"testRunEnd": {"status": "COMPLETE", "result": "PASS"}}, "sequenceNumber": 22, "timestamp": "2024-08-19T00:30:42.287470Z"}

Signed-off-by: Sathish Sekar <[email protected]>
@sksekar sksekar requested a review from mimir-d August 19, 2024 01:20
@sksekar sksekar self-assigned this Aug 19, 2024
@sksekar sksekar requested a review from caggio August 19, 2024 01:21
Copy link
Contributor

@mimir-d mimir-d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the design of the OctTvOutputReporter thin wrapper breaks fundamental properties coming from the ocptv api lib

while this isnt mandatory to fix, i would advise to consider them since this introduces code failure modes and tech debt, which may explode later and you or another maintainer will have to address

src/pci_lmt/results.py Outdated Show resolved Hide resolved
src/pci_lmt/results.py Outdated Show resolved Hide resolved
src/pci_lmt/results.py Outdated Show resolved Hide resolved
src/pci_lmt/results.py Outdated Show resolved Hide resolved
@sksekar
Copy link
Collaborator Author

sksekar commented Aug 20, 2024

Thanks for your time & support. Need some help!

the design of the OctTvOutputReporter thin wrapper breaks fundamental properties coming from the ocptv api lib

I'm trying to add the OCPTV output format natively but with very minimal changes to the main code. The thin wrapper allows me to handle OCPTV related code in a single class with very little modification to the main code.

while this isnt mandatory to fix, i would advise to consider them since this introduces code failure modes and tech debt, which may explode later and you or another maintainer will have to address

Understood. I would love to know if there is a way to retain the benefits of the OCPTV APIs (esp contextmanager) and still contain the implementation in it's own Reporter sub-class.

@mimir-d
Copy link
Contributor

mimir-d commented Aug 20, 2024

i left a link with api examples above; it would be sufficient to follow that (how the api uses contextmanager, eg. https://github.com/opencomputeproject/ocp-diag-core-python/blob/dev/src/ocptv/output/run.py#L147) in your wrapper class, since you want to keep that reporter interface. At the end of the day, try to write some code and then do the mental exercise of seeing what you can do with it. It doesnt have to be perfect, but if there's easy ways to misuse it (as i demonstrated above), then it can likely be improved

Sathish Sekar and others added 2 commits August 20, 2024 11:10
- this fixes the upstream typing check

Signed-off-by: mimir-d <[email protected]>
@mimir-d mimir-d merged commit 0f2e04a into dev Aug 21, 2024
5 checks passed
@mimir-d mimir-d deleted the feature/ocptv-output branch August 21, 2024 01:43
@sksekar
Copy link
Collaborator Author

sksekar commented Aug 21, 2024

Thank you so much @mimir-d. Appreciate your support (during the review) and quick turnaround (in fixing the APIs).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants