Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Job metrics #2036

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

Feature: Job metrics #2036

wants to merge 4 commits into from

Conversation

LuiggiTenorioK
Copy link
Member

@LuiggiTenorioK LuiggiTenorioK commented Dec 17, 2024

PR for #2031

Example of usage on: #2031 (comment)

Process:

  1. Should be triggered once the job finishes in COMPLETED or FAILED
  2. Read the JOBS.<JOB_SECTION>.METRICS section of the job and parse it
  3. Build the remote file path and read it
  4. Store it on DB

@LuiggiTenorioK LuiggiTenorioK linked an issue Dec 17, 2024 that may be closed by this pull request
@LuiggiTenorioK LuiggiTenorioK force-pushed the gh-2031-metric-retrieval branch from 88535eb to 4e37772 Compare December 17, 2024 12:15
@codecov-commenter
Copy link

codecov-commenter commented Dec 17, 2024

Codecov Report

Attention: Patch coverage is 65.57377% with 63 lines in your changes missing coverage. Please review.

Project coverage is 49.92%. Comparing base (80e6f37) to head (e57278d).

Files with missing lines Patch % Lines
autosubmit/job/metrics_processor.py 68.54% 36 Missing and 3 partials ⚠️
autosubmit/platforms/locplatform.py 26.66% 11 Missing ⚠️
autosubmit/platforms/paramiko_platform.py 35.29% 11 Missing ⚠️
autosubmit/job/job.py 85.71% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2036      +/-   ##
==========================================
+ Coverage   49.80%   49.92%   +0.11%     
==========================================
  Files          72       73       +1     
  Lines       17569    17734     +165     
  Branches     3418     3437      +19     
==========================================
+ Hits         8751     8853     +102     
- Misses       8007     8067      +60     
- Partials      811      814       +3     
Flag Coverage Δ
fast-tests 49.92% <65.57%> (+0.11%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@LuiggiTenorioK LuiggiTenorioK self-assigned this Dec 17, 2024
@LuiggiTenorioK LuiggiTenorioK added the new feature Use this label to plan and request new features label Dec 17, 2024
@LuiggiTenorioK LuiggiTenorioK force-pushed the gh-2031-metric-retrieval branch 6 times, most recently from cd0271d to a735d80 Compare December 23, 2024 14:19
@LuiggiTenorioK LuiggiTenorioK force-pushed the gh-2031-metric-retrieval branch 3 times, most recently from ce0cb6c to c933de6 Compare December 30, 2024 19:28
Comment on lines 146 to 147
with open(path, "r") as f:
content = f.read()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dbeltrankyl is there any defined method to read the file's content on the remote?

I saw the paramiko_platform.py and there are just methods to move files from remote to local. Still, I'm confused about how to use the Platform class children correctly 😵‍💫

cc @kinow

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think not. Talking by memory, I believe Autosubmit doesn't read anything from the remote; it only checks some files ( completed, failed wrapper... ) but doesn't read the content

@LuiggiTenorioK LuiggiTenorioK force-pushed the gh-2031-metric-retrieval branch from 5f16ddb to d624b8b Compare January 30, 2025 15:39
@LuiggiTenorioK LuiggiTenorioK force-pushed the gh-2031-metric-retrieval branch from d624b8b to e57278d Compare January 30, 2025 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature Use this label to plan and request new features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Job metrics retrieval
3 participants