Extract and log warnings from forward model steps #10050
Conversation
CodSpeed Performance ReportMerging #10050 will not alter performanceComparing Summary
|
berland
force-pushed
the
extract_warnings
branch
2 times, most recently
from
7613b99 to
7ade322
Compare
berland
added
the
release-notes:new-feature
src/ert/scheduler/job.py
Outdated
| "Warning:" in line | ||
| or "FutureWarning" in line | ||
| or "DeprecationWarning" in line | ||
| or "WARNING" in line |
There was a problem hiding this comment.
| "Warning:" in line | |
| or "FutureWarning" in line | |
| or "DeprecationWarning" in line | |
| or "WARNING" in line | |
| "Warning" in line | |
| or "warning" in line |
Would just checking this be sufficient, or would we pick up unwanted things?
There was a problem hiding this comment.
That is an open question for now :) . Probably no and yes.
There was a problem hiding this comment.
I will add some test code that makes it more specific what we want to include and not, based on what I can find in real life examples on scratch.
There was a problem hiding this comment.
Potentially we just want FutureWarnings first. Then we can add more later. The most permissive would be re.search('warning', line, re.IGNORECASE).
There was a problem hiding this comment.
In one example runpath..
$ grep -i warning *stdout* *stderr* | wc -l
15587
(of which 7000 are unique). I think we must be more specific.
berland
force-pushed
the
extract_warnings
branch
from
7ade322 to
bd3d11f
Compare
Each individual forward model step is run in a separate subprocess and has no relation to Erts root logger. Thus, in order to log warnings, there is no other way than parsing the stdout and stderr files for anything that looks interesting. It is assumed that each realization will trigger approximately the same set of warnings, so the first realization to be finished will get the honour of exposing its warnings. Warning lines are truncated to 2048 characters in length, and identical warnings from the same file will be deduplicated (their count will be shown).
berland
force-pushed
the
extract_warnings
branch
from
bd3d11f to
4b95223
Compare
| "Warning:" in line | ||
| or "FutureWarning" in line | ||
| or "DeprecationWarning" in line | ||
| or "UserWarning" in line |
There was a problem hiding this comment.
Are we sure why we want all these? If any one of those are from a know source, adding a comment about the intended source would be a good idea.
There was a problem hiding this comment.
It depends on our purpose, is this only for getting Future/DeprecationWarnings or do we also want to make a "service" to the users to be able to show warnings in the GUI at a later stage? (#9381 )
Each individual forward model step is run in a separate subprocess and has no relation to Erts root logger. Thus, in order to log warnings, there is no other way than parsing the stdout and stderr files for anything that looks interesting.
It is assumed that each realization will trigger approximately the same set of warnings, so the first realization to be finished will get the honour of exposing its warnings.
Issue
Resolves #10039
git rebase -i main --exec 'just rapid-tests')When applicable