Date filter

[shashank-boyapally]

Type of change

• Refactor
• New feature
• Bug fix
• Optimization
• Documentation Update

Description

You can now constrain your look-back period using the --lookback option. The format for look-back is XdYh, where X represents the number of days and Y represents the number of hours. Do not merge until #43 gets merged, and fmatch 0.0.8 is released

(venv) sboyapal orion (date-filter) >> orion cmd --hunter-analyze --lookback="10d"
2024-06-25 15:51:47,771 - Orion - INFO - file: orion.py - line: 94 - 🏹 Starting Orion in command-line mode
2024-06-25 15:51:47,773 - Orion - INFO - file: utils.py - line: 213 - The test aws-small-scale-cluster-density-v2 has started
2024-06-25 15:51:47,774 - Matcher - INFO - Executing query against index=ospst-perf-scale-ci-*
2024-06-25 15:51:47,861 - Matcher - INFO - Executing query against index=ospst-perf-scale-ci-*
2024-06-25 15:51:47,891 - Matcher - INFO - Executing query against index=ospst-ripsaw-kube-burner*
2024-06-25 15:51:47,943 - Orion - INFO - file: utils.py - line: 50 - Collecting podReadyLatency
2024-06-25 15:51:47,943 - Matcher - INFO - Executing query against index=ospst-ripsaw-kube-burner*
2024-06-25 15:51:47,991 - Orion - INFO - file: utils.py - line: 50 - Collecting apiserverCPU
2024-06-25 15:51:47,991 - Matcher - INFO - Executing query against index=ospst-ripsaw-kube-burner*
2024-06-25 15:51:49,211 - Orion - INFO - file: utils.py - line: 50 - Collecting ovnCPU
2024-06-25 15:51:49,211 - Matcher - INFO - Executing query against index=ospst-ripsaw-kube-burner*
2024-06-25 15:51:50,568 - Orion - INFO - file: utils.py - line: 50 - Collecting etcdCPU
2024-06-25 15:51:50,568 - Matcher - INFO - Executing query against index=ospst-ripsaw-kube-burner*
2024-06-25 15:51:52,410 - Orion - INFO - file: utils.py - line: 50 - Collecting etcdDisk
2024-06-25 15:51:52,410 - Matcher - INFO - Executing query against index=ospst-ripsaw-kube-burner*
aws-small-scale-cluster-density-v2
==================================
time                       uuid                                  buildUrl                        podReadyLatency_P99    apiserverCPU_avg    ovnCPU_avg    etcdCPU_avg    etcdDisk_avg
-------------------------  ------------------------------------  ----------------------------  ---------------------  ------------------  ------------  -------------  --------------
2024-06-17 01:11:35 +0000  22dbe8e9-fe5f-4bc4-976e-2443ad5c5848  https://tinyurl.com/2cfnf4yz                  13000            14.5708        3.55213        9.60174       0.0229627
2024-06-23 13:45:25 +0000  72ae2705-d49c-4107-a552-3609c1c6ea40  https://tinyurl.com/2qpjlbnr                  13000             9.78274       3.33798        7.497         0.015988
2024-06-23 16:19:46 +0000  c322787f-4d2a-4132-ba52-118db655a33b  https://tinyurl.com/2kl88q5w                  13000             9.5838        3.3216         7.24948       0.0158418

Related Tickets & Documents

• Related Issue Timestamping uuid/hunter results #29
• Closes #

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.

Testing

• Please describe the System Under Test.
• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

[vishnuchalla]

With the changes in fmatch, looks good to me. I just doubt the SingletonLogger

[jtaleric]

lgtm! Tested! Nice work!

[paigerube14]

/lgtm

[shahsahil264]

LGTM

[vishnuchalla]

With the changes in fmatch, looks good to me. I just doubt the SingletonLogger

Merging the PR for now, as this can be revisited later as an improvement.

[dry923]

so generally LGTM. i just have 1 question. why are we limiting it to just days/hours and not opening it up to "normal" time inputs. ie 1Y2M3h4m2s ? and/or to expand on that allowing a straight date input if you know the cutoff date and dont want to math the days/hours.

[shashank-boyapally]

The idea was to have a lookback period, and run orion as a post step of our jobs. The lookback period will act as a sliding window. We can have cutoff date too in the next feature add. And the precision of seconds, since our jobs are atleast 1h apart, I assumed a precision of hours should work, no harm in having it upto seconds too.