Adding method to ack

[jtaleric]

Type of change

We need a mechanism to ack issues we have opened issues for, or issues we have brought to the attention of dev teams.

• Refactor
• New feature
• Bug fix
• Optimization
• Documentation Update

Description

Related Tickets & Documents

• Related Issue #
• Closes #

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.

Testing

• Please describe the System Under Test.
• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

[josecastillolema]

Closes #78

[josecastillolema]

So the ack.yml file will live in the orion repository right?
How often do you envision this file will need to be updated?

[vishnuchalla]

This approach will bring us overhead to specify yaml manually to acknowledge regressions which are just being deleted from our list of changepoints. Instead we need to think of an automated way of acknowledging regressions so that there is no manual intervention.

One possible idea could be, Why not have orion publish all the regressions detected to a separate elastic search index automatically? it can query list of existing regressions from the persistent store (.i.e es index that it has) and delete them from its reporting list to avoid duplicates every single time it notifies regressions.

[jtaleric]

This approach will bring us overhead to specify yaml manually to acknowledge regressions which are just being deleted from our list of changepoints. Instead we need to think of an automated way of acknowledging regressions so that there is no manual intervention.

IMHO Performance Regressions should be manually ack'ed.

I don't think automated acks are the right way.

One possible idea could be, Why not have orion publish all the regressions detected to a separate elastic search index automatically? it can query list of existing regressions from the persistent store (.i.e es index that it has) and delete them from its reporting list to avoid duplicates every single time it notifies regressions.

This sounds like more overhead and additional accounting due to yet another index in OpenSearch. Today Orion is purely RO when it comes to OpenSearch.

Let's jump in a huddle so I can better explain what we are trying to accomplish! Thanks!

[jtaleric]

So the ack.yml file will live in the orion repository right?

How often do you envision this file will need to be updated?

The workflow I proposed to @shashank-boyapally :

1. Regression is found
2. PerfEng User engages with component team, determines to open a bug
3. PerfEng who engaged with component team opens PR to add the ack.

This way we know who is acking in the event we need to look back and ask questions.

[vishnuchalla]

This approach will bring us overhead to specify yaml manually to acknowledge regressions which are just being deleted from our list of changepoints. Instead we need to think of an automated way of acknowledging regressions so that there is no manual intervention.

IMHO Performance Regressions should be manually ack'ed.

I don't think automated acks are the right way.

One possible idea could be, Why not have orion publish all the regressions detected to a separate elastic search index automatically? it can query list of existing regressions from the persistent store (.i.e es index that it has) and delete them from its reporting list to avoid duplicates every single time it notifies regressions.

This sounds like more overhead and additional accounting due to yet another index in OpenSearch. Today Orion is purely RO when it comes to OpenSearch.

Okay, lets say if we ack regressions manually. What is the plan to make a note of all the previously detected regressions somewhere and feed them as input yaml for every single run of orion?

[vishnuchalla]

So the ack.yml file will live in the orion repository right?
How often do you envision this file will need to be updated?

The workflow I proposed to @shashank-boyapally :

1. Regression is found
2. PerfEng User engages with component team, determines to open a bug
3. PerfEng who engaged with component team opens PR to add the ack.

This way we know who is acking in the event we need to look back and ask questions.

So, do you mean to say that we will have a yaml in this repo itself which will keep on updating manually with the list of regressions every single time they are detected?

[jtaleric]

So the ack.yml file will live in the orion repository right?
How often do you envision this file will need to be updated?

The workflow I proposed to @shashank-boyapally :

1. Regression is found
2. PerfEng User engages with component team, determines to open a bug
3. PerfEng who engaged with component team opens PR to add the ack.

This way we know who is acking in the event we need to look back and ask questions.

So, do you mean to say that we will have a yaml in this repo itself which will keep on updating manually with the list of regressions every single time they are detected?

It doesn't need to be in this repo, but yes, the workflow should take a manual effort.. Regressions shouldn't be automagically silenced, IMHO.

[vishnuchalla]

lgtm