Update Term Design and add Confidence Level

[dinogun]

The current design [v0.0.20] related to terms has a couple of issues

1. Term durations are not strictly enforced, we can go back in time (upto a threshold level) to satisfy requirements related to a term duration. This means that the user is not clear what duration was considered to arrive at a recommendation
2. If enough data is not available that satisfies the entire term duration, then no recommendation is generated. eg, for short term, we need a full 96 data points with a measurement duration of 15 mins. This means that for any workload that is short lived, say 3 hours, we cannot generate any recommendations

To prevent this, we need to update the term design as follows

1. Fix term durations as follows (Assuming a measurement duration of 15 mins: short term = 24 hours (Max: 96 data points), medium term = 7 days (Max: 672 data points), long term = 15 days (Max: 1440 data points).
2. Remove buffer thresholds. This means that we do not go further back in time, beyond the prescribed term durations above, to get data for a given duration
3. Add a minimum data threshold. This represents the minimum amount of data needed for generating a recommendation associated with a given duration term. We can start with the following defaults. Minimum data points for: Short term: 30 mins (Min: 2 data points), Medium term: 2 days (Min: 192 data points), Long Term: 8 days (Min: 768 data points)
4. The design should allow for the minimum thresholds to be configurable for each individual experiment.
5. Add a confidence level associated with a recommendation. The confidence level is a value between 0 and 1 where 1 represents the highest confidence in the recommendation. The initial implementation will be based entirely on the number of data points used to generate the recommendation. It can be confidence_level = actual_data_points / term_max_data_points

[chandrams]

If a user has to configure thresholds for more than a term, how can this be done with a single threshold field?

Should we express threshold in terms of data points required and internally calculate it as measurement_duration * threshold as measurement_duration is also configurable? This can be taken up later if required.

[chandrams]

Test scenarios for validating this:

• Validate recommendations & confidence level, when the specified no. of data points are available within the term duration

  • With 2 data points within 24 hrs for short term

  • With 96 data points within 24 hrs for short term

  • With 1 data point within 24 hrs for short term

  • With 2 data points beyond 24 hrs for short term

  • With no data points within 24 hrs for short term

    Test above for medium & long term

• If min data threshold is configurable then validate the min data threshold configured for each of the terms is reflected, test any ranges and ensure the expected error messages are displayed

• Validate user documentation for min data threshold and json sample output

[rbadagandi1]

Moving this issue to 0.0.21 since Term Design changes are going as part of 1081 PR and 0.0.20.2 release. This issue will be closed as part of Confidence level implementation