Test scenarios for Bulk API

[chandrams]

End-to-End test

• Deploy kruize, create metric profile, invoke the crawler API with the specified crawler json, get the job status using the jobID and invoke listRecommendations using the completed experiment name

• Test with following inputs in the crawler json

  • Specify only timestamp range
  • Specify only labels
  • Datasource as prometheus
  • Datasource as thanos
  • Ensure exclude section is taken into account by specifying:
    • Containers to exclude
    • Namespaces to exclude
    • Workloads to exclude
  • Ensure include section is taken into account by specifying:
    • Containers to include
    • Namespaces to include
    • Workloads to include

• Add validations for:

  • API response codes
  • Validate the job status json
  • Validate the recommendations returned by list recommendations json

• End-to-End test - invoke listRecommendations for not completed experiments, what is the behavior?

• Invoke the crawler API with invalid json parameters & validate the response status code & message

  • Invalid labels
    • Invalid org_id - blank, null, invalid, non-existing
    • Invalid cluster_id - blank, null, invalid, non-existing
    • Same labels in exclude and include, what is the behavior?
  • Invalid time range
    • Invalid time range - blank, null, invalid format
    • Valid time range but with no usage data available
  • Empty crawler input json
  • Invalid datasource
    • Non-existing datasource
    • Datasource that is not reachable
  • Mandatory fields in the crawler input json
    • Are any fields in the crawler json mandatory? If so, test by omitting the mandatory fields
    • Invalid Exclusions / Inclusions
    • What if the same containers/namespaces/workloads are present in the exclusions / inclusions?
    • What is the behavior when the namespaces excluded are present in multiple clusters, will it be excluded irrespective of the inclusion labels?