E2E Test failures on 8.9.0

[pebrc]

TestFilebeatAutodiscoverRecipe,

.........................................................................................................................................................................................................................................................................................................
    step.go:51: 
        	Error Trace:	/go/src/github.com/elastic/cloud-on-k8s/test/e2e/test/utils.go:94
        	Error:      	Received unexpected error:
        	            	hit count should be more than 0 for /*beat*/_search?q=agent.type:filebeat
        	Test:       	TestFilebeatAutodiscoverRecipe/ES_data_should_pass_validations

TestKubernetesIntegrationRecipe

.........................................................................................................................................................................................................................................................................................
    step.go:51: 
        	Error Trace:	/go/src/github.com/elastic/cloud-on-k8s/test/e2e/test/utils.go:94
        	Error:      	Received unexpected error:
        	            	elasticsearch client failed for https://elasticsearch-zk64-es-internal-http.e2e-2lvs6-mercury.svc:9200/_data_stream/metrics-kubernetes.container-k8s: 404 Not Found: {Status:404 Error:{CausedBy:{Reason: Type:} Reason:no such index [metrics-kubernetes.container-k8s] Type:index_not_found_exception StackTrace: RootCause:[{Reason:no such index [metrics-kubernetes.container-k8s] Type:index_not_found_exception}]}}
        	Test:       	TestKubernetesIntegrationRecipe/ES_data_should_pass_validations

TestMetricbeatStackMonitoringRecipe.

........................................................................................................................................................................................................................................................................................................
    step.go:51: 
        	Error Trace:	/go/src/github.com/elastic/cloud-on-k8s/test/e2e/test/utils.go:94
        	Error:      	Received unexpected error:
        	            	hit count should be more than 0 for /*beat*/_search?q=agent.type:metricbeat
        	Test:       	TestMetricbeatStackMonitoringRecipe/ES_data_should_pass_validations

[pebrc]

TestFilebeatAutodiscoverRecipe passes when adding new RBAC permissions which are apparently new in 8.9.0 (I am not sure why we did not catch this in the snapshot builds)

+- apiGroups: ["apps"]
+  resources:
+  - replicasets
+  verbs:
+  - get
+  - list
+  - watch
+- apiGroups: ["batch"]
+  resources:
+  - jobs
+  verbs:
+  - get
+  - list
+  - watch

[thbkrkr]

It's strange, we catched it partially, for TestFilebeatDefaultConfig per example, see #6946 and #6946 (comment) to read the story behind this.

[pebrc]

Same for TestKubernetesIntegrationRecipe