Kube-burner workload plugin for Arcaflow

arca-kube-burner is a workload plugin which can run kube-burner benchmarks or the web-burner workloads using the Arcaflow python SDK.

Documentation for Kube-burner workloads can be found here: Workloads Documentation

Documentation for web-burner workloads can be found here: Workloads Documentation

To test:

In order to run the kube-burner plugin run the following steps:

Native

Note: The plugin should be able to access the kubeconfig of your openshift cluster and the kube-burner binary must be downloaded locally. Install poetry(curl -sSL https://install.python-poetry.org | python3 - ). Poetry requires python version > 3.7, recommended to use >3.9

1. Clone this repository
2. Create a venv in the current directory with python3.9 -m venv $(pwd)/venv
3. Activate the venv by running source venv/bin/activate
4. cd arcaflow-plugin-kube-burner
5. curl -L https://github.com/cloud-bulldozer/kube-burner/releases/download/v1.4.2/kube-burner-1.4.2-Linux-x86_64.tar.gz | tar xz -C . kube-burner
6. Run poetry install
7. Copy and Paste the openshift cluster's kubeconfig file content into the kubeburner_input.yaml file
8. To run a kube-burner workload python3.9 ./arcaflow_plugin_kubeburner/kubeburner_plugin.py -f configs/kubeburner_input.yaml -s kube-burner --debug

Containerized

1. Clone this repository
2. cd arcaflow-plugin-kube-burner
3. Copy and Paste the openshift cluster's kubeconfig file content into the kubeburner_input.yaml file
4. Create the container with docker build -t arca-kube-burner -f Dockerfile
5. Run cat configs/kubeburner_input.yaml | docker run -i arca-kube-burner -s kube-burner --debug -f -

In order to run the web-burner plugin run the following steps:

Prerequisites

Note: This is for ICNI2 worklaods

1. Enable sr-iov on the baremetal nodes from the node management console or using badfish.
2. Install the openshift-sriov-network-operator on the openshift cluster using the cli or the operatorhub GUI.
3. Identify and label a specific number of nodes with the node-role.kubernetes.io/worker-spk="" label.
4. check if all labelled worker nodes have the same sr-iov PF(this is done by sshing into each node from the provisoner node to get the PF of a node, command: nic=$(ssh -i /home/kni/.ssh/id_rsa -o StrictHostKeyChecking=no core@{worker-node name} "sudo ovs-vsctl list-ports br-ex | head -1") eg: $nic = ens7f0
5. Apply the sriov node policy using the $nic obtained from step 4.
6. wait for sriov nodes to be ready

Native

Note: The plugin should be able to access the kubeconfig of your openshift cluster and the kube-burner binary must be downloaded locally. Rename the kube-burner binary as web-burner or follow step number 7&8 below. Install poetry(curl -sSL https://install.python-poetry.org | python3 - ). Poetry requires python version > 3.7, recommended to use >3.9

1. Clone this repository
2. Create a venv in the current directory with python3.9 -m venv $(pwd)/venv
3. Activate the venv by running source venv/bin/activate
4. Run git clone https://github.com/redhat-performance/web-burner.git --branch v1.0
5. Run cp -r web-burner/workload web-burner/objectTemplates arcaflow-plugin-kube-burner/
6. cd arcaflow-plugin-kube-burner
7. curl -L https://github.com/cloud-bulldozer/kube-burner/releases/download/v0.14.2/kube-burner-0.14.2-Linux-x86_64.tar.gz | tar xz -C . kube-burner
8. mv kube-burner kube-burner-0.14.2
9. Run poetry install
10. Copy and Paste the openshift cluster's kubeconfig file content into the configs/webburner_input.yaml file
11. To run a web-burner workload python3.9 ./arcaflow_plugin_kubeburner/kubeburner_plugin.py -f configs/webburner_input.yaml -s run-web-burner --debug
12. To delete a web-burner workload python3.9 ./arcaflow_plugin_kubeburner/kubeburner_plugin.py -f configs/webburner_input.yaml -s delete-web-burner --debug

Containerized

1. Clone this repository
2. cd arcaflow-plugin-kube-burner
3. Copy and Paste the openshift cluster's kubeconfig file content into the configs/webburner_input.yaml and configs/webburner_cleanup.yaml files.
4. Create the container with docker build -t arca-web-burner .
5. To run a web-burner workload cat configs/webburner_input.yaml | docker run -i arca-web-burner -s run-web-burner --debug -f -
6. To delete a web-burner workload cat configs/webburner_input.yaml | docker run -i arca-kube-burner -s delete-web-burner --debug -f -

Image Building

You can change this plugin's image version tag in .github/workflows/carpenter.yaml by editing the IMAGE_TAG variable, and pushing that change to the branch designated in that workflow.

Autogenerated Input/Output Documentation by Arcaflow-Docsgen Below

Web-Burner Workload (delete-web-burner)

Plugin to delete resources created by the web-burner workload

Input

Type:	scope
Root object:	WebBurnerInputParams
Properties	bfd_enabled (string) Name: BFD Description: Bidirectional Forwarding Detection Required: No Default (JSON encoded): "false" Type: string bridge (string) Name: BRIDGE Description: The network bridge to use. breth0 for kind.sh ovn-kubernetes clusters Required: No Default (JSON encoded): "br-ex" Type: string burst (int) Name: Burst Description: Maximum burst for throttle Required: No Default (JSON encoded): 20 Type: int es_index (string) Name: es-index Description: The ElasticSearch index used to index the metrics Required: No Type: string es_server (string) Name: es-server Description: List of ES instances Required: No Type: string indexing (string) Name: INDEXING Description: To enable or disable indexing in elasticsearch(true/false) Required: No Default (JSON encoded): "false" Type: string kubeconfig (string) Name: kubeconfig Description: Openshift cluster kubeconfig file content as a string Required: Yes Type: string number_of_nodes (int) Name: Number of nodes Description: Size of cluster/ number of nodes in the cluster Required: Yes Type: int qps (int) Name: QPS Description: Max number of queries per second Required: No Default (JSON encoded): 20 Type: int scale_factor (int) Name: Scale Factor Description: Scaling factor for the workload Required: No Default (JSON encoded): 1 Type: int sriov (string) Name: SRIOV Description: To enable or disable sriov, disabling it will create macvlan network attachment definitions instead Required: No Default (JSON encoded): "true" Type: string uuid (string) Name: uuid Description: uuid to be used for the job Required: No Type: string workload_template (string) Name: Workload Template Description: Kube-burner Template to use Required: Yes Type: string
Objects WebBurnerInputParams (object) Type: object Properties bfd_enabled (string) Name: BFD Description: Bidirectional Forwarding Detection Required: No Default (JSON encoded): "false" Type: string bridge (string) Name: BRIDGE Description: The network bridge to use. breth0 for kind.sh ovn-kubernetes clusters Required: No Default (JSON encoded): "br-ex" Type: string burst (int) Name: Burst Description: Maximum burst for throttle Required: No Default (JSON encoded): 20 Type: int es_index (string) Name: es-index Description: The ElasticSearch index used to index the metrics Required: No Type: string es_server (string) Name: es-server Description: List of ES instances Required: No Type: string indexing (string) Name: INDEXING Description: To enable or disable indexing in elasticsearch(true/false) Required: No Default (JSON encoded): "false" Type: string kubeconfig (string) Name: kubeconfig Description: Openshift cluster kubeconfig file content as a string Required: Yes Type: string number_of_nodes (int) Name: Number of nodes Description: Size of cluster/ number of nodes in the cluster Required: Yes Type: int qps (int) Name: QPS Description: Max number of queries per second Required: No Default (JSON encoded): 20 Type: int scale_factor (int) Name: Scale Factor Description: Scaling factor for the workload Required: No Default (JSON encoded): 1 Type: int sriov (string) Name: SRIOV Description: To enable or disable sriov, disabling it will create macvlan network attachment definitions instead Required: No Default (JSON encoded): "true" Type: string uuid (string) Name: uuid Description: uuid to be used for the job Required: No Type: string workload_template (string) Name: Workload Template Description: Kube-burner Template to use Required: Yes Type: string

Outputs

error

Type:	scope
Root object:	ErrorOutput
Properties	error (string) Name: Failure Error Description: Reason for failure Required: Yes Type: string exit_code (int) Name: Exit Code Description: Exit code returned by the program in case of a failure Required: Yes Type: int
Objects ErrorOutput (object) Type: object Properties error (string) Name: Failure Error Description: Reason for failure Required: Yes Type: string exit_code (int) Name: Exit Code Description: Exit code returned by the program in case of a failure Required: Yes Type: int

success

Type:	scope
Root object:	SuccessOutput
Properties	output (string) Name: Kube burner workload output Description: Output generated by the kube burner workload Required: Yes Type: string uuid (string) Name: UUID Description: UUID used for this workload run Required: Yes Type: string
Objects SuccessOutput (object) Type: object Properties output (string) Name: Kube burner workload output Description: Output generated by the kube burner workload Required: Yes Type: string uuid (string) Name: UUID Description: UUID used for this workload run Required: Yes Type: string

Kube-Burner Workload (kube-burner)

Kube-burner Workloads: node-density, node-density-cni, node-density-heavy, cluster-density, cluster-density-v2, cluster-density-ms

Input

Type:	scope
Root object:	KubeBurnerInputParams
Properties	alerting (string) Name: alerting Description: Enable alerting(true/false) Required: No Default (JSON encoded): "true" Type: string burst (int) Name: Burst Description: Maximum burst for throttle Required: No Default (JSON encoded): 20 Type: int churn (string) Name: churn Description: Enable churning(true/false) Required: No Default (JSON encoded): "true" Type: string churn_delay (string) Name: churn-delay Description: Time to wait between each churn Required: No Default (JSON encoded): "2m0s" Type: string churn_duration (string) Name: churn-duration Description: Churn duration Required: No Default (JSON encoded): "1h0m0s" Type: string churn_percent (int) Name: churn-percent Description: Percentage of job iterations that kube-burner will churn each round Required: No Default (JSON encoded): 10 Type: int es_index (string) Name: es-index Description: The ElasticSearch index used to index the metrics Required: No Type: string es_server (string) Name: es-server Description: List of ES instances Required: No Type: string gc (string) Name: gc Description: Garbage collect created namespaces(true/false) Required: No Default (JSON encoded): "true" Type: string iterations (int) Name: iterations Description: Cluster-density iterations Required: No Default (JSON encoded): 500 Type: int kubeconfig (string) Name: kubeconfig Description: Openshift cluster kubeconfig file content as a string Required: Yes Type: string local_indexing (string) Name: local-indexing Description: Enable local indexing Required: No Default (JSON encoded): "false" Type: string log_level (string) Name: log-level Description: Allowed values: debug, info, warn, error, fatal Required: No Default (JSON encoded): "info" Type: string network_policies (string) Name: network-policies Description: Enable network policies in the workload Required: No Default (JSON encoded): "true" Type: string pod_ready_threshold (string) Name: pod-ready-threshold Description: Pod ready timeout threshold for node-density workload Required: No Default (JSON encoded): "5s" Type: string pods_per_node (int) Name: pods-per-node Description: Pods per node for node-density* workloads Required: No Default (JSON encoded): 245 Type: int probes_period (int) Name: probes-period Description: Perf app readiness/livenes probes period in seconds Required: No Default (JSON encoded): 10 Type: int qps (int) Name: QPS Description: Max number of queries per second Required: No Default (JSON encoded): 20 Type: int timeout (string) Name: timeout Description: Benchmark timeout Required: No Default (JSON encoded): "2h" Type: string uuid (string) Name: uuid Description: uuid to be used for the job Required: No Type: string workload (string) Name: Name Description: workload name Required: Yes Type: string
Objects KubeBurnerInputParams (object) Type: object Properties alerting (string) Name: alerting Description: Enable alerting(true/false) Required: No Default (JSON encoded): "true" Type: string burst (int) Name: Burst Description: Maximum burst for throttle Required: No Default (JSON encoded): 20 Type: int churn (string) Name: churn Description: Enable churning(true/false) Required: No Default (JSON encoded): "true" Type: string churn_delay (string) Name: churn-delay Description: Time to wait between each churn Required: No Default (JSON encoded): "2m0s" Type: string churn_duration (string) Name: churn-duration Description: Churn duration Required: No Default (JSON encoded): "1h0m0s" Type: string churn_percent (int) Name: churn-percent Description: Percentage of job iterations that kube-burner will churn each round Required: No Default (JSON encoded): 10 Type: int es_index (string) Name: es-index Description: The ElasticSearch index used to index the metrics Required: No Type: string es_server (string) Name: es-server Description: List of ES instances Required: No Type: string gc (string) Name: gc Description: Garbage collect created namespaces(true/false) Required: No Default (JSON encoded): "true" Type: string iterations (int) Name: iterations Description: Cluster-density iterations Required: No Default (JSON encoded): 500 Type: int kubeconfig (string) Name: kubeconfig Description: Openshift cluster kubeconfig file content as a string Required: Yes Type: string local_indexing (string) Name: local-indexing Description: Enable local indexing Required: No Default (JSON encoded): "false" Type: string log_level (string) Name: log-level Description: Allowed values: debug, info, warn, error, fatal Required: No Default (JSON encoded): "info" Type: string network_policies (string) Name: network-policies Description: Enable network policies in the workload Required: No Default (JSON encoded): "true" Type: string pod_ready_threshold (string) Name: pod-ready-threshold Description: Pod ready timeout threshold for node-density workload Required: No Default (JSON encoded): "5s" Type: string pods_per_node (int) Name: pods-per-node Description: Pods per node for node-density* workloads Required: No Default (JSON encoded): 245 Type: int probes_period (int) Name: probes-period Description: Perf app readiness/livenes probes period in seconds Required: No Default (JSON encoded): 10 Type: int qps (int) Name: QPS Description: Max number of queries per second Required: No Default (JSON encoded): 20 Type: int timeout (string) Name: timeout Description: Benchmark timeout Required: No Default (JSON encoded): "2h" Type: string uuid (string) Name: uuid Description: uuid to be used for the job Required: No Type: string workload (string) Name: Name Description: workload name Required: Yes Type: string

Outputs

error

Type:	scope
Root object:	ErrorOutput
Properties	error (string) Name: Failure Error Description: Reason for failure Required: Yes Type: string exit_code (int) Name: Exit Code Description: Exit code returned by the program in case of a failure Required: Yes Type: int
Objects ErrorOutput (object) Type: object Properties error (string) Name: Failure Error Description: Reason for failure Required: Yes Type: string exit_code (int) Name: Exit Code Description: Exit code returned by the program in case of a failure Required: Yes Type: int

success

Type:	scope
Root object:	SuccessOutput
Properties	output (string) Name: Kube burner workload output Description: Output generated by the kube burner workload Required: Yes Type: string uuid (string) Name: UUID Description: UUID used for this workload run Required: Yes Type: string
Objects SuccessOutput (object) Type: object Properties output (string) Name: Kube burner workload output Description: Output generated by the kube burner workload Required: Yes Type: string uuid (string) Name: UUID Description: UUID used for this workload run Required: Yes Type: string

Web-Burner Workload (run-web-burner)

Plugin to run the Web-burner workload

Input

Type:	scope
Root object:	WebBurnerInputParams
Properties	bfd_enabled (string) Name: BFD Description: Bidirectional Forwarding Detection Required: No Default (JSON encoded): "false" Type: string bridge (string) Name: BRIDGE Description: The network bridge to use. breth0 for kind.sh ovn-kubernetes clusters Required: No Default (JSON encoded): "br-ex" Type: string burst (int) Name: Burst Description: Maximum burst for throttle Required: No Default (JSON encoded): 20 Type: int es_index (string) Name: es-index Description: The ElasticSearch index used to index the metrics Required: No Type: string es_server (string) Name: es-server Description: List of ES instances Required: No Type: string indexing (string) Name: INDEXING Description: To enable or disable indexing in elasticsearch(true/false) Required: No Default (JSON encoded): "false" Type: string kubeconfig (string) Name: kubeconfig Description: Openshift cluster kubeconfig file content as a string Required: Yes Type: string number_of_nodes (int) Name: Number of nodes Description: Size of cluster/ number of nodes in the cluster Required: Yes Type: int qps (int) Name: QPS Description: Max number of queries per second Required: No Default (JSON encoded): 20 Type: int scale_factor (int) Name: Scale Factor Description: Scaling factor for the workload Required: No Default (JSON encoded): 1 Type: int sriov (string) Name: SRIOV Description: To enable or disable sriov, disabling it will create macvlan network attachment definitions instead Required: No Default (JSON encoded): "true" Type: string uuid (string) Name: uuid Description: uuid to be used for the job Required: No Type: string workload_template (string) Name: Workload Template Description: Kube-burner Template to use Required: Yes Type: string
Objects WebBurnerInputParams (object) Type: object Properties bfd_enabled (string) Name: BFD Description: Bidirectional Forwarding Detection Required: No Default (JSON encoded): "false" Type: string bridge (string) Name: BRIDGE Description: The network bridge to use. breth0 for kind.sh ovn-kubernetes clusters Required: No Default (JSON encoded): "br-ex" Type: string burst (int) Name: Burst Description: Maximum burst for throttle Required: No Default (JSON encoded): 20 Type: int es_index (string) Name: es-index Description: The ElasticSearch index used to index the metrics Required: No Type: string es_server (string) Name: es-server Description: List of ES instances Required: No Type: string indexing (string) Name: INDEXING Description: To enable or disable indexing in elasticsearch(true/false) Required: No Default (JSON encoded): "false" Type: string kubeconfig (string) Name: kubeconfig Description: Openshift cluster kubeconfig file content as a string Required: Yes Type: string number_of_nodes (int) Name: Number of nodes Description: Size of cluster/ number of nodes in the cluster Required: Yes Type: int qps (int) Name: QPS Description: Max number of queries per second Required: No Default (JSON encoded): 20 Type: int scale_factor (int) Name: Scale Factor Description: Scaling factor for the workload Required: No Default (JSON encoded): 1 Type: int sriov (string) Name: SRIOV Description: To enable or disable sriov, disabling it will create macvlan network attachment definitions instead Required: No Default (JSON encoded): "true" Type: string uuid (string) Name: uuid Description: uuid to be used for the job Required: No Type: string workload_template (string) Name: Workload Template Description: Kube-burner Template to use Required: Yes Type: string

Outputs

error

Type:	scope
Root object:	ErrorOutput
Properties	error (string) Name: Failure Error Description: Reason for failure Required: Yes Type: string exit_code (int) Name: Exit Code Description: Exit code returned by the program in case of a failure Required: Yes Type: int
Objects ErrorOutput (object) Type: object Properties error (string) Name: Failure Error Description: Reason for failure Required: Yes Type: string exit_code (int) Name: Exit Code Description: Exit code returned by the program in case of a failure Required: Yes Type: int

success

Type:	scope
Root object:	SuccessOutput
Properties	output (string) Name: Kube burner workload output Description: Output generated by the kube burner workload Required: Yes Type: string uuid (string) Name: UUID Description: UUID used for this workload run Required: Yes Type: string
Objects SuccessOutput (object) Type: object Properties output (string) Name: Kube burner workload output Description: Output generated by the kube burner workload Required: Yes Type: string uuid (string) Name: UUID Description: UUID used for this workload run Required: Yes Type: string