stress-ng workload plugin for Arcaflow

This plugin provides the functionality of stress-ng with various stressors using the Arcaflow python SDK.

Currently-supported stressors

• cpu
• vm
• matrix
• mq
• hdd

To run directly with the Arcaflow engine:

In order to run the arca-stressng plugin follow these steps:

Containerized

1. Clone this repository
2. Create the container with either docker build -t arca-stressng <clone-directory> or podman build -t arca-stressng <clone-directory>
3. Run the container with either cat stressng_example.yaml | docker run -i arca-stressng -f - or cat stressng_example.yaml | podman run -i arca-stressng -f -

Native

• Prerequisite: stress-ng needs to be installed on your system *

1. Clone this repository
2. Create a python venv in the current directory with python3 -m venv ./venv
3. Activate the venv by running source ./venv/bin/activate
4. Install poetry pip install poetry
5. Run poetry install
6. Edit stressng_example.yaml to suit your needs
7. Run ./stressng_plugin.py -f stressng_example.yaml

Autogenerated Input/Output Documentation by Arcaflow-Docsgen Below

stress-ng workload (workload)

Run the stress-ng workload with the given parameters

Input

Type:	scope
Root object:	WorkloadParams
Properties	StressNGParams (reference[StressNGParams]) Name: Stress-NG Job Parameters Description: Global workload parameters and list of stressors for the stress-ng job Required: Yes Type: reference[StressNGParams] Referenced object: StressNGParams cleanup (bool) Name: Cleanup Description: Cleanup artifacts after the plugin run Required: No Default (JSON encoded): false Type: bool
Objects CpuStressorParams (object) Type: object Properties cpu_count (int) Name: CPU count Description: Number of CPU cores to be used (0 means all) Required: Yes Type: int cpu_load (int) Name: CPU load Description: load CPU by percentage Required: No Type: int cpu_method (string) Name: CPU stressor method Description: fine grained control of which cpu stressors to use (ackermann, cfloat etc. Required: No Default (JSON encoded): "all" Type: string stressor (enum[string]) Name: Stressor Description: Stressor for the benchmark workload Required: Yes Type: enum[string] Values cpu: CPU hdd: HDD matrix: MATRIX mq: MQ vm: VM HDDStressorParams (object) Type: object Properties hdd (int) Name: HDD workers Description: start N workers continually writing, reading and removing temporary files Required: Yes Type: int hdd_bytes (string) Name: Bytes per worker Description: write N bytes for each hdd process, the default is 1 GB. One can specify the size in units of Bytes, KBytes, MBytes and GBytes using the suffix b, k, m or g. Required: Yes Type: string hdd_write_size (string) Name: Write Size Description: specify size of each write in bytes. Size can be from 1 byte to 4MBOne can specify the size in units of Bytes, KBytes, MBytes using the suffix b, k, m Required: Yes Type: string stressor (enum[string]) Name: Stressor Description: Stressor for the benchmark workload Required: Yes Type: enum[string] Values cpu: CPU hdd: HDD matrix: MATRIX mq: MQ vm: VM MatrixStressorParams (object) Type: object Properties matrix (int) Name: Matrix count Description: Number of Matrix stressors to be run (0 means 1 stressor per CPU Required: Yes Type: int stressor (enum[string]) Name: Stressor Description: Stressor for the benchmark workload Required: Yes Type: enum[string] Values cpu: CPU hdd: HDD matrix: MATRIX mq: MQ vm: VM MqStressorParams (object) Type: object Properties mq (int) Name: MQ count Description: Number of MQ stressors to be run (0 means 1 stressor per CPU) Required: Yes Type: int stressor (enum[string]) Name: Stressor Description: Stressor for the benchmark workload Required: Yes Type: enum[string] Values cpu: CPU hdd: HDD matrix: MATRIX mq: MQ vm: VM StressNGParams (object) Type: object Properties metrics_brief (bool) Name: brief metrics Description: Brief version of the metrics output Required: No Type: bool stressors (list[one of[string]]) Required: Yes Type: list[one of[string]] List items Type: one of[string] timeout (string) Name: Runtime Description: Time to run the benchmark test Required: Yes Type: string verbose (bool) Name: verbose Description: verbose output Required: No Type: bool workdir (string) Name: Working Dir Description: Path were stress-ng will be executed (example to target a specific volume) Required: No Type: string VmStressorParams (object) Type: object Properties mmap (string) Name: mmap Description: Number of stressors per CPU Required: No Type: string mmap_bytes (string) Name: Allocation of memory per stressor Required: No Type: string stressor (enum[string]) Name: Stressor Description: Stressor for the benchmark workload Required: Yes Type: enum[string] Values cpu: CPU hdd: HDD matrix: MATRIX mq: MQ vm: VM vm (int) Name: VM count Description: Number of VM stressors to be run (0 means 1 stressor per CPU Required: Yes Type: int vm_bytes (string) Name: VM memory Description: Amount of memory a single VM stressor will use Required: Yes Type: string WorkloadParams (object) Type: object Properties StressNGParams (reference[StressNGParams]) Name: Stress-NG Job Parameters Description: Global workload parameters and list of stressors for the stress-ng job Required: Yes Type: reference[StressNGParams] Referenced object: StressNGParams cleanup (bool) Name: Cleanup Description: Cleanup artifacts after the plugin run Required: No Default (JSON encoded): false Type: bool

Outputs

error

Type:	scope
Root object:	WorkloadError
Properties	error (string) Required: Yes Type: string
Objects WorkloadError (object) Type: object Properties error (string) Required: Yes Type: string

success

Type:	scope
Root object:	WorkloadResults
Properties	cpuinfo (reference[CPUOutput]) Name: CPU Output Description: CPU stressor output object Required: No Type: reference[CPUOutput] Referenced object: CPUOutput hddinfo (reference[HDDOutput]) Name: HDD Output Description: HDD stressor output object Required: No Type: reference[HDDOutput] Referenced object: HDDOutput matrixinfo (reference[MatrixOutput]) Name: Matrix Output Description: Matrix stressor output object Required: No Type: reference[MatrixOutput] Referenced object: MatrixOutput mqinfo (reference[MQOutput]) Name: MQ Output Description: MQ stressor output object Required: No Type: reference[MQOutput] Referenced object: MQOutput systeminfo (reference[SystemInfoOutput]) Name: System Info Description: System info output object Required: Yes Type: reference[SystemInfoOutput] Referenced object: SystemInfoOutput vminfo (reference[VMOutput]) Name: VM Output Description: VM stressor output object Required: No Type: reference[VMOutput] Referenced object: VMOutput
Objects CPUOutput (object) Type: object Properties bogo-ops (int) Name: Bogus Operations Description: Number of stressor loop iterations Required: Yes Type: int bogo-ops-per-second-real-time (float) Name: Bogus operations per second in real time Description: real time measurement is how long the run took based on the wall clock time (that is, the time the stressor took to run). Required: Yes Type: float bogo-ops-per-second-usr-sys-time (float) Name: Bogus operations per second per user and sys time Description: is the bogo-ops rate divided by the user + system time.This is the real per CPU throughput taking into consideration all the CPUs used and all the time consumed by the stressor and kernel time. Required: Yes Type: float cpu-usage-per-instance (float) Name: CPU usage per instance Description: is the amount of CPU used by each stressor instance Required: Yes Type: float max-rss (string) Name: Max RSS Description: Maximum resident set size Required: Yes Type: string stressor (string) Name: Stressor Description: Type of stressor for workload Required: Yes Type: string system-time (float) Name: CPU System Time Description: The CPU time spent in kernel space Required: Yes Type: float user-time (float) Name: CPU User Time Description: The CPU time spent in user space Required: Yes Type: float wall-clock-time (float) Name: Wall Clock Time Description: The time the stressor took to run Required: Yes Type: float HDDOutput (object) Type: object Properties bogo-ops (int) Name: Bogus Operations Description: Number of stressor loop iterations Required: Yes Type: int bogo-ops-per-second-real-time (float) Name: Bogus operations per second in real time Description: real time measurement is how long the run took based on the wall clock time (that is, the time the stressor took to run). Required: Yes Type: float bogo-ops-per-second-usr-sys-time (float) Name: Bogus operations per second per user and sys time Description: is the bogo-ops rate divided by the user + system time.This is the real per CPU throughput taking into consideration all the CPUs used and all the time consumed by the stressor and kernel time. Required: Yes Type: float cpu-usage-per-instance (float) Name: CPU usage per instance Description: is the amount of CPU used by each stressor instance Required: Yes Type: float max-rss (string) Name: Max RSS Description: Maximum resident set size Required: Yes Type: string stressor (string) Name: Stressor Description: Type of stressor for workload Required: Yes Type: string system-time (float) Name: CPU System Time Description: The CPU time spent in kernel space Required: Yes Type: float user-time (float) Name: CPU User Time Description: The CPU time spent in user space Required: Yes Type: float wall-clock-time (float) Name: Wall Clock Time Description: The time the stressor took to run Required: Yes Type: float MQOutput (object) Type: object Properties bogo-ops (int) Name: Bogus Operations Description: Number of stressor loop iterations Required: Yes Type: int bogo-ops-per-second-real-time (float) Name: Bogus operations per second in real time Description: real time measurement is how long the run took based on the wall clock time (that is, the time the stressor took to run). Required: Yes Type: float bogo-ops-per-second-usr-sys-time (float) Name: Bogus operations per second per user and sys time Description: is the bogo-ops rate divided by the user + system time.This is the real per CPU throughput taking into consideration all the CPUs used and all the time consumed by the stressor and kernel time. Required: Yes Type: float cpu-usage-per-instance (float) Name: CPU usage per instance Description: is the amount of CPU used by each stressor instance Required: Yes Type: float max-rss (string) Name: Max RSS Description: Maximum resident set size Required: Yes Type: string stressor (string) Name: Stressor Description: Type of stressor for workload Required: Yes Type: string system-time (float) Name: CPU System Time Description: The CPU time spent in kernel space Required: Yes Type: float user-time (float) Name: CPU User Time Description: The CPU time spent in user space Required: Yes Type: float wall-clock-time (float) Name: Wall Clock Time Description: The time the stressor took to run Required: Yes Type: float MatrixOutput (object) Type: object Properties bogo-ops (int) Name: Bogus Operations Description: Number of stressor loop iterations Required: Yes Type: int bogo-ops-per-second-real-time (float) Name: Bogus operations per second in real time Description: real time measurement is how long the run took based on the wall clock time (that is, the time the stressor took to run). Required: Yes Type: float bogo-ops-per-second-usr-sys-time (float) Name: Bogus operations per second per user and sys time Description: is the bogo-ops rate divided by the user + system time.This is the real per CPU throughput taking into consideration all the CPUs used and all the time consumed by the stressor and kernel time. Required: Yes Type: float cpu-usage-per-instance (float) Name: CPU usage per instance Description: is the amount of CPU used by each stressor instance Required: Yes Type: float max-rss (string) Name: Max RSS Description: Maximum resident set size Required: Yes Type: string stressor (string) Name: Stressor Description: Type of stressor for workload Required: Yes Type: string system-time (float) Name: CPU System Time Description: The CPU time spent in kernel space Required: Yes Type: float user-time (float) Name: CPU User Time Description: The CPU time spent in user space Required: Yes Type: float wall-clock-time (float) Name: Wall Clock Time Description: The time the stressor took to run Required: Yes Type: float SystemInfoOutput (object) Type: object Properties bufferram (int) Name: bufferram Description: amount of buffer RAM the test machine had Required: Yes Type: int cpus (int) Name: cpus Description: number of CPU cores the test machine had Required: Yes Type: int cpus-online (int) Name: cpus_online Description: number of online CPUs the test machine had Required: Yes Type: int date-yyyy-mm-dd (string) Name: date Description: date on which the test was run Required: Yes Type: string epoch-secs (int) Name: epoch Description: epoch at which the test was run Required: Yes Type: int freeram (int) Name: freeram Description: amount of free RAM the test machine had Required: Yes Type: int freeswap (int) Name: freeswap Description: amount of free swap the test machine had Required: Yes Type: int hostname (string) Name: hostname Description: host on which the test was run Required: Yes Type: string machine (string) Name: machine Description: machine type on which the test was run Required: Yes Type: string nodename (string) Name: nodename Description: name of the node on which the test was run Required: Yes Type: string pagesize (int) Name: pagesize Description: memory page size the test machine used Required: Yes Type: int release (string) Name: release Description: kernel release on which the test was run Required: Yes Type: string run-by (string) Name: run_by Description: username of the person who ran the test Required: Yes Type: string sharedram (int) Name: sharedram Description: amount of shared RAM the test machine had Required: Yes Type: int stress-ng-version (string) Name: stress_ng_version Description: version of the stressng tool used Required: Yes Type: string sysname (string) Name: system name Description: System name Required: Yes Type: string ticks-per-second (int) Name: ticks_per_second Description: ticks per second used on the test machine Required: Yes Type: int time-hh-mm-ss (string) Name: time Description: time at which the test was run Required: Yes Type: string totalram (int) Name: totalram Description: total amount of RAM the test machine had Required: Yes Type: int totalswap (int) Name: totalswap Description: total amount of swap the test machine had Required: Yes Type: int uptime (int) Name: uptime Description: uptime of the machine the test was run on Required: Yes Type: int version (string) Name: version Description: version on which the test was run Required: Yes Type: string VMOutput (object) Type: object Properties bogo-ops (int) Name: Bogus Operations Description: Number of stressor loop iterations Required: Yes Type: int bogo-ops-per-second-real-time (float) Name: Bogus operations per second in real time Description: real time measurement is how long the run took based on the wall clock time (that is, the time the stressor took to run). Required: Yes Type: float bogo-ops-per-second-usr-sys-time (float) Name: Bogus operations per second per user and sys time Description: is the bogo-ops rate divided by the user + system time.This is the real per CPU throughput taking into consideration all the CPUs used and all the time consumed by the stressor and kernel time. Required: Yes Type: float cpu-usage-per-instance (float) Name: CPU usage per instance Description: is the amount of CPU used by each stressor instance Required: Yes Type: float max-rss (string) Name: Max RSS Description: Maximum resident set size Required: Yes Type: string stressor (string) Name: Stressor Description: Type of stressor for workload Required: Yes Type: string system-time (float) Name: CPU System Time Description: The CPU time spent in kernel space Required: Yes Type: float user-time (float) Name: CPU User Time Description: The CPU time spent in user space Required: Yes Type: float wall-clock-time (float) Name: Wall Clock Time Description: The time the stressor took to run Required: Yes Type: float WorkloadResults (object) Type: object Properties cpuinfo (reference[CPUOutput]) Name: CPU Output Description: CPU stressor output object Required: No Type: reference[CPUOutput] Referenced object: CPUOutput hddinfo (reference[HDDOutput]) Name: HDD Output Description: HDD stressor output object Required: No Type: reference[HDDOutput] Referenced object: HDDOutput matrixinfo (reference[MatrixOutput]) Name: Matrix Output Description: Matrix stressor output object Required: No Type: reference[MatrixOutput] Referenced object: MatrixOutput mqinfo (reference[MQOutput]) Name: MQ Output Description: MQ stressor output object Required: No Type: reference[MQOutput] Referenced object: MQOutput systeminfo (reference[SystemInfoOutput]) Name: System Info Description: System info output object Required: Yes Type: reference[SystemInfoOutput] Referenced object: SystemInfoOutput vminfo (reference[VMOutput]) Name: VM Output Description: VM stressor output object Required: No Type: reference[VMOutput] Referenced object: VMOutput