Skip to content

v3.5 Scenarios

Jump to bottom
Verobika Kochugova edited this page Jun 4, 2018 · 15 revisions

Contents

  1. Introduction
  2. Syntax
  3. Configuration
    3.1. Inheritance
    3.2. Overriding
    3.3. Values Substitution
  4. Step Types
    4.1. External Command
    4.2. Load
    4.3. Precondition
    4.4. Composite
    4.4.1. Sequential
    4.4.2. Parallel
    4.4.3. Loop
    4.4.3.1. By Count
    4.4.3.2. By Range
    4.4.3.3. By Sequence
    4.4.3.4. Infinite
    4.5. Mixed Load
    4.5.1. Weighted Load
    4.6. Chain Load
    4.6.1. Delay Between Operations
  5. Scenarios Validation

1. Introduction

Mongoose supports custom user scenarios. Currently it supports the JSON based scenarios syntax which is described in this document.

2. Syntax

Basically, the scenario describes a tree of steps. Some kind of steps may include another step either a list of the steps. Each scenario step may contain its own config section describing this step configuration.

3. Configuration

Mongoose can not run without a scenario. So it uses the default scenario implicitly if the scenario file to run is not specified obviously. The file containing the default scenario is located at scenario/default.json.

The default scenario contents:

{
    "type" : "load"
}

To run a custom scenario the following CLI option should be used:

java -jar <MONGOOSE_DIR>/mongoose.jar \
    --test-scenario-file=<PATH_TO_JSON_SCENARIO_FILE>

3.1. Inheritance

The configuration values from the step's configuration are inherited by all child steps (and possibly overridden).

{
   "type" : "sequential",
   "config" : {
      // the configuration specified here will be inherited by the child steps
      "load" : {
          "type" : "read"
      }
   },
   "steps" : [
      {
          // will use the inherited "load-type=read" value
          "type" : "load",
      },
      {
          "type" : "load",
          "config" : {
              "load" : {
                  // overrides the inherited "load-type=read" value
                  "type" : "update"
              }
          }
      }
      ...
   ]
}

3.2. Overriding

The configuration values from the step's configuration override the default configuration and the CLI options:

{
   "type" : "load",
   "config" : {
      // here are the configuration hierarchy
      "test" : {
          "step" : {
              "id" : "step_0"
          }
      }
   }
}

In the case above doesn't matter which test-step-id CLI option is specified, the value "step_0" will override this.

Inheritance and overriding order:

  1. Configuration loaded from config/defaults.json
  2. Override with CLI arguments.
  3. Override with the parent (composite) step configuration values.
  4. Override with the current step configuration values.

3.3. Values Substitution

Values substitution allows to parameterize the scenarios using environment variables. The environment variables values should be referred "${<ENV_VAR_NAME>}" in the scenario.

{
    "type" : "load",
    "config" : {
        "item" : {
            "data" : {
                "size" : "${ITEM_DATA_SIZE}"
            },
            "output" : {
                "path" : "${ITEM_OUTPUT_PATH}"
            },
            "input" : {
                "file" : "${ITEM_INPUT_FILE}"
            }
        },
        "load" : {
            "type" : "${LOAD_TYPE}",
            "limit" : {
                "concurrency" : "${STORAGE_DRIVER_CONCURRENCY}"
            }
        }
    }
}

Then the commands to run this scenario may look like:

export ITEM_OUTPUT_PATH=bucket1
export ITEM_INPUT_FILE=items.csv
export STORAGE_DRIVER_CONCURRENCY=100
export ITEM_DATA_SIZE=1KB
export LOAD_TYPE=create
java -jar mongoose-3.5.1/mongoose.jar \
    --scenario-file=mongoose-3.5.1/scenario/misc/env-vars-substitution.json

4. Step Types

Quick reference table

Type Description Mandatory Optional
command Execute the external command type, value blocking
load Execute the load operations type config
precondition Same as "load" but w/o persistent metrics output type config
sequential Execute the child steps sequentially type, steps config
parallel Execute the chile step in parallel type, steps config
for Execute the child steps sequence multiple times in a loop type, steps config, value, in
mixed Execute the load operations of different kinds type config
chain Execute the chain of the load operations type config

4.1. External Command

The step allows to execute the external commands.

{
   "type" : "command",
   "value" : "sleep 10"
}

The step executes in the blocking manner by default. To execute the command asynchronously (don't wait for its finish):

{
   "type" : "command",
   "value" : "find /",
   "blocking" : false
}

4.2. Load

Executes the load operations on the storage. A load step also produces the metrics as a result.

The example of the typical load step configuration is below:

{
    "type" : "load",
    "config" : {
        "item" : {
            "input" : {
                "file" : "items2read.csv"
            }
        },
        "load" : {
            "type" : "read"
        },
        "storage" : {
            "auth" : {
                "uid" : "user1",
                "secret" : "**********",
                "token" : "42b16b00b542b16b00b542b16b00b542b16b00b5"
            },
            "driver" : {
                "addrs" : [
                    "remote-driver-0.com",
                    "remote-driver-1.com",
                    "remote-driver-2.com",
                    "remote-driver-3.com"
                ],
                "remote" : true,
                "type" : "atmos"
            },
            "load" : {
                "generator" : {
                    "recycle" : {
                        "enabled" : true
                    }
                },
                "limit" : {
                    "concurrency" : 1000
                }
            },
            "net" : {
                "node" : {
                    "addrs" : [
                        "storage-node-0.com",
                        "storage-node-1.com",
                        "storage-node-2.com",
                        "storage-node-3.com"
                    ],
                    "port" : 8080
                }
            },
            "test" : {
                "step" : {
                    "limit" : {
                        "time" : "10m"
                    }
                }
            }
        }
    }
}

4.3. Precondition

Deprecated. Use the output configuration options to control the metrics output for a load step.

4.4. Composite

Composite scenario steps include child steps. Any composite step should have steps node with an array of the child steps.

4.4.1. Sequential

Executes the child steps sequentially

{
   "type" : "sequential",
   "steps" : [
      {
         "type" : "load",
         ...
      },
      {
         "type" : "command",
         ...
      }
      ...
   ]
}

4.4.2. Parallel

Executes the child steps in parallel

{
   "type" : "parallel",
   "steps" : [
      {
         "type" : "sequential",
         ...
      },
      {
         "type" : "chain",
         ...
      }
      ...
   ]
}

4.4.3. Loop

Executes the child steps multiple times sequentially. The loop conditions are controlled by value and in nodes.

4.4.3.1. By Count

Only value node is present and set to an integer number > 0.

{
    "type" : "for",
    "value" : 10,
    "steps" : [
        {
            "type" : "load"
        }
    ]
}

4.4.3.2. By Range

  • value is set to a name of the counter.
  • in is set to a range expression ("-," or "<START_INT>-<END_INT>"). 
{
    "type" : "for",
    "value" : "i",
    "in" : "2.71828182846-3.1415926,0.1",
    "steps" : [
        {
            "type" : "command",
            "value" : "echo ${i}"
        }
    ]
}

4.4.3.3. By Sequence

  • value is set to a name of the parameter.
  • in is set to an array of parameter values.
{
   "type" : "for",
   "value" : "concurrency",
   "in" : [
      1, 10, 100, 1000, 10000, 100000
   ],
   "config" : {
      "load" : {
         "limit" : {
            "concurrency" : "${concurrency}"
         }
      }
   },
   "steps" : [
      {
         "type" : "load"
      }
   ]
}

4.4.3.4. Infinite

Nor value neither in nodes are set.

{
   "type" : "for",
   "steps" : [
      {
            "type" : "load"
      }
   ]
}

4.5. Mixed Load

Executes different kinds of the I/O tasks (produced by different load generators) using the shared load controller. Using the same load controller allows to set the shared step limits, step id, etc.

List of the shared options:

  1. item-output-file
  2. load-batch-size
  3. load-rate-limit
  4. load-queue-size
  5. test-step-id
  6. test-step-limit-count
  7. test-step-limit-fail-count
  8. test-step-limit-fail-rate
  9. test-step-limit-size
  10. test-step-limit-time

Example:

{
    "type" : "mixed",
    "config" : [
        {
            // shared options
            "item" : {
                "output" : {
                    "file" : "mixed-load-42-output-items.csv"
                }
            },
            "test" : {
                "step" : {
                    "id" : "mixed-load-42",
                    "limit" : {
                        "count" : 1000000
                    }
                }
            },
            // 1st operation kind config
            ...
        },
        {
            // 2nd operation kind config
            ...
        },
        {
            // 3rd operation kind config
            ...
        }
    ]
}

4.5.1. Weighted Load

Weighted load is the extension of the mixed load. Additional node weights specifies the integer weights for each kind of operations.

{
    "type" : "mixed",
    "weights" : [
        20, 80
    ],
    "config" : [
        // create, weight = 20%
        {
            "item" : {
                "data" : {
                    "size" : "4KB-16KB"
                }
            },
            "test" : {
                "step" : {
                    "limit" : {
                        "time" : "90s"
                    }
                }
            }
        },
        // read, weight = 80%
        {
            "item" : {
                "input" : {
                    "file" : "weighted-load-items2read.csv"
                }
            },
            "load" : {
                "circular" : true,
                "type" : "read"
            }
        }
    ]
}

4.6. Chain Load

Executes a sequence of the load operations for each item independently. This means that, for example to read the created items it's not necessary to wait until the load step for create load type is finished. Each item is read/updated/etc as soon as possible after it has been created. It should be useful to test if the item is readable from one set of the nodes of a distributed storage after it is created somewhere on the other set of the nodes.

List of the shared options:

  1. item-output-file
  2. test-step-id
  3. test-step-limit-time
{
    "type" : "chain",
    "config" : [
        {
            "item" : {
                "output" : {
                    "path" : "/bucket1"
                }
            },
            "storage" : {
                "net" : {
                    "node" : {
                        "addrs" : "10.123.45.67"
                    }
                }
            }
        },
        {
            "load" : {
                "type" : "read"
            },
            "storage" : {
                "net" : {
                    "node" : {
                        "addrs" : "10.123.45.68"
                    }
                }
            }
        }
    ]
}

4.6.1. Delay Between Operations

The item-output-delay option allows to set a minimal time before the next operation is performed.

{
    "type" : "chain",
    "config" : [
        {
            "item" : {
                "output" : {
                    "delay" : "1m",
                    "path" : "/default"
                }
            },
            "storage" : {
                "net" : {
                    "node" : {
                        "addrs" : [
                            "10.123.45.67",
                            "10.123.45.68",
                            "10.123.45.69",
                            "10.123.45.70"
                        ]
                    }
                }
            }
        },
        {
            "load" : {
                "type" : "read"
            },
            "storage" : {
                "net" : {
                    "node" : {
                        "addrs" : [
                            "10.234.56.78",
                            "10.234.56.79",
                            "10.234.56.80",
                            "10.244.56.81"
                        ]
                    }
                }
            }
        }
    ]
}

Note

  • Minimum delay is 1 second, 0 value means no delay.
  • In the distributed mode the system clocks should be synchronized precisely.

5. Scenarios Validation

There are a JSON schema file in the distribution: scenario/scenario-schema.json. An user may automatically validate the scenarios using this schema. This should help to write one's own custom scenario correctly.

About

Documentation

Development

Notes

Archive

Links

Clone this wiki locally