A Packer.io build for Grinder. Packer.io is a tool for creating identical machine images for multiple platforms from a single source configuration. It supports build products for Amazon EC2, Digital Ocean, Docker, VirtualBox, VMWare, and others. Grinder is load testing framework that makes it easy to run a distributed test using many load injector machines.
For this project, however, only the AWS EC2 builder is used in the Packer.io configuration. Support for the other Packer.io builders is not planned.
This project uses Packer.io and a couple of Bash scripts to build and start up Grinder in the AWS cloud. The build.sh script creates AMIs for the Grinder console and agents. The included, but optional, start.sh script starts the Grinder cloud and prepares it to receive your tests. Likewise, the stop.sh script brings down your AWS Grinder cloud.
Make sure to read the warnings about the start and stop scripts in the Deployment section below.
You should install these prerequisites before running any of the packer-aws-grinder scripts.
- Packer.io
- AWS CLI
- Basic System tools (e.g., echo, openssl, awk, tail, tr, tee, bash)
- strip-json-comments (Optional: Only needed if you plan to edit the Packer.io script)
Before you run the build script, you'll need to configure a few important variables. The packer build uses three configuration files (vars.json, agent-vars.json, and console-vars.json).
The vars/vars.json file contains the generic configurations (like AWS access key, AWS security group, AWS region, etc.) The vars/console-vars.json and vars/agent-vars.json files allow you to configure the Grinder console and agents differently (for instance, using different source AMIs, AWS instance types, etc.)
To get you started, the project has an vars/example-vars.json file which can be copied to vars/vars.json and then edited. The same is true for vars/example-agent-vars.json (to be copied to vars/agent-vars.json) and vars/example-console-vars.json (to be copied to vars/console-vars.json). The build script will then inject these variables into the build, when appropriate.
Note: When running the build script, any variable in the project's vars files that ends with _password will get an automatically generated value. To have the build script regenerate the _password values with a new build, delete the .passwords file before re-running the build script. You can also just skip this automated step by providing your own passwords in those files.
General variables are those that are consistent across the Grinder console and agent machines.
- aws_access_key
- A valid AWS_ACCESS_KEY that will be used to interact with Amazon Web Services (AWS).
- aws_secret_key
- The AWS_SECRET_KEY that corresponds to the supplied AWS_ACCESS_KEY.
- aws_security_group_id
- A pre-configured AWS Security Group ID (not Name) that will allow SSH access to the EC2 instance. It should also allow access to ports 6372 and 6373. Access can be restricted to the machine from which you will submit the tests. Access for the Grinder agents is dynamically granted (and revoked) as they are spun up (and shutdown).
- aws_region
- The AWS region to use. For instance: us-east-1 or us-west-2.
- packer_build_name
- A name that will distinguish your build products from someone else's. It can be a simple string like `Fedora` or `UCLA`.
The agent-vars.json and console-vars.json files have the same set of variables, but are split into two files because one might want different types of AWS instances for the console and agents.
- aws_instance_type
- The AWS instance type to use. For instance: t2.medium or m3.medium.
- aws_virtualization_type
- The AWS virtualization type to use. For instance: hvm or pv.
- aws_source_ami
- The source AMI to use as a base. Note that the source AMI, virtualization type, and instance type must be compatible. If you use a different AMI from the default, make sure it's an Ubuntu image (as that's what the Packer.io build expects).
To run the packer-aws-grinder build (creating the Grinder AMIs and Console instance), type the following (from within the project directory):
./build.sh
Note: To have the build script use the packer-aws-grinder.json file, you'll need to have strip-json-comments installed. If you don't have that installed, the build script will use the pre-generated aws-grinder.json file. Any changes to the build script meant to persist between builds should be made to the packer-aws-grinder.json file.
To deploy the Grinder machines to the AWS cloud, type:
./start.sh <NUMBER_OF_AGENTS>
This will bring up one console and the specified number of agents. So, for instance, one would type ./start.sh 5 to start one console and five agents. The agents will be automatically connected to the console. Grinder tests can then be sent to the console and they will be distributed to the attached agents.
To bring down the Grinder cloud, type:
./stop.sh
This will stop the console and agents, keeping both around to be run later. If you want to actually get rid of the Grinder Agent instances, tell the stop.sh script that you want to terminate them by typing:
./stop.sh terminate
This will terminate the agent instances, but not the console instance. To do a complete clean, removing the agent instances, the console instance, and the generated EC2 AMIs and related snapshots, type:
./stop.sh clean
You will, after that, have to re-run the ./build.sh script before running the start.sh and stop.sh scripts again.
Warning: Running Grinder in the AWS cloud costs money. The start and stop scripts are provided as a convenience, but you should confirm that they've actually worked. I'm not responsible for any hours you incur through use of the AWS cloud. If you don't agree to that, don't use the supplied start and stop scripts. As an alternative to the provided scripts, you can bring up the console and agents through AWS' Web interface. The agents will automatically connect to console as long as they're brought up after the console is fully functional.
To deploy the Fedora's fcrepo4 to the AWS cloud, type:
./fedora.sh start
To bring down Fedora in the AWS cloud, type:
./fedora.sh stop
To clean up all the resources related to Fedora (removing the EC2 instance and local file system artifacts), type:
./fedora.sh clean
To change any of the configuration options (e.g., select a different Fedora AMI), edit the vars/fedora-vars.json file.
For more detailed instructions about how to run Fedora in the AWS Grinder cloud, consult the Running Fedora in the AWS Grinder Cloud document