An experiment at running benchmarks on multiple EC2 instances #151
Conversation
| ## The user and IP address or hostname of the server | ||
| HOSTNAME=$2 | ||
|
|
||
| USER="ubuntu" |
There was a problem hiding this comment.
Minor nit: This can vary depending on AMI. Suggest ${3:-ubuntu}.
| fi | ||
|
|
||
| ## The path of the private key for ssh authentication | ||
| PRIVATE_KEY=$1 |
There was a problem hiding this comment.
I know this is only the first position argument, but is it reasonable to default to ~/.ssh/id_rsa here?
There was a problem hiding this comment.
Sounds reasonable, I can make that change
|
|
||
| ## TODO: This only works if the user's name is ubuntu (and also is tied to the value in experiment-script.sh) | ||
| ## Is there any better way to refactor this so that this is not an issue? | ||
| COMPLETION_FILE="/home/ubuntu/pash/done.txt" |
| aws ec2 start-instances --instance-ids "$instance_id" | ||
| local ip=$(wait-for-instance-ip "$instance_id"); | ||
| echo "$ip" | ||
| trap "aws ec2 stop-instances --instance-ids $instance_id" EXIT | ||
| if [ "$stop_flag" -eq 1 ]; then | ||
| trap "echo Stopping instance: $instance_id; aws ec2 stop-instances --instance-ids $instance_id" EXIT |
There was a problem hiding this comment.
I wrote this script so that it can either be run directly, or sourced. If this script is sourced, then this trap will trigger only when the shell that sourced it exits. You can support both modes by changing the function to run in a subshell (Turn the {} wrapping the function body into ())
That might be better anyway given the set -e at the top.
There was a problem hiding this comment.
That is a good idea, I will do that too!
This PR includes scripts that allow running experiments on multiple ec2-instances.
The infrastructure is in
evaluation/multi-instance-experimentand assuming that the instances referred to inmain.shexist, we just have to callmain.sh.It now works, and can execute the eurosys one-liners experiment on two EC2 instances. The infrastructure spawns background jobs on the server and then uses a polling process to check when the experiment is done. The reason is that experiments might take a long time and if the connection drops, the experiment will stop.
What is missing:
Especially for the second point, I would need your @zyrolasting and @nvasilakis and that is why I added you as reviewers. Any thoughts and suggestions on how to refactor this properly would be great :)