Skip to content
This repository was archived by the owner on Jan 25, 2025. It is now read-only.

Commit

Permalink
Merge pull request #4 from andypern/regions
Browse files Browse the repository at this point in the history
Regions
  • Loading branch information
nranchev authored Feb 3, 2017
2 parents aafa129 + 9be4e6d commit bffad6f
Show file tree
Hide file tree
Showing 2 changed files with 27 additions and 10 deletions.
34 changes: 25 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# S3 Bench
This tool offers the ability to run very basic throughput benchmarking against
an S3-compatible endpoint. It does a series of put operations followed by a
series of get operations and displays the corresponding statistics. The tool
This tool offers the ability to run very basic throughput benchmarking against
an S3-compatible endpoint. It does a series of put operations followed by a
series of get operations and displays the corresponding statistics. The tool
uses the AWS Go SDK.

## Requirements
* Go

## Installation
Run the following command to build the binary.
Run the following command to build the binary.

```
go get github.com/igneous-systems/s3bench
Expand All @@ -20,19 +20,35 @@ The s3bench command is self-describing. In order to see all the available option
just run s3bench -help.

### Example input
The following will run a benchmark from 2 concurrent clients, which in
aggregate will put a total of 10 unique new objects. Each object will be
exactly 1024 bytes. The objects will be placed in a bucket named loadgen.
The S3 endpoint will be ran against http://endpoint1:80 and
The following will run a benchmark from 2 concurrent clients, which in
aggregate will put a total of 10 unique new objects. Each object will be
exactly 1024 bytes. The objects will be placed in a bucket named loadgen.
The S3 endpoint will be ran against http://endpoint1:80 and
http://endpoint2:80. Object name will be prefixed with loadgen.

```
./s3bench -accessKey=KEY -accessSecret=SECRET -bucket=loadgen -endpoint=http://endpoint1:80,http://endpoint2:80 -numClients=2 -numSamples=10 -objectNamePrefix=loadgen -objectSize=1024
```

#### Note on regions & endpoints
By default, the region used will be `igneous-test` , a fictitious region which
is suitable for using with the Igneous Data Service. However, you can elect to
use this tool with Amazon S3, in which case you will need to specify the proper region.

It is also important when using Amazon S3 that you specify the proper endpoint, which
will generally be `http://s3-regionName.amazonaws.com:80`. EG: if the bucket which you are
testing is in Oregon, you would specify:

```
-endpoint http://s3-us-west-2.amazonaws.com:80 -region us-west-2
```

For more information on this, please refer to [AmazonS3 documentation.](https://aws.amazon.com/documentation/s3/)



### Example output
The output will consist of details for every request being made as well as the
The output will consist of details for every request being made as well as the
current average throughput. At the end of the run summaries of the put and get
operations will be displayed.

Expand Down
3 changes: 2 additions & 1 deletion s3bench.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ var bufferBytes []byte

func main() {
endpoint := flag.String("endpoint", "", "S3 endpoint(s) comma separated - http://IP:PORT,http://IP:PORT")
region := flag.String("region", "igneous-test", "AWS region to use, eg: us-west-1|us-east-1, etc")
accessKey := flag.String("accessKey", "", "the S3 access key")
accessSecret := flag.String("accessSecret", "", "the S3 access secret")
bucketName := flag.String("bucket", "bucketname", "the bucket for which to run the test")
Expand Down Expand Up @@ -79,7 +80,7 @@ func main() {
// Start the load clients and run a write test followed by a read test
cfg := &aws.Config{
Credentials: credentials.NewStaticCredentials(*accessKey, *accessSecret, ""),
Region: aws.String("igneous-test"),
Region: aws.String(*region),
S3ForcePathStyle: aws.Bool(true),
}
params.StartClients(cfg)
Expand Down

0 comments on commit bffad6f

Please sign in to comment.