Pravega Release Validation Checklist

Verify Pravega checksum (.tgz)

md5:

$ md5sum pravega.tgz

$ cat pravega.tgz.md5

should match.

sha1:

$ sha1sum pravega.tgz

$ cat pravega.tgz.sha1

should match.

sha256:

$ sha256sum pravega.tgz

$ cat pravega.tgz.sha256

should match.

Verify Pravega checksum (.zip)

md5:

$ md5sum pravega.zip

$ cat pravega.zip.md5

should match.

sha1:

$ sha1sum pravega.zip

$ cat pravega.zip.sha1

should match.

sha256:

$ sha256sum pravega.zip

$ cat pravega.zip.sha256

should match.

Verify Pravega Javadoc checksum (.tgz)

md5:

$ md5sum pravega-javadoc.tgz

$ cat pravega-javadoc.tgz.md5

should match.

sha1:

$ sha1sum pravega-javadoc.tgz

$ cat pravega-javadoc.tgz.sha1

should match.

sha256:

$ sha256sum pravega-javadoc.tgz

$ cat pravega-javadoc.tgz.sha256

should match.

Verify Pravega Javadoc checksum (.zip)

md5:

$ md5sum pravega-javadoc.zip

$ cat pravega-javadoc.zip.md5

should match.

sha1:

$ sha1sum pravega-javadoc.zip

$ cat pravega-javadoc.zip.sha1

should match.

sha256:

$ sha256sum pravega-javadoc.zip

$ cat pravega-javadoc.zip.sha256

should match.

Explore Pravega Client API

Unzip/untar pravega-javadoc.tgz/.zip

Go through the javadocs

Running pravega:

Standalone

These ports should be occupied: zookeeper on 4000 , controller on 9090, rest server on 9091, segmentstore on 6000

pravega.tgz:

$ tar xvf pravega.tgz

$ cd pravega

$ ./bin/pravega-standalone

pravega.zip :

$ unzip pravega.zip

$ cd pravega

$ ./bin/pravega-standalone

From source code(.zip):

$ unzip pravega.zip

$ cd pravega

$ gradle startStandalone

From source code(.tgz):

$ tar xvf pravega.tgz

$ cd pravega

$ gradle startStandalone

From docker images:

$ docker run -it -e HOST_IP=<ip> -p 9090:9090 -p 12345:12345 pravega/pravega:latest standalone

Docker compose:

$ wget https://github.com/pravega/pravega/tree/master/docker/compose/docker-compose.yml

$ HOST_IP=<host-ip-of-your-local-machine> docker-compose up

Controller can be accessed at: ${HOST_IP}:9090

Distributed

Using Docker Swarm:

$ docker stack up --compose-file hdfs.yml pravega // Deploying HDFS

$ docker stack up --compose-file zookeeper.yml pravega // Deploying Zookeeper

$ ZK_URL=zookeeper:2181 HDFS_URL=hdfs:8020 docker stack up --compose-file pravega.yml pravega // Deploying Pravega

Manual installation:

• Set up HDFS(2.7+) cluster
• Install Java8
• Install Zookeeper 3.5.1-alpha+

Create following paths in Zookeeper:

$ bin/zkCli.sh -server $ZK_URL create /pravega

$ bin/zkCli.sh -server $ZK_URL create /pravega/bookkeeper

• Install Bookkeeper 4.4.0+

Change the Bookkeeper config in conf/bk_server.conf as follows:

# Comma separated list of <zp-ip>:<port> for all ZK servers
zkServers=localhost:2181

# Alternatively specify a different path to the storage for /bk
journalDirectory=/bk/journal
ledgerDirectories=/bk/ledgers
indexDirectories=/bk/index

zkLedgersRootPath=/pravega/bookkeeper/ledgers

Format bookie:

$ bin/bookkeeper shell metaformat –nonInteractive

Start Bookie:

$ bin/bookkeeper bookie

Install Controller:

$ untar/unzip pravega.tgz/pravega.zip

$ cd pravega

$ ZK_URL=<zk-ip>:2181 bin/pravega-controller

Install SegmentStore:

Edit the conf/config.properties file to set the zk, hdfs and controller ips.

pravegaservice.zkURL=<zk-ip>:2181
bookkeeper.zkAddress=<zk-ip>:2181
autoScale.controllerUri=tcp://<controller-ip>:9090

# Settings required for HDFS
hdfs.hdfsUrl=<hdfs-ip>:8020

$ bin/pravega-segmentstore

Running in the cloud(AWS)

Prerequisties:

• Terraform needs to be installed
• AWS account

$ cd deployment/aws
$ sudo terraform apply 

Following inputs need to be provided when prompted:

• AWS access key, AWS secret key
• AWS key pair absolute path
• AWS region (Only us-east-1 and us-west-1 are supported as of now)

Customise Pravega Cluster

Change default value of pravega_num in variable.tf

Verify Pravega Client checksum(.tgz)

md5:

$ md5sum pravega-client.tgz

$ cat pravega-client.tgz.md5

should match.

sha1:

$ sha1sum pravega-client.tgz

$ cat pravega-client.tgz.sha1

should match.

sha256:

$ sha256sum pravega-client.tgz

$ cat pravega-client.tgz.sha256

should match.

Verify Pravega Client checksum(.zip)

md5:

$ md5sum pravega-client.zip

$ cat pravega-client.zip.md5

should match.

sha1:

$ sha1sum pravega-client.zip

$ cat pravega-client.zip.sha1

should match.

sha256:

$ sha256sum pravega-client.zip

$ cat pravega-client.zip.sha256

should match.

Use Pravega Client API

Unzip/untar pravega-client.tgz/.zip Try importing client libraries and check if you are able to use some api

• Create a gradle/maven project
• Load client libraries
• Try using Stream Manager / Client Factory.

Create a stream, write and read:

After deploying Pravega:

Run some sample code snippets from pravega-samples to check basic functionality from

https://github.com/pravega/pravega-samples/blob/master/standalone-examples/README.md

By default pravega-samples use the latest version of pravega from maven central repo.

$ cd pravega

$ ./gradlew install

$ mkdir pravega-samples

$ cd standalone-examples

$ git clone https://github.com/pravega/pravega-samples.git

$ ./gradlew installDist

Replace 127.0.0.1 with host ip of your local machine where the controller is running.

Start HelloWorld Writer

$ ./bin/helloWorldWriter [-scope myScope] [-name myStream] [-uri tcp://127.0.0.1:9090] [-routingkey myRK] [-message 'hello world']

Start HelloWorld Reader:

$ ./bin/helloWorldReader [-scope myScope] [-name myStream] [-uri tcp://127.0.0.1:9090]

Start ConsoleWriter:

$ ./bin/consoleWriter [-scope myScope] [-name myStream] [-uri tcp://127.0.0.1:9090]

Start ConsoleReader:

$ ./bin/consoleReader [-scope myScope] [-name myStream] [-uri tcp://127.0.0.1:9090]