Skip to content

Latest commit

 

History

History
20 lines (14 loc) · 691 Bytes

README.md

File metadata and controls

20 lines (14 loc) · 691 Bytes

Spark-Docker-Example

An example of using Semantive/Spark Docker image with sbt-docker plugin to create and submit a Spark App.

Requirements

  1. Docker 1.10.0+ and docker-compose 1.6.0+
  2. SBT

Running

  1. Create docker image of the sbt-application by executing following command inside sbt-application directory:

sbt docker

  1. Run whole cluster using docker-compose (from the directory containing docker-compose.yml file):

docker-compose up

Docker-compose will run three images:

  • Spark master
  • Spark worker
  • sbt-application (Spark job)