add deployment requirements to readme

* add necessary information for deploy preparations to readme
* add service dependency variables in compose file and dependency wait script
* fix wso2is image to not use internal addresses for resource download

github: #2

0-wait-dependencies.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env sh
+
+set -e
+
+if [ -n "$SERVICE_DEPENDENCIES" ]; then
+
+	for dep in $SERVICE_DEPENDENCIES; do
+
+		host="$(echo $dep | cut -d':' -f1)"
+		port="$(echo $dep | cut -d':' -f2)"
+
+		wait-for-service.sh -h $host -p $port
+	done
+fi

README.md

@@ -27,18 +27,42 @@ To build all artefacts and docker images run:
 ./build.sh
 ```
 
-# Deploying
+# Deployment
 
-Deployment is achieve using a docker compose file. For it to work a docker swarm needs to be initialized and all required nodes to be joined in the swarm.
+Deployment is achieved using a docker compose file. For it to work a docker swarm needs to be initialized and all required nodes to be joined in the swarm.
 Note that images need to be in a docker registry (so that docker can pull them) or to be present on all machines.
 
-There is also a proxy service (nginx) that is used for ssl termination and to proxy most services through a single address. It requires the `NGINX_SERVER_NAME` to be set to the public hostname of the machine running it. You can either `export` it beforehand or pass it in when deploying.
-
 For ssl to work the certificate and public key must be placed inside `/etc/sep/keystores` and be named nginx.crt and nginx.key.
 
 Also for graphdb to work you need to place your license file in `/etc/sep/licenses/graphdb.license`.
 
+## Service placement
+The compose file uses docker's placement constraints. These are labels added to a service to constrain where it could be deployed.
+Before deploying you need to set the appropriate node labels to the nodes in your swarm.
+
+To do that - on the master node execute `docker node update --label-add <label> <node id | hostname>`. Where `label` is the label name + value that needs to be set e.g. `com.sirma.sep.solr.audit=yes` and `node id` is the given by docker to the target node (you can list all nodes using `docker node ls` command).
+
+## Environment variables
+Some services require variables that specify external to the swarm host names and address - update the following services by adding the specified env vars before deploying the compose file.
+
+1. proxy
+  1.1 `NGINX_SERVER_NAME` - this is the host name of the machine service all http requests e.g. `example.com`
+  1.2. `NGINX_SERVER_NAME_INTERNAL` - some http services are proxied through port 8080 which should not be exposed to the world, usually this is paired with an internal address e.g. `internal.example.com`
+  1.3. `PROXY_INTERNAL_SERVICE_ADDR` - the full base url for acccessing internal services e.g. `http://internal.example.com:8080`
+  1.4. `PROXY_EXTERNAL_SERVICE_ADDR` - the base url for accessing public services e.g. https://example.com
+2. wildfly
+  2.1. `IMAGE_SERVER_BASE_URL` - base address at where the iiif server serves images - usually this is the same as `PROXY_EXTERNAL_SERVICE_ADDR`
+3. iiif
+  3.1. `BASE_URL` - bese url for iiif images - should be in the form <IMAGE_SERVER_BASE_URL>/iiif/fcgi-bin/iipsrv.fcgi?IIIF=
+
+## Service dependencies
+Some services require other to be up and running before they can start properly e.g. `wildfly` depends on the `db` service. This is controlled by an environment variable called `SERVICE_DEPENDENCIES`. It's value is a space separated list of `service_name:port` e.g. `SERVICE_DEPENDENCIES=db:5432 keycloak:8080`.
+
+Also each service that has dependencies mounts the [0-wait-dependencies.sh](./0-wait-dependencies.sh) script, which actually parses the variable and checks it the service is up.
+This script must be placed under `/etc/sep/stacks/bin` directory on all nodes deploying services with dependencies.
+
+## Deploying the stack
 To deploy the compose file (stack) run:
 ```bash
-NGINX_SERVER_NAME="example.com" docker stack deploy --compose-file ./docker-stack.yml sep
+docker stack deploy --compose-file ./docker-stack.yml sep
 ```

docker-stack.yml

@@ -29,8 +29,10 @@ services:
           constraints:
               - node.labels.com.sirma.sep.solr.audit==yes
     environment:
+      - SERVICE_DEPENDENCIES=db:5432
       - PROXY_SERVICE_DEF_MAIN=internal:solr-audit:/solr/audit/:http://solr-audit:8983
     volumes:
+      - /etc/sep/stacks/bin/0-wait-dependencies.sh:/docker-init.d/0-wait-dependencies.sh:ro
       - solr_audit_data:/var/lib/solr
       - solr_audit_logs:/opt/solr/server/logs
 
@@ -74,6 +76,7 @@ services:
         constraints:
           - node.labels.com.sirma.sep.idp==yes
     environment:
+      - SERVICE_DEPENDENCIES=db:5432 ldap:10389
       - HOST_NAME=idp
       - LDAP_USE_EMBEDDED=false
       - LDAP_HOST=ldap
@@ -85,15 +88,16 @@ services:
       - idp_data:/opt/wso2is/repository/data
       - idp_logs:/opt/wso2is/repository/logs
       - /etc/sep/certs:/opt/wso2is/repository/resources/keystore
+      - /etc/sep/stacks/bin/0-wait-dependencies.sh:/docker-init.d/0-wait-dependencies.sh:ro
 
   keycloak:
     image: docker-reg.sirmaplatform.com/cs-keycloak:2.28.0-SNAPSHOT
     deploy:
       placement:
         constraints:
           - node.labels.com.sirma.sep.idp==yes
-    env_file: keycloak.env
     environment:
+      SERVICE_DEPENDENCIES: db:5432
       PROXY_SERVICE_DEF_MAIN: secure:keycloak:/auth:http://keycloak:8080
       MASTER_DISPLAY_NAME: "Conservation Space"
       MASTER_DISPLAY_NAME_HTML: "Conservation Space"
@@ -102,6 +106,7 @@ services:
     volumes:
       - keycloak_data:/opt/keycloak/standalone/data
       - keycloak_logs:/opt/keycloak/standalone/log
+      - /etc/sep/stacks/bin/0-wait-dependencies.sh:/docker-init.d/0-wait-dependencies.sh:ro
 
   ldap:
     image: docker-reg.sirmaplatform.com/sep-openldap:2.25.0
@@ -120,6 +125,7 @@ services:
         constraints:
           - node.labels.com.sirma.sep.dms==yes
     environment:
+      - SERVICE_DEPENDENCIES=db:5432 idp:9443
       - PROXY_SERVICE_DEF_MAIN=internal:alfresco:/alfresco:http://alfresco:8080
       - LDAP_HOST=ldap
       - LDAP_PORT=10389
@@ -130,6 +136,7 @@ services:
       - alfresco_logs:/opt/tomcat/logs
       - alfresco_data:/var/lib/alfresco
       - alfresco_solr_data:/var/lib/alfresco-solr
+      - /etc/sep/stacks/bin/0-wait-dependencies.sh:/docker-init.d/0-wait-dependencies.sh:ro
 
   graphdb:
     image: docker-reg.sirmaplatform.com/graphdb:8.3.1-u5
@@ -183,7 +190,6 @@ services:
       placement:
         constraints:
           - node.labels.com.sirma.sep.iiif==yes
-    env_file: iiif.env
     environment:
       - PROXY_SERVICE_DEF_MAIN=secure:iiif:/iiif:http://iiif
     volumes:
@@ -224,8 +230,9 @@ services:
       - /etc/sep/keystores:/opt/wildfly/standalone/configuration/sep/certs
       - /etc/sep/assets:/etc/sep/assets
       - wildfly_content:/opt/wildfly/standalone/data
-    env_file: wildfly.env
+      - /etc/sep/stacks/bin/0-wait-dependencies.sh:/docker-init.d/0-wait-dependencies.sh:ro
     environment:
+      - SERVICE_DEPENDENCIES=alfresco:8080 graphdb:8080 keycloak:8080
       - DB_NAME_AUDIT=sep
       - IDP_HOST=idp
       - IDP_PORT=9443
@@ -242,13 +249,16 @@ services:
         constraints:
           - node.labels.com.sirma.sep.web==yes
     environment:
+      - SERVICE_DEPENDENCIES=wildfly:8080
       - BACKEND_PROTOCOL=http
       - BACKEND_ADDR=wildfly:8080
       - BACKEND_CONTEXT_PATH=emf
       - PROXY_SERVICE_DEF_MAIN=secure:ui:/:http://ui
+    volumes:
+      - /etc/sep/stacks/bin/0-wait-dependencies.sh:/docker-init.d/0-wait-dependencies.sh:ro
+
   proxy:
     image: docker-reg.sirmaplatform.com/http-proxy:2.28.0-SNAPSHOT
-    env_file: proxy.env
     deploy:
       placement:
         constraints:

docker/sep-wso2is/Dockerfile

@@ -32,15 +32,17 @@ ENV ADMIN_USERNAME="admin" \
 	MAX_PASS_LENGTH=30 \
 	VOLUME_WSOIDP_KEYSTORE="/opt/wso2is/repository/resources/keystore"
 
+COPY patch0009/ /opt/wso2is/repository/components/patches/patch0009/
+
 RUN \
 	apk --no-cache add unzip && \
 	mkdir /opt/wso2is/repository/data-dir-init && \
-	wget -O - http://fs.ruse.ittbg.com/Programs/wso2/WSO2-IS-5.0.0-SP01.tar.gz | tar zx -C /opt && \
+	curl -sfL --referer "http://connect.wso2.com/wso2/getform/reg/new_product_download" https://product-dist.wso2.com/products/identity-server/5.0.0/service-pack/WSO2-IS-5.0.0-SP01.zip > /opt/sp01.zip && \
+	(cd /opt; unzip sp01.zip) && \
 	ln -s /opt/wso2is /opt/wso2is-5.0.0 && \
 	chmod +x /opt/WSO2-IS-5.0.0-SP01/install_sp.sh && \
 	echo "y" > /opt/yes_install_wso_service_pack && \
 	(cd /opt/WSO2-IS-5.0.0-SP01; sh /opt/WSO2-IS-5.0.0-SP01/install_sp.sh < /opt/yes_install_wso_service_pack) && \
-	wget -O - http://fs.ruse.ittbg.com/Programs/wso2/patch0009.tar.gz | tar xz -C repository/components/patches && \
 	mv /opt/wso2is/repository/data/is-default-schema.zip /opt/wso2is/repository/data-dir-init/is-default-schema.zip && \
 	mkdir /opt/wso2is/repository/deployment/server/webapps/authenticationendpoint/ && \
 	unzip /opt/wso2is/repository/deployment/server/webapps/authenticationendpoint.war -d /opt/wso2is/repository/deployment/server/webapps/authenticationendpoint/ && \