Merge remote-tracking branch 'origin/develop' for release 0.3.0

CHANGELOG.md

@@ -1,7 +1,38 @@
 # Change Log
 
 ## UNRELEASED [x.y.z](https://github.com/davidalger/warden/tree/x.y.z) (yyyy-mm-dd)
-[All Commits](https://github.com/davidalger/warden/compare/0.2.4..develop)
+[All Commits](https://github.com/davidalger/warden/compare/0.3.0..develop)
+
+## Version [0.3.0](https://github.com/davidalger/warden/tree/0.3.0) (2020-03-06)
+[All Commits](https://github.com/davidalger/warden/compare/0.2.4..0.3.0)
+
+**Upgrade Notes:**
+
+The fix for issue [#65](https://github.com/davidalger/warden/issues/65) required removing the `warden` network from each environment's services (see commit [36cb0174](https://github.com/davidalger/warden/commit/36cb0174399a40c7f3eb4c39ae70d33afd39c4a3)) and as a result environments referencing the `warden` network in per-project `.warden/*.yml` configuration files may need to be updated for compatibility with Warden 0.3.0.
+
+Should you see an error like the following when running `warden env ...` then this applies to you:
+
+```
+ERROR: Service "nginx" uses an undefined network "warden"
+```
+
+To resolve this issue, simply remove the following from each service defined in `.warden/*.yml` files on the project similar to what was done in commit [36cb0174](https://github.com/davidalger/warden/commit/36cb0174399a40c7f3eb4c39ae70d33afd39c4a3) on the base environment definitions:
+
+```
+networks:
+  - warden
+  - default
+```
+
+**Bug Fixes:**
+
+* Updated to no longer connect environment containers to `warden` network and instead peer `traefik` and `tunnel` containers with each project when it is started (issue [#65](https://github.com/davidalger/warden/issues/65))
+
+**Enhancements:**
+
+* Added automatic start of Mutagen sync on `env up` and `env start` when sync is not connected (issue [#90](https://github.com/davidalger/warden/issues/90))
+* Added automatic stop of Mutagen sync on `env down` and `env stop` (issue [#90](https://github.com/davidalger/warden/issues/90))
+* Updated routing rules so Traefik will now by default route both example.com and *.example.com to application ([#111](https://github.com/davidalger/warden/pull/111) by [davidalger](https://github.com/davidalger))
 
 ## Version [0.2.4](https://github.com/davidalger/warden/tree/0.2.4) (2020-02-29)
 [All Commits](https://github.com/davidalger/warden/compare/0.2.3..0.2.4)

README.md

@@ -7,6 +7,7 @@ Warden is a CLI utility for orchestrating Docker based developer environments, a
 * [Warden Website](https://warden.dev/)
 * [Warden Documentation](https://docs.warden.dev/)
 * [Installing Warden](https://docs.warden.dev/installing.html)
+* [Environment Types](https://docs.warden.dev/environments/types.html)
 * [Initializing An Environment](https://docs.warden.dev/environments/initializing.html)
 * [Docker Images](https://docs.warden.dev/images.html)
 

commands/debug.cmd

@@ -16,7 +16,8 @@ if [[ ${WARDEN_ENV_DEBUG_HOST} == "" ]]; then
         WARDEN_ENV_DEBUG_HOST=host.docker.internal
     else
         WARDEN_ENV_DEBUG_HOST=$(
-            docker container inspect traefik --format '{{.NetworkSettings.Networks.warden.Gateway}}'
+            docker container inspect $(warden env ps -q php-debug) \
+                --format '{{range .NetworkSettings.Networks}}{{println .Gateway}}{{end}}' | head -n1
         )
     fi
 fi

commands/env-init.cmd

@@ -20,6 +20,8 @@ WARDEN_ENV_TYPE="${WARDEN_PARAMS[1]:-}"
 cat > "${WARDEN_ENV_PATH}/.env" <<EOF
 WARDEN_ENV_NAME=${WARDEN_ENV_NAME}
 WARDEN_ENV_TYPE=${WARDEN_ENV_TYPE}
+WARDEN_WEB_ROOT=/
+
 TRAEFIK_DOMAIN=${WARDEN_ENV_NAME}.test
 TRAEFIK_SUBDOMAIN=app
 EOF

commands/env.cmd

@@ -1,7 +1,9 @@
 #!/usr/bin/env bash
 [[ ! ${WARDEN_COMMAND} ]] && >&2 echo -e "\033[31mThis script is not intended to be run directly!" && exit 1
 
+source "${WARDEN_DIR}/utils/core.sh"
 source "${WARDEN_DIR}/utils/env.sh"
+
 WARDEN_ENV_PATH="$(locateEnvPath)" || exit $?
 loadEnvConfig "${WARDEN_ENV_PATH}" || exit $?
 
@@ -64,11 +66,47 @@ else
     export WARDEN_SELENIUM_DEBUG=
 fi
 
-## lookup internal (warden docker network) IP address of traefik container (do not fail if traefik is stopped)
+## disconnect peered service containers from environment network
+if [[ "${WARDEN_PARAMS[0]}" == "down" ]]; then
+    disconnectPeeredServices "${WARDEN_ENV_NAME}_default"
+fi
+
+## connect peered service containers to environment network
+if [[ "${WARDEN_PARAMS[0]}" == "up" ]]; then
+    ## create environment network for attachments if it does not already exist
+    if [[ $(docker network ls -f "name=${WARDEN_ENV_NAME}_default" -q) == "" ]]; then
+        docker-compose \
+            --project-directory "${WARDEN_ENV_PATH}" -p "${WARDEN_ENV_NAME}" \
+            "${DOCKER_COMPOSE_ARGS[@]}" up --no-start
+    fi
+
+    ## connect globally peered services to the environment network
+    connectPeeredServices "${WARDEN_ENV_NAME}_default"
+fi
+
+## lookup address of traefik container on environment network
 export TRAEFIK_ADDRESS="$(docker container inspect traefik \
-    --format '{{.NetworkSettings.Networks.warden.IPAddress}}' 2>/dev/null || true)"
+    --format "{{if .NetworkSettings.Networks.${WARDEN_ENV_NAME}_default}} \
+        {{.NetworkSettings.Networks.${WARDEN_ENV_NAME}_default.IPAddress}} \
+    {{end}}" 2>/dev/null || true \
+)"
 
 ## anything not caught above is simply passed through to docker-compose to orchestrate
 docker-compose \
     --project-directory "${WARDEN_ENV_PATH}" -p "${WARDEN_ENV_NAME}" \
     "${DOCKER_COMPOSE_ARGS[@]}" "${WARDEN_PARAMS[@]}" "$@"
+
+## start mutagen sync if needed
+if ([[ "${WARDEN_PARAMS[0]}" == "up" ]] || [[ "${WARDEN_PARAMS[0]}" == "start" ]]) \
+    && [[ $OSTYPE =~ ^darwin ]] \
+    && [[ $(warden sync list | grep -i 'Connection state: Connected' | wc -l | awk '{print $1}') != "2" ]]
+then
+    warden sync start
+fi
+
+## stop mutagen sync if needed
+if ([[ "${WARDEN_PARAMS[0]}" == "down" ]] || [[ "${WARDEN_PARAMS[0]}" == "stop" ]]) \
+    && [[ $OSTYPE =~ ^darwin ]]
+then
+    warden sync stop
+fi

commands/up.cmd

@@ -1,6 +1,7 @@
 #!/usr/bin/env bash
 [[ ! ${WARDEN_COMMAND} ]] && >&2 echo -e "\033[31mThis script is not intended to be run directly!" && exit 1
 
+source "${WARDEN_DIR}/utils/core.sh"
 source "${WARDEN_DIR}/utils/install.sh"
 assertWardenInstall
 
@@ -39,3 +40,8 @@ done
 
 pushd "${WARDEN_HOME_DIR}" >/dev/null
 docker-compose -p warden -f "${WARDEN_DIR}/docker/docker-compose.yml" up -d "${WARDEN_PARAMS[@]}" "$@"
+
+## connect peered service containers to environment networks
+for network in $(docker network ls -f label=dev.warden.environment.name --format {{.Name}}); do
+  connectPeeredServices "${network}"
+done

commands/usage.help

@@ -24,6 +24,7 @@ Warden version $(cat ${WARDEN_DIR}/version)
   down              Stop and remove containers, networks, and services
   env-init          Configure environment by adding '.env' file to the current working directory
   env               Controls an environment from any point within the root project directory
+  db                Interacts with the db service on an environment (see 'warden db -h' for details)
   install           Initializes or updates warden configuration on host machine
   restart           Restarts warden managed containers
   shell             Launches into a shell within the current project environment

docs/environments/initializing.md

@@ -2,6 +2,10 @@
 
 The below example demonstrates the from-scratch setup of the Magento 2 application for local development. A similar process can easily be used to configure an environment of any other type. This assumes that Warden has been previously started via `warden up` as part of the installation procedure.
 
+``` note::
+    In addition to the below manual process, there is a `Github Template available for Magento 2 <https://github.com/davidalger/warden-env-magento2>`_ allowing for quick setup of new Magento projects. To use this, click the green "Use this template" button to create your own repository based on the template repository, run the init script and update the README with any project specific information.
+```
+
 1. Create a new directory on your host machine at the location of your choice and then jump into the new directory to get started:
 
        mkdir -p ~/Sites/exampleproject
@@ -15,6 +19,8 @@ The below example demonstrates the from-scratch setup of the Magento 2 applicati
 
        WARDEN_ENV_NAME=exampleproject
        WARDEN_ENV_TYPE=magento2
+       WARDEN_WEB_ROOT=/
+
        TRAEFIK_DOMAIN=exampleproject.test
        TRAEFIK_SUBDOMAIN=app
 
@@ -29,14 +35,15 @@ The below example demonstrates the from-scratch setup of the Magento 2 applicati
        VARNISH_VERSION=6.0
 
        WARDEN_SELENIUM=0
+       WARDEN_SELENIUM_DEBUG=0
        WARDEN_BLACKFIRE=0
        WARDEN_SPLIT_SALES=0
        WARDEN_SPLIT_CHECKOUT=0
 
-       BLACKFIRE_CLIENT_ID="<client_id>"
-       BLACKFIRE_CLIENT_TOKEN="<client_token>"
-       BLACKFIRE_SERVER_ID="<server_id>"
-       BLACKFIRE_SERVER_TOKEN="<server_token>"
+       BLACKFIRE_CLIENT_ID=
+       BLACKFIRE_CLIENT_TOKEN=
+       BLACKFIRE_SERVER_ID=
+       BLACKFIRE_SERVER_TOKEN=
 
 3. Sign an SSL certificate for use with the project (the input here should match the value of `TRAEFIK_DOMAIN` in the above `.env` example file):
 
@@ -45,20 +52,29 @@ The below example demonstrates the from-scratch setup of the Magento 2 applicati
 4. Next you'll want to start the project environment:
 
        warden env up -d
-       warden sync start   ## Omit this if running on a Linux host (or if not used by env type)
 
-   If you encounter an error about `Mounts denied…`, follow the instructions in the error message and run `warden env up -d` again.
+   ``` warning::
+       If you encounter an error about ``Mounts denied``, follow the instructions in the error message and run ``warden env up -d`` again.
+   ```
+
+   ``` note::
+       On macOS when using Warden versions prior to 0.3.0, you will need to start the Mutagen sync session manually be running ``warden sync start`` whenever starting the environment. In Warden 0.3.0 and later the sync session is started and stopped automatically when running commands such as ``up``, ``start``, ``down``, and ``stop``.
+   ```
 
 5. Drop into a shell within the project environment. Commands following this step in the setup procedure will be run from within the `php-fpm` docker container this launches you into:
 
        warden shell
 
-6. If you already have Magento Marketplace credentials configured, you may skip this step (`~/.composer/` on the host is mounted into the container to share composer cache between projects, and has the effect of persisting the `auth.json` on the host machine as well):
-
-    Note: To locate your authentication keys for Magento 2 repository, reference [this page on DevDocs](https://devdocs.magento.com/guides/v2.3/install-gde/prereq/connect-auth.html).
+6. Configure global Magento Marketplace credentials
 
        composer global config http-basic.repo.magento.com <username> <password>
 
+    ``` note::
+        To locate your authentication keys for Magento 2 repository, `reference DevDocs <https://devdocs.magento.com/guides/v2.3/install-gde/prereq/connect-auth.html>`_.
+
+        If you have previously configured global credentials, you may skip this step, as ``~/.composer/`` is mounted into the container from the host machine in order to share composer cache between projects, and also shares the global ``auth.json`` from the host machine.
+    ```
+
 7. Initialize project source files using composer create-project and then move them into place:
 
        composer create-project --repository-url=https://repo.magento.com/ \
@@ -141,10 +157,12 @@ The below example demonstrates the from-scratch setup of the Magento 2 applicati
 
 9. Launch the application in your browser:
 
-    * https://app.exampleproject.test/
-    * https://app.exampleproject.test/backend/
-    * https://mailhog.exampleproject.test/
-    * https://rabbitmq.exampleproject.test/
-    * https://elasticsearch.exampleproject.test/
+    * [https://app.exampleproject.test/](https://app.exampleproject.test/)
+    * [https://app.exampleproject.test/backend/](https://app.exampleproject.test/backend/)
+    * [https://mailhog.exampleproject.test/](https://mailhog.exampleproject.test/)
+    * [https://rabbitmq.exampleproject.test/](https://rabbitmq.exampleproject.test/)
+    * [https://elasticsearch.exampleproject.test/](https://elasticsearch.exampleproject.test/)
 
-Note: To completely destroy the `exampleproject` environment we just created, run `warden env down -v` to tear down the project's Docker containers, volumes, etc, then `warden sync stop` as needed to terminate the Mutagen session.
+``` note::
+    To completely destroy the ``exampleproject`` environment we just created, run ``warden env down -v`` to tear down the project's Docker containers, volumes, etc.
+```

docs/environments/types.md

@@ -1,12 +1,59 @@
 ### Environment Types
 
-Warden currently supports three environment types. These types are passed to `env-init` when configuring a project for local development for the first time. This list of environment types can also be seen by running `warden env-init --help` on your command line.
+Warden currently supports three environment types. These types are passed to `env-init` when configuring a project for local development for the first time. This list of environment types can also be seen by running `warden env-init --help` on your command line. The `docker-compose` configuration used to assemble each environment type can be found in the [environments directory](https://github.com/davidalger/warden/tree/master/environments) on Github.
 
-* [`local`](https://github.com/davidalger/warden/blob/master/environments/local.base.yml) This environment type does nothing more than declare the `docker-compose` version and declare the external `warden` network which Traefik uses to proxy requests into the project's containers. When this is used, a `.warden/warden-env.yml` may be placed in the root directory of the project workspace to define the desired containers, volumes, etc needed for the project. An example of a `local` environment type being used can be found in the [m2demo project](https://github.com/davidalger/m2demo).
-* `magento2` Provides the necessary containerized services for running Magento 2 in a local development context including Nginx, Varnish, php-fpm (PHP 7.0+), MariaDB, Elasticsearch, RabbitMQ and Redis. In order to achieve a well performing experience on macOS, source files are synced into the container using a Mutagen sync session (`pub/media` remains mounted using a delegated mount).
-* `magento1` Supports development of Magento 1 projects, launching containers for Nginx, php-fpm (PHP 5.5, 5.6 or 7.0+), MariaDB and Redis. Files mounted using delegated mount on macOS and natively on Linux.
-* `laravel` Supports development of Laravel projects, launching containers for Nginx, php-fpm, MariaDB and Redis. Files mounted using delegated mount on macOS and natively on Linux.
+#### Local
 
-All environment types (other than `local`) come pre-configured with a `mailhog` container, with `fpm` services configured to use `mhsendmail` to ensure outbound email does not inadvertently send out, and allows for simpler testing of email functionality on projects (you can use [Traefik](https://traefik.warden.test/) to find the `mailhog` url for each project). There are also two `fpm` containers, `php-fpm` and `php-debug` (more on this later) to provide Xdebug support enabled via nothing more than setting the `XDEBUG_SESSION` cookie in your browser to direct the request to the `php-debug` container.
+The `local` environment type does nothing more than declare the `docker-compose` version and label the project network so Warden will recognize it as belonging to an environment orchestrated by Warden.
 
-For full details and a complete list of variables which may be used to adjusting things such as PHP or MySQL versions (by setting them in the project's `.env` file), and to see the `docker-compose` definitions used to assemble each environment type, look at the contents of the [environments directory](https://github.com/davidalger/warden/tree/master/environments) in this repository. Each environment has a `base` configuration YAML file, and optionally a `darwin` and `linux-gnu` file which add to the `base` definitions anything specific to a given host architecture (this is, for example, how the `magento2` environment type works seamlessly on macOS with Mutagen sync sessions while using native filesystem mounts on Linux hosts). This directory also houses the configuration used for starting Mutagen sync sessions on a project via the `warden sync start` command.
+When this type is used, a `.warden/warden-env.yml` may be placed in the root directory of the project workspace to define the desired containers, volumes, etc needed for the project. An example of a `local` environment type being used can be found in the [m2demo project](https://github.com/davidalger/m2demo).
+
+Similar to the other environment type's base definitions, Warden supports a `warden-env.darwin.yml` and `warden-env.linux-gnu.yml`
+
+#### Magento 2
+
+The `magento2` environment type provides necessary containerized services for running Magento 2 in a local development context including:
+
+* Nginx
+* Varnish
+* PHP-FPM (7.0+)
+* MariaDB
+* Elasticsearch
+* RabbitMQ
+* Redis
+
+In order to achieve a well performing experience on macOS, files in the webroot are synced into the container using a Mutagen sync session with the exception of `pub/media` which remains mounted using a delegated mount.
+
+#### Magento 1
+
+The `magento1` environment type supports development of Magento 1 projects, launching containers including:
+
+* Nginx
+* PHP-FPM (5.5, 5.6 or 7.0+)
+* MariaDB
+* Redis
+
+Files are currently mounted using a delegated mount on macOS and natively on Linux.
+
+#### Laravel
+
+The `laravel` environment type supports development of Laravel projects, launching containers including:
+
+* Nginx
+* PHP-FPM
+* MariaDB
+* Redis
+
+Files are currently mounted using a delegated mount on macOS and natively on Linux.
+
+#### Commonalities
+
+In addition to the above, each environment type (with the exception of the `local` type) come with a pre-configured Mailhog service, with PHP setup to use `mhsendmail` to ensure outbound email does not inadvertently leave your network and to support simpler testing of email functionality. You can use [Traefik](https://traefik.warden.test/) to find the Mailhog URL for each project.
+
+Where PHP is specified in the above list, there should be two `fpm` containers, `php-fpm` and `php-debug` in order to provide Xdebug support. Use of Xdebug is enabled by setting the `XDEBUG_SESSION` cookie in your browser to direct the request to the `php-debug` container. Shell sessions opened in the debug container via `warden debug` will also connect PHP process for commands on the CLI to Xdebug.
+
+The configuration of each environment leverages a `base` configuration YAML file, and optionally a `darwin` and `linux-gnu` file to add to `base` configuration anything which may be specific to a given host architecture (this is, for example, how the `magento2` environment type works seamlessly on macOS with Mutagen sync sessions while using native filesystem mounts on Linux hosts).
+
+### Environment Templates
+
+There is a [Github Template available for Magento 2](https://github.com/davidalger/warden-env-magento2) allowing for quick setup of new Magento projects. To use this, click the green "Use this template" button to create your own repository based on the template repository, run the init script and update the README with any project specific information.