Skip to content

Commit

Permalink
Merge pull request #176 from tapis-project/staging
Browse files Browse the repository at this point in the history
Staging
  • Loading branch information
mpackard authored Apr 21, 2023
2 parents fa91ac6 + d0f5463 commit 6e6af01
Show file tree
Hide file tree
Showing 100 changed files with 693 additions and 714 deletions.
51 changes: 39 additions & 12 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,19 @@ Notable changes between versions.

## 1.3.5

Image updates
**Breaking Changes**

Previous versions of Tapis Deployer have placed important vault configs in ~/vault, ~/vault-token. These files are now moved to a configurable directory set by the `tapisdatadir` variable. This should be set to the directory on your deployment machine that contains data for your installation and should

**If you migrating from an existing Tapis deployement** be sure to copy:
- create the `tapisdatadir`/vault directory
- copy ~/vault file to `tapisdatadir`/vault/vault-init file
- copy ~/vault-token file to `tapisdatadir`/vault/vault-token file


Other changes:

Image updates for:

- Systems: 1.3.0 to 1.3.1 (tapis/systems)
- Apps: 1.3.1 to 1.3.2 (tapis/apps)
Expand All @@ -27,29 +39,45 @@ Image updates
- Updated several image minor release versions.
- Added VERSION file to reflect which version of tapis-deployer was used.

## 1.3.0

**Breaking Changes**

- Tapis Deployer 1.3.0 differs greatly from 1.2.x. Please refer to the documentation https://tapis.readthedocs.io/en/latest/technical/index.html for migration guide.
- The template generation backend was redone using Ansible.
- The input generator is deprecated.
- If you have an existing Tapis deployment, you may be using the "file" storage type for vault. In the future the default will use "raft" storage type. For new installs, no action is required. Follow Migration steps below to migrate from file to raft storage.

### Migration from 1.2.x steps

- Check your existing vault for storage type. Exec into the container and get storage type

## 1.3.0 - 20230106
kubectl exec -it deploy/vault vault status | grep "Storage Type"
Storage Type file

### Breaking Changes:
If your storage type if "file", include this in your host_vars:

vault_raft_storage: false

### New features:
If your storage type is "raft", no further action required. Ensure that "vault_raft_storage" var is undefined in your host_vars.

- Proxy/Nginx: Moved each location stanza to its own file.
## 1.3.0

**Breaking Changes**

- Tapis Deployer 1.3.0 differs greatly from 1.2.x. Please refer to the documentation https://tapis.readthedocs.io/en/latest/technical/index.html for migration guide.
- The template generation backend was redone using Ansible.
- The input generator is deprecated.

### Migration from 1.2.x steps

- Remove "tokens_tenants" var from the tokens section of your input file. Is now set to ["*"] by default, meaning tokens will get a list of tenants from tenants service.
- Remove "authenticator_service_tenants" var from the authenticator section of your input file. Is now set to ["*"] by default, meaning authenticator will get a list of tenants from tenants service.
- Update your host_vars (deployer input file) to include the following new variables. (see inventory_examples for reference):

# Choose where your deployment files should be created.
tapisdir: '{{ ansible_env.HOME }}/tmp/{{ inventory_hostname }}'
tapisdatadir: '{{ ansible_env.HOME }}/tmp/{{ inventory_hostname }}-data'

### Bug fixes:

## 1.2.x - 2023-01-06

- Docker flavor: Proxy/Nginx: Moved each location stanza to its own file.


## 1.2.0 - 2022-05-31
Expand Down Expand Up @@ -92,4 +120,3 @@ Initial pre-release of Tapis Deployer for generating Tapis deployment YAML & scr
### Bug fixes:

- None.
[tapistest@cic02 tapis-deployer]$
95 changes: 48 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,65 +1,66 @@
# Tapisctl
# Tapis Deployer

Based on Ansible playbooks and roles. Used for generating deployment files for tapis.
The Tapis Deployer software generates scripts for deploying and manging a Tapis installation.
Tapis Deployer is based on the Ansible project. Currently, Tapis Deployer targets Kubernetes clusters
for deployment of Tapis services, but we plan to support using Docker Compose instead of Kubernetes
in a future release.

Includes both Docker and Kubernetes deployment flavors.
## User Documentation

## Install
Full documentation is being developed for Tapis Deployer on the Tapis ReadTheDocs site. See
the [Deployment & Administration Guide](https://tapis.readthedocs.org/deployment/index.html).

Run as regular user (e.g. rocky).

Install Ansible using your Linux distro package manager. e.g. For Rocky Linux:
## Developing Deployer

# sudo dnf install ansible libselinux-python3
This section describes the process by which changes are made to Deployer itself.

Install Ansible community module:
_In progress..._

# ansible-galaxy collection install community.general
### Procedures for normal development (By Developers and Deployer Maintainers together)

For testing you can use one of the inventory_example inventories. For example
DEVs: create a temporary feature branch off of tapis-deployer dev branch (https://github.com/tapis-project/tapis-deployer)
could be named for feature i.e. "refactor-tapisui"
put all changes you wish to make into this branch
push your branch to github
when happy with these changes as a batch, create a PR from this branch to dev
Maintainers
Maintainers: folks will review and iterate on the PR, maybe decline, ask for edits, etc
Maintainers: once happy with the changes, folks will merge into dev
Maintainers will create PRs to merge dev into staging
After some amount of testing and iteration
Create a PR to bring changes from dev branch to staging
Additional folks may review PR or just merge it.

# ansible-playbook -i inventory_example/docker-inventory.yml playbooks/generate.yml
### Procedures for NEW TAPIS RELEASE (IN PROGRESS, draft)

After running you should find ~/tmp/tapisquickstart-docker1 containing docker compose.yml files for each Tapis component.
Performed by Deployer Maintainer Persons

There is a similar example playbook for Kubernetes inventory_example/kube-inventory.yml.
This applies to a quick image-version-only increment or a more substantial release after some development by above procedure.

## Customize Your Install
Maintainers gather feedback and change requirements for new release. i.e. Ask developers, maintainers, users. This can be a longer dev process or a quick version-only change.
For image-version-only changes, developers request an update to one or more production Tapis services. Requests are informal and take the form of a slack message on the tapis-deployer channel.
Create a TEMPORARY branch from main called <next vesion of deployer>
i.e. 1.3.99-rc
bring in desired changes
if it's a quick image version adjustment, maintainers change image numbers in tapis-deployer/playbooks/roles/<component>/defaults/main/images.yml
The only changes to deployer are new image tags for the target services.
if more substantial release, manually curate the changes from the staging branch into this RC branch
update release notes

Copy the inventory_example dir out of the repo dir and start modifying the hosts and host_vars files to suit your needs.
increment the deployer version in https://github.com/tapis-project/tapis-deployer/blob/main/playbooks/roles/baseburnup/defaults/main/vars.yml (e.g.

# cp -r inventory_example ~/inventory
# mv ~/inventory/docker-inventory.yml ~/inventory/hosts
baseburnup_tapis_deployer_version: 1.3.99
merge to main & immediately tag this new commit as a version "v1.3.99"
send announcement of new release <boilerplate template HERE TBD>?

For example if you want to change the location of the generated files, add a var to the ~/inventory/hosts file:
## Developer Guide

tapis_installs:
hosts:
tapisquickstart-docker1:
ansible_connection: local
tapisflavor: docker
tapisdir: '{{ ansible_env.HOME }}/tmp/tapis-1.3'

(See Ansible docs for more setup.)

Run the playbook after making changes:

# ansible-playbook -i ~/inventory/hosts playbooks/generate.yml

## Docker vs. Kubernetes Installation

You can generate either Docker- or Kubernetes-specific deployment files using the same playbooks. You specify this in your inventory (hosts file) by specifying the `tapisflavor` variable one of 2 ways.

tapisflavor: docker

or:

tapisflavor: kube


## Other Playbooks

There are several other playbooks to help with other things, for example for printing all the config variables:

# ansible-playbook -i ~/inventory/hosts playbooks/showconf.yml
- Create a branch off of the appropriate trunk branch
- If you want a very simple change, like a change to a component image version, quick bugfix, etc., branch off of *main* branch.
- If it's a more elaborate change, branch off of *dev* branch.
- Make and commit your changes
- For example, to increase the image version for the Apps API images, edit the file `playbooks/roles/apps/defaults/main/images.yml`
- Create a Pull Request from your branch against the branch you created the branch from, *main* or *dev*
- Testers can pull and test your changes before merging into the common branches and creating a new release.

Loading

0 comments on commit 6e6af01

Please sign in to comment.