-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MG-153 - Add Vault scripts as git subtree #157
base: master
Are you sure you want to change the base?
Conversation
git-subtree-dir: docker/addons/vault git-subtree-split: 54a134e72f47e51acfbced9a94c1b22e13c3e461
Signed-off-by: JeffMboya <[email protected]>
Signed-off-by: JeffMboya <[email protected]>
git-subtree-dir: vault git-subtree-split: a32634a1e90508f08a75081b0e595a427d3cbb00
Signed-off-by: JeffMboya <[email protected]>
git-subtree-dir: vault git-subtree-split: a32634a1e90508f08a75081b0e595a427d3cbb00
git-subtree-dir: scripts/vault git-subtree-split: a32634a1e90508f08a75081b0e595a427d3cbb00
Signed-off-by: JeffMboya <[email protected]>
Signed-off-by: JeffMboya <[email protected]>
git-subtree-dir: scripts/vault git-subtree-split: a32634a1e90508f08a75081b0e595a427d3cbb00
Signed-off-by: JeffMboya <[email protected]>
Signed-off-by: JeffMboya <[email protected]>
Signed-off-by: JeffMboya <[email protected]>
@arvindh123 There’s a small issue that needs your input:
How should we proceed? |
I hope we had provision to pass env file via flage |
Yes, we have the provision |
Signed-off-by: JeffMboya <[email protected]>
Signed-off-by: JeffMboya <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.env can be kept at vault/.env,
The script folder should be exact duplicate of magistrala
README.md
Outdated
|
||
Since we have added the `magistrala` Vault directory as a subtree in the `devops` repository, we only include the `docker/addons/vault` directory and its contents in `scripts/vault` directory. We do not include `docker/.env` or other unrelated directories from the `magistrala` repository. | ||
|
||
As a result, running the Vault setup scripts from within the `scripts/vault/scripts` directory may throw the following error: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is better to mention only how to run scripts with --env-files
Signed-off-by: JeffMboya <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need docker-compose and vault config files in scripts/vault ?
scripts/vault/.env
Outdated
@@ -40,4 +40,4 @@ MG_VAULT_PKI_INT_CA_ST='PARIS' | |||
MG_VAULT_PKI_INT_CA_ADDR='5 Av. Anatole' | |||
MG_VAULT_PKI_INT_CA_PO='75007' | |||
MG_VAULT_PKI_INT_CLUSTER_PATH=http://localhost | |||
MG_VAULT_PKI_INT_CLUSTER_AIA_PATH=http://localhost | |||
MG_VAULT_PKI_INT_CLUSTER_AIA_PATH=http://localhost |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add new line at end of file
scripts/vault/README.md
Outdated
@@ -45,44 +45,24 @@ When the Vault service is started, some initialization steps need to be done to | |||
|
|||
## Setup | |||
|
|||
The following scripts are provided, which work on the running Vault service in Docker. | |||
The following scripts are provided, which work on the running Vault service from within the `docker/addons/vault/scripts` directory. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doc is relative to magistrala, may be simply we should point MG vault docs for usage
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doc is relative to magistrala, may be simply we should point MG vault docs for usage
Yes, the README
is relative to Magistrala because adding docker/addons/vault
as a subtree in the DevOps repository imports content relative to Magistrala. The issue is that when the subtree is updated, it will continue importing files tied to Magistrala. A possible solution is to import docker/addons/vault
locally, remove unnecessary files, and then push the filtered contents to GitHub. This way, the DevOps repository only includes the required files.
Signed-off-by: JeffMboya <[email protected]>
We do not need it. It's there because we have added all the contents of |
Then is it possible to remove them and have only vault/scripts ? |
Signed-off-by: JeffMboya <[email protected]>
git-subtree-dir: scripts/vault git-subtree-split: a32634a1e90508f08a75081b0e595a427d3cbb00
git-subtree-dir: vault git-subtree-split: a32634a1e90508f08a75081b0e595a427d3cbb00
Signed-off-by: JeffMboya <[email protected]>
git-subtree-dir: vault git-subtree-split: a32634a1e90508f08a75081b0e595a427d3cbb00
Signed-off-by: JeffMboya <[email protected]>
git-subtree-dir: scripts/vault git-subtree-split: a32634a1e90508f08a75081b0e595a427d3cbb00
Signed-off-by: JeffMboya <[email protected]>
Signed-off-by: JeffMboya <[email protected]>
git-subtree-dir: docker/addons/vault/scripts git-subtree-split: d9f3bca47842e392383039b84e6b6e223de79e10
…dons/vault/scripts'
Signed-off-by: JeffMboya <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if these files are no needed scripts/vault/config.hcl
, scripts/vault/entrypoint.sh
, please remove them.
README.md
Outdated
To run a Vault setup script, use the `--env-file` option to specify the path to your `.env` file: | ||
|
||
```bash | ||
./<script-name>.sh --env-file <path-to-your-env-file> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Move this part to scripts/vault/Readme.md and add words like refer magistrala vault document (and its link)
Signed-off-by: JeffMboya <[email protected]>
README.md
Outdated
@@ -57,6 +57,20 @@ git push origin <your-branch> | |||
|
|||
Replace `<your-branch>` with the branch you are working on. | |||
|
|||
### Running Vault Setup Scripts with `--env-file` | |||
|
|||
To run a Vault setup script, use the `--env-file` option to specify the path to your `.env` file: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JeffMboya This part is moved scripts/vault/Readme.md
, so i think it is no more needed here
What do you think ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JeffMboya This part is moved
scripts/vault/Readme.md
, so i think it is no more needed here What do you think ?
Yes, it should be removed.
Signed-off-by: JeffMboya <[email protected]>
scripts/vault/README.md
Outdated
## Vault Web UI | ||
|
||
If the Vault is setup through `docker/addons/vault`, Then Vault Web UI is accessible by default on `http://localhost:8200/ui`. | ||
For detailed documentation on the available scripts and their usage, visit the [Vault Addon Documentation](https://github.com/absmach/magistrala/tree/main/docker/addons/vault#readme). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add following guide
# Developer Guide: Updating Vault Scripts from Magistrala
This guide provides step-by-step instructions to update the Vault scripts in your local Magistrala DevOps repository, sync them with the `main` branch of the Magistrala repository, and create a pull request (PR) to merge the changes.
## Prerequisites
Make sure you have the following:
- A local clone of the Magistrala DevOps repository.
- Access to the Magistrala GitHub repository (`https://github.com/absmach/magistrala.git`).
## Step 1: Create new branch
Create a new branch from `master` brach
```bash
git checkout -b <your-branch-name>
``
Replace `<your-branch-name>` with the name a new branch name.
## Step 2: Add the Magistrala Remote
If the Magistrala remote is not already added to your local repository, use the following command to add it:
```bash
git remote add -f magistrala https://github.com/absmach/magistrala.git
``
## Step 3: Add Subtree for Vault Scripts
If the subtree for the Vault scripts has not been added, execute the following command:
```bash
git subtree add --prefix=scripts/vault/scripts magistrala main --squash --prefix=docker/addons/vault/scripts
``
- `--prefix=scripts/vault/scripts`: Specifies the target directory in your local repository where the Vault scripts will be added.
- `magistrala main`: Refers to the `main` branch of the Magistrala repository.
- `--squash`: Combines all commits from the Magistrala `main` branch into a single commit when adding the subtree.
## Step 4: Update Vault Scripts to the Latest Version
To update the Vault scripts to the latest version and synchronize with the `main` branch of the Magistrala repository, use the following command:
```bash
git subtree pull --prefix=scripts/vault/scripts magistrala main --squash --prefix=docker/addons/vault/scripts
``
- This command pulls the latest changes from the `main` branch of the Magistrala repository.
- `--squash` creates a single commit for the changes, making the history simpler to manage.
## Step 5: Push Changes and Create a Pull Request
After syncing with the Magistrala `main` branch, push the changes to your working branch:
```bash
git push origin <your-branch-name>
``
Replace `<your-branch-name>` with the name of the branch you're working on.
Once the changes are pushed, go to your GitHub repository and create a pull request (PR) to merge the updates.
## Summary of Commands related to Git subtree
### Add Magistrala Remote
```bash
git remote add -f magistrala https://github.com/absmach/magistrala.git
``
### Add Subtree for Vault Scripts
```bash
git subtree add --prefix=scripts/vault/scripts magistrala main --squash --prefix=docker/addons/vault/scripts
``
### Update Vault Scripts to the Latest
```bash
git subtree pull --prefix=scripts/vault/scripts magistrala main --squash --prefix=docker/addons/vault/scripts
``
Signed-off-by: JeffMboya <[email protected]>
Signed-off-by: JeffMboya <[email protected]>
Signed-off-by: JeffMboya <[email protected]>
Signed-off-by: JeffMboya <[email protected]>
Signed-off-by: JeffMboya <[email protected]>
Signed-off-by: JeffMboya <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add following guide
# Developer Guide: Updating Vault Scripts from Magistrala
This guide provides step-by-step instructions to update the Vault scripts in your local Magistrala DevOps repository, sync them with the `main` branch of the Magistrala repository, and create a pull request (PR) to merge the changes.
## Prerequisites
Make sure you have the following:
- A local clone of the Magistrala DevOps repository.
- Access to the Magistrala GitHub repository (`https://github.com/absmach/magistrala.git`).
## Step 1: Create new branch
Create a new branch from `master` brach
```bash
git checkout -b <your-branch-name>
Replace <your-branch-name>
with the name a new branch name.
Step 2: Add the Magistrala Remote
If the Magistrala remote is not already added to your local repository, use the following command to add it:
git remote add -f magistrala https://github.com/absmach/magistrala.git
Step 3: Add Subtree for Vault Scripts
If the subtree for the Vault scripts has not been added, execute the following command:
git subtree add --prefix=scripts/vault/scripts magistrala main --squash --prefix=docker/addons/vault/scripts
--prefix=scripts/vault/scripts
: Specifies the target directory in your local repository where the Vault scripts will be added.magistrala main
: Refers to themain
branch of the Magistrala repository.--squash
: Combines all commits from the Magistralamain
branch into a single commit when adding the subtree.
Step 4: Update Vault Scripts to the Latest Version
To update the Vault scripts to the latest version and synchronize with the main
branch of the Magistrala repository, use the following command:
git subtree pull --prefix=scripts/vault/scripts magistrala main --squash --prefix=docker/addons/vault/scripts
- This command pulls the latest changes from the
main
branch of the Magistrala repository. --squash
creates a single commit for the changes, making the history simpler to manage.
Step 5: Push Changes and Create a Pull Request
After syncing with the Magistrala main
branch, push the changes to your working branch:
git push origin <your-branch-name>
Replace <your-branch-name>
with the name of the branch you're working on.
Once the changes are pushed, go to your GitHub repository and create a pull request (PR) to merge the updates.
Summary of Commands related to Git subtree
Add Magistrala Remote
git remote add -f magistrala https://github.com/absmach/magistrala.git
Add Subtree for Vault Scripts
git subtree add --prefix=scripts/vault/scripts magistrala main --squash --prefix=docker/addons/vault/scripts
Update Vault Scripts to the Latest
git subtree pull --prefix=scripts/vault/scripts magistrala main --squash --prefix=docker/addons/vault/scripts
git checkout -b <your-branch-name> | ||
`` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
git checkout -b <your-branch-name> | |
`` | |
```bash | |
git checkout -b <your-branch-name> |
### 4. `vault_set_pki.sh` | ||
```bash | ||
git remote add -f magistrala https://github.com/absmach/magistrala.git | ||
`` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Environmental variables starting with`MG_VAULT_PKI_INT` in `docker/.env` file are used by `vault_set_pki.sh` to generate intermediate CA. | ||
```bash | ||
git subtree add --prefix=scripts/vault/scripts magistrala main --squash --prefix=docker/addons/vault/scripts | ||
`` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To skip enable auth approle step use the following `vault_create_approle.sh --skip-enable-approle` | ||
```bash | ||
git subtree pull --prefix=scripts/vault/scripts magistrala main --squash --prefix=docker/addons/vault/scripts | ||
`` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Requirement: [VAULT CLI](https://developer.hashicorp.com/vault/tutorials/getting-started/getting-started-install) | ||
```bash | ||
git push origin <your-branch-name> | ||
`` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
token Interact with tokens | ||
``` | ||
git remote add -f magistrala https://github.com/absmach/magistrala.git | ||
`` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
### Add Subtree for Vault Scripts | ||
```bash | ||
git subtree add --prefix=scripts/vault/scripts magistrala main --squash --prefix=docker/addons/vault/scripts | ||
`` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does this do?
This PR adds vault scripts as a git subtree in the devops repo
Which issue(s) does this PR fix/relate to?
List any changes that modify/break current functionality
None
Have you included tests for your changes?
No, manually tested
Did you document any new/modified functionality?
Yes
Notes