Skip to content

Commit

Permalink
Add documentation
Browse files Browse the repository at this point in the history
Signed-off-by: Hoang Quoc Trung <[email protected]>
  • Loading branch information
ichbinfrog committed Apr 14, 2024
1 parent b7cde0a commit 7b8747f
Show file tree
Hide file tree
Showing 4 changed files with 41 additions and 63 deletions.
9 changes: 5 additions & 4 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@ jobs:
uses: actions/setup-go@v4
with:
go-version: '1.22.x'
cache-dependency-path: go.sum
- name: Install dependencies
run: go get .
- name: Build
run: go build -v ./...
- name: Test with the Go CLI
run: go test
- name: Build and test
run: |
go build -v ./...
go test ./...
27 changes: 27 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Contributing guidelines

Feel free to open PRs to improve the code base, tests or add further preflight and postflight checks.

Currently E2E tests are run manually based on the infrastructure bootstrapped in `e2e/terraform`. It's highly recommended that you run both unit tests and integration tests on your PRs


**Unit test (also on the CI)**

```console
go test -v ./...
```

**Integration tests (WIP)**

```console
# Initial setup
cd e2e/terraform
terraform apply

# Running tests
go run main.go \
--src-project=$SRC_PROJECT \
--src-instance=$SRC_INSTANCE \
--dst-project=$DST_PROJECT \
--dst-instance=$DST_INSTANCE
```
13 changes: 9 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# CloudSQL Migration utility

A tool for quickly migrating CloudSQL instances to each other.
**A tool for quickly migrating CloudSQL instances to each other.**

To run a migration, simply run:

Expand All @@ -23,11 +23,16 @@ This approach is favorable the classic [sql export-import](https://cloud.google.

- It's much **faster** :zap:. The time to create and restore snapshots are quite well optimized is inversely proportional to the instance's resources.

- Users with their credentials, database schemas, permissions, extensions are preserved and transferred without issue from the source to the target instance.
- It's **safer** :lock: as data does not persist on a temporary bucket but on snapshots that are best practices to have anyways. On top of that, users with their credentials, database schemas, permissions, extensions are preserved and transferred without issue from the source to the target instance.

- Not having to deal with the complexities of dumping and importing SQL (e.g. extension, user permission and other issues that arises from the lack of control of the dump generated)
- It's **simpler** :baby:; Not having to deal with the complexities of dumping and importing SQL (e.g. extension, user permission and other issues that arises from the lack of control of the dump generated)

- Does not stress the source instance if [automated backups](https://cloud.google.com/sql/docs/mysql/backup-recovery/backing-up#set-retention) is enabled. The latter is incremental thus generating a new snapshot from the last point in time is significantly faster and less resource intensive than performing an sql dump.


## Getting started

Build your and install own binary with:

```console
go install github.com/ichbinfrog/cloudsqlmigrate
```
55 changes: 0 additions & 55 deletions e2e/terraform/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -49,59 +49,4 @@ resource "google_sql_database_instance" "target" {
disk_type = "PD_HDD"
disk_size = 10
}
}


// E2E Tests
resource "google_service_account" "e2e" {
project = google_project.self.project_id
account_id = "sa-e2e"
}

resource "google_project_iam_member" "e2e" {
// Technically, the least privilege role would only be a subset of the CloudSQL Viewer Role with
// "cloudsql.backupRuns.list", "cloudsql.backupRuns.get", "cloudsql.backupRuns.create", "cloudsql.backupRuns.restoreBackup",
// but I'm lazy :)
for_each = toset([
"roles/cloudsql.admin",
])
project = google_project.self.project_id
member = "serviceAccount:${google_service_account.e2e.email}"
role = each.key
}

resource "google_iam_workload_identity_pool" "github" {
project = google_project_service.self["iamcredentials.googleapis.com"].project
workload_identity_pool_id = "github-pool"
display_name = "Github E2E Tests pipeline"
}

resource "google_iam_workload_identity_pool_provider" "github" {
project = google_project.self.project_id

workload_identity_pool_id = google_iam_workload_identity_pool.github.workload_identity_pool_id
workload_identity_pool_provider_id = "github-provider"
description = "OIDC identity pool provider for e2e tests"
disabled = false

attribute_mapping = {
"google.subject" = "assertion.sub"
"attribute.actor" = "assertion.actor"
"attribute.repository_owner" = "assertion.repository_owner"
"attribute.repository" = "assertion.repository"
}

oidc {
issuer_uri = "https://token.actions.githubusercontent.com"
}
}

resource "google_service_account_iam_member" "identity_federation_principalset" {
service_account_id = google_service_account.e2e.name
role = "roles/iam.workloadIdentityUser"
member = "principalSet://iam.googleapis.com/${google_iam_workload_identity_pool.github.name}/attribute.repository/${var.github_username}/${var.github_repo}"

depends_on = [
google_iam_workload_identity_pool_provider.github
]
}

0 comments on commit 7b8747f

Please sign in to comment.