Skip to content

Commit

Permalink
feat: store artifacts in OCI registry (#3328)
Browse files Browse the repository at this point in the history
Initial draft of storing artifacts in the OCI registry, not really ready
for review and needs a lot of work:

Basically:

- Starts a registry container in dev mode that is used to store the
deployment content
- Deletes the artefacts table
- The controller uploads artifacts to the registry on deployment
- The runner pulls from the registry to download its artifacts
  • Loading branch information
stuartwdouglas authored Nov 15, 2024
1 parent 876aab0 commit b02c014
Show file tree
Hide file tree
Showing 43 changed files with 647 additions and 586 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,8 @@ jobs:
run: just pnpm-install
- name: Build Language Plugins
run: just build-language-plugins
- name: Pull Registry
run: docker image pull registry:2
- name: Console e2e
run: just e2e-frontend
integration-shard:
Expand Down
10 changes: 4 additions & 6 deletions backend/controller/artefacts/artefact.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import (
"context"
"io"

"github.com/TBD54566975/ftl/internal/model"
"github.com/TBD54566975/ftl/internal/sha256"
)

Expand All @@ -31,15 +30,14 @@ type ReleaseArtefact struct {
Executable bool
}

type Registry interface {
type Service interface {

// GetDigestsKeys locates the `digests` corresponding `ArtefactKey`s and identifies the missing ones
GetDigestsKeys(ctx context.Context, digests []sha256.SHA256) (keys []ArtefactKey, missing []sha256.SHA256, err error)

// Upload pushes the specified media, and metadata, to the registry and returns the computed digest
Upload(context context.Context, artefact Artefact) (sha256.SHA256, error)

// Download performs a streaming download of the artefact identified by the supplied digest
Download(context context.Context, digest sha256.SHA256) (io.ReadCloser, error)
// GetReleaseArtefacts locates the artefacts metadata corresponding with the specified release
GetReleaseArtefacts(ctx context.Context, releaseID int64) ([]ArtefactKey, error)
// AddReleaseArtefact associates the given `release` with the artefact associated with the given `digest`
AddReleaseArtefact(ctx context.Context, key model.DeploymentKey, ra ReleaseArtefact) error
}
137 changes: 0 additions & 137 deletions backend/controller/artefacts/dal_registry.go

This file was deleted.

31 changes: 0 additions & 31 deletions backend/controller/artefacts/internal/sql/db.go

This file was deleted.

5 changes: 0 additions & 5 deletions backend/controller/artefacts/internal/sql/models.go

This file was deleted.

22 changes: 0 additions & 22 deletions backend/controller/artefacts/internal/sql/querier.go

This file was deleted.

29 changes: 0 additions & 29 deletions backend/controller/artefacts/internal/sql/queries.sql

This file was deleted.

Loading

0 comments on commit b02c014

Please sign in to comment.