Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli: support for GCP marketplace images #2792

Merged
merged 9 commits into from
Jan 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/actions/constellation_create/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,13 @@ runs:
yq eval -i "(.provider.azure.useMarketplaceImage) = true" constellation-conf.yaml
yq eval -i "(.image) = \"${{ inputs.marketplaceImageVersion }}\"" constellation-conf.yaml

- name: Set marketplace image flag (GCP)
if: inputs.marketplaceImageVersion != '' && inputs.cloudProvider == 'gcp'
shell: bash
run: |
yq eval -i "(.provider.gcp.useMarketplaceImage) = true" constellation-conf.yaml
yq eval -i "(.image) = \"${{ inputs.marketplaceImageVersion }}\"" constellation-conf.yaml

- name: Update measurements for non-stable images
if: inputs.fetchMeasurements
shell: bash
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/e2e-test-marketplace-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ on:
type: choice
options:
- "azure"
default: "azure"
- "gcp"
required: true
runner:
description: "Architecture of the runner that executes the CLI"
Expand Down
11 changes: 10 additions & 1 deletion dev-docs/workflows/marketplace-images.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,13 @@ And ensure that the cluster uses a release image (i.e. `.image=vX.Y.Z` in the `c

## GCP

Marketplace Images on GCP are not available yet.
On GCP, to use a marketplace image, ensure that the account is entitled to use marketplace images by Edgeless Systems
by accepting the terms through the [web portal](https://console.cloud.google.com/marketplace/vm/config/edgeless-systems-public/constellation).

Then, set the VMs to use the marketplace image in the `constellation-conf.yaml` file:

```bash
yq eval -i ".provider.gcp.useMarketplaceImage = true" constellation-conf.yaml
```

And ensure that the cluster uses a release image (i.e. `.image=vX.Y.Z` in the `constellation-conf.yaml` file). Afterwards, proceed with the cluster creation as usual.
4 changes: 2 additions & 2 deletions docs/docs/overview/license.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,6 @@

Once you have received your Enterprise License file, place it in your [Constellation workspace](../architecture/orchestration.md#workspaces) in a file named `constellation.license`.

## Azure Marketplace
## CSP Marketplaces

Constellation is available through the Azure Marketplace. This allows you to create self-managed Constellation clusters that are billed on a pay-per-use basis (hourly, per vCPU) with your Azure account. You can still get direct support by Edgeless Systems. For more information, please [contact us](https://www.edgeless.systems/enterprise-support/).
Constellation is available through the Marketplaces of Azure and GCP. This allows you to create self-managed Constellation clusters that are billed on a pay-per-use basis (hourly, per vCPU) with your CSP account. You can still get direct support by Edgeless Systems. For more information, please [contact us](https://www.edgeless.systems/enterprise-support/).

Check warning on line 33 in docs/docs/overview/license.md

View workflow job for this annotation

GitHub Actions / prose

[vale] reported by reviewdog 🐶 [Microsoft.We] Try to avoid using first-person plural like 'us'. Raw Output: {"message": "[Microsoft.We] Try to avoid using first-person plural like 'us'.", "location": {"path": "docs/docs/overview/license.md", "range": {"start": {"line": 33, "column": 306}}}, "severity": "WARNING"}
7 changes: 6 additions & 1 deletion internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,9 @@ type GCPConfig struct {
// description: |
// Deploy Persistent Disk CSI driver with on-node encryption. For details see: https://docs.edgeless.systems/constellation/architecture/encrypted-storage
DeployCSIDriver *bool `yaml:"deployCSIDriver" validate:"required"`
// description: |
// Use the specified GCP Marketplace image offering.
UseMarketplaceImage *bool `yaml:"useMarketplaceImage" validate:"omitempty"`
}

// OpenStackConfig holds config information for OpenStack based Constellation deployments.
Expand Down Expand Up @@ -349,6 +352,7 @@ func Default() *Config {
Zone: "",
ServiceAccountKeyPath: "",
DeployCSIDriver: toPtr(true),
UseMarketplaceImage: toPtr(false),
},
OpenStack: &OpenStackConfig{
DirectDownload: toPtr(true),
Expand Down Expand Up @@ -699,7 +703,8 @@ func (c *Config) DeployYawolLoadBalancer() bool {

// UseMarketplaceImage returns whether a marketplace image should be used.
func (c *Config) UseMarketplaceImage() bool {
return c.Provider.Azure != nil && c.Provider.Azure.UseMarketplaceImage != nil && *c.Provider.Azure.UseMarketplaceImage
return (c.Provider.Azure != nil && c.Provider.Azure.UseMarketplaceImage != nil && *c.Provider.Azure.UseMarketplaceImage) ||
(c.Provider.GCP != nil && c.Provider.GCP.UseMarketplaceImage != nil && *c.Provider.GCP.UseMarketplaceImage)
}

// Validate checks the config values and returns validation errors.
Expand Down
7 changes: 6 additions & 1 deletion internal/config/config_doc.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

46 changes: 38 additions & 8 deletions internal/imagefetcher/imagefetcher.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import (
"fmt"
"io/fs"
"regexp"
"strings"

"github.com/edgelesssys/constellation/v2/internal/api/fetcher"
"github.com/edgelesssys/constellation/v2/internal/api/versionsapi"
Expand Down Expand Up @@ -52,10 +53,6 @@ func (f *Fetcher) FetchReference(ctx context.Context,
return "", fmt.Errorf("parsing config image short path: %w", err)
}

if useMarketplaceImage {
return buildMarketplaceImage(ver, provider)
}

imgInfoReq := versionsapi.ImageInfo{
Ref: ver.Ref(),
Stream: ver.Stream(),
Expand Down Expand Up @@ -85,21 +82,54 @@ func (f *Fetcher) FetchReference(ctx context.Context,
return "", fmt.Errorf("validating image info file: %w", err)
}

if useMarketplaceImage {
return buildMarketplaceImage(marketplaceImagePayload{
ver: ver,
provider: provider,
attestationVariant: attestationVariant,
imgInfo: imgInfo,
filters: filters(provider, region),
})
}

return getReferenceFromImageInfo(provider, attestationVariant.String(), imgInfo, filters(provider, region)...)
}

// marketplaceImagePayload is a helper struct to pass around the required information to build a marketplace image URI.
type marketplaceImagePayload struct {
ver versionsapi.Version
provider cloudprovider.Provider
attestationVariant variant.Variant
imgInfo versionsapi.ImageInfo
filters []filter
}

// buildMarketplaceImage returns a marketplace image URI for the given CSP and version.
func buildMarketplaceImage(ver versionsapi.Version, provider cloudprovider.Provider) (string, error) {
sv, err := semver.New(ver.Version())
func buildMarketplaceImage(payload marketplaceImagePayload) (string, error) {
sv, err := semver.New(payload.ver.Version())
if err != nil {
return "", fmt.Errorf("parsing image version: %w", err)
}

switch provider {
if sv.Prerelease() != "" {
return "", fmt.Errorf("marketplace images are not supported for prerelease versions")
}

switch payload.provider {
case cloudprovider.Azure:
// For Azure, multiple fields of information are required to use marketplace images,
// so we pack them in a custom URI.
return mpimage.NewAzureMarketplaceImage(sv).URI(), nil
case cloudprovider.GCP:
// For GCP, we just need to replace the GCP project name (constellation-images) to the public project that
// hosts the marketplace images (mpi-edgeless-systems-public).
imageRef, err := getReferenceFromImageInfo(payload.provider, payload.attestationVariant.String(), payload.imgInfo, payload.filters...)
if err != nil {
return "", fmt.Errorf("getting image reference: %w", err)
}
return strings.Replace(imageRef, "constellation-images", "mpi-edgeless-systems-public", 1), nil
default:
return "", fmt.Errorf("marketplace images are not supported for csp %s", provider.String())
return "", fmt.Errorf("marketplace images are not supported for csp %s", payload.provider.String())
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ See the [full list of CSPs](https://docs.edgeless.systems/constellation/overview

### Optional

- `marketplace_image` (Boolean) Whether a marketplace image should be used. Currently only supported for Azure.
- `marketplace_image` (Boolean) Whether a marketplace image should be used. Currently only supported for Azure and GCP.
- `region` (String) Region to retrieve the image for. Only required for AWS.
The Constellation OS image must be [replicated to the region](https://docs.edgeless.systems/constellation/workflows/config),and the region must [support AMD SEV-SNP](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/snp-requirements.html), if it is used for Attestation.
- `version` (String) Version of the Constellation OS image to use. (e.g. `v2.13.0`). If not set, the provider version value is used.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,8 @@ func (d *ImageDataSource) Schema(_ context.Context, _ datasource.SchemaRequest,
},
"csp": newCSPAttributeSchema(),
"marketplace_image": schema.BoolAttribute{
Description: "Whether a marketplace image should be used. Currently only supported for Azure.",
MarkdownDescription: "Whether a marketplace image should be used. Currently only supported for Azure.",
Description: "Whether a marketplace image should be used. Currently only supported for Azure and GCP.",
MarkdownDescription: "Whether a marketplace image should be used. Currently only supported for Azure and GCP.",
msanft marked this conversation as resolved.
Show resolved Hide resolved
Optional: true,
},
"region": schema.StringAttribute{
Expand Down Expand Up @@ -128,11 +128,11 @@ func (d *ImageDataSource) ValidateConfig(ctx context.Context, req datasource.Val
)
}

// Marketplace image is only supported for Azure
if !data.CSP.Equal(types.StringValue("azure")) && !data.MarketplaceImage.IsNull() {
resp.Diagnostics.AddAttributeWarning(
// Marketplace image is only supported for Azure and GCP
if data.CSP.Equal(types.StringValue("aws")) && !data.MarketplaceImage.IsNull() {
resp.Diagnostics.AddAttributeError(
path.Root("marketplace_image"),
"Marketplace images are currently only supported on Azure", "When another CSP than Azure is used, setting 'marketplace_image' has no effect.",
"Marketplace images are currently only supported on Azure and GCP", "When another CSP than Azure or GCP is used, marketplace images are unavailable.",
)
}

Expand Down