Templating for root-level keys

[cueckoo]

Originally opened by @narqo in cuelang/cue#738

Following the thread on Twitter /w @myitcv and @rogpeppe.

I'm trying to come up with schema in cue, that will help us with catching typos in the existing yaml configs.

Below is a simplified example of a config. This is a single yaml file, where the root-level key can a be any string, but the value must match a predefined struct.

# config.yml

testing:
  redis:
    network: tcp
    addr:    localhost:9092

production_eu:
  stats:
    output1: graphite  # (sic) a typo here, the key must be "output"
  redis:
    network: unix
    addr:    /run/redis.sock

The schema I use:

// schema.cue

{[string]: #Config}

#Config: {
  stats?: #StatsConfig
  redis: {
    network: "tcp" | "unix"
    address: string
  }
}

#StatsConfig: {
  output: "graphite"  // only one value is supported
}

I expected, that running cue vet against the config, would raise the case about an extra field production_eu.stats.output1 (helping us to spot the typo). But the vet has no complains:

%!c(MISSING)ue version
cue version 0.3.0-beta.3 darwin/amd64
%!c(MISSING)ue vet config.yml config.cue
%!e(MISSING)cho $status
0

---

If I change the schema, specifying the exact top-level keys, vet raises the error as I expected:

// schema.cue

testing: #Config
production_eu: #Config

(redacted)

%!c(MISSING)ue vet config.yml config.cue
production_eu.stats: field `output1` not allowed:
⋯

---

If I try to update the "shape" of the config, moving testing, production_eu, etc keys under a single root-level key, vet also works as I expected it:

# config.yml

env:
  testing:
    (redacted)
  production_eu:
    (redacted)

# schema.cue

env: [string]: #Config

(redacted)

%!c(MISSING)ue vet config.yml config.cue
env.production_eu.stats: field `output1` not allowed:
⋯

[cueckoo]

Original reply by @ostcar in cuelang/cue#738 (comment)

I am also trying to validate an existing yaml file where the root-level-keys can be any string but have to have a special value. I came to the conclusion, that this is not so easy to do in cue.

Therefore I use cue import -f -l '"root"' config.yml

This creates a new file config.cue that contains the data of your config file converted to cue. But all root-keys are under the key root. This workaround helped in my case.

[cueckoo]

Original reply by @myitcv in cuelang/cue#738 (comment)

Hopefully @mpvl's answer after yours answered the question?

[cueckoo]

Original reply by @mpvl in cuelang/cue#738 (comment)

@ostcar @myitcv @narqo
I would say this is a bug: like in Go you embed types to extend them. If can embed the definition #Foo: foo: int in a struct, like this

#Bar: {
    #Foo
    bar: int
}

after which #Bar would allow both foo and bar.

This is what is happening in your example. However, embeddings across files should only be allowed between files of the same package. The same holds for cross-file references. Files without a package clause (or package _), are treated as standalone. So in that case, combining the two files should not behave like embedding, but rather as unification, as you rightfully expected.

Some of these concepts post-date the command so not all command have been brought in line with this behavior yet.

[cueckoo]

Original reply by @ostcar in cuelang/cue#738 (comment)

Is is possible to but the yaml-file and the cue-schema-fiele in the same package? cue vet has the flag -p, --package string package name for non-CUE files. But it seem this has no effect.

[cueckoo]

Original reply by @myitcv in cuelang/cue#738 (comment)

@ostcar please can you raise a separate issue for this with a repro of what you tried? Thanks

[cueckoo]

Original reply by @rdlaitila in cuelang/cue#738 (comment)

@mpvl thanks for the additional clarification on embedding and intra-package and non-package behavior. Is the answer that this use case is not supported?

I'm experiencing the same issue as the original question. I have a properly declared package with all my definitions and schemata, and use a non-package cue file that "embeds" in the top level to validate against actual json. Here is a simple example:

types/types.cue

package types

#SchemataRoot: {
	a: string
	b: int
}

#SchemataExt: {
	#SchemataRoot
	c: bool
}

schema/schema.cue

import "rootexample.com/types:types"
{
	types.#SchemataExt
}

data.json

{
	"a": "a",
	"b": 100,
	"c": true,
	"d": "should be not allowed"
}

cue vet

%!c(MISSING)ue vet data.json schema/schema.cue
%!e(MISSING)cho $?
0

this of course works if I have a "root" key:

schema/schema.cue

import "rootexample.com/types:types"
root: types.#SchemataExt

data.json

{
	"root": {
		"a": "a",
		"b": 100,
		"c": true,
		"d": "should be not allowed"
	}
}

cue vet

%!c(MISSING)ue vet data.json schema/schema.cue
root: field `d` not allowed:
    ./data.json:6:3
    ./schema/schema.cue:2:7
    ./types/types.cue:3:16
    ./types/types.cue:8:15
    ./types/types.cue:9:2

[cueckoo]

Original reply by @myitcv in cuelang/cue#738 (comment)

@rdlaitila apologies you have run into a bit of a rough edge here. Please see the discussion in cuelang/cue#750.

TL;DR - it is not as easy as it should be today to achieve what you want and we're looking to fix that.

[cueckoo]

Original reply by @rdlaitila in cuelang/cue#738 (comment)

@myitcv thanks for the reply I look forward to one day perhaps having support for validating data we may not have control over generating but still want strictness at the top level. I just want to say cue is a fantastic body of work and regardless of this behavior I'm super excited for its future!

[fab29p]

On the first example (and presumably the last one), just add close(). Tested on Cue 0.6.0

close({[string]: #Config})