DONT MERGE. Vam.

[jamii]

(I'm just opening this to get the review ui).

[jamii]

How about

type Nullable struct {
    mask []byte
    vector Any
}

Then we don't have to embed the null mask in every other type.

[jamii]

zed.DecodeString(val)

If I'm reading this right this is not a copy - all the resulting strings share the same underlying byte buffer?

[nwt]

That's what we want but there's an implicit copy here as the compiler will insert a call to runtime.slicebytetostring when it sees this. Our usual way around that is this: https://github.com/brimdata/zed/blob/40639dce6c333da1c55844b026052dc3718cc2d2/pkg/byteconv/byteconv.go#L11-L16

[jamii]

Notes:

• Adds a command that tests the aggs.
• Alters the project command to use vectors.
• Adds a VecScan op which projects some paths.
  • Exec in compiler/kernel/op
  • Work happens in VecScanner
• Adds some hacky aggregate ops.
  • (Ignoring these for now)
• Adds vcache to lake.
• Adds Index - a selection vector.
  • Adds commented out materialization code.
• vam/projection splits into vectors by type (should be replaced by producing Union vector where appropriate)
• Adds VecScanner.
  • parent feeds lake uris
  • (Don't follow the cancellation logic yet)
  • Loads a vector for each (type, path) combo
    • Probably path should include union branches instead.
• Changes vcache loading code to call loadVector.
  • array/set (only handles ints)
  • map
  • nulls (not finished)
  • object (on demand?)
  • primitives (only a few types)
    • stored as zng plus precomputed offsets
    • commented out code for dictionary compression?
  • records (comment about missing locking?)
  • union
• Adds a mutex to vcache cache fetch.
• Deletes old vcache projection code.
• Comments out vcache reader.
• Adds vector.
  • Any is the interface.
  • Builder takes a zcode.Builder and adds 1 value to it.
    • (zcode.Builder just helps track lengths of nested trees in zng)
  • mem ???
  • Const probably needs a length field.
  • String vector