Compact Storage

[pmenon]

Materializing sql::Val into memory is inefficient due to its layout. Because the null indicator byte is stored before the type-specific data, larger SQL types (e.g., ints and big ints) require a 3-byte and 7-byte padding. This wasted space effectively doubles the size of these types. Thus, hash-tables and sorters can effectively halve their size by using a more efficient layout.

What we need is a Storage responsible with converting sql::Val used during query processing into a compact form when writing to memory. Ideally, Storage is instantiated with a
schema a priori, and determines an optimal alignment to minimize wasted space. A simple approach is to sort by data type size with a NULL bitmap tacked onto the end.