Using dictionary array to store low cardinality tag column

[evenyag]

What type of enhancement is this?

Performance

What does the enhancement do?

Creating a repeated vector for each batch is costly. It needs to clone the string value many times.

greptimedb/src/mito2/src/read/projection.rs

Lines 198 to 207 in 9af9c02

	let vector = match cache_manager {
	Some(cache) => repeated_vector_with_cache(
	&column_schema.data_type,
	value,
	num_rows,
	cache,
	)?,
	None => new_repeated_vector(&column_schema.data_type, value, num_rows)?,
	};
	columns.push(vector);

What's more, it might speed up some binary operations while evaluating expressions in the query engine.

We might use dictionary arrays to hold tag values to reduce the overhead. Then we can remove the repeated vector cache.

Implementation challenges

We might replace the schema of string tag columns with dictionary(string) during the optimization phase and tell the mito engine to return dictionary arrays.

For simplicity, we can simply map a string tag column to a dictionary(string) column without considering its cardinality.