PgValue variant bypassing QValue for more efficient & correct PG<>PG mirroring

[serprex]

Currently enabled by passing system = 'PG' in CREATE MIRROR command

[serprex]

Schema setup/changes needs to be corrected for PG<>PG & pulling in extra info like collation etc would be nice

[serprex]

Problem: table schema deltas don't care about numeric precision/scale

[serprex]

Problem: TypeModifier only used for numeric; should be used for all types such as varchar

[iskakaushik]

can you share some examples where text representation of QValue differs from PGValue?

[serprex]

can you share some examples where text representation of QValue differs from PGValue?

This the wrong question, we could map everything to QValueInvalid, but that won't work for schema changes / creating correct matching tables

That said, NaN/inf/-inf for doubles & numerics

Similarly, we map json/jsonb both to QValueJSON, not preserving their difference

Arrays are a whole thing

[serprex]

Thoughts/listing things which can be done as followup:

1. types have typmod_in/typmod_out which can be used to properly propagate to destination, need to transmit string instead of raw integer value (hack: only propagate integer value for text types)
2. not null & other constraints/indexes, complicates schema changes since not sure what to do about ingesting after a column drop if column is not null, similarly constraints may be violated by deleted columns, so constraints would have to ignore soft deleted columns
3. process type messages in replication stream, adding to custom type map
4. support types without binary serialization (aclitem, seg) which doesn't work with pgx CopyFrom in initial copy. Also think there's issues with custom enums. Need some fallback method
5. TRUNCATE
6. scripting will get all columns as text, transforms will be responsible for preserving valid text representation
7. collations in table schema
8. should propagate typmod in schema changes

addressed