You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
nadove-ucsc opened this issue
Nov 1, 2024
· 2 comments
Labels
-[priority] Mediumbug[type] A defect preventing use of the system as specifiedmanifests[subject] Generation and contents of manifestsorange[process] Done by the Azul team
We derive the PFB from the AnVIL schema where possible, but for non-schema tables, we fall back to our old approach of building the schema dynamically based on the observed shape of the replicas' contents. These dynamic schemas may differ from one manifest to the next if the replicas in one manifest exhibit shapes not observed in the other.
Since the replica shapes are constrained by the BigQuery table schema, I expect that the only place this will be observable is with nullable columns. A BigQuery column with type NULLABLE STRING may manifest in the PFB schema with the type null, [null, string], or string, depending on whether all/some/none of the values for that column are NULL within a given PFB manifest.
The text was updated successfully, but these errors were encountered:
hannes-ucsc
added
bug
[type] A defect preventing use of the system as specified
manifests
[subject] Generation and contents of manifests
-
[priority] Medium
labels
Nov 5, 2024
-[priority] Mediumbug[type] A defect preventing use of the system as specifiedmanifests[subject] Generation and contents of manifestsorange[process] Done by the Azul team
We derive the PFB from the AnVIL schema where possible, but for non-schema tables, we fall back to our old approach of building the schema dynamically based on the observed shape of the replicas' contents. These dynamic schemas may differ from one manifest to the next if the replicas in one manifest exhibit shapes not observed in the other.
Since the replica shapes are constrained by the BigQuery table schema, I expect that the only place this will be observable is with nullable columns. A BigQuery column with type
NULLABLE STRING
may manifest in the PFB schema with the typenull
,[null, string]
, orstring
, depending on whether all/some/none of the values for that column are NULL within a given PFB manifest.The text was updated successfully, but these errors were encountered: