-
Notifications
You must be signed in to change notification settings - Fork 67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow CSV output when columns are consistent even if types aren't #4781
Labels
Comments
mattnibs
added a commit
that referenced
this issue
Nov 17, 2023
Adjust the csvio.Writer so that it can handle records with the same field names but different types. Closes #4781
mattnibs
added a commit
that referenced
this issue
Nov 17, 2023
Adjust the csvio.Writer so that it can handle records with the same field names but different types. Closes #4781
mattnibs
added a commit
that referenced
this issue
Nov 17, 2023
Adjust the csvio.Writer so that it can handle records with the same field names but different types. Closes #4781
mattnibs
added a commit
that referenced
this issue
Nov 17, 2023
Adjust the csvio.Writer so that it can handle records with the same field names but different types. Closes #4781
mattnibs
added a commit
that referenced
this issue
Nov 17, 2023
Adjust the csvio.Writer so that it can handle records with the same field names but different types. Closes #4781
mattnibs
added a commit
that referenced
this issue
Nov 17, 2023
Adjust the csvio.Writer so that it can handle records with the same field names but different types. Closes #4781
mattnibs
added a commit
that referenced
this issue
Nov 22, 2023
Adjust the csvio.Writer so that it can handle records with the same field names but different types. Closes #4781
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Repro is with Zed commit 80dbcda.
Consider this community user's CSV output example that motivated the changes in #4773
Or here's another for a non-
null
case:As the error messages indicate, adding
fuse
does work around the problem in both cases by taking a first pass through the data to coerce all the input values to construct a single merged record type. However, since CSV effectively lacks real data typing, what's truly important in the record type is that the fields are consistent, since once the field names in the header row are output there's no way to deal with additional fields later encountered. But using the examples above, anull
value is going to be output the same in CSV (i.e., "nothing" between the comma delimiters) regardless of whether it had a particular type in Zed. Similarly, if we runfuse
on the second example, indeed theunion
type is established to indicate the field could hold an integer or string:but when those values are being output on the second pass, they're still printed in the same column of the CSV output as a number or string.
i.e., the brief existence of the
union
type was just to satisfy a current constraint of the CSV writer, but didn't really enhance the output in any way.In a discussion with @nwt he agreed that we could probably relax this constraint of the CSV writer. This would reduce the number of times that new users encounter the
(consider 'fuse')
message and have to take a detour to learn about why it's needed and how to use it.The text was updated successfully, but these errors were encountered: