Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The file with non-standard newline character can't be read when sqllogictests testing #12328

Closed
goldmedal opened this issue Sep 4, 2024 · 0 comments · Fixed by #13677
Closed
Labels
bug Something isn't working

Comments

@goldmedal
Copy link
Contributor

Describe the bug

As the discussion in #12263 (comment), this behavior seems a bug. If the input file with non-standard newline (e.g. CR), we will get the object store error

DataFusion error: Object Store error: Generic LocalFileSystem error: Requested range was invalid

To Reproduce

Provide a file using CR as the newline character, then use this file in sqllogcitests.

Expected behavior

#12263 implement the custom terminator for the csv format. If the terminator is correct, the record should be read well. Otherwise, the error message should like

Arrow error: Csv error: incorrect number of fields for line 1, expected 2 got more than 2

Additional context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant