Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request for IGNORE_ERROR=>NumErrors for COPY or external table #56

Open
rferraton opened this issue May 6, 2022 · 2 comments
Open

Request for IGNORE_ERROR=>NumErrors for COPY or external table #56

rferraton opened this issue May 6, 2022 · 2 comments

Comments

@rferraton
Copy link

It would usefull (i think) to have an IGNORE_ERROR option for COPY and external table with à number of errors that can be ignored during the process.

@vogelsgesang
Copy link
Contributor

can you describe your use case in more detail? Which types of errors are you referring to? Individual values which are invalid? Rows which don't have enough columns? Complete files which are missing on disk? What is the benefit of specifying an exact number of expected incorrect values over just ignoring just all such errors?

In case you aren't aware of it, yet: ON_CAST_FAILURE allows you to ignore all cast errors for CSVs

@rferraton
Copy link
Author

rferraton commented May 8, 2022

I am currently facing incompleted rows (a row full of NUL 00 ascii characters due to the mode of extraction of the original data...but nevermind)
As we want to use external table with array we will have the problem in each file of the array, so we should have n "acceptables" errors no more...

It is a classical parameter you can find in commercial ETL/ELT solutions like informatica, datastage, sap bods...
On theses tools you can sometimes have the possibility to set the parameter to -1 to allow errors without limit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants