Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Direct URL can't be mapped to file system #1606

Open
5 of 6 tasks
bobir01 opened this issue Aug 26, 2024 · 2 comments
Open
5 of 6 tasks

Direct URL can't be mapped to file system #1606

bobir01 opened this issue Aug 26, 2024 · 2 comments

Comments

@bobir01
Copy link

bobir01 commented Aug 26, 2024

URL that does not end with simple file link query cant be read to system files
data url is changed to public csv file due to risks of privacy, but the core problem is not about file content here ;(

  • pgloader --version

    pgloader version "3.6.7~devel"
    compiled with SBCL 2.1.11.debian
    
  • [] did you test a fresh compile from the source tree?

    Compiling pgloader from sources is documented in the
    README, it's
    easy to do, and if patches are to be made to fix your bug, you're going
    to have to build from sources to get the fix anyway…

  • did you search for other similar issues?

  • how can I reproduce the bug?

    I am trying to insert csv file that has tailing tokens after its name, so pgloader could not write/read this file


LOAD CSV
    FROM 'https://www.stats.govt.nz/assets/Uploads/Annual-enterprise-survey/Annual-enterprise-survey-2023-financial-year-provisional/Download-data/annual-enterprise-survey-2023-financial-year-provisional-size-bands.csv?sv=2023-11-03&st=2024-08-26T11%3A45%3A43Z&se=2024-08-30T11%3A45%3A00Z' WITH ENCODING utf-8
    INTO postgresql://user:*****@localhost:5432/db
    TARGET TABLE cdm_commerce.dm_basic_osa_hourly

    WITH
        skip header = 1,
        fields terminated by ','
        SET work_mem to '1GB', maintenance_work_mem to '2GB';

BORDET|Bobirjon Mardonov
  • pgloader output you obtain
2024-08-26T13:09:57.012000Z LOG pgloader version "3.6.7~devel"
2024-08-26T13:09:57.012000Z LOG Data errors in '/tmp/pgloader/'
2024-08-26T13:09:57.012000Z LOG Parsing commands from file #P"/home/b_mardonov/csv.load"
KABOOM!
SIMPLE-FILE-ERROR: Can't find the TRUENAME of wild pathname /home/b_mardonov/https://www.stats.govt.nz/assets/Uploads/Annual-enterprise-survey/Annual-enterprise-survey-2023-financial-year-provisional/Download-data/annual-enterprise-survey-2023-financial-year-provisional-size-bands.csv?sv=2023-11-03&st=2024-08-26T11%3A45%3A43Z&se=2024-08-30T11%3A45%3A00Z) (physicalized from /home/b_mardonov/https://www.stats.govt.nz/assets/Uploads/Annual-enterprise-survey/Annual-enterprise-survey-2023-financial-year-provisional/Download-data/annual-enterprise-survey-2023-financial-year-provisional-size-bands.csv?sv=2023-11-03&st=2024-08-26T11%3A45%3A43Z&se=2024-08-30T11%3A45%3A00Z).
An unhandled error condition has been signalled:
   Can't find the TRUENAME of wild pathname /home/b_mardonov/https://www.stats.govt.nz/assets/Uploads/Annual-enterprise-survey/Annual-enterprise-survey-2023-financial-year-provisional/Download-data/annual-enterprise-survey-2023-financial-year-provisional-size-bands.csv?sv=2023-11-03&st=2024-08-26T11%3A45%3A43Z&se=2024-08-30T11%3A45%3A00Z (physicalized from /home/b_mardonov/https://www.stats.govt.nz/assets/Uploads/Annual-enterprise-survey/Annual-enterprise-survey-2023-financial-year-provisional/Download-data/annual-enterprise-survey-2023-financial-year-provisional-size-bands.csv?sv=2023-11-03&st=2024-08-26T11%3A45%3A43Z&se=2024-08-30T11%3A45%3A00Z).




What I am doing here?

Can't find the TRUENAME of wild pathname #1=/home/b_mardonov/https://www.stats.govt.nz/assets/Uploads/Annual-enterprise-survey/Annual-enterprise-survey-2023-financial-year-provisional/Download-data/annual-enterprise-survey-2023-financial-year-provisional-size-bands.csv?sv=2023-11-03&st=2024-08-26T11%3A45%3A43Z&se=2024-08-30T11%3A45%3A00Z (physicalized from #1#).
  • data that is being loaded, if relevant
[data.csv](https://www.stats.govt.nz/assets/Uploads/Annual-enterprise-survey/Annual-enterprise-survey-2023-financial-year-provisional/Download-data/annual-enterprise-survey-2023-financial-year-provisional-size-bands.csv?sv=2023-11-03&st=2024-08-26T11%3A45%3A43Z&se=2024-08-30T11%3A45%3A00Z)
  • How the data is different from what you expected, if relevant
@bobir01
Copy link
Author

bobir01 commented Aug 26, 2024

this seems very silly issue why there is no option to rename blob downloading ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@bobir01 and others