Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build(datasets): upgrade s3fs to newer calver #348

Closed
wants to merge 24 commits into from
Closed

build(datasets): upgrade s3fs to newer calver #348

wants to merge 24 commits into from

Commits on Sep 21, 2023

  1. build(datasets): upgrade to newer, CalVered s3fs 

    Signed-off-by: Matthias Roels <[email protected]>
    @MatthiasRoels
    MatthiasRoels committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    dcef71d View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2023

  1. fix(datasets): unit tests for matplotlib writer leverage moto server 

    Since s3fs is switched to aiobotocore, tests started to fail with message
`'MockRawResponse' object has no attribute 'raw_headers'`. This is a bug
in aiobotocore. Using moto server does seem to resolve the issues

Signed-off-by: Matthias Roels <[email protected]>
    @MatthiasRoels
    MatthiasRoels committed Sep 26, 2023
    Configuration menu
    Copy the full SHA
    5d87b28 View commit details
    Browse the repository at this point in the history

  2. fix(datasets): unit tests for dask pq dataset now leverage moto server 

    Signed-off-by: Matthias Roels <[email protected]>
    @MatthiasRoels
    MatthiasRoels committed Sep 26, 2023
    Configuration menu
    Copy the full SHA
    dd5e433 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2023

  1. build(datasets): bump moto version for server mode 

    Signed-off-by: Matthias Roels <[email protected]>
    @MatthiasRoels
    MatthiasRoels committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    73a5ad2 View commit details
    Browse the repository at this point in the history

  2. fix(datasets): apply linting to test files 

    Signed-off-by: Matthias Roels <[email protected]>
    @MatthiasRoels
    MatthiasRoels committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    f08fce6 View commit details
    Browse the repository at this point in the history

  3. fix(datasets): unit tests for video dataset now leverage moto server 

    Signed-off-by: Matthias Roels <[email protected]>
    @MatthiasRoels
    MatthiasRoels committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    5109fb5 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2023

  1. Fix(datasets): linting errors 

    Signed-off-by: Matthias Roels <[email protected]>
    @MatthiasRoels
    MatthiasRoels committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    a5bfd4c View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2023

  1. Merge branch 'main' into build/datasets-upgrade-s3fs-to-newer-calver 

    Signed-off-by: Matthias Roels <[email protected]>
    @MatthiasRoels
    MatthiasRoels committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    51859ad View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2023

  1. Merge branch 'main' into build/datasets-upgrade-s3fs-to-newer-calver

    @MatthiasRoels
    MatthiasRoels authored Oct 11, 2023
    Configuration menu
    Copy the full SHA
    e578ad3 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2023

  1. fix(datasets): set correct CalVer for s3fs in setup.py 

    Signed-off-by: Matthias Roels <[email protected]>

Co-authored-by: Nok Lam Chan <[email protected]>
    @MatthiasRoels @noklam
    MatthiasRoels and noklam authored Oct 12, 2023
    Configuration menu
    Copy the full SHA
    308fe19 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into build/datasets-upgrade-s3fs-to-newer-calver

    @MatthiasRoels
    MatthiasRoels authored Oct 12, 2023
    Configuration menu
    Copy the full SHA
    4fb1b02 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2023

  1. Merge branch 'main' into build/datasets-upgrade-s3fs-to-newer-calver

    @MatthiasRoels
    MatthiasRoels authored Oct 18, 2023
    Configuration menu
    Copy the full SHA
    3e43613 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2023

  1. Merge branch 'main' into build/datasets-upgrade-s3fs-to-newer-calver

    @noklam
    noklam authored Nov 20, 2023
    Configuration menu
    Copy the full SHA
    31510e8 View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2023

  1. Merge branch 'main' into build/datasets-upgrade-s3fs-to-newer-calver 

    Signed-off-by: Merel Theisen <[email protected]>
    @merelcht
    merelcht authored Nov 27, 2023
    Configuration menu
    Copy the full SHA
    3bfbfdc View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2023

  1. Merge branch 'main' into build/datasets-upgrade-s3fs-to-newer-calver 

    Signed-off-by: Merel Theisen <[email protected]>
    @merelcht
    merelcht authored Nov 29, 2023
    Configuration menu
    Copy the full SHA
    66931e8 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2023

  1. Add patching for AWSResponse to make moto and aiobotocore work 

    Signed-off-by: Merel Theisen <[email protected]>
    @merelcht
    merelcht committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    619213c View commit details
    Browse the repository at this point in the history

  2. Fix lint 

    Signed-off-by: Merel Theisen <[email protected]>
    @merelcht
    merelcht committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    cee9b80 View commit details
    Browse the repository at this point in the history

  3. Add AWS mocking to lazy polars dataset test + temporarily ignore spar… 

    …k tests

Signed-off-by: Merel Theisen <[email protected]>
    @merelcht
    merelcht committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    dabd773 View commit details
    Browse the repository at this point in the history

  4. Try use same moto for all python versions 

    Signed-off-by: Merel Theisen <[email protected]>
    @merelcht
    merelcht committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    3bc77e0 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into build/datasets-upgrade-s3fs-to-newer-calver 

    Signed-off-by: Merel Theisen <[email protected]>
    @merelcht
    merelcht authored Nov 30, 2023
    Configuration menu
    Copy the full SHA
    e5b551d View commit details
    Browse the repository at this point in the history

  6. Undo polars changes 

    Signed-off-by: Merel Theisen <[email protected]>
    @merelcht
    merelcht committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    cf32731 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'build/datasets-upgrade-s3fs-to-newer-calver' of github.… 

    …com:MatthiasRoels/kedro-plugins into build/datasets-upgrade-s3fs-to-newer-calver
    @merelcht
    merelcht committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    32e0e31 View commit details
    Browse the repository at this point in the history

  8. chore(datasets): fix accidental reference to NumPy (kedro-org#450) 

    Signed-off-by: Deepyaman Datta <[email protected]>
    @deepyaman @merelcht
    deepyaman authored and merelcht committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    402439e View commit details
    Browse the repository at this point in the history

  9. Merge branch 'main' into build/datasets-upgrade-s3fs-to-newer-calver

    @merelcht
    merelcht authored Nov 30, 2023
    Configuration menu
    Copy the full SHA
    1857887 View commit details
    Browse the repository at this point in the history