Skip to content

Commit

Permalink
Merge pull request #162 from thomasmarwitz/fix-hard-dependency
Browse files Browse the repository at this point in the history
Fix hard dependency in s3fsstore
  • Loading branch information
xhochy authored Jan 24, 2025
2 parents fe23bda + e960e57 commit c590f0d
Show file tree
Hide file tree
Showing 4 changed files with 21 additions and 3 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ jobs:
run: pixi run -e build build-wheel
- name: Check package
run: pixi run -e build check-wheel
- name: Check store creation (verify that packaged version is working)
run: pixi exec -s python=3.9 -s pip sh check_store_creation.sh
- name: Upload package
uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
with:
Expand Down
11 changes: 11 additions & 0 deletions check_store_creation.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Run in a temporary pixi env to check whether the installed package can be imported and
# used to create an hfs store that doesn't require any additional dependencies.

# This check is useful because we don't want to have hard-coded dependencies in the package
# apart from "uritools". Everything else should be optional and only raise an ImportError
# if the user tries to use a specific store backend, e.g. s3://.

whl_file=$(ls dist/*.whl)
pip install $whl_file
python -c 'from minimalkv import get_store; get_store("hfs", path=".")'
echo "Store creation successful"
4 changes: 4 additions & 0 deletions docs/changes.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
Changelog
*********

1.11.1
======
* Remove hard dependency of aiobotocore in the S3FSStore.

1.11.0
======
* Add option to pass STS credentials to ``s3://`` store by setting the ``is_sts_credentials=true`` url param and passing role params via ``sts_assume_role__*``, e.g. ``sts_assume_role__RoleArn=...``.
Expand Down
7 changes: 4 additions & 3 deletions minimalkv/net/s3fsstore.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,14 @@
from minimalkv import UrlMixin
from minimalkv._url_utils import _get_password, _get_username
from minimalkv.fsspecstore import FSSpecStore
from minimalkv.net._aws_refreshable_session import (
create_aio_session_w_refreshable_credentials,
)

try:
from s3fs import S3FileSystem

from minimalkv.net._aws_refreshable_session import (
create_aio_session_w_refreshable_credentials,
) # aws refreshable session only is of interest in conjunction with s3fs

has_s3fs = True
except ImportError:
has_s3fs = False
Expand Down

0 comments on commit c590f0d

Please sign in to comment.