Skip to content

Commit

Permalink
Fixes bug introduced in issue/#34
Browse files Browse the repository at this point in the history
Due to a bug all public bucket urls were generated using the internal client. Due to this fault public urls did not respect the value of MINIO_EXTERNAL_ENDPOINT.
The old fake client was repurposed as an external client.

Additional changes:
* Dropped the `v` prefix from version numbers
* GH workflows updated to the latest Ubuntu and Python versions and updated the watched tags
* README.md updated with new Python version recommendations
  • Loading branch information
theriverman committed Mar 26, 2023
1 parent abeed73 commit 1bd19be
Show file tree
Hide file tree
Showing 5 changed files with 42 additions and 36 deletions.
8 changes: 4 additions & 4 deletions .github/workflows/publish-to-pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,20 @@ on:
- 'develop'
- 'feature/*'
tags:
- v**
- '**'

jobs:
build-n-publish:
name: Build and publish Python 🐍 distributions 📦 to PyPI and TestPyPI
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
runs-on: ubuntu-18.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@master
# Setup Python
- name: Set up Python 3.8
- name: Set up Python 3.11
uses: actions/setup-python@v1
with:
python-version: 3.8
python-version: 3.11
# Install Dependencies
- name: Install pypa/build
run: >-
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/publish-to-test-pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,20 @@ on:
branches:
- master
tags:
- v**
- '**'

jobs:
build-n-publish:
name: Build and publish Python 🐍 distributions 📦 to PyPI and TestPyPI
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
runs-on: ubuntu-18.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@master
# Setup Python
- name: Set up Python 3.8
- name: Set up Python 3.11
uses: actions/setup-python@v1
with:
python-version: 3.8
python-version: 3.11
# Install Dependencies
- name: Install pypa/build
run: >-
Expand Down
5 changes: 1 addition & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,12 +206,9 @@ See [README.Docker.md](README.Docker.md) for a real-life Docker Compose demonstr

## Compatibility
* Django 2.2 or later
* Python 3.6.0 or later
* Python 3.8.0 or later
* MinIO SDK 7.0.2 or later

**Note:** This library relies heavily on [PEP 484 -- Type Hints](https://www.python.org/dev/peps/pep-0484/)
which was introduced in *Python 3.5.0*.

## Contribution
Please find the details in [CONTRIBUTE.md](CONTRIBUTE.md)

Expand Down
51 changes: 29 additions & 22 deletions django_minio_backend/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,13 +87,13 @@ def __init__(self,
self._REPLACE_EXISTING = kwargs.get('replace_existing', False)

self.__CLIENT: Union[minio.Minio, None] = None # This client is used for internal communication only. Communication this way should not leave the host network's perimeter
self.__CLIENT_FAKE: Union[minio.Minio, None] = None # This fake client is used for pre-signed URL generation only; it does not execute HTTP requests
self.__CLIENT_EXT: Union[minio.Minio, None] = None # This client is used for external communication. This client is necessary for creating region-aware pre-signed URLs
self.__MINIO_ENDPOINT: str = get_setting("MINIO_ENDPOINT", "")
self.__MINIO_EXTERNAL_ENDPOINT: str = get_setting("MINIO_EXTERNAL_ENDPOINT", self.__MINIO_ENDPOINT)
self.__MINIO_ACCESS_KEY: str = get_setting("MINIO_ACCESS_KEY")
self.__MINIO_SECRET_KEY: str = get_setting("MINIO_SECRET_KEY")
self.__MINIO_USE_HTTPS: bool = get_setting("MINIO_USE_HTTPS")
self.__MINIO_REGION: str = get_setting("MINIO_REGION", "us-east-1") # MINIO defaults to "us-east-1" when region is set to None
self.__MINIO_REGION: str = get_setting("MINIO_REGION", "us-east-1") # MINIO defaults to "us-east-1" when region is set to None
self.__MINIO_EXTERNAL_ENDPOINT_USE_HTTPS: bool = get_setting("MINIO_EXTERNAL_ENDPOINT_USE_HTTPS", self.__MINIO_USE_HTTPS)
self.__MINIO_BUCKET_CHECK_ON_SAVE: bool = get_setting("MINIO_BUCKET_CHECK_ON_SAVE", False)

Expand Down Expand Up @@ -207,7 +207,7 @@ def stat(self, name: str) -> Union[minio.datatypes.Object, bool]:
def delete(self, name: str):
"""
Deletes an object in Django and MinIO.
This method is called only when an object is deleted from it's own `change view` ie.:
This method is called only when an object is deleted from its own `change view` ie.:
http://django.test/admin/upload/privateattachment/13/change/
This method is NOT called during a bulk_delete order!
:param name: File object name
Expand Down Expand Up @@ -248,15 +248,14 @@ def url(self, name: str):
:param name: (str) file path + file name + suffix
:return: (str) URL to object
"""
client = self.client if self.same_endpoints else self.client_external

if self.is_bucket_public:
base_url = self.client._base_url.build("GET", self.__MINIO_REGION).geturl()
# noinspection PyProtectedMember
base_url = client._base_url.build("GET", self.__MINIO_REGION).geturl()
return f'{base_url}{self.bucket}/{name}'
if self.same_endpoints:
# in this scenario the fake client is not needed
client = self.client
else:
client = self.client_fake

# private bucket
try:
u: str = client.presigned_get_object(
bucket_name=self.bucket,
Expand Down Expand Up @@ -347,13 +346,16 @@ def is_minio_available(self) -> MinioServerStatus:

@property
def client(self) -> minio.Minio:
"""Get handle to an (already) instantiated minio.Minio instance"""
"""
Get handle to an (already) instantiated minio.Minio instance. This is the default Client.
If "MINIO_EXTERNAL_ENDPOINT" != MINIO_ENDPOINT, this client is used for internal communication only
"""
return self.__CLIENT or self._create_new_client()

@property
def client_fake(self) -> minio.Minio:
"""Get handle to an (already) instantiated FAKE minio.Minio instance for generating signed URLs for external access"""
return self.__CLIENT_FAKE or self._create_new_client(fake=True)
def client_external(self) -> minio.Minio:
"""Get handle to an (already) instantiated EXTERNAL minio.Minio instance for generating pre-signed URLs for external access"""
return self.__CLIENT_EXT or self._create_new_client(external=True)

@property
def base_url(self) -> str:
Expand All @@ -365,23 +367,28 @@ def base_url_external(self) -> str:
"""Get external base URL to MinIO"""
return self.__BASE_URL_EXTERNAL

def _create_new_client(self, fake: bool = False) -> minio.Minio:
def _create_new_client(self, external: bool = False) -> minio.Minio:
"""
Instantiates a new Minio client and assigns it to their respective class variable
:param external: If True, the returned value is self.__CLIENT_EXT instead of self.__CLIENT
"""
mc = minio.Minio(
endpoint=self.__MINIO_EXTERNAL_ENDPOINT if fake else self.__MINIO_ENDPOINT,
self.__CLIENT = minio.Minio(
endpoint=self.__MINIO_ENDPOINT,
access_key=self.__MINIO_ACCESS_KEY,
secret_key=self.__MINIO_SECRET_KEY,
secure=self.__MINIO_EXTERNAL_ENDPOINT_USE_HTTPS if fake else self.__MINIO_USE_HTTPS,
secure=self.__MINIO_USE_HTTPS,
http_client=self.HTTP_CLIENT,
region=self.__MINIO_REGION,
)
if fake:
self.__CLIENT_FAKE = mc
else:
self.__CLIENT = mc
return mc
self.__CLIENT_EXT = minio.Minio(
endpoint=self.__MINIO_EXTERNAL_ENDPOINT,
access_key=self.__MINIO_ACCESS_KEY,
secret_key=self.__MINIO_SECRET_KEY,
secure=self.__MINIO_EXTERNAL_ENDPOINT_USE_HTTPS,
http_client=self.HTTP_CLIENT,
region=self.__MINIO_REGION,
)
return self.__CLIENT_EXT if external else self.__CLIENT

# MAINTENANCE
def check_bucket_existence(self):
Expand Down
6 changes: 4 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,16 @@
'Framework :: Django :: 2.2',
'Framework :: Django :: 3.0',
'Framework :: Django :: 3.1',
'Framework :: Django :: 4.0',
'Framework :: Django :: 4.1',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
'Topic :: Internet :: WWW/HTTP :: Dynamic Content',
'Topic :: Internet :: WWW/HTTP :: Dynamic Content :: Content Management System',
'Topic :: Internet :: WWW/HTTP :: WSGI :: Application',
Expand Down

0 comments on commit 1bd19be

Please sign in to comment.