Skip to content

django-postgres-extra with support for Django 5+ #257

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 56 additions & 68 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,101 +1,89 @@
# django-pg-extra-extended (Fork for Django 5+)

<h1 align="center">
<img width="400" src="https://i.imgur.com/79S6OVM.png" alt="django-postgres-extra">
</h1>


This is a fork of `django-postgres-extra`, updated to support Django 5+ while maintaining PostgreSQL enhancements for the Django ORM.

| | | |
|--------------------|---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| :white_check_mark: | **Tests** | [![CircleCI](https://circleci.com/gh/SectorLabs/django-postgres-extra/tree/master.svg?style=svg)](https://circleci.com/gh/SectorLabs/django-postgres-extra/tree/master) |
| :memo: | **License** | [![License](https://img.shields.io/:license-mit-blue.svg)](http://doge.mit-license.org) |
| :package: | **PyPi** | [![PyPi](https://badge.fury.io/py/django-postgres-extra.svg)](https://pypi.python.org/pypi/django-postgres-extra) |
| :four_leaf_clover: | **Code coverage** | [![Coverage Status](https://coveralls.io/repos/github/SectorLabs/django-postgres-extra/badge.svg?branch=coveralls)](https://coveralls.io/github/SectorLabs/django-postgres-extra?branch=master) |
| <img src="https://cdn.iconscout.com/icon/free/png-256/django-1-282754.png" width="22px" height="22px" align="center" /> | **Django Versions** | 2.0, 2.1, 2.2, 3.0, 3.1, 3.2, 4.0, 4.1, 4.2, 5.0 |
| <img src="https://cdn3.iconfinder.com/data/icons/logos-and-brands-adobe/512/267_Python-512.png" width="22px" height="22px" align="center" /> | **Python Versions** | 3.6, 3.7, 3.8, 3.9, 3.10, 3.11 |
| :package: | **PyPi** | (Coming soon) |
| <img src="https://cdn.iconscout.com/icon/free/png-256/django-1-282754.png" width="22px" height="22px" align="center" /> | **Django Versions** | 5.0+ |
| <img src="https://cdn3.iconfinder.com/data/icons/logos-and-brands-adobe/512/267_Python-512.png" width="22px" height="22px" align="center" /> | **Python Versions** | 3.8, 3.9, 3.10, 3.11, 3.12 |
| <img src="https://pbs.twimg.com/profile_images/1152122059/psycopg-100_400x400.png" width="22px" height="22px" align="center" /> | **Psycopg Versions** | 2, 3 |
| :book: | **Documentation** | [Read The Docs](https://django-postgres-extra.readthedocs.io/en/master/) |
| :warning: | **Upgrade** | [Upgrade from v1.x](https://django-postgres-extra.readthedocs.io/en/master/major_releases.html#new-features)
| :checkered_flag: | **Installation** | [Installation Guide](https://django-postgres-extra.readthedocs.io/en/master/installation.html) |
| :fire: | **Features** | [Features & Documentation](https://django-postgres-extra.readthedocs.io/en/master/index.html#features) |
| :droplet: | **Future enhancements** | [Potential features](https://github.com/SectorLabs/django-postgres-extra/issues?q=is%3Aopen+is%3Aissue+label%3Aenhancement) |

`django-postgres-extra` aims to make all of PostgreSQL's awesome features available through the Django ORM. We do this by taking care of all the hassle. As opposed to the many small packages that are available to try to bring a single feature to Django with minimal effort. ``django-postgres-extra`` goes the extra mile, with well tested implementations, seamless migrations and much more.

With seamless we mean that any features we add will work truly seamlessly. You should not have to manually modify your migrations to work with fields and objects provided by this package.

---

:warning: **This README is for v2. See the `v1` branch for v1.x.**

---

## Major features
| :book: | **Documentation** | (Coming soon) |
| :fire: | **Features** | [Features & Documentation](https://github.com/MONSTER-HARSH/django-pg-extra-extended/) |
| :droplet: | **Future enhancements** | [Potential features](https://github.com/MONSTER-HARSH/django-pg-extra-extended/issues?q=is%3Aopen+is%3Aissue+label%3Aenhancement) |

[See the full list](http://django-postgres-extra.readthedocs.io/#features)
## About

* **Native upserts**
This fork of `django-postgres-extra` extends support for Django 5+, keeping all the powerful PostgreSQL features, including:

* Single query
* Concurrency safe
* With bulk support (single query)
- **Native upserts** with bulk support
- **Extended support for HStoreField** (unique constraints, null constraints, etc.)
- **Declarative table partitioning** for PostgreSQL 11+
- **Faster deletes** using table truncation
- **Advanced indexing options** (conditional and case-sensitive unique indexes)

* **Extended support for HStoreField**
## Installation

* Unique constraints
* Null constraints
* Select individual keys using ``.values()`` or ``.values_list()``
Coming soon to PyPI.

* **PostgreSQL 11.x declarative table partitioning**
For now, install directly from GitHub:

* Supports both range and list partitioning
```sh
pip install git+https://github.com/MONSTER-HARSH/django-pg-extra-extended.git
```

* **Faster deletes**
## Getting Started

* Truncate tables (with cascade)

* **Indexes**

* Conditional unique index.
* Case sensitive unique index.

## Working with the code
### Prerequisites
1. Clone the repository:

* PostgreSQL 10 or newer.
* Django 2.0 or newer (including 3.x, 4.x).
* Python 3.6 or newer.
```sh
git clone https://github.com/MONSTER-HARSH/django-pg-extra-extended.git
```

### Getting started
2. Create a virtual environment:

1. Clone the repository:
```sh
cd django-pg-extra-extended
python -m venv env
source env/bin/activate
```

λ git clone https://github.com/SectorLabs/django-postgres-extra.git
3. Create a PostgreSQL user for testing:

2. Create a virtual environment:
```sh
createuser --superuser psqlextra --pwprompt
export DATABASE_URL=postgres://psqlextra:<password>@localhost/psqlextra
```

λ cd django-postgres-extra
λ virtualenv env
λ source env/bin/activate
4. Install dependencies:

3. Create a postgres user for use in tests (skip if your default user is a postgres superuser):
```sh
pip install .[test] .[analysis]
```

λ createuser --superuser psqlextra --pwprompt
λ export DATABASE_URL=postgres://psqlextra:<password>@localhost/psqlextra
5. Run tests:

Hint: if you're using virtualenvwrapper, you might find it beneficial to put
the ``export`` line in ``$VIRTUAL_ENV/bin/postactivate`` so that it's always
available when using this virtualenv.
```sh
tox
```

4. Install the development/test dependencies:
## Migration from Original django-postgres-extra

λ pip install .[test] .[analysis]
If you're upgrading from `django-postgres-extra` (SectorLabs version), ensure that:

5. Run the tests:
- You update the package source to this fork.
- You check for any API changes due to Django 5+ compatibility adjustments.

λ tox
## Contributing

6. Run the benchmarks:
Contributions are welcome! Please open an issue or submit a pull request.

λ py.test -c pytest-benchmark.ini
## License

7. Auto-format code, sort imports and auto-fix linting errors:
MIT License. See [LICENSE](LICENSE) for details.

λ python setup.py fix
15 changes: 0 additions & 15 deletions psqlextra/__init__.py
Original file line number Diff line number Diff line change
@@ -1,15 +0,0 @@
import django

from ._version import __version__

if django.VERSION < (3, 2): # pragma: no cover
default_app_config = "psqlextra.apps.PostgresExtraAppConfig"

__all__ = [
"default_app_config",
"__version__",
]
else:
__all__ = [
"__version__",
]
23 changes: 0 additions & 23 deletions psqlextra/backend/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@
from typing import TYPE_CHECKING

from django.conf import settings
from django.contrib.postgres.signals import (
get_hstore_oids,
register_type_handlers,
)
from django.db import ProgrammingError

from . import base_impl
Expand Down Expand Up @@ -98,22 +94,3 @@ def prepare_database(self):
"or add the extension manually.",
exc_info=True,
)
return

# Clear old (non-existent), stale oids.
get_hstore_oids.cache_clear()

# Verify that we (and Django) can find the OIDs
# for hstore.
oids, _ = get_hstore_oids(self.alias)
if not oids:
logger.warning(
'"hstore" extension was created, but we cannot find the oids'
"in the database. Something went wrong.",
)
return

# We must trigger Django into registering the type handlers now
# so that any subsequent code can properly use the newly
# registered types.
register_type_handlers(self)
2 changes: 1 addition & 1 deletion psqlextra/backend/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -1045,7 +1045,7 @@ def _partitioning_properties_for_model(model: Type[Model]):
% (model.__name__, meta.method)
)

if not isinstance(meta.key, (list, tuple)):
if not isinstance(meta.key, list):
raise ImproperlyConfigured(
(
"Model '%s' is not properly configured to be partitioned."
Expand Down
14 changes: 3 additions & 11 deletions psqlextra/manager/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from psqlextra.query import PostgresQuerySet


class PostgresManager(Manager.from_queryset(PostgresQuerySet)): # type: ignore[misc]
class PostgresManager(Manager.from_queryset(PostgresQuerySet)):
"""Adds support for PostgreSQL specifics."""

use_in_migrations = True
Expand Down Expand Up @@ -37,10 +37,7 @@ def __init__(self, *args, **kwargs):
)

def truncate(
self,
cascade: bool = False,
restart_identity: bool = False,
using: Optional[str] = None,
self, cascade: bool = False, using: Optional[str] = None
) -> None:
"""Truncates this model/table using the TRUNCATE statement.

Expand All @@ -54,19 +51,14 @@ def truncate(
False, an error will be raised if there
are rows in other tables referencing
the rows you're trying to delete.
restart_identity:
Automatically restart sequences owned by
columns of the truncated table(s).
"""

connection = connections[using or "default"]
table_name = connection.ops.quote_name(self.model._meta.db_table)

with connection.cursor() as cursor:
sql = f"TRUNCATE TABLE {table_name}"
sql = "TRUNCATE TABLE %s" % table_name
if cascade:
sql += " CASCADE"
if restart_identity:
sql += " RESTART IDENTITY"

cursor.execute(sql)
5 changes: 1 addition & 4 deletions psqlextra/models/base.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,4 @@
from typing import Any

from django.db import models
from django.db.models import Manager

from psqlextra.manager import PostgresManager

Expand All @@ -13,4 +10,4 @@ class Meta:
abstract = True
base_manager_name = "objects"

objects: "Manager[Any]" = PostgresManager()
objects = PostgresManager()
10 changes: 2 additions & 8 deletions psqlextra/models/partitioned.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
from typing import Iterable

from django.db.models.base import ModelBase

from psqlextra.types import PostgresPartitioningMethod
Expand All @@ -17,7 +15,7 @@ class PostgresPartitionedModelMeta(ModelBase):
"""

default_method = PostgresPartitioningMethod.RANGE
default_key: Iterable[str] = []
default_key = []

def __new__(cls, name, bases, attrs, **kwargs):
new_class = super().__new__(cls, name, bases, attrs, **kwargs)
Expand All @@ -34,14 +32,10 @@ def __new__(cls, name, bases, attrs, **kwargs):
return new_class


class PostgresPartitionedModel(
PostgresModel, metaclass=PostgresPartitionedModelMeta
):
class PostgresPartitionedModel(PostgresModel, metaclass=PostgresPartitionedModelMeta):
"""Base class for taking advantage of PostgreSQL's 11.x native support for
table partitioning."""

_partitioning_meta: PostgresPartitionedModelOptions

class Meta:
abstract = True
base_manager_name = "objects"
Loading