Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python312Packages.pytest-django: drop xdist again due to too many rac… #336569

Closed

Conversation

SuperSandro2000
Copy link
Member

…e conditions

Description of changes

I think we exposed some racy tests here: I tried building it multiple times and every time some other tests have failed:

=================================== FAILURES ===================================
_________________ Test_django_db_blocker.test_unblock_manually _________________
[gw87] linux -- Python 3.12.4 /nix/store/04gg5w1s662l329a8kh9xcwyp0k64v5a-python3-3.12.4/bin/python3.12

self = <django.db.backends.utils.CursorWrapper object at 0x7ffff4a9a3f0>
sql = 'SELECT %s AS "a" FROM "app_item" LIMIT 1', params = (1,)
ignored_wrapper_args = (False, {'connection': <DatabaseWrapper vendor='sqlite' alias='default'>, 'cursor': <django.db.backends.utils.CursorWrapper object at 0x7ffff4a9a3f0>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
                # params default might be backend specific.
                return self.cursor.execute(sql)
            else:
>               return self.cursor.execute(sql, params)

/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/utils.py:89: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <django.db.backends.sqlite3.base.SQLiteCursorWrapper object at 0x7ffff47ad6d0>
query = 'SELECT ? AS "a" FROM "app_item" LIMIT 1', params = (1,)

    def execute(self, query, params=None):
        if params is None:
            return super().execute(query)
        # Extract names if params is a mapping, i.e. "pyformat" style is used.
        param_names = list(params) if isinstance(params, Mapping) else None
        query = self.convert_query(query, param_names=param_names)
>       return super().execute(query, params)
E       sqlite3.OperationalError: no such table: app_item

/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/sqlite3/base.py:328: OperationalError

The above exception was the direct cause of the following exception:

self = <tests.test_fixtures.Test_django_db_blocker object at 0x7ffff477e3c0>
django_db_blocker = <pytest_django.plugin.DjangoDbBlocker object at 0x7ffff5cd7d10>

    def test_unblock_manually(self, django_db_blocker: DjangoDbBlocker) -> None:
        try:
            django_db_blocker.unblock()
>           Item.objects.exists()

tests/test_fixtures.py:719: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/models/manager.py:87: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/models/query.py:1241: in exists
    return self.query.has_results(using=self.db)
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/models/sql/query.py:598: in has_results
    return compiler.has_results()
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/models/sql/compiler.py:1530: in has_results
    return bool(self.execute_sql(SINGLE))
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/models/sql/compiler.py:1562: in execute_sql
    cursor.execute(sql, params)
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/utils.py:67: in execute
    return self._execute_with_wrappers(
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/utils.py:80: in _execute_with_wrappers
    return executor(sql, params, many, context)
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/utils.py:84: in _execute
    with self.db.wrap_database_errors:
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/utils.py:91: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/utils.py:89: in _execute
    return self.cursor.execute(sql, params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <django.db.backends.sqlite3.base.SQLiteCursorWrapper object at 0x7ffff47ad6d0>
query = 'SELECT ? AS "a" FROM "app_item" LIMIT 1', params = (1,)

    def execute(self, query, params=None):
        if params is None:
            return super().execute(query)
        # Extract names if params is a mapping, i.e. "pyformat" style is used.
        param_names = list(params) if isinstance(params, Mapping) else None
        query = self.convert_query(query, param_names=param_names)
>       return super().execute(query, params)
E       django.db.utils.OperationalError: no such table: app_item

/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/sqlite3/base.py:328: OperationalError
________________ Test_django_db_blocker.test_unblock_with_block ________________
[gw87] linux -- Python 3.12.4 /nix/store/04gg5w1s662l329a8kh9xcwyp0k64v5a-python3-3.12.4/bin/python3.12

self = <django.db.backends.utils.CursorWrapper object at 0x7ffff4ac65d0>
sql = 'SELECT %s AS "a" FROM "app_item" LIMIT 1', params = (1,)
ignored_wrapper_args = (False, {'connection': <DatabaseWrapper vendor='sqlite' alias='default'>, 'cursor': <django.db.backends.utils.CursorWrapper object at 0x7ffff4ac65d0>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
                # params default might be backend specific.
                return self.cursor.execute(sql)
            else:
>               return self.cursor.execute(sql, params)

/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/utils.py:89: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <django.db.backends.sqlite3.base.SQLiteCursorWrapper object at 0x7ffff461f020>
query = 'SELECT ? AS "a" FROM "app_item" LIMIT 1', params = (1,)

    def execute(self, query, params=None):
        if params is None:
            return super().execute(query)
        # Extract names if params is a mapping, i.e. "pyformat" style is used.
        param_names = list(params) if isinstance(params, Mapping) else None
        query = self.convert_query(query, param_names=param_names)
>       return super().execute(query, params)
E       sqlite3.OperationalError: no such table: app_item

/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/sqlite3/base.py:328: OperationalError

The above exception was the direct cause of the following exception:

self = <tests.test_fixtures.Test_django_db_blocker object at 0x7ffff477e750>
django_db_blocker = <pytest_django.plugin.DjangoDbBlocker object at 0x7ffff5cd7d10>

    def test_unblock_with_block(self, django_db_blocker: DjangoDbBlocker) -> None:
        with django_db_blocker.unblock():
>           Item.objects.exists()

tests/test_fixtures.py:725: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/models/manager.py:87: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/models/query.py:1241: in exists
    return self.query.has_results(using=self.db)
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/models/sql/query.py:598: in has_results
    return compiler.has_results()
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/models/sql/compiler.py:1530: in has_results
    return bool(self.execute_sql(SINGLE))
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/models/sql/compiler.py:1562: in execute_sql
    cursor.execute(sql, params)
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/utils.py:67: in execute
    return self._execute_with_wrappers(
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/utils.py:80: in _execute_with_wrappers
    return executor(sql, params, many, context)
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/utils.py:84: in _execute
    with self.db.wrap_database_errors:
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/utils.py:91: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/utils.py:89: in _execute
    return self.cursor.execute(sql, params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <django.db.backends.sqlite3.base.SQLiteCursorWrapper object at 0x7ffff461f020>
query = 'SELECT ? AS "a" FROM "app_item" LIMIT 1', params = (1,)

    def execute(self, query, params=None):
        if params is None:
            return super().execute(query)
        # Extract names if params is a mapping, i.e. "pyformat" style is used.
        param_names = list(params) if isinstance(params, Mapping) else None
        query = self.convert_query(query, param_names=param_names)
>       return super().execute(query, params)
E       django.db.utils.OperationalError: no such table: app_item

/nix/store/d9nnvdisc2d5dvqsgv48niqrp9fxsqx9-python3.12-django-4.2.15/lib/python3.12/site-packages/django/db/backends/sqlite3/base.py:328: OperationalError
=========================== short test summary info ============================
SKIPPED [2] pytest_django_test/db_helpers.py:91: Do not test db reuse since database does not support it
FAILED tests/test_fixtures.py::Test_django_db_blocker::test_unblock_manually - django.db.utils.OperationalError: no such table: app_item
FAILED tests/test_fixtures.py::Test_django_db_blocker::test_unblock_with_block - django.db.utils.OperationalError: no such table: app_item
=================== 2 failed, 213 passed, 2 skipped in 8.05s ===================

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

Copy link
Member

@dotlambda dotlambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment why pytest-xdist can't be added.

@Titaniumtown
Copy link
Contributor

I can reproduce the race-y tests. Thank you for the fix.

@SuperSandro2000
Copy link
Member Author

Integrating into the 4.9.0 update.

@SuperSandro2000 SuperSandro2000 deleted the pytest-django-xdist branch October 29, 2024 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants