Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix UUID handling in OData queries #60

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

AHMETCEMRE
Copy link

Pull Request Description

This pull request addresses the issue of UUID handling in odata-query with SQLAlchemy, ensuring that UUIDs are treated correctly in query filters, which was previously causing errors. My solution is particularly relevant to the ongoing discussion in Issue #25 ("Filter not working for uuid column") raised by schwingkopf and others, where similar issues were encountered when filtering on UUID columns.

Problem Statement:

In my case, I encountered an issue where filtering by UUID columns in a PostgreSQL database resulted in the following error:

sqlalchemy.exc.ProgrammingError: (psycopg.errors.UndefinedFunction) operator does not exist: uuid = character varying
HINT: No operator matches the given name and argument types.

This occurred because odata-query was generating SQL queries that incorrectly treated UUIDs as VARCHAR types, leading to invalid SQL statements. The error message highlighted that the UUID type was being compared with a VARCHAR, which is not allowed.

This issue mirrors the one reported in Issue #25, where filtering by UUID columns produced no results due to a similar type mismatch.

Changes Made:

  1. grammar.py Changes:

    • Introduced a new UUID pattern (_UUID_PATTERN) to properly recognize UUIDs with or without quotes.
      _UUID_PATTERN = r"'?[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12}'?"
    • Updated the GUID method to remove unnecessary quotes and parse UUIDs as GUID objects:
      t.value = ast.GUID(t.value.strip("'"))

    This change ensures that UUIDs are treated correctly in the query parsing phase.

  2. common.py Changes:

    • Modified the visit_GUID method to ensure UUIDs are passed as UUID objects in SQLAlchemy queries instead of VARCHAR strings:
      return literal(node.py_val, type_=Uuid())

    This is essential to prevent the type mismatch that was causing the errors. As mentioned in Issue Filter not working for uuid column #25, SQLAlchemy expects UUIDs to be handled in a specific way, and this change resolves the mismatch by ensuring that the UUID remains in its proper form throughout query generation.

  3. Test Changes:
    Updated tests in both test_odata_to_sqlalchemy_core.py and test_odata_to_sqlalchemy_orm.py:

    • Updated UUID comparisons to use UUID objects:

      BlogPost.c.id == uuid.UUID("a7af27e6-f5a0-11e9-9649-0a252986adba")
    • Updated IN clause tests:

      BlogPost.c.id.in_([
          literal(uuid.UUID('a7af27e6-f5a0-11e9-9649-0a252986adba')),
          literal(uuid.UUID('800c56e4-354d-11eb-be38-3af9d323e83c'))
      ])

    These changes ensure that the UUIDs are passed in their correct type during tests, which prevents the type mismatch errors that were mentioned in Issue Filter not working for uuid column #25 when handling UUIDs in IN clauses.


Contribution to the Discussion:

This pull request contributes directly to the ongoing conversation in Issue #25, where both I and other users faced similar UUID-related problems. The problem, as described in the issue, revolves around handling UUIDs properly in queries, particularly ensuring they are treated as UUID types instead of strings or VARCHARs.

In my solution, I upgraded SQLAlchemy to version ^2.0, which offers better support for UUIDs. I have tested my changes with both PostgreSQL and SQLite databases, and in both cases, I was able to successfully filter records using UUID values.

Thank you for providing such a useful library, and I hope this PR helps to resolve the issue or at least opens further discussion on UUID handling.


@AHMETCEMRE
Copy link
Author

Hi @OliverHofkens,

Again, thank you for this great project! It's been very useful for our work, but this uuid issue is quite important for us to resolve. Could you kindly review the PR at your earliest convenience?

@huaxing-w
Copy link

Thanks @OliverHofkens and @AHMETCEMRE , this is a great fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants