Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve correctness and error messages for JSON functions. #2517

Merged
merged 12 commits into from
May 29, 2024

Conversation

nicktobey
Copy link
Contributor

MySQL doesn't do this and neither should we.

MySQL:

mysql> select JSON_INSERT("null", "$.a", 1);
+-------------------------------+
| JSON_INSERT("null", "$.a", 1) |
+-------------------------------+
| null                          |
+-------------------------------+
1 row in set (0.00 sec)

mysql> select JSON_INSERT("null", "$.a", 1) is null;
+---------------------------------------+
| JSON_INSERT("null", "$.a", 1) is null |
+---------------------------------------+
|                                     0 |
+---------------------------------------+

The only time we should be coercing a JSON-null document into SQL-null is for JSON_EXTRACT (for paths other than "$") and JSON_VALUE (for all paths). But these are already handled separately.

nicktobey added 2 commits May 24, 2024 15:08
…or lookup, which is handled separately.)

MySQL doesn't do this and neither should we.
@nicktobey nicktobey changed the title on't coerce JSON values containing JSON null into SQL null. Don't coerce JSON values containing JSON null into SQL null. May 24, 2024
nicktobey and others added 8 commits May 25, 2024 13:28
…ON_CONTAINS, and JSON_CONTAINS_PATH.

JSON_VALUE is the only function that always returns SQL NULL if the document is JSON NULL.
…JSON, or are the wrong type.

They now more closely match MySQL by printing the name of the function and the position of the incorrect parameter.
@nicktobey
Copy link
Contributor Author

So actually, it turns out that the only function that has any exceptional behavior around null documents is JSON_VALUE, which always returns SQL NULL for lookups on a document that is JSON NULL.

In all other cases, what we're seeing is the natural consequence of the fact that:

  • Doing a lookup on a path that doesn't exist results in SQL NULL
  • For a scalar document, the only valid lookup path is "$"

This PR now incorporates a couple of fixes that are just related enough that separating them into individual PRs would be a hassle:

  • We're now stricter about types: most JSON functions require that the input is either a JSON column, or a string which can be coerced to JSON. Other types should result in an error, and should not be implicitly converted to JSON.
  • Improved error messages to match MySQL when JSON functions are called with incorrect input.
  • Avoid coercing JSON NULL to SQL NULL in the results of these functions: MySQL doesn't do it and neither should we.

There's another issue I encountered during this, which I documented but am not attempting to fix with this PR: dolthub/dolt#7905

@nicktobey nicktobey changed the title Don't coerce JSON values containing JSON null into SQL null. Improve correctness and error messages for JSON functions. May 25, 2024
Copy link

Additional work is required for integration with Dolt.
Additional work is required for integration with DoltgreSQL.

Copy link

Additional work is required for integration with Dolt.
Additional work is required for integration with DoltgreSQL.

Copy link
Contributor

@max-hoffman max-hoffman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, i didn't individually check the errors against MySQL but the messages look more useful. I have also noticed weird path behavior, I haven't spent time trying to find an optimal solution but nudging in the right direction seems good.

@nicktobey nicktobey merged commit 5e36924 into main May 29, 2024
8 checks passed
@nicktobey nicktobey deleted the nicktobey/sql-null branch May 29, 2024 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants