Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Levels of SQL Execution blog post #6821

Merged
merged 1 commit into from
Jan 23, 2025
Merged

Conversation

joellabes
Copy link
Contributor

What are you changing in this pull request and why?

We did a cool acquisition and I think people should know about it

Checklist

@joellabes joellabes requested a review from jasnonaz January 23, 2025 17:33
@joellabes joellabes requested a review from a team as a code owner January 23, 2025 17:33
Copy link

vercel bot commented Jan 23, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
docs-getdbt-com ✅ Ready (Inspect) Visit Preview Jan 23, 2025 5:36pm

@github-actions github-actions bot added content Improvements or additions to content developer blog This content fits on the developer blog. size: medium This change will take up to a week to address labels Jan 23, 2025

<!-- truncate -->

## What is SQL comprehension?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[custom.SentenceCaseHeaders] 'What is SQL comprehension?' should use sentence-style capitalization. Try 'What is SQL comprehension' instead.


By building on top of tools that truly understand SQL, it is possible to create systems that are much more capable, resilient and flexible than we’ve seen to date.

## The Levels of SQL Comprehension
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[custom.SentenceCaseHeaders] 'The Levels of SQL Comprehension' should use sentence-style capitalization. Try 'The levels of SQL comprehension' instead.


Finally, at Level 3, you can actually **execute a query and modify data**, because it understands all the complexities involved in answering the question "how does the exact data passed into this query get transformed/mutated".

## Can I see an example?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[custom.SentenceCaseHeaders] 'Can I see an example?' should use sentence-style capitalization. Try 'Can i see an example' instead.


In a toy example like this one, the distinctions between the different levels might feel subtle. As you move away from a single query and into a full-scale project, the functionality gaps become more pronounced. That’s hard to demonstrate in a blog post, but fortunately there’s another easier option: look at some failing queries. How the query is broken impacts what level of tool is necessary to recognize the error.

## So let’s break things
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[custom.SentenceCaseHeaders] 'So let’s break things' should use sentence-style capitalization. Try 'So let s break things' instead.


Each of the above queries requires progressively greater SQL comprehension abilities to identify the mistake.

### Parser (Level 1): Capture Syntax Errors
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[custom.SentenceCaseHeaders] 'Parser (Level 1): Capture Syntax Errors' should use sentence-style capitalization. Try 'Parser level 1): Capture syntax errors' instead.


Parsers know that `selecte` is **not a valid keyword** in Snowflake SQL, and will reject it.

### Compiler (Level 2): Capture Compilation Errors
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[custom.SentenceCaseHeaders] 'Compiler (Level 2): Capture Compilation Errors' should use sentence-style capitalization. Try 'Compiler level 2): Capture compilation errors' instead.

- It knows that the second argument (`value`) needs to be a number, but that `getdate()` returns a `timestamp_ltz`.
- Likewise, it knows that a number is not a valid date/time expression for the third argument.

### Executor (Level 3): Capture Data Errors
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[custom.SentenceCaseHeaders] 'Executor (Level 3): Capture Data Errors' should use sentence-style capitalization. Try 'Executor level 3): Capture data errors' instead.


But this time the compiler also thinks everything is fine! Remember that a compiler checks the signature of a function. It knows that `cast` takes a source expression and a target datatype as arguments, and it's checked that both these arguments are of the correct type.

It even has an overload that knows that strings can be cast into dates, but since it can’t do any validation of those strings’ *values* it doesn’t know **January 32nd isn’t a valid date**.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[custom.Typos] Oops there's a typo -- did you really mean '32nd'?

Copy link
Contributor

@jasnonaz jasnonaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@jasnonaz jasnonaz enabled auto-merge January 23, 2025 17:35
@jasnonaz jasnonaz merged commit 749c2af into current Jan 23, 2025
11 checks passed
@jasnonaz jasnonaz deleted the joellabes-sql-comprehension-blog branch January 23, 2025 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content Improvements or additions to content developer blog This content fits on the developer blog. size: medium This change will take up to a week to address
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants