Consolidate sql examples #11173
Consolidate sql examples #11173
Conversation
| @@ -1,51 +0,0 @@ | |||
| // Licensed to the Apache Software Foundation (ASF) under one | |||
There was a problem hiding this comment.
examples were inlined
| @@ -109,3 +129,135 @@ async fn example_read_csv_file_with_schema(file_path: &str) -> DataFrame { | |||
| // Register a lazy DataFrame by using the context and option provider | |||
| ctx.read_csv(file_path, csv_read_option).await.unwrap() | |||
| } | |||
|
|
|||
There was a problem hiding this comment.
I think putting the related functionality together in one example will make it easier for people to find what they are looking for (especially as the number of examples in DataFusion continues to grow)
| @@ -15,6 +15,10 @@ | |||
| // specific language governing permissions and limitations | |||
| // under the License. | |||
|
|
|||
| //! This example shows how to use (only) the DataFusion SQL parser | |||
There was a problem hiding this comment.
I renamed this example (not it is in datafusion-sql not datafusion core) and added some comments about what it does
| // specific language governing permissions and limitations | ||
| // under the License. | ||
|
|
||
| //! This file contains several examples of how to run SQL queries using DataFusion |
There was a problem hiding this comment.
I moved all the sql examples here and added some introductory material
| @@ -109,3 +129,135 @@ async fn example_read_csv_file_with_schema(file_path: &str) -> DataFrame { | |||
| // Register a lazy DataFrame by using the context and option provider | |||
| ctx.read_csv(file_path, csv_read_option).await.unwrap() | |||
| } | |||
|
|
|||
| /// This example demonstrates how to use the to_date series | |||
| /// of functions in the DataFrame API | |||
There was a problem hiding this comment.
Some of the consolidated examples had both SQL and dataframe examples, so I split such examples into dataframe.rs and sql.rs
|
@lewiszlw do you have time to review this large (but mechanical) PR? |
|
I agree current examples are hard to navigate. What about moving these examples under sql directory? I am concerned that the |
That is a good point 🤔 And I suppose a single large .rs file is unlikely to be very easy to use / find either 🤔 Perhaps what would be ideal would be to move the examples into the library guide so they can live alongside the actual docs. https://github.com/apache/datafusion/tree/main/docs/source/library-user-guide That way it would be easier to find them. However, I think we would have to make sure they are automatically tested: #1813 I did do some digging through old PRs, here is an example that seems to do such a test #2018 I'll see if I can use that same approach... |
|
Converting to a draft while working on this one |
|
I played around with what we have and it turns out I think we can test examples in the user guide. I made a PR with docs of how to do it: #11178 Perhaps with that as a framework, we can actually move many of the rust examples inline into the user guide -- I think that would be a better docs experience |
This looks like a massive change, but it is really just moving code around
Which issue does this PR close?
Part of #11172
Rationale for this change
Fewer example files will make it easier to find what you are looking for
What changes are included in this PR?
Consolidate sql examples into
datafusion-examples/examples/sql.rsinstead ofAre these changes tested?
By existing CI job
Are there any user-facing changes?
Hopefully easier to navigate / find examples