Documentation Updates

[Rachel-Reverie]

Hello!

We've been working on improving the condition of our documentation (you may have seen the updated tutorial) and we have another big update coming out. Since this is intended to make things easier for the community, we'd like to get your feedback on the changes.

The work is in PR 4292, and you can see a netlify preview of the documentation as well.

There are a few changes and improvements this time around.

First, we've reorganized the table of contents to follow the "Setup, Connect to data, Create Expectations, Validate data" workflow that was introduced in the updated tutorial. Each step on that path has it's own overview page to accompany the more detailed how-to guides.

Second, we've updated our reference section! There is now a Glossary of Terms that covers important Great Expectations technical terms, all in one place, that you can reference while going through the docs.

Third, every technical term in the Glossary of Terms links to a Technical Term page. These pages give an in depth view of the object being discussed: it's definition, it's purpose, what it interacts with, as well as how and when you'll use it in the "Setup, Connect to data, Create Expectations, Validate data" workflow.

And finally, something that we're previewing in the Technical Term pages but hope to include in all of our documentation soon: When a technical term is used for the first time on a page it is automatically linked to the in-depth Technical Term page, and given hover-text with it's definition. We hope that this will facilitate reading the docs when you just need a quick reminder of what a term means as well as making it easier to dig deep into the workings of Great Expectations if you decide to look for more detail on any given object.

Once more, here's the link to the netlify preview for the documentation update.

Please check these changes out, and let us know what you think!

[jakeberesford-palmetto]

This seems like a solid improvement to the docs, thank you!

One thing I'd call out is there are links to documentation scattered throughout the source code. Having worked with great_expectations for ~6months I've come to learn that many of these links are dead or link to legacy docs. With this change, I suspect many more of those links will be dead. Updating / removing those links throughout the codebase would be a great follow-up PR.

[Rachel-Reverie]

Thanks for the feedback! Yes, that is one of the items we need to address with our documentation -- and it is in my queue. None of the links that previously existed will "die" from this update, though: I removed some pages from the online documentation's table of contents, but those pages are still being hosted (and will continue to be until I've gone through the links in our source code and updated them for the new pages, and added a comprehensive suite of redirects for anyone who is using an older version of Great Expectations that doesn't have the updated links). After that I'll make a pass looking for any links to legacy docs that need updated.

[lucasasmith]

Nice 😃. I like the division of the topics into the explicit steps of the GE workflow. It seems like it'll be more clear for newcomers.

Just a personal prefernce but I actually like the Changelog in its current existing section, but understand if it's moved under another section.

[Rachel-Reverie]

Thanks for the input! My thought was that having the changelog alongside the migration guide would be beneficial since the two are related. An alternative, I suppose, would be to put the changelog back at the top level of the table of contents and move the migration guide into the "supplemental documentation" under the reference section -- but I do think that keeping the two closely associated with each other would be more beneficial for anyone who realizes they need to find the migration guide after taking a look at the changelog!

[john-schwegler-vectorsolutions]

This definitely looks like a nice upgrade! I'm just introducing our developers to using Great Expectations, and I think this will make it easier to get them started.

On the links in code (etc) issue, you might want to have a generic "landing zone" of standard link URLs, that can then be repointed to the actual documentation as it gets changed, restructured, etc.

[Rachel-Reverie]

I'm glad this looks to be helpful for your developers!

Interesting thought on the links in code. If I'm understanding you right, you're suggesting having a set of links that don't actually correspond to any of the existing pages as our "in code" links, and then handle getting anyone who clicks on them with a redirect list? (If I'm misunderstanding, please let me know!)

That would be easier than hunting for links in the code in the future, since they'd all be in one place to edit in the redirects file. And it would save us from an ever-growing redirects file, since it would save us from having to add a redirect rule every time a document was replaced with a new page to account for anyone using older code with the old links.

I'll bring that up as an option to consider, thanks!

[john-schwegler-vectorsolutions]

Yes, that's right. You can decide on a set of "standard" links that you need, and then redirect those to actual relevant pages at the time they're used. If you have a list of those standard links, it should be easier to manage over time (and keep relevant).

[Rachel-Reverie]

Absolutely brilliant. Thank you for introducing me to this method for keeping our links manageable in the future!

[john-schwegler-vectorsolutions]

Happy to help, I think your team is developing great product.

[Calvo94]

This is an amazing upgrade 🙌 I loved the diagrames on the overviews.

[Rachel-Reverie]

Thank you! One of the items in our ongoing updates queue is adding additional diagrams into the Technical Terms pages as well.