Create a proposal for a notebooks repository

[alimaredia]

See docs/notebooks/notebooks-repo.md in this pull request for more details.

[cdoern]

I think a decent counterproposal to this is that each component would house its own notebooks in their own repos given that it's likely they will become upstream projects in their own right housed under the instructlab org. WDYT about that?

I think the orchestrated ones make sense to be put in a centralized repo, but unsure if putting component notebooks in a repo other than the component repos themselves makes sense from the standpoint of a contributor looking for examples.

[RobotSail]

LGTM

[kelbrown20]

I had a thought similar to what @cdoern was saying. I think this makes sense for this repo to maybe contain the combined workflow notebooks. Where the stages would go in the library repos.

[bbrowning]

At a high level, we already have one example notebook in our current SDG repo and are planning multiple more, along with integration tests that test these notebooks, in the updated SDG repo. I think we'll always want our notebooks that showcase the best of SDG in an SDG-specific repo, because they will only touch on SDG and also show ways SDG may be used that are outside of just the end-to-end opinionated InstructLab flow for producing models.

With that said, I do think it's reasonable that we have some content (including notebooks) that show how all of the individual libraries can be used together. I would support a centralized place to collect some of these end-to-end examples, showing how someone can implement the opinionated InstructLab flow using notebooks as well as how they can plug their own steps and requirements into notebooks by starting with our opinions and customizing things.

So, +1 from me on a place for these cross-cutting types of notebooks, but -1 from me if the desire is to house notebooks specific to individual libraries here.

[alimaredia]

I've taken the feedback on this PR and removed the stages directory. The reason I included that was so that users had a centralized place to discover all of the libraries notebooks but there are other ways to do this, like on the community's website or on google colab that accomplish the same goal.

[nimbinatus]

A bit of bikeshedding on my part, but perhaps examples is better than notebooks so it can be more of a catchall for any and all types of examples, like a one-stop shop? Notebooks are one type of artifact, but what about if we have code examples once the SDK from https://github.com/instructlab/dev-docs/blob/main/docs/sdk/instructlab-sdk.md is built? I would think it would be good to have all of these things in one place for every user to use.

[alimaredia]

@nimbinatus great idea. I changed the repo name and dev doc a bit for flexibility sake.