IA for Bokeh demos and examples

I'd like to propose the simplifying the IA of our main repos demos and examples. The reasons are:

• We have way too much duplication and lack of focus that contributes heavily to the testing and maintenance burden. In particular something should be in notebooks because that format offers something in particular, not just to duplicate other existing examples in a notebook.

• It is way too hard for users to navigate the docs and examples in general. Need to streamline the examples but also make them searchable and indexable.

My proposal is as follows:

bokeh
   |
   +---tests
   |      +---models         # dull, for strict image diff tests / refguide
   |      +---notebooks      # strict tests when/where feasible
   |
   +---examples
          +---app           
          +---charts       
          +---compat         # combine all examples into one dir
          +---embed
          +---interactions   # all demo notebooks in main repo go here
          +---models         # replaces glyphs for docs consistency
          +---plotting

• Under bokeh/tests there are single-purpose scripts and notebooks that are intended for strict image diff testing, as well as for use in documentation (e.g. bokeh-plot directives in the reference guide). We can certainly reference/link them as well, but they are not especially interesting.

• For the sake of documentation consistency, examples/glyphs will be renamed examples/models. Will make an effort to search #bokeh tagged discussions on SO and publicize on ML and Release Notes and Twitter.

• The charts, models, and plotting directories have examples that output HTML file output.

• the subdirectories in compat only have a handful of files each. They are merged, and filenames are used to differentiate examples. The MPL examples are improved to only contain "interesting" (less synthetic/arbitrary) examples, ideally using verbatim examples from MPL gallery.

• The embed contains examples of using bokeh.embed (what it is now, essentially)

• The interactions directory will be renamed interactive and contains examples that involve or highlight user interactions in some way:

  • Bokeh in Jupyter notebooks at all
  • Widgets examples that are not apps
  • Server examples that are not apps
  • Linked behaviors (explanatory notebook)
  • Hover tools (explanatory notebook)
  • Jupyter notebook comms / interactors
  • JS callbacks (explanatory notebook)
  • Others as appropriate

  Might make sense to have subdirectories here.

• The disposition of any charts/noteboooks examples that we'd like to keep would be to go to bokeh-notebooks repo so that they can be presented and linked to on NBViewer / Anaconda Cloud.

Questions / Misc Comments

Where do examples like the AJAX data source once go? It is currently in interactions but that's not a very good description for it. Make a separate streaming or animated subdirectory? Rename interactions to something more general?

All script examples should get a standard docstring (or possibly comment block) that is appropriate for it so that additional information can go in the gallery page for the example. Is something similar possible for the notebooks? What does this standard block look like, roughly?

All script examples should be indexed for functions they use as well as any human curated tags (perhaps in a special comment or the docstring), so that links to examples for specific topics or API functions can be generated automatically. What is the format of the custom topic tagging? Is something similar possible for the notebooks?