Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multichartz #420

Closed
wants to merge 231 commits into from
Closed

Multichartz #420

wants to merge 231 commits into from

Conversation

goodboy
Copy link
Contributor

@goodboy goodboy commented Nov 15, 2022

The UI follow up to #414.

Hopefully this is the final core-UX feature you always wanted 🏄🏼

mult-instrument overlayed real-time and historical data feeds with simultaneous interaction and "current symbol" selectable order mode control


THE CMD YOU NEED:

piker chart btcusdt.binance ethusdt.binance xmrusdt.binance


First draft TODO

There are still tons of refinements and bugs to fix as part of this first draft

Init basic funtionality / usage:

  • basic initial usage is to start a chart with multiple input symbols:
    piker chart btcusdt.binance ethusdt.binance xmrusdt.binance
  • dynamic managment of symbols per chart view (i.e. overlayed symbols can be added/removed from a running LinkedSplits
  • more or less the idea of the currently selected symbol state as described in sub-sections below
    • select by y-axis
    • select by double click on curve?
    • select by hot key (some kinda tab or space global hotkey?)
  • loading and saving of watchlists by name which contain multiple fqsns

Fast view

  • selected feed should be closest to curve?
  • L1 labels can be simplified
    • place them all x-offset to the inner most y-axis?
    • only show (humanized) size on bid/ask not price?
  • draw little line from each last label to curve?
  • some kind of label system to easily know which curve is which feed?
    • last label should include fqsn?
    • each y-axis should have fqsn?
  • for cross-brokerd symbols should we use a single axis?
    • how to determine this automatically? Do a in the same range calc?
  • how should last datum graphics be offset, eg. should they be in reverse order of the y-axis from left-to-right?

History view

  • curves need to be epoch time aligned instead shm index aligned since obviously it will result in non-time aligned data sets as it is now using the 'index' basis..
  • fix pg.LinearRegion item to show all background overlay curves transparently

Graphics loop

  • use single Feed.open_multi_steam() for the update task and have it look up appropriate graphics update apis per symbol-quote
  • refactor a bunch of the charting/graphics apis to mostly to remove the global-ish ChartPlotWidget instance and API calls, instead moving all this state into Flow / Flume items and possibly add a Viz type to replace the Flow and instead make the Flow a table of associated src data -> procesor pipelines?
    • more to write up as individual subtasks here

Search UI/UX

  • make search results and caching work with multi-symbol selections by the GodWidget
  • all selected symbols should be shown in fqsn form in the window title bar
    • what about the symbol meta-data we used to have? -> maybe just put it in the feed label in the sidepane?
  • allow switching between single-symbol selection
  • add support for multichart "feed groups" using the existing results tree view UI
    • adding and removing items from a overlay group
    • context menu options to add/remove symbols from a group
    • mouse UX (shift-click-drag-release?) to create a group from a single/flat results tree?
    • hot keys for adding a new search result to a group instead of a new widget set (maybe ctl-o or ctl-enter on result row?)
  • add a "selected sub-symbol" UX where whatever highlighted symbol is in the results view changes the state of the "currently active contract":
    • highlight the feed's graphics curves and y-axis vs. the other overlays
    • activate order mode control for that symbol
    • show feed meta-data in sidepane label for selected

FSP (subcharts)

  • should fsps like vlm which require a subchart be overlayed or only the currently selected shown by default?
    • we can make this configurable and maybe offer a sidepane drop down showing all combos and/or a toggle set of symbols from the overlay set?
  • for ohlc main chart overlays how should we do coloring?

Order mode

  • allow passing a fqsns: list[str] to open_order_mode() and having this setup the client loop and the ems tasks to use feed.open_feed() with the same fqsns passed through?
  • pp line arrow-marker should sit on the specific y-axis?
  • depending on "selected symbol" only show that pp?
  • show active order summary markers (for those not in view) on each sub-yaxis
  • support submitting orders per symbol this is currently mega borked
  • how/when to show all pps?

@goodboy goodboy force-pushed the agg_feedz branch 2 times, most recently from e206a24 to 88979ec Compare November 17, 2022 20:00
@goodboy goodboy force-pushed the agg_feedz branch 2 times, most recently from 4a35fcb to 13d81eb Compare December 10, 2022 21:15
@goodboy goodboy force-pushed the agg_feedz branch 2 times, most recently from 791d214 to 963e5bd Compare January 10, 2023 16:09
@goodboy goodboy force-pushed the multichartz branch 4 times, most recently from 9589896 to 91275dc Compare January 13, 2023 17:19
Base automatically changed from agg_feedz to master January 13, 2023 17:20
We have this method on our `ChartPlotWidget` but it makes more sense to
directly associate axis-labels with, well, the label's parent axis XD.

We add `._stickies: dict[str, YAxisLabel]` to replace
`ChartPlotWidget._ysticks` and pass in the `pg.PlotItem` to each axis
instance, stored as `Axis.pi` instead of handing around linked split
references (which are way out of scope for a single axis).

More work needs to be done to remove dependence on `.chart:
ChartPlotWidget` references in the date axis type as per comments.
Main "public" API change is to make `GodWidget.get/set_chart_symbol()`
accept and cache-on fqsn tuples to allow handling overlayed chart groups
and adjust method names to be plural to match.

Wrt `LinkedSplits`,
- create all chart widget axes with a `None` plotitem argument and set
  the `.pi` field after axis creation (since apparently we have another
  object reference causality dilemma..)
- set a monkeyed `PlotItem.chart_widget` for use in axes that still need
  the widget reference.
- drop feed pause/resume for now since it's leaking feed tasks on the
  `brokerd` side and we probably don't really need it any more, and if
  we still do it should be done on the feed not the flume.

Wrt `ChartPlotItem`,
- drop `._add_sticky()` and use the `Axis` method instead and add some
  overlay + axis sanity checks.
- refactor `.draw_ohlc()` to be a lighter wrapper around a call to
  `.add_plot()`.
use the new `do_overlay_scaling: bool` since we know each feed will have
its own updates (cuz multiplexed by feed..) and we can avoid
ranging/scaling overlays that will make their own calls.

Also, pass in the last datum "brighter" color for ohlc curves as it was
originally (and now that we can pass that styling bit through).
A super snappy `numpy.median()` calculator (per input range) which we
slap an `lru_cache` on thanks to handy dunder method hacks for such
things on mutable types XD
Massively speeds up scaling transform cycles (duh).

Also includes a draft for an "overlay transform" type/api; obviously
still a WIP 🏄..
In the (incrementally updated) display loop we have range logic that is
incrementally updated in real-time by streams, as such we don't really
need to update all linked chart's (for any given, currently updated
chart) y-ranges on calls of each separate (sub-)chart's
`ChartView.interact_graphics_cycle()`. In practise there are plenty of
cases where resizing in one chart (say the vlm fsps sub-plot) requires
a y-range re-calc but not in the OHLC price chart. Therefore
we always avoid doing more resizing then necessary despite it resulting
in potentially more method call overhead (which will later be justified
by better leveraging incrementally updated `Viz.maxmin()` and
`media_from_range()` calcs).
It's way faster since it uses a uniform time arithmetic to narrow the
`numpy.searchsorted()` range before actually doing the index search B)
This was a subtle logic error when building the `plots: dict` we weren't
adding the "main (ohlc or other source) chart" from the `LinkedSplits`
set when interacting with some sub-chart from `.subplots`..

Further this tries out bypassing `numpy.median()` altogether by just
using `median = (ymx - ymn) / 2` which should be nearly the same?
In situations where clients are (dynamically) subscribing *while*
broadcasts are starting to taking place we need to handle the
`set`-modified-during-iteration case. This scenario seems to be more
common during races on concurrent startup of multiple symbols. The
solution here is to use another set to take note of subscribers which
are successfully sent-to and then skipping them on re-try.

This also contains an attempt to exception-handle throttled stream
overruns caused by higher frequency feeds (like binance) pushing more
quotes then can be handled during (UI) client startup.
Was broken since the `_adhoc_futes_set` rework a while back. Removes the
cmdty symbols from that set into a new one and fixes the contract
case block to catch `Contract(secType='CMDTY')` case. Also makes
`Client.search_symbols()` return details `dict`s so that `piker search`
will work again..
@goodboy
Copy link
Contributor Author

goodboy commented Feb 13, 2023

This is going to get closed as it was replaced by a sequence of 7 (or so) smaller PRs ending with #455.

Also going to make a follow issue which will include the bullet set above rewritten and checked off as we finish follow up refinement tasks.

@goodboy
Copy link
Contributor Author

goodboy commented Feb 14, 2023

Replaced by the slew of PRs and follow up (copy pasted from here to start) task set in #461.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant