Roadmap

[jonasvdd]

This issue is a request to the community to submit what their vision of plotly-resampler is, which features are still worth implementing?

Some features which I find worth pursuing:

• Summary dataset statistics of the respective view (initially in a table)
  -> e.g. a Table with df.describe() for each series that is shown.
• Options to annotate your data
  No idea how to implement this with plotly, would think that you would need to define your own OO structure to do so (which re-uses underlying annotations), but this would still imply a lot of logic and design decisions (e.g. loading annotations, saving annotations, ...)
• Other aggregation methods (e.g. using width of trace for uncertainty / downsampling rate)
  just a gist, but using a plot-per-pixel ratio of 2 with LTTB seems a rather good IDEA to me.
  Also, playing with the line-width seems a valid path to embark upon.

[szkjn]

Hi, thanks for the great work.

I'm currently working on a data viz app and came accross plotly_resampler a couple of weeks ago. I'm trying to find a way to incorporate it in my already existing Dash app. Would there be a straightforward way to do that ?

Thanks for your time.

[jonasvdd]

Hi @szkjn,

Lovely to hear that!

Valid question, actually there is a (straightforward) way!
Look at this example: https://github.com/predict-idlab/trace-updater/blob/master/usage.py, here we use the register_update_graph_callback on a new, standalone dash-app layout. I acknowledge that this is not yet documented that well, so thanks for pointing this out, I will handle the documentation issue in #17.

Additionally, a new release this week will also support non-scatter-like traces (e.g., bar-charts, histograms) in your figure 🔥.

Hope this helps you further!

Kind regards,
Jonas

[szkjn]

Hey @jonasvdd, thanks for your quick answer !

I had a look at usage.py. I don't quite get how usual callbacks can be used alongside the register_update_graph_callback method.

For e.g, if I want to dynamically plug the sampler to say - a dropdown that selects the y-axis between noisy_sin and noise, how would I wire that ? I tried putting the figure constructor and register_update_graph_callback in a callback but it doesn't seem to work. Am I missing something ?

Also, if there's a better place to ask these questions, please point me in the right direction :)

Regards.

[jonasvdd]

Hi,

I think that I understand your problem. You have multiple callback functions, which want to output/update the figure-related components. I would suggest that you create a separate issue, where you describe / copy-paste the text above!
You can also DM me on Twitter so we can chat further on discord :)!

[jonasvdd]

@szkjn, we added a dash-example on how to use plotly-resampler within a dash app! 😃

[jayceslesar]

Annotations in a sense of being able to mark some arbitrary x_start and x_end? Very useful but maybe out of scope with this package as that would require a database + some messy plotly code (for now). Happy to hack on that and this in the future....I think annotations should be essentially a start and a stop time and some text that is fed by an Enum the user defines for strict categories, and should be supported by many plotting libraries, as well as abstracted to not use any plotting libraries.

Another interesting idea is integrating this with a tool like https://github.com/ibis-project/ibis, say where you have some dashboard fed by ibis code, and you wanted to implement downsampling against that backend.

[cjstudioz]

Additionally, a new release this week will also support non-scatter-like traces (e.g., bar-charts, histograms) in your figure 🔥.

may I ask, does that mean bar-charts are not yet supported? this post was quite a while ago.
I'm testing this out on a dash app and seems to work fine for area and lines, but bar charts are still in the original frequency and hence take a long time to render

I'm using plotly express to build my charts

fig.replace(px.bar(noisy_sin))

[jvdd]

Hi @cjstudioz,

Plotly-Resampler (at the moment) only aggregates data for scatter-like traces.

I believe earlier versions of plotly-resampler crashed when non-scatter-like traces were present in the figure - which the release @jonasvdd is talking about solved.

P.S. If you have any proposal of how bar charts should be rendered (with aggregation functionality), you may always create a seperate issue or pull request 🙏

[cjstudioz]

happy to create a new feature request / bug report but just wanted to check. Does plotly-resampler currently meant to support bar charts? so I know whether to raise a bug or feature req.

also thanks for this great tool obviously

[jvdd]

We intended to focus initially on scatter traces (as we mainly work on very large time series datasets in our job). However, we are open to extending the functionality of plotly-resampler :) (although this is less of a priority for us)

[jonasvdd]

Hi @cjstudioz, feel free to do so! 😄 Could you then also provide a minimal (slow-)working example?

Plotly-Resampler its FigureResampler allows using other visualization types, but as @jvdd stated, we only focused (so far) on the scalability of the scatter traces and have not found significant use-cases in our day-to-day work for adding scalability to other ones.

[AdrianSkib]

Hi! I love your work; it's making my work so much easier!

As for the road ahead, I really wish there was a way to avoid the gaps in line plots. Some sort of interpolation between the rendered points could maybe work.

[jonasvdd]

Hi,

Do you mean: setting interleave_gaps to False for the AbstractSeriesAggregators?

This would boil down to this:

# create a x-index with a gap
x = np.concatenate((np.arange(0, 1_000_000), np.arange(2_000_000, 3_000_000)))
y = np.random.randn(2_000_000)


# Default figure -> uses the default aggregation method with interleave gaps set to True
fr_autogap = FigureResampler(default_downsampler=EfficientLTTB(interleave_gaps=True))
fr_autogap.add_trace({}, hf_x=x, hf_y=y)
fr_autogap.show()

# Interleave gaps set to False
fr_nogap = FigureResampler(default_downsampler=EfficientLTTB(interleave_gaps=False))
fr_nogap.add_trace({}, hf_x=x, hf_y=y)
fr_nogap.show()

📷 ⬇️

Maybe we can add this to the FAQ section of our docs? @jvdd @AdrianSkib

[AdrianSkib]

That's perfect - thanks a bunch!

[juan11iguel]

I do not know it this is already possible, sorry if I missed it in the documentation. I have an existing figure and I want to replace the data of a particular trace.

This would be useful in the following case: There is a figure with two subplots, in the top one the original signal is shown, while on the bottom one a denoised version of the original signal. The denoising can be adjusted by changing the value of a parameter, in a slider for example, the denoised signal should then be recalculated (so a new signal) and the trace data should be replaced with the new one.

The only approach that comes to my mind right now is taking the previous figure as a State in a callback, deleting the previous trace and with the add_trace method add the updated one. Finally, return the new figure.

Great work on the library btw 😁

EDIT: I realise that instead of deleting the trace and adding a new one the update_traces method can be used, but still remains the problem of returning a new figure. Also, would the updated trace respect the resampler optimization? (I will try and check)

[alyashenko]

@Juasmis you can use the FigureWidget class instead of Figure and update the data series in it to redraw. The plotly_resamper wrapper works great with it.

[jvdd]

Hey @Juasmis

That is a very cool use case you are describing!! 👀

If you are working in a notebook environment I would also suggest using the FigureWidgetResampler (as @alyashenko suggested).

To support reloading when data is added / changed we added the reload_data and reset_axes methods to the FigureWidgetResampler

P.S.: You can see an example of FigureWidgetResampler being used to dynamically update the trace data in this (old) issue #58

[juan11iguel]

I am trying to implement this functionality in a Dash app and not sure what is the alternative to the reload_data method. If I understand correctly the dynamic updating in Dash is carried out by TraceUpdater. From the Dash app example the updated data needs to be processed by construct_update_data function and it only handles relayoutData like datatype (I think? 😅).

Maybe I am polluting this thread with these questions, if it is the case, I am happy to move the conversation to a different place

Thanks for the help! @alyashenko @jvdd

[jonasvdd]

Hi @Juasmis,

I found it a fun challenge to create a dash app concerning your problem!
I think the dash app example which I made in #159 meets your requirements.

Kind regards,
Jonas

[jayceslesar]

Sort of an out there feature but lets say I did have an infinite stream of timeseries data that I wanted to plot and use the functionality of this library.

1. Could we suggest optimal ways to partition that data given we know the frequency of it?
2. Could actually give plotly-resampler a directory of these partitioned files to draw from and plot.

Lots of unknowns there but I think interesting

Sort of related:
Getting the downsampling implemented in some sort of SQL-like recipe either exposed through https://github.com/ibis-project/ibis or as some documentation in this repo (or maybe can port the tsdownsample to sqlalchemy or something?)