Streaming plots

[KristinaPlazonic]

I would like to use lightning-scala for streaming plots, but couldn't make it work. Finally I found out that if you use PUT request instead of a POST request, you can update the data on the server, but I still need a method to automatically update the plot - right now after data update, the plot is not redrawn automatically (though it is redrawn with fresh data if you hit the refresh button in the browser).

What I would like to use Lightning for is real-time dashboards for streaming data from (Scala) Spark Streaming. Do you think this is a good idea or have any tips or thoughts on what your further development roadmap will be in this regard?

I would also like to contribute code if possible.

Thanks so much!

[mathisonian]

Hey @KristinaPlazonic, awesome to hear what you are using lightning for, and it would be great to have your help!

You can update or append data to visualizations on the server, and depending on where you are viewing the chart, it will be updated automatically. For example if you are on the lightning web interface or are embedding charts in iframes the are updated automatically thru websockets. If you are in a jupyter notebook they wont automatically update because the semantics don't exactly make sense there.

This is sounds like a great use case, and one that I know many are interested in. I'm more than happy to answer any specific questions that you come across while setting this up.

As for code contribution, any updates to the Scala client would be greatly appreciated (there are some TODOs in the README, any of which would be good to tackle. also issues). The python client is our most feature-rich client at the moment, so it may be worthwhile checking out some examples in python to see the features that it has

Also updating anything in the documentation that you find to be lacking. The source for lightning-viz.org is here. Of specific interest may be the "streaming" visualization section and the REST api section. (for example the rest API doesn't make not of being able to PUT data right now, but it would be nice to mention that).

We have an open chatroom here: https://gitter.im/lightning-viz/lightning that you are more than welcome to visit and ask any questions, etc.