Streaming from serial port?

[ubaldot]

Is it possible to plot datastream coming from the serial port? I have a MCU connected to the serial port that stream data read from a sensor towards my macbook and I would like to plot them in real-time.

At the moment the easiest it to use a python script to convert the received data from the serial port to some data stream readable by plotjuggler (e.g. UDP), but I am wondering if there is a native/plugin solution.

[facontidavide]

It is possible, but the challenge is that when using the serial port, it is not clear where the message starts and ends.

A would love to add a native plugin, but that aspect of the protocol should be defined first

[ubaldot]

Ciao Davide, I think that we could make a decision based on a maximum-likelihood usage and keep it bare-bones.

I propose two solutions:

1. Messages start with '[' and it end with "]". All the list elements are interpreted as floats separated by commas. It will be on the user side to secure that all the sent list have the same length. We could add some robustness such that if a message has length 3 and next has length 5, then the last two values will be discarded AND if a message has length 5 and the next has length 3, then the 4th and 5th missing elements in the list are either padded to 0.0 or they are kept to their previous value. The user may be warned that the messages have not the same length. This would already cover lots of use cases. We have to keep in mind that the scope of this tools is to plot numbers on a graph.

2. Use ASCII format to add a pinch of generality - but also complexity. Message starts with any character x such that x != '\0' and it ends with the terminating character y = '\0'. We all need an ASCII parser in the reception side, that distinguish between integers, floats or chars. Mind that there is an army of Arduino users out there that are used to this kind of format.

I don't think we need more complexity than that and yet we can cover zillions of use-cases.
At most we could provide a spot and a button to send characters back to the UART: the user write something in the box and when pushing the button the message is sent back to UART as a string.

What do you think?

[facontidavide]

I will not make this an official driver. First, I would like to support binary format too, in particular in a bandwidth constrained channel like the serial port.

At the architectural level, transport mechanism is orthogonal to protocol.

The only requirement of the transport mechanism is to be message based.

I am not against creating some solution that may make users like you happy, but since this is not interesting for me in this very moment, I will not devote time to it.

[ubaldot]

Fair enough!
Right, I haven't considered the binary format as well.
Shall we create an issue so that some contributor may pick up?
Unfortunately I am not a C++ guy at all, nor I have time at all... too many projects and too many life duties :(

[Bartimaeus-]

@ubaldot if you just need something simple the Arduino IDE serial plotter might do the trick: https://docs.arduino.cc/software/ide-v2/tutorials/ide-v2-serial-plotter/

[ubaldot]

Thanks for the tip! Actually, I found a tool called seriaplot which is great for plotting serial data. The only limitation is that (at least for serialplot) that you cannot associate timeseries to graphs arbitrarily.
Nevertheless, this afternoon I found a serial plotter written in Rust than seems to meet my requirements.
I’ll check it out. Alternatively, I can run a python script to encapsulate the serial data into the payload of an e.g. UDP packet and send it to plotjuggler.

[RallyTronics]

COBS works well for packet framing, here's an implementation https://github.com/charlesnicholson/nanocobs

[cHemingway]

I have a rough implementation in a branch here (my first time working with the Plotjuggler codebase) based on DataStreamUDP.

There is a lot of duplicate code and I am curently using newlines to split messages, so doesn't work with any binary protocols, I've only tested it with JSON so far.
It also chokes if you pass in any incorrect messages which likely needs dealt with, serial often has other output on it (e.g. at startup) that isn't the desired protocol.