Expose Note Events via TCP/UDP to Enable Live Visualizers & Teaching Tools

[jimishol]

Background

In my previous post How MuseScore motivated me to read about harmony, I introduced a 3D harmony structure that projects notes, chords, and scales as geometric entities in real time. Despite my limited programming background, I turned that idea into a working prototype:

Cholidean Harmony Structure
https://github.com/jimishol/cholidean-harmony-structure#cholidean-harmony-structure

The Challenge

Today, no MIDI application I know—MuseScore included—ever exposes the fundamental Note On/Off (or velocity=0) events outside its own audio engine. My visualizer (written in Lua) can only abuse FluidSynth’s console output.

Feature Request

Edit: After testing, I found TCP latency on Windows too high for real‑time use, so I’ve settled on UDP as the practical solution.

Could MuseScore offer an optional TCP-based MIDI event stream? In practice, this would be a lightweight server you toggle in Preferences (or via a tiny QML plugin) that broadcasts every note-on/note-off over a socket. My Lua tool could then connect with plain TCP—no extra libraries—let the events flow, and render harmony in 3D live.

Basic Requirements

• Host default: 127.0.0.1
• Port default: 9000
• Data format: raw MIDI bytes (hex/decimal) or simple text lines, for example:

  90 3C 64   ← noteOn, channel 1, Middle C, velocity 100  
  80 3C 40   ← noteOff, channel 1, Middle C, velocity 64
  

• Zero impact on existing audio/MIDI pipeline
• Minimal UI: just a checkbox and host/port fields

Why This Matters

• Enables truly live external visualizers and teaching tools
• Eliminates file-export and console scraping hacks
• Fosters innovative plugins and community-built performance interfaces.
• Puts MuseScore ahead of other apps by offering a real-time MIDI feed

Questions for the Community

1. Has anyone else needed raw MIDI event streaming from MuseScore?
2. Are there hidden hooks or plugin APIs I’ve missed?
3. Would a raw-byte TCP approach fit MuseScore’s architecture?
4. What minimal UI/UX would you recommend for configuration?

Thank you for reading—and for any pointers on making MuseScore a real-time MIDI server. I believe this small addition could unlock a whole new class of educational and creative tools!

[jimishol]

Since opening this request, I rejected TCP due to its high latency on Windows and instead implemented a UDP‑based backend in my Cholidean Harmony Structure project:

https://github.com/jimishol/cholidean-harmony-structure/blob/main/src/backends/udpMidi/track_active_notes_thread.lua

It listens for raw MIDI bytes over UDP and merges them into a live note state at ~50 Hz. This has proven that a socket‑based event stream is both lightweight and reliable for real‑time visualization.

▶ See it in action: Windows demo video]

The visualization does not attempt to label chords or assign harmonic functions. Instead, the 12‑tone equal temperament system is projected into 3D space, where harmonic relationships and progressions become visible as natural geometric movements. Just as theorists interpret what they hear, they can also interpret what they see — the system provides the raw, immediate structure, leaving analysis open to the observer.

If MuseScore exposed its note‑on/note‑off events over TCP/UDP in a similar way, external tools could connect directly without hacks like scraping FluidSynth output. This would open the door for live visualizers, educational aids, and experimental performance interfaces. For beginner composers in particular, being able to see harmony unfold in real time while editing or playing back a score could make abstract concepts far more intuitive.

Has there been any movement on this idea, or interest from developers who might want to prototype such a feature?