feat(agent): use multistream and yamux muxer instead of HTTP server #5201
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
henrybarreto
force-pushed
the
feat/use-multistream-as-mux
branch
6 times, most recently
from
1ac9dfc to
12f001d
Compare
henrybarreto
force-pushed
the
feat/use-multistream-as-mux
branch
9 times, most recently
from
4c17372 to
d2d6431
Compare
henrybarreto
force-pushed
the
feat/use-multistream-as-mux
branch
3 times, most recently
from
473e89b to
dfb1470
Compare
otavio
requested changes
Sep 9, 2025
henrybarreto
force-pushed
the
feat/use-multistream-as-mux
branch
from
6df3f30 to
33e6e30
Compare
henrybarreto
force-pushed
the
feat/use-multistream-as-mux
branch
2 times, most recently
from
2618c1b to
a90cc47
Compare
henrybarreto
force-pushed
the
feat/use-multistream-as-mux
branch
from
c3d3f74 to
ebefc5f
Compare
henrybarreto
force-pushed
the
feat/use-multistream-as-mux
branch
2 times, most recently
from
b8ee206 to
29a42de
Compare
henrybarreto
force-pushed
the
feat/use-multistream-as-mux
branch
from
29a42de to
6f5aca6
Compare
This change replaces the legacy per-connection HTTP/revdial reverse-tunnel flow with a multiplexed, per-stream protocol (v2) based on HashiCorp's yamux and multistream-select. Yamux sessions allow a single agent websocket to host many logical streams without repeating the HTTP reverse-listen handshake for each session. Multistream-select provides explicit per-stream protocol negotiation, making each logical stream's intent (SSH open/close, HTTP proxy) explicit and versionable. The new approach reduces handshake overhead and makes future protocol extensions easier and safer. Over time the legacy reverse-tunnel implementation (per-request HTTP/revdial) has become a maintenance burden and a performance limiter: each logical session required a full HTTP handshake and a separate reverse listener, which increased latency and duplicated logic across server components. This change replaces that model with a single multiplexed transport per agent using HashiCorp's yamux, and introduces explicit per-stream protocol negotiation via multistream-select. Practically, when an agent connects we now bind its websocket to a yamux session (Manager.Bind) and keep that session alive with a light ping loop. Individual logical operations—opening an SSH session, closing it, or proxying HTTP—are created as yamux streams. Each stream negotiates its intent using multistream identifiers (`/ssh/open/1.0.0`, `/ssh/close/1.0.0`, `/http/proxy/1.0.0`) and then exchanges a small JSON envelope for any parameters. The dialer package centralizes version handling and prepares connections for callers via DialTo(ctx, tenant, uid, target), while Target implementations encapsulate the version-specific bootstrap (legacy HTTP v1 vs yamux+multistream v2). We kept backward compatibility in mind: the Manager still recognizes existing revdial-based connections and will return a v1 connection when appropriate. That lets the system run both v1 and v2 agents concurrently and supports staged rollouts. I also removed the duplicated tunnel wiring and replaced it with a focused HTTP sidecar (http) that exposes both the legacy endpoints and the new v2 bind endpoint; server and session code now call into dialer.Dialer and Target helpers rather than performing raw HTTP handshakes.
Updated the tunnel listening mechanism to utilize a configuration loaded from authentication data instead of the default configuration. This change enhances flexibility by allowing device-specific settings to be applied dynamically. The NewConfigFromMap function was added to create a new configuration from a map, ensuring that default values are used when specific keys are missing. Additionally, improved error handling during session creation to attempt a fallback to the default configuration if the custom one fails.
henrybarreto
force-pushed
the
feat/use-multistream-as-mux
branch
from
6f5aca6 to
db8435d
Compare
…v2 tunnels This change refactors the agent to support multiple connection protocol versions, introducing a new configuration option ConnectionVersion to select between them. Version 1 uses an Echo-based tunnel with HTTP-like handlers, while version 2 leverages a Yamux-based tunnel integrated with the client. The agent’s Listen method now delegates to version-specific listeners, and related handlers were updated accordingly. The tunnel abstraction was also revised to adopt generics, renaming Handler to HandlerFunc and splitting out implementations for v1 and v2. Additionally, reverse listener creation was modernized in the API client with explicit NewReverseListenerV1 and NewReverseListenerV2 methods, each with their own configuration management. These changes simplify the agent’s architecture while allowing incremental migration to the new tunnel protocol.
henrybarreto
force-pushed
the
feat/use-multistream-as-mux
branch
from
db8435d to
dc25c91
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.