enhancement(config): Generate Grafana dashboards using CUE

[lucperkins]

As we'll be using Grafana as our default dashboard solution, we need to be able to easily generate and iterate on dashboards for specific use cases:

• Dashboards for specific roles (agent, aggregator, etc.)
• Within those roles we'll need dashboards for different platforms (Kubernetes et al)
• We may want to explore defaults for metrics sources beyond internal_metrics
• We should have a "just exploring" dashboard (single Vector instance running on your laptop)

To that end, we need to have a CUE schema for dashboards here in the repo plus a script that will enable us to generate all of the dashboards we need using one command (so that they can be either committed to Git or generated as part of build process).

The current plan:

• One dashboard.cue file that provides the schema. It will share the metadata namespace with the other CUE sources, in case we end up sharing information with the docs sources.
• Other places in the repo can have their own CUE files that define the needed dashboard (e.g. k8s-agent-internal-metrics.cue)
• A single shell script will be able to generate all dashboards in the repo in one command

[binarylogic]

This is worth establishing a consensus before we proceed. I think a simple RFC is in order. I agree with this in theory but I think we should also:

1. Not make these Grafana specific. We should establish a simpler, higher-level definition that we can then translate into Grafana (as well as others). Grafana's definitions, in my opinion, include way too many details, and many of them are specific to Grafana. We'll want definitions that we can translate to a variety of platforms.
2. Broadly agree on the dashboards and charts before defining them. We should have a drill-down methodology to our dashboards that start high level. There are also questions around variable use and so on.

[MOZGIII]

1. Not make these Grafana specific. We should establish a simpler, higher-level definition that we can then translate into Grafana (as well as others). Grafana's definitions, in my opinion, include way too many details, and many of them are specific to Grafana. We'll want definitions that we can translate to a variety of platforms.

This sounds like a challenge far greater than building Grafana-specific dashboards. I think we should do Grafana-only for v1, and then, after we have the set of dashboards in mind, try and address the portable implementation in v2.

Either way, I agree that an RFC and research process is worth it here.

[lucperkins]

As with #4838, I'm going to close this for now and potentially re-open if it's decided that this specific approach is the right way to go for providing a default OSS solution for observability of Vector itself.