[API Proposal]: Kubernetes specialized Resource Monitoring #6496
Description
Important
There's new proposal in comments below
Background and motivation
Current implementation of Microsoft.Extensions.Diagnostics.ResourceMonitoring for Windows container environment does not support requests.cpu and requests.memory in calculations. For limits it reads JobObject now, which is suitable for non-Kubernetes environments.
In Kubernetes, both requests and limits can be exposed via environment variables if cluster is configured to use Downward API. If the cluster is configured accordingly and assuming that engineer knows that and knows the names of the environment variables, ResourceMonitoring could be extended to use these values utilization calculations.
Example yaml with Downward API that injects values into pods' environment variables
env:
- name: MY_CLUSTER_CPU_REQUEST
valueFrom:
resourceFieldRef:
resource: requests.cpu
- name: MY_CLUSTER_CPU_LIMIT
valueFrom:
resourceFieldRef:
resource: limits.cpu
- name: MY_CLUSTER_MEMORY_REQUEST
valueFrom:
resourceFieldRef:
resource: requests.memory
- name: MY_CLUSTER_MEMORY_LIMIT
valueFrom:
resourceFieldRef:
resource: limits.memoryLinux is just fine, so that applies only for Windows.
API Proposal
Configuration class extension of K8s config.
public class ResourceMonitoringOptions
{
... other configs
/// <summary>
/// Configuration for reading CPU and memory limits and requests from Kubernetes environment variables.
/// This is supported only on Windows containers.
/// </summary>
public KubernetesResourceConfig? KubernetesResourceConfig { get; set; }
}K8s config details
public sealed class KubernetesResourceConfig
{
/// <summary>
/// Enables reading CPU/memory utilization context from Kubernetes via environment variables.
/// </summary>
public bool Enabled { get; set; } = false;
/// <summary>
/// The environment variable name for CPU limit (e.g., "MY_CLUSTER_CPU_LIMIT").
/// </summary>
public string CpuLimitEnvVar { get; set; } = "MY_CLUSTER_CPU_LIMIT";
/// <summary>
/// The environment variable name for memory limit (e.g., "MY_CLUSTER_MEMORY_LIMIT").
/// </summary>
public string MemoryLimitEnvVar { get; set; } = "MY_CLUSTER_MEMORY_LIMIT";
/// <summary>
/// The environment variable name for CPU request (e.g., "MY_CLUSTER_CPU_REQUEST").
/// </summary>
public string CpuRequestEnvVar { get; set; } = "MY_CLUSTER_CPU_REQUEST";
/// <summary>
/// The environment variable name for memory request (e.g., "MY_CLUSTER_MEMORY_REQUEST").
/// </summary>
public string MemoryRequestEnvVar { get; set; } = "MY_CLUSTER_MEMORY_REQUEST";
}API Usage
How to configure Resource Monitoring with K8s config
services.AddResourceMonitoring(options =>
{
options.KubernetesResourceConfig = new KubernetesResourceConfig
{
Enabled = true,
CpuLimitEnvVar = "MY_CLUSTER_CPU_LIMIT",
MemoryLimitEnvVar = "MY_CLUSTER_MEMORY_LIMIT",
CpuRequestEnvVar = "MY_CLUSTER_CPU_REQUEST",
MemoryRequestEnvVar = "MY_CLUSTER_MEMORY_REQUEST"
};
});Alternative Designs
No response
Risks
If improperly configured it should fallback to existing behavior or throw on startup, although it seems extensively too conservative.
Env variables parsing needs to be done well to handle all possible options how K8s can be configured
It's Windows only => have a validation that throws on non-Windows