-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Open
Labels
kind/featureCategorizes issue or PR as related to a new feature.Categorizes issue or PR as related to a new feature.
Description
Bug Description
Logs and TargetGroupBinding objects are flooded with errors when target pods are deployed on hybrid nodes.
Steps to Reproduce
- Step-by-step guide to reproduce the bug:
- Deploy EKS cluster with hybrid nodes as per: https://docs.aws.amazon.com/eks/latest/userguide/hybrid-nodes-overview.html
- Configure CNI and Ingress
- Manifests applied while reproducing the issue:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
nodeSelector:
eks.amazonaws.com/compute-type: hybrid
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx
labels:
name: nginx
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
spec:
ingressClassName: alb
rules:
- http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: nginx
port:
number: 80
---
- Controller logs/error messages while reproducing the issue:
{"level":"error","ts":"2025-08-21T13:17:46Z","msg":"Requesting network requeue due to error from ReconcileForPodEndpoints","tgb":{"name":"k8s-default-nginx-1234567890","namespace":"default"},"error":"providerID eks-hybrid:///ap-northeast-1/eks-test/mi-01c49bb1234567890 is invalid for EC2 instances, node: mi-01c49bb1234567890"}
{"level":"info","ts":"2025-08-21T13:17:46Z","msg":"Requeue for networking requeue","tgb":{"name":"k8s-default-nginx-1234567890","namespace":"default"}}
Expected Behavior
ALB forwarding traffic to Nginx pod, no errors in controller logs, TargetGroupBinding resource is healthy,
Actual Behavior
It works, but controller logs and k8s events are flooded with reconcile errors.
Current Workarounds
Environment
- AWS Load Balancer controller version: v2.13.3
- Kubernetes version: 1.32
- Using EKS (yes/no), if so version?: Yes, 1.32
- Using Service or Ingress: Ingress
Possible Solution (Optional)
Hybrid nodes use different ProviderID and InstanceID format (mi-
)
Hybrid node: eks-hybrid:///ap-northeast-1/eks-test/mi-03a43a81234567890
EC2 node: aws:///ap-northeast-1c/i-05c8de21234567890
So this check is not working:
var awsInstanceIDRegex = regexp.MustCompile("^i-[^/]*$") |
Hybrid pods and nodes don't have ENIs, so the resolver should account for this.
Contribution Intention (Optional)
- Yes, I'm willing to submit a PR to fix this issue
- No, I cannot work on a PR at this time
Additional Context
Metadata
Metadata
Assignees
Labels
kind/featureCategorizes issue or PR as related to a new feature.Categorizes issue or PR as related to a new feature.