help！Traffic is interrupted during the delayed stop of the old pod

[jixinzhang001]

I use the rollout and istio Subset-level traffic splitting function to implement online management, but when the traffic is completely switched to the new pod, the old pod enters the scale down delay, which is 30s.

During this 30s period, my new and old pods are not Traffic comes in until the old pod enters the terminated state.

This process affects the online business for 30 seconds. Does anyone know what causes this problem?

[Ilhan-Personal]

Hey @jixinzhang001 can you describe what your rollout steps look like and from where you gathered the data from?

[kostis-codefresh]

This is currently an open bug. See also #2507

The workaround is to use Host Level Splitting https://argo-rollouts.readthedocs.io/en/stable/features/traffic-management/istio/#host-level-traffic-splitting

[jixinzhang001]

when I upgrade rollout to 1.8.1 , this issue has not been resloved.

[jixinzhang001]

@Ilhan-Personal monitor data from prometheus。 when canary weight from 100 to 0 and stable weight from 0 to 100 ，the caller will have a timeout of 30 second。

workload rollout yaml：
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: test-module
spec:
progressDeadlineSeconds: 3600
replicas: 3
minReadySeconds: 1
workloadRef:
apiVersion: apps/v1
kind: Deployment
name: test-module
strategy:
canary:
scaleDownDelaySeconds: 30
analysis:
templates:
- templateName: test-module-monitor

  canaryMetadata:
    annotations:
      version: canary
    labels:
      version: canary
      service.istio.io/canonical-revision: canary
  stableMetadata:
    annotations:
      version: stable
    labels:
      version: stable
      service.istio.io/canonical-revision: stable
  trafficRouting:
    istio:
      virtualService:
        name: test-module        # required
      destinationRule:
        name: test-module    # required
        canarySubsetName: canary  # required
        stableSubsetName: stable  # required
  steps:

  - setCanaryScale:
      weight: 100
  - setWeight: 5
  - pause:
      duration: 30
  - setWeight: 10
  - pause:
      duration: 30
  - setWeight: 20
  - pause:
      duration: 20
  - setWeight: 40
  - pause:
      duration: 20
  - setWeight: 80
  - pause:
      duration: 20
  - setWeight: 100
  - pause:
      duration: 60