kubelet parameter(eviction-max-pod-grace-period ), not work as expected like officical comment.

[rainbowBPF2]

What happened?

Hey brothers , feedback a confusing point about kubelet soft eviction:

Just like kubelet --help show:
--eviction-max-pod-grace-period int32 Maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met. If negative, defer to pod specified value.

I guess if this parameter is set to negative ,such as -1, soft eviction would use pod specified value(TerminationGracePeriodSeconds),

But When I try to evict pod by creating node pressure(such as memory pressure), I found it's always -1 send to CRI runtime.
Then pod container stopped immediately with sigkill, exit 137.

In short， this parameter：
（1）set as active number：work as expected；
（2）no set， kubelet read its default number：0， work as expected；
（3）set as negative number: not work as kubelet help show.

Thanks for your response. May it be a bug ?

What did you expect to happen?

--eviction-max-pod-grace-period set to negative, then kubelet syncPod logic send pod‘s TerminationGracePeriodSeconds as gracePeriod to CRI runtime to stop container.

How can we reproduce it (as minimally and precisely as possible)?

(1) set this parameter to a negative number, such as -1;
(2) config kubelet soft eviction ;
(3) running a pod asking a lot memory , to trigger node memory pressure;
(4) check CRI runtime log, such as containerd ;
(5) then you will find timeout parameter -1 sent to pod container , then container exit with code 137(sigkill);

Anything else we need to know?

I running these tests on Tencent Cloud TKE.

And I thought it seem not related to Cloud Service Provider.

Kubernetes version

Client Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.3-tke.34", GitCommit:"61cc96d2f7e9277e89e29b4b04d045f27d6e75df", GitTreeState:"clean", BuildDate:"2023-03-08T07:40:04Z", GoVersion:"go1.12.17", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.3-tke.32", GitCommit:"8ca5807ae095365d68754a2ba20b10bbe5a8998c", GitTreeState:"clean", BuildDate:"2022-10-08T04:00:33Z", GoVersion:"go1.12.14", Compiler:"gc", Platform:"linux/amd64"}

Cloud provider

Tencent Cloud TKE

OS version

AME="Ubuntu" VERSION="18.04.4 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.4 LTS" VERSION_ID="18.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=bionic UBUNTU_CODENAME=bionic

Install tools

Container runtime (CRI) and version (if applicable)

containerd --version
containerd github.com/containerd/containerd v1.4.3-tke.2 a11500cbfa0b4d2fc9b905e03c35f349ef5b1a9f

Related plugins (CNI, CSI, ...) and versions (if applicable)

[tzneal]

/sig node

[SergeyKanzhelev]

/triage accepted

The question is whether we fix code and make the backward incompatible change or fix a doc. I'd vote for a doc change which is less work. But blocks an interesting scenarios.

@rainbowBPF2 were you looking for the described behavior? Does your scenario requires this?

/priority important-longterm

[rainbowBPF2]

@SergeyKanzhelev

Thanks for brothers‘ attention on this point.

1. For daily business running, temporarily no bad effects, and seems not required. Doc change is also a choice.
2. But look at it in the long run, it will be an amazing/exciting feature.
   Because we have a more reasonable choice to terminate pods in case of soft-eviction, to realize Pod-Graceful-Shutdown

In short，no required but deep expected, 😄。

[fgksgf]

/assign