Skip to content

hudson.plugins.ec2.EC2RetentionStrategyTest.testRetentionDespiteIdleWithMinimumInstanceActiveTimeRange is flaky #3944

Open
@basil

Description

Since #3900 I have observed hudson.plugins.ec2.EC2RetentionStrategyTest.testRetentionDespiteIdleWithMinimumInstanceActiveTimeRange failing multiple times in CI (but not locally) with

java.lang.AssertionError: expected:<2> but was:<3>
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:633)
	at hudson.plugins.ec2.EC2RetentionStrategyTest.testRetentionDespiteIdleWithMinimumInstanceActiveTimeRange(EC2RetentionStrategyTest.java:612)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.jvnet.hudson.test.JenkinsRule$1.evaluate(JenkinsRule.java:706)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.lang.Thread.run(Thread.java:1583)

We run BOM tests with surefire.rerunFailingTestsCount=1 to retry flaky tests one time, which prevents mildly flaky tests from creating too much noise but is ineffective against very flaky tests. The fact that it is ineffective here indicates this is a very flaky test.

We should look into making the test less flaky or exclude it if we cannot figure out how.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions