Skip to content

Commit 2c6dbcc

Browse files
kchristidiskchristidis
committed
[FAB-1658] Fix Deliver failures on BDD tests
https://jira.hyperledger.org/browse/FAB-1658 In FAB-1382 [1] the Deliver API was modified so as to work with a range. The BDD implementation of Deliver was also modified in that changeset so as to take an end block of `NEWEST`. Occasionally, the Kafka orderer will fail on one of these BDD tests because the Deliver request comes before all the incoming (via Broadcast) messages have been processed. In that case, the `NEWEST` value is translated to a lower-numbered block than the expected one. As an example, consider the case where we send 100 messages to the orderer, and each block has 10 messages. When we send the Deliver call, Kafka has processed 90 messages, so it translates NEWEST to 9. Our test expects NEWEST to be 10 and thus fails. Until we find an elegant fix for this, I am adding a 1-second sleep before the Deliver call is invoked on the Kafka cases. This should add 14 seconds to the duration of these tests. I stress that this is a temporary patch, and I am open to a more elegant solution, but until then, the default should not be a setup where the orderer fails occasionally. [1] https://gerrit.hyperledger.org/r/#/c/3271/ Change-Id: I77a091546e77981838a4b8ee29019178ddf0d49b Signed-off-by: Kostas Christidis <[email protected]>
1 parent ba288e1 commit 2c6dbcc

File tree

2 files changed

+39
-30
lines changed

2 files changed

+39
-30
lines changed

‎bddtests/features/orderer.feature

+33-30
Original file line numberDiff line numberDiff line change
@@ -15,29 +15,30 @@ Feature: Orderer
1515
And I wait "<BootTime>" seconds
1616
And user "binhn" is an authorized user of the ordering service
1717
When user "binhn" broadcasts "<NumMsgsToBroadcast>" unique messages on "orderer0"
18+
And user "binhn" waits "<WaitTime>" seconds
1819
And user "binhn" connects to deliver function on "orderer0"
1920
And user "binhn" sends deliver a seek request on "orderer0" with properties:
2021
| Start | End |
2122
| 1 | Newest |
2223
Then user "binhn" should get a delivery from "orderer0" of "<ExpectedBlocks>" blocks with "<NumMsgsToBroadcast>" messages within "<BatchTimeout>" seconds
2324

2425
Examples: Solo Orderer
25-
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime |
26-
| docker-compose-orderer-solo.yml | 20 | 2 | 10 | .5 |
27-
| docker-compose-orderer-solo.yml | 40 | 4 | 10 | .5 |
28-
| docker-compose-orderer-solo.yml | 60 | 6 | 10 | .5 |
26+
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime | WaitTime |
27+
| docker-compose-orderer-solo.yml | 20 | 2 | 10 | .5 | 0 |
28+
| docker-compose-orderer-solo.yml | 40 | 4 | 10 | .5 | 0 |
29+
| docker-compose-orderer-solo.yml | 60 | 6 | 10 | .5 | 0 |
2930

3031
Examples: 1 Kafka Orderer and 1 Kafka Broker
31-
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime |
32-
| environments/orderer-1-kafka-1 | 20 | 2 | 10 | 5 |
33-
| environments/orderer-1-kafka-1 | 40 | 4 | 10 | 5 |
34-
| environments/orderer-1-kafka-1 | 60 | 6 | 10 | 5 |
32+
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime | WaitTime |
33+
| environments/orderer-1-kafka-1 | 20 | 2 | 10 | 5 | 1 |
34+
| environments/orderer-1-kafka-1 | 40 | 4 | 10 | 5 | 1 |
35+
| environments/orderer-1-kafka-1 | 60 | 6 | 10 | 5 | 1 |
3536

3637
Examples: 1 Kafka Orderer and 3 Kafka Brokers
37-
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime |
38-
| environments/orderer-1-kafka-3 | 20 | 2 | 10 | 5 |
39-
| environments/orderer-1-kafka-3 | 40 | 4 | 10 | 5 |
40-
| environments/orderer-1-kafka-3 | 60 | 6 | 10 | 5 |
38+
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime | WaitTime |
39+
| environments/orderer-1-kafka-3 | 20 | 2 | 10 | 5 | 1 |
40+
| environments/orderer-1-kafka-3 | 40 | 4 | 10 | 5 | 1 |
41+
| environments/orderer-1-kafka-3 | 60 | 6 | 10 | 5 | 1 |
4142

4243
# @doNotDecompose
4344
Scenario Outline: Basic seek orderer function (Utilizing properties for atomic broadcast)
@@ -46,6 +47,7 @@ Feature: Orderer
4647
And I wait "<BootTime>" seconds
4748
And user "binhn" is an authorized user of the ordering service
4849
When user "binhn" broadcasts "<NumMsgsToBroadcast>" unique messages on "orderer0"
50+
And user "binhn" waits "<WaitTime>" seconds
4951
And user "binhn" connects to deliver function on "orderer0"
5052
And user "binhn" sends deliver a seek request on "orderer0" with properties:
5153
| Start | End |
@@ -57,22 +59,22 @@ Feature: Orderer
5759
Then user "binhn" should get a delivery from "orderer0" of "<ExpectedBlocks>" blocks with "<NumMsgsToBroadcast>" messages within "1" seconds
5860

5961
Examples: Solo Orderer
60-
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime |
61-
| docker-compose-orderer-solo.yml | 20 | 2 | 10 | .5 |
62-
| docker-compose-orderer-solo.yml | 40 | 4 | 10 | .5 |
63-
| docker-compose-orderer-solo.yml | 60 | 6 | 10 | .5 |
62+
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime | WaitTime |
63+
| docker-compose-orderer-solo.yml | 20 | 2 | 10 | .5 | 0 |
64+
| docker-compose-orderer-solo.yml | 40 | 4 | 10 | .5 | 0 |
65+
| docker-compose-orderer-solo.yml | 60 | 6 | 10 | .5 | 0 |
6466

6567
Examples: 1 Kafka Orderer and 1 Kafka Broker
66-
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime |
67-
| environments/orderer-1-kafka-1 | 20 | 2 | 10 | 5 |
68-
| environments/orderer-1-kafka-1 | 40 | 4 | 10 | 5 |
69-
| environments/orderer-1-kafka-1 | 60 | 6 | 10 | 5 |
68+
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime | WaitTime |
69+
| environments/orderer-1-kafka-1 | 20 | 2 | 10 | 5 | 1 |
70+
| environments/orderer-1-kafka-1 | 40 | 4 | 10 | 5 | 1 |
71+
| environments/orderer-1-kafka-1 | 60 | 6 | 10 | 5 | 1 |
7072

7173
Examples: 1 Kafka Orderer and 3 Kafka Brokers
72-
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime |
73-
| environments/orderer-1-kafka-3 | 20 | 2 | 10 | 5 |
74-
| environments/orderer-1-kafka-3 | 40 | 4 | 10 | 5 |
75-
| environments/orderer-1-kafka-3 | 60 | 6 | 10 | 5 |
74+
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime | WaitTime |
75+
| environments/orderer-1-kafka-3 | 20 | 2 | 10 | 5 | 1 |
76+
| environments/orderer-1-kafka-3 | 40 | 4 | 10 | 5 | 1 |
77+
| environments/orderer-1-kafka-3 | 60 | 6 | 10 | 5 | 1 |
7678

7779

7880
# @doNotDecompose
@@ -82,20 +84,21 @@ Feature: Orderer
8284
And I wait "<BootTime>" seconds
8385
And user "binhn" is an authorized user of the ordering service
8486
When user "binhn" broadcasts "<NumMsgsToBroadcast>" unique messages on "orderer0"
87+
And user "binhn" waits "<WaitTime>" seconds
8588
And user "binhn" connects to deliver function on "orderer0"
8689
And user "binhn" sends deliver a seek request on "orderer0" with properties:
8790
| Start | End |
8891
| Oldest | 2 |
8992
Then user "binhn" should get a delivery from "orderer0" of "<ExpectedBlocks>" blocks with "<NumMsgsToBroadcast>" messages within "<BatchTimeout>" seconds
9093

9194
Examples: Solo Orderer
92-
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime |
93-
| docker-compose-orderer-solo.yml | 20 | 3 | 10 | .5 |
95+
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime | WaitTime |
96+
| docker-compose-orderer-solo.yml | 20 | 3 | 10 | .5 | 0 |
9497

9598
Examples: 1 Kafka Orderer and 1 Kafka Broker
96-
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime |
97-
| environments/orderer-1-kafka-1 | 20 | 3 | 10 | 5 |
99+
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime | WaitTime |
100+
| environments/orderer-1-kafka-1 | 20 | 3 | 10 | 5 | 1 |
98101

99102
Examples: 1 Kafka Orderer and 3 Kafka Brokers
100-
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime |
101-
| environments/orderer-1-kafka-3 | 20 | 3 | 10 | 5 |
103+
| ComposeFile | NumMsgsToBroadcast | ExpectedBlocks | BatchTimeout | BootTime | WaitTime |
104+
| environments/orderer-1-kafka-3 | 20 | 3 | 10 | 5 | 1 |

‎bddtests/steps/orderer_impl.py

+6
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616

1717
import os
1818
import re
19+
import time
1920
import subprocess
2021
import devops_pb2
2122
import fabric_pb2
@@ -50,6 +51,11 @@ def step_impl(context, enrollId, composeService):
5051
streamHelper = userRegistration.connectToDeliverFunction(context, composeService)
5152

5253

54+
@when(u'user "{enrollId}" waits "{waitTime}" seconds')
55+
def step_impl(context, enrollId, waitTime):
56+
time.sleep(float(waitTime))
57+
58+
5359
@then(u'user "{enrollId}" should get a delivery from "{composeService}" of "{expectedBlocks}" blocks with "{numMsgsToBroadcast}" messages within "{batchTimeout}" seconds')
5460
def step_impl(context, enrollId, expectedBlocks, numMsgsToBroadcast, batchTimeout, composeService):
5561
userRegistration = orderer_util.getUserRegistration(context, enrollId)

0 commit comments

Comments
 (0)