Skip to content

Commit 29954d6

Browse files
scottz64scottz64
committed
Orderer Traffic Engine (OTE) FAB-1805
This solves FAB-1805, and partially solves FAB-972 by providing some of the tests for our Orderer Functional Test Plan. Updated 2/16 to add linter corrections and "% complete" progress logs. Updated 2/17 to allow running on multiple architectures and to clean up some tests under development. This OTE tool is useful for Functional Testing the Orderer Service. The user can specify network test configuration parameters, such as the number of orderers, ordererType, total number of transactions to send, etc., and also may set environment variables to override several of the configuration parameters defined in orderer/orderer.yaml such as batchSize and batchTimeout. OTE leverages another tool driver.sh to launch a network, using docker containers for the orderers and kafka-brokers. The driver.sh tool is under development at https://github.com/dongmingh/v1framework and some days is unstable, so we include a working version of driver.sh here, along with its helper file scripts network.json and json2yml.js. It creates a docker-compose file and uses that to launch a network. OTE also creates a CA and PEER, but those are not currently used by OTE. OTE borrows the orderer/sample_clients when it creates Producers to send/broadcast transactions and Consumers to receive/deliver batches. These are created as go-threads for concurrency. OTE counts and verifies totals of transactions and batches, and provides report logs. Users can run OTE from the command line, "go build; ./ote", as well as from "go test" functional tests in file ote_test.go, which will be useful for continuous integration test suites. OTE also supports multi-channels. Below are all the environment variables which are currently supported (and defaults): OTE_TXS 55 OTE_CHANNELS 1 OTE_ORDERERS 1 OTE_KAFKABROKERS 0 OTE_MASTERSPY false CONFIGTX_ORDERER_ORDERERTYPE solo CONFIGTX_ORDERER_BATCHSIZE_MAXMESSAGECOUNT 10 CONFIGTX_ORDERER_BATCHTIMEOUT 10 Examples: git clone https://github.com/hyperledger-fabric.git cd $GOPATH/src/github.com/hyperledger/fabric make native make docker cd $GOPATH/src/github.com/hyperledger/fabric/bddtests/regression/ote go build ./ote CONFIGTX_ORDERER_BATCHSIZE_MAXMESSAGECOUNT=20 ./ote OTE_TXS=100 OTE_CHANNELS=4 ./ote CONFIGTX_ORDERER_ORDERERTYPE=kafka OTE_KAFKABROKERS=3 ./ote go test -run ORD7[79] go test -timeout 90m go get github.com/jstemmer/go-junit-report go test -run ORD7 -v | go-junit-report > report.xml Change-Id: I327c926bf51ac348d61fbe29b4888dbeaf8f55b2 Signed-off-by: Scott Zwierzynski <[email protected]> Signed-off-by: vagrant <[email protected]> Signed-off-by: Scott Zwierzynski <[email protected]>
1 parent 60ecaf1 commit 29954d6

File tree

9 files changed

+2596
-3
lines changed

9 files changed

+2596
-3
lines changed

‎bddtests/regression/go/README.md

+16-3
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,19 @@
1-
## GO tests, scripts, and supporting files
2-
(For all scripts added to this directory, add information below to
3-
describe pre-requisites, setup steps, and usage examples.)
1+
## GO tests, scripts, tools
2+
3+
OTE is the Orderer Traffic Engine application that
4+
launches a network as specified with environment variable parameters,
5+
creates Producer clients to broadcast transactions to all channels on
6+
all orderers,
7+
creates Consumer clients to deliver/receive batches of transactions,
8+
and counts and reports all results.
9+
10+
TDK is the Test Development Kit application test tool and suite of tests for
11+
high level system testing with an eye towards consensus and reliability.
12+
This tool is still under development; it requires enhancements to be converted
13+
from v0.6 to v1.0 interfaces.
14+
15+
For all scripts added to this directory, refer to information below
16+
describing pre-requisites, setup steps, and usage examples.
417

518
## SETUP
619

‎bddtests/regression/go/ote/README.md

+164
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# Orderer Traffic Engine (OTE)
2+
3+
## What does OTE do?
4+
5+
+ This Orderer Traffic Engine (OTE) tool creates and tests the operation of a
6+
hyperledger fabric ordering service.
7+
+ The focus is strictly on the orderers themselves.
8+
No peers are involved: no endorsements or validations or committing to ledgers.
9+
No SDK is used.
10+
11+
+ OTE sends transactions to
12+
every channel on every orderer, and verifies that the correct number
13+
of transactions and blocks are delivered on every channel from every orderer.
14+
+ OTE generates report logs and returns
15+
a pass/fail boolean and a resultSummaryString.
16+
17+
## How does OTE do it?
18+
19+
+ OTE invokes a local copy of the tool driver.sh (including helper files
20+
network.json and json2yml.js) -
21+
which is a close copy of the original version at
22+
https://github.com/dongmingh/v1FabricGenOption.
23+
+ The driver.sh launches an orderer service network per the user-provided
24+
parameters including number of orderers, orderer type,
25+
number of channels, and more.
26+
+ Producer clients are created to connect via
27+
grpc ports to the orderers to concurrently send traffic until the
28+
requested number of transactions are sent.
29+
Each client generates unique transactions - a fraction of the total
30+
requested number of transactions.
31+
+ Consumer clients are created to connect via
32+
grpc ports to the orderers to concurrently receive delivered traffic
33+
until all batches of transactions are tallied.
34+
OTE checks if the correct number of blocks and TXs are delivered
35+
by all the orderers on all the channels
36+
37+
## Prerequisites
38+
- <a href="https://git-scm.com/downloads" target="_blank">Git client</a>
39+
- <a href="https://www.docker.com/products/overview" target="_blank">Docker v1.12 or higher</a>
40+
- [Docker-Compose v1.8 or higher](https://docs.docker.com/compose/overview/)
41+
- GO
42+
43+
Check your Docker and Docker-Compose versions with the following commands:
44+
```bash
45+
docker version
46+
docker-compose version
47+
```
48+
49+
### Prepare binaries and images:
50+
51+
- Alternative 1: Prepare all binaries and images using a script
52+
```bash
53+
cd $GOPATH/src/github.com/hyperledger/fabric/bddtests/regression/ote
54+
./docker_images.sh
55+
```
56+
57+
- Alternative 2: Prepare binaries and images manually
58+
- - Clone the fabric repository, build the binaries and images
59+
```bash
60+
cd $GOPATH/src/github.com/hyperledger/fabric
61+
make native docker
62+
```
63+
- - Clone the fabric-ca repository, build the images
64+
```bash
65+
cd $GOPATH/src/github.com/hyperledger/
66+
67+
# Use ONE of these methods to clone the repository:
68+
go get github.com/hyperledger/fabric-ca
69+
git clone https://github.com/hyperledger/fabric-ca.git
70+
git clone ssh://[email protected]:29418/fabric-ca
71+
72+
cd $GOPATH/src/github.com/hyperledger/fabric-ca
73+
make docker
74+
```
75+
76+
### Environment Variables for test setup, with defaults:
77+
```
78+
OTE_TXS 55
79+
OTE_CHANNELS 1
80+
OTE_ORDERERS 1
81+
OTE_KAFKABROKERS 0
82+
OTE_MASTERSPY false
83+
OTE_PRODUCERS_PER_CHANNEL 1
84+
```
85+
86+
### Environment Variables for configuration
87+
Find default values of all variables in hyperledger/fabric/orderer/orderer.yaml
88+
and hyperledger/fabric/peer/core.yaml.
89+
```
90+
CONFIGTX_ORDERER_ORDERERTYPE solo
91+
CONFIGTX_ORDERER_BATCHSIZE_MAXMESSAGECOUNT 10
92+
CONFIGTX_ORDERER_BATCHTIMEOUT 10
93+
94+
Others:
95+
CORE_LOGGING_LEVEL <unset>
96+
CORE_LEDGER_STATE_STATEDATABASE leveldb
97+
CORE_SECURITY_LEVEL 256
98+
CORE_SECURITY_HASHALGORITHM SHA2
99+
```
100+
101+
## Execute OTE on shell command line
102+
There are several environment variables to control the test parameters,
103+
such as number of transactions, number of orderers, ordererType, and more.
104+
To see an example test using default settings, simply execute the following.
105+
```bash
106+
cd $GOPATH/src/github.com/hyperledger/fabric/bddtests/regression/ote
107+
go build
108+
./ote
109+
CONFIGTX_ORDERER_BATCHSIZE_MAXMESSAGECOUNT=20 ./ote
110+
OTE_TXS=100 OTE_CHANNELS=4 ./ote
111+
CONFIGTX_ORDERER_ORDERERTYPE=kafka OTE_KAFKABROKERS=3 ./ote
112+
```
113+
114+
Choose which variables to modify from default values. For example:
115+
116+
+ This test will create eight Producer clients.
117+
+ Each Producer will send 125 transactions to a different orderer and channel.
118+
+ 250 total TXs will be broadcast on each channel.
119+
+ 500 total TXs will be broadcast to each orderer.
120+
+ Four Consumer clients will be created to receive the delivered
121+
batches on each channel on each orderer.
122+
+ 50 batches (with 10 TX each) will be delivered on channel 0, and
123+
a different 50 batches will be delivered on channel 1. On both Orderers.
124+
+ 100 batches will be received on every orderer; this is the sum of the
125+
totals received on each channel on the orderer.
126+
```bash
127+
OTE_TXS=1000 OTE_CHANNELS=4 OTE_ORDERERS=2 CONFIGTX_ORDERER_ORDERERTYPE=kafka ./ote
128+
```
129+
130+
## Execute OTE GO Tests
131+
The tester may optionally define environment variables to
132+
set the test parameters and to
133+
override certain orderer configuration parameters.
134+
Then use "go test" to execute Test functions
135+
to execute either one test, or all go tests, or
136+
a subset of existing functional go tests using a regular expression
137+
to choose tests in local test files.
138+
```bash
139+
cd $GOPATH/src/github.com/hyperledger/fabric/bddtests/regression/ote
140+
go test -run ORD77
141+
go test -run ORD7[79]
142+
go test -run batchSz -timeout 20m
143+
go test -timeout 90m
144+
145+
go get github.com/jstemmer/go-junit-report
146+
go test -run ORD7 -v | go-junit-report > report.xml
147+
148+
```
149+
150+
## Execute OTE GO Tests for Continuous Improvement
151+
Optionally, one can translate the "go test" output to xml for reports.
152+
This is useful for automated test suites that are automatically
153+
executed from Jenkins by Continuous Improvement processes.
154+
155+
#### Pre-requisite to convert "go test" output to xml
156+
```bash
157+
cd $GOPATH/src/github.com/hyperledger/fabric/bddtests/regression/ote
158+
go get github.com/jstemmer/go-junit-report
159+
```
160+
#### Example command to execute all "go tests" and convert to xml:
161+
```
162+
cd $GOPATH/src/github.com/hyperledger/fabric/bddtests/regression/ote
163+
go test -v -timeout 120m | go-junit-report > ote_report.xml
164+
```

‎bddtests/regression/go/ote/docker_images.sh

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
#!/bin/bash
2+
# Clone fabric git repository
3+
#############################
4+
echo "Fabric Images"
5+
cd $GOPATH/src/github.com/hyperledger/fabric
6+
#FABRIC_COMMIT=$(git log -1 --pretty=format:"%h")
7+
make docker && make native
8+
9+
# Clone fabric-ca git repository
10+
################################
11+
echo "Ca Images"
12+
CA_REPO_NAME=fabric-ca
13+
cd $GOPATH/src/github.com/hyperledger/
14+
if [ ! -d "$CA_REPO_NAME" ]; then
15+
git clone --depth=1 https://github.com/hyperledger/$CA_REPO_NAME.git
16+
#CA_COMMIT=$(git log -1 --pretty=format:"%h")
17+
fi
18+
cd $CA_REPO_NAME
19+
make docker
20+
echo "List of all Images"
21+
docker images | grep hyperledger

‎bddtests/regression/go/ote/driver.sh

+170
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
#!/bin/bash
2+
3+
InvalidArgs=0
4+
5+
#init var
6+
nBroker=0
7+
nPeer=1
8+
9+
while getopts ":l:d:w:x:b:c:t:a:o:k:p:" opt; do
10+
case $opt in
11+
# peer environment options
12+
l)
13+
CORE_LOGGING_LEVEL=$OPTARG
14+
export CORE_LOGGING_LEVEL=$CORE_LOGGING_LEVEL
15+
echo "CORE_LOGGING_LEVEL: $CORE_LOGGING_LEVEL"
16+
;;
17+
d)
18+
db=$OPTARG
19+
echo "ledger state database type: $db"
20+
;;
21+
w)
22+
CORE_SECURITY_LEVEL=$OPTARG
23+
export CORE_SECURITY_LEVEL=$CORE_SECURITY_LEVEL
24+
echo "CORE_SECURITY_LEVEL: $CORE_SECURITY_LEVEL"
25+
;;
26+
x)
27+
CORE_SECURITY_HASHALGORITHM=$OPTARG
28+
export CORE_SECURITY_HASHALGORITHM=$CORE_SECURITY_HASHALGORITHM
29+
echo "CORE_SECURITY_HASHALGORITHM: $CORE_SECURITY_HASHALGORITHM"
30+
;;
31+
32+
# orderer environment options
33+
b)
34+
CONFIGTX_ORDERER_BATCHSIZE_MAXMESSAGECOUNT=$OPTARG
35+
export CONFIGTX_ORDERER_BATCHSIZE_MAXMESSAGECOUNT=$CONFIGTX_ORDERER_BATCHSIZE_MAXMESSAGECOUNT
36+
echo "CONFIGTX_ORDERER_BATCHSIZE_MAXMESSAGECOUNT: $CONFIGTX_ORDERER_BATCHSIZE_MAXMESSAGECOUNT"
37+
;;
38+
c)
39+
CONFIGTX_ORDERER_BATCHTIMEOUT=$OPTARG
40+
export CONFIGTX_ORDERER_BATCHTIMEOUT=$CONFIGTX_ORDERER_BATCHTIMEOUT
41+
echo "CONFIGTX_ORDERER_BATCHTIMEOUT: $CONFIGTX_ORDERER_BATCHTIMEOUT"
42+
;;
43+
t)
44+
CONFIGTX_ORDERER_ORDERERTYPE=$OPTARG
45+
export CONFIGTX_ORDERER_ORDERERTYPE=$CONFIGTX_ORDERER_ORDERERTYPE
46+
echo "CONFIGTX_ORDERER_ORDERERTYPE: $CONFIGTX_ORDERER_ORDERERTYPE"
47+
if [$nBroker == 0 ] && [ $CONFIGTX_ORDERER_ORDERERTYPE == 'kafka' ]; then
48+
nBroker=1 # must have at least 1
49+
fi
50+
;;
51+
52+
# network options
53+
a)
54+
Req=$OPTARG
55+
echo "action: $Req"
56+
;;
57+
k)
58+
nBroker=$OPTARG
59+
echo "# of Broker: $nBroker"
60+
;;
61+
p)
62+
nPeer=$OPTARG
63+
echo "# of peer: $nPeer"
64+
;;
65+
o)
66+
nOrderer=$OPTARG
67+
echo "# of orderer: $nOrderer"
68+
;;
69+
70+
# else
71+
\?)
72+
echo "Invalid option: -$OPTARG" >&2
73+
InvalidArgs=1
74+
;;
75+
:)
76+
echo "Option -$OPTARG requires an argument." >&2
77+
InvalidArgs=1
78+
;;
79+
esac
80+
done
81+
82+
83+
if [ $InvalidArgs == 1 ]; then
84+
echo "Usage: "
85+
echo " ./driver.sh [opt] [value] "
86+
echo " network variables"
87+
echo " -a: action [create|add] "
88+
echo " -p: number of peers "
89+
echo " -o: number of orderers "
90+
echo " -k: number of brokers "
91+
echo " "
92+
echo " peer environment variables"
93+
echo " -l: core logging level [(deafult = not set)|CRITICAL|ERROR|WARNING|NOTICE|INFO|DEBUG]"
94+
echo " -w: core security level [256|384]"
95+
echo " -x: core security hash algorithm [SHA2|SHA3]"
96+
echo " -d: core ledger state DB [goleveldb|couchdb] "
97+
echo " "
98+
echo " orderer environment variables"
99+
echo " -b: batch size [10|msgs in batch/block]"
100+
echo " -t: orderer type [solo|kafka] "
101+
echo " -c: batch timeout [10s|max secs before send an unfilled batch] "
102+
echo " "
103+
exit
104+
fi
105+
106+
if [ $nBroker -gt 0 ] && [ $CONFIGTX_ORDERER_ORDERERTYPE == 'solo' ]; then
107+
echo "reset Kafka Broker number to 0 due to the CONFIGTX_ORDERER_ORDERERTYPE=$CONFIGTX_ORDERER_ORDERERTYPE"
108+
nBroker=0
109+
fi
110+
111+
#OS
112+
##OSName=`uname`
113+
##echo "Operating System: $OSName"
114+
115+
116+
dbType=`echo "$db" | awk '{print tolower($0)}'`
117+
echo "action=$Req nPeer=$nPeer nBroker=$nBroker nOrderer=$nOrderer dbType=$dbType"
118+
VP=`docker ps -a | grep 'peer node start' | wc -l`
119+
echo "existing peers: $VP"
120+
121+
122+
echo "remove old docker-composer.yml"
123+
rm -f docker-compose.yml
124+
125+
# form json input file
126+
if [ $nBroker == 0 ]; then
127+
#jsonFILE="network_solo.json"
128+
jsonFILE="network.json"
129+
else
130+
# jsonFILE="network_kafka.json"
131+
jsonFILE="network.json"
132+
fi
133+
echo "jsonFILE $jsonFILE"
134+
135+
# create docker compose yml
136+
if [ $Req == "add" ]; then
137+
N1=$[nPeer+VP]
138+
N=$[N1]
139+
VPN="peer"$[N-1]
140+
else
141+
N1=$nPeer
142+
N=$[N1 - 1]
143+
VPN="peer"$N
144+
fi
145+
146+
## echo "N1=$N1 VP=$VP nPeer=$nPeer VPN=$VPN"
147+
148+
node json2yml.js $jsonFILE $N1 $nOrderer $nBroker $dbType
149+
150+
## sed 's/-x86_64/TEST/g' docker-compose.yml > ss.yml
151+
## cp ss.yml docker-compose.yml
152+
# create network
153+
if [ $Req == "create" ]; then
154+
155+
#CHANGED FROM ORIG SCRIPT
156+
#docker-compose -f docker-compose.yml up -d --force-recreate $VPN cli
157+
docker-compose -f docker-compose.yml up -d --force-recreate $VPN
158+
for ((i=1; i<$nOrderer; i++))
159+
do
160+
tmpOrd="orderer"$i
161+
docker-compose -f docker-compose.yml up -d $tmpOrd
162+
done
163+
fi
164+
165+
if [ $Req == "add" ]; then
166+
docker-compose -f docker-compose.yml up -d $VPN
167+
168+
fi
169+
170+
exit

0 commit comments

Comments
 (0)