Skip to content

Commit 1d87797

Browse files
lhaskinslhaskins
committed
[FAB-3753]: Readme for Behave System Tests
This is the readme for the system and feature behave tests. Also updated Makefile to build cryptogen and configtxgen in the behave-deps target. Added a few packages to the install_behave.sh and a comment that this script is for linux distros that support the apt package manager. Change-Id: I468ee6a8369425705f1680e26c32aad46239e64d Signed-off-by: Latitia M Haskins <[email protected]>
1 parent 5a27382 commit 1d87797

File tree

3 files changed

+287
-1
lines changed

3 files changed

+287
-1
lines changed

‎Makefile

+1-1
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,7 @@ behave-environments: $(BEHAVE_ENVIRONMENT_TARGETS)
157157
$(BEHAVE_ENVIRONMENT_TARGETS):
158158
@docker-compose --file $@/docker-compose.yml build
159159

160-
behave-deps: docker peer build/bin/block-listener behave-environments
160+
behave-deps: docker peer build/bin/block-listener behave-environments configtxgen cryptogen
161161
behave: behave-deps
162162
@echo "Running behave tests"
163163
@cd bddtests; behave $(BEHAVE_OPTS)

‎scripts/install_behave.sh

+10
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
#!/bin/bash
22

3+
#
4+
# This script is used on Debian based linux distros.
5+
# (i.e., linux that supports the apt packaging manager.)
6+
#
7+
38
# Update system
49
apt-get update -qq
510

@@ -11,6 +16,7 @@ apt-get install --yes libyaml-dev
1116

1217
apt-get install --yes python-setuptools
1318
apt-get install --yes python-pip
19+
apt-get install --yes build-essential
1420
pip install --upgrade pip
1521
pip install behave
1622
pip install nose
@@ -25,6 +31,10 @@ pip install --upgrade 'grpcio==0.13.1'
2531

2632
# Pip packages required for some behave tests
2733
pip install ecdsa python-slugify b3j0f.aop
34+
pip install google
35+
pip install protobuf
36+
pip install pyyaml
37+
pip install pykafka
2838

2939
# install ruby and apiaryio
3040
#apt-get install --yes ruby ruby-dev gcc

‎test/feature/README.rst

+276
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,276 @@
1+
Behave tests for Hyperledger Fabric Feature and System Tests
2+
============================================================
3+
4+
.. image:: http://cdn.softwaretestinghelp.com/wp-content/qa/uploads/2007/08/regression-testing.jpg
5+
6+
Behave is a tool used for Behavior Driven Development (BDD) testing. It uses tests (feature files) written in a natural language called Gherkin. The tests are executed using python as the supporting code.
7+
8+
BDD is an agile software development technique that encourages collaboration between developers, QA and non-technical or business participants in a software project. Feel free to read more about `BDD`_.
9+
10+
.. _BDD: http://pythonhosted.org/behave/philosophy.html
11+
12+
13+
This drectory contains a behave implementation of system and feature file testing for Hyperledger Fabric.
14+
15+
Full documentation and usage examples for Behave can be found in the `online documentation`_.
16+
17+
.. _online documentation: http://pythonhosted.org/behave/
18+
19+
20+
Continuous Integration (CI) Execution
21+
-------------------------------------
22+
The following are links to the Jenkins execution of these tests:
23+
* `daily`_
24+
* `weekly`_
25+
* `release`_
26+
27+
.. _daily: https://jenkins.hyperledger.org/view/Daily
28+
.. _weekly: https://jenkins.hyperledger.org/view/Weekly
29+
.. _release: https://jenkins.hyperledger.org/view/Release
30+
31+
32+
Pre-requisites
33+
--------------
34+
You must have the following installed:
35+
* `python`_
36+
* `docker`_
37+
* `docker-compose`_
38+
39+
Ensure that you have Docker for `Linux`_, `Mac`_ or `Windows`_ 1.12 or higher properly installed on your machine.
40+
41+
.. _python: https://www.python.org/
42+
.. _docker: https://www.docker.com/
43+
.. _docker-compose: https://docs.docker.com/compose/
44+
.. _Linux: https://docs.docker.com/engine/installation/#supported-platforms
45+
.. _Mac: https://docs.docker.com/engine/installation/mac/
46+
.. _Windows: https://docs.docker.com/engine/installation/windows/
47+
48+
You can install Behave and additional packages either using the ``scripts/install_behave.sh`` (useful for linux distros that use the apt packaging manager) or following the links for your specific OS environment.
49+
* `pip`_
50+
* `python-setuptools`_
51+
* `python-dev`_
52+
* `build-essential`_
53+
* `behave`_
54+
* `google`_
55+
* `protobuf`_
56+
* `pyyaml`_
57+
58+
.. _pip: https://packaging.python.org/installing/#requirements-for-installing-packages
59+
.. _python-setuptools: https://packaging.python.org/installing/
60+
.. _python-dev: https://packaging.python.org/installing/
61+
.. _build-essential: http://py-generic-project.readthedocs.io/en/latest/installing.html
62+
.. _behave: http://pythonhosted.org/behave/install.html
63+
.. _google: https://pypi.python.org/pypi/google
64+
.. _protobuf: https://pypi.python.org/pypi/protobuf/2.6.1
65+
.. _pyyaml: https://pypi.python.org/pypi/PyYAML
66+
.. _pykafka: https://pypi.python.org/pypi/pykafka
67+
68+
You should also clone the following repositories
69+
* `hyperledger-fabric`_
70+
* `hyperledger-fabric-ca`_
71+
72+
.. _hyperledger-fabric: https://github.com/hyperledger/fabric
73+
.. _hyperledger-fabric-ca: https://github.com/hyperledger/fabric-ca
74+
75+
76+
Getting Started
77+
---------------
78+
Before executing the behave tests, it is assumed that there are docker images and tools that have already been built.
79+
80+
================
81+
Areas of Testing
82+
================
83+
BDD tests are testing functionality and feature behavior. With this in mind, the following are areas that we plan to be covered in these BDD tests:
84+
* Basic setup (Happy Path)
85+
* Orderer Functionality
86+
* solo
87+
* kafka
88+
* Ledgers
89+
* Endorser and committer peers
90+
* Fabric-CA (used for SSL connections)
91+
* Upgrades and Fallbacks
92+
* Bootstrapping
93+
* configtxgen
94+
* cryptogen
95+
* Growing and shrinking networks
96+
* Stopping and Starting components
97+
* … and more (such as different tooling, messages sizes, special scenarios)
98+
99+
The following are not covered in these BDD tests:
100+
* scalability
101+
* performance
102+
* long running tests
103+
* stress testing
104+
* timed tests
105+
106+
107+
======================
108+
Building docker images
109+
======================
110+
When executing tests that are using docker-compose fabric-ca images, be sure to have the fabric-ca docker images built. You must perform a ``make docker`` in the ``/path/to/hyperledger/fabric-ca`` directory.
111+
112+
The docker images for ``peer``, ``orderer``, ``kafka``, and ``zookeeper`` are needed. You must perform a ``make docker`` in the ``/path/to/hyperledger/fabric`` directory.
113+
114+
115+
=========================
116+
Building tool executables
117+
=========================
118+
The **configtxgen** and **cryptogen** tools are used when bootstrapping the networks in these tests. As a result, you must perform a ``make configtxgen && make cryptogen`` in the ``/path/to/hyperledger/fabric`` directory.
119+
120+
121+
How to Contribute
122+
--------------------------
123+
124+
.. image:: http://i.imgur.com/ztYl4lG.jpg
125+
126+
There are different ways that you can contribute in this area.
127+
* Writing feature files
128+
* Writing python test code to execute the feature files
129+
* Adding docker-compose files for different network configurations
130+
131+
===================================
132+
How Do I Write My Own Feature File?
133+
===================================
134+
The feature files are written by anyone who understands the requirements. This can be a business analyst, quality analyst, manager, developer, customer. The file describes a feature or part of a feature with representative examples of expected outcomes and behaviors. These files are plain-text and do not require any compilation. Each feature step maps to a python step implementation.
135+
136+
The following is an example of a simple feature file:
137+
138+
.. sourcecode:: gherkin
139+
140+
Feature: Test to ensure I take the correct accessory
141+
Scenario: Test what happens on a rainy day
142+
Given it is a new day
143+
When the day is rainy
144+
And the day is cold
145+
Then we should bring an umbrella
146+
Scenario Outline: Test what to bring
147+
Given it is a new day
148+
When the day is <weather>
149+
Then we should bring <accessory>
150+
Examples: Accessories
151+
| weather | accessory |
152+
| hot | swimsuit |
153+
| cold | coat |
154+
| cloudy | nothing |
155+
156+
157+
Keywords that are used when writing feature files:
158+
* **Feature**
159+
* The introduction of the different feature test scenarios
160+
* You can have multiple scenarios for a single feature
161+
* **Scenario/Scenario Outline**
162+
* The title and description of the test
163+
* You can run the same test with multiple inputs
164+
* **Given**
165+
* Indicates a known state before any interaction with the system.
166+
* **Avoid talking about user interaction.**
167+
* **When**
168+
* Key actions are performed on the system.
169+
* This is the step which may or may not cause some state to change in your system.
170+
* **Then**
171+
* The observed and expected outcomes.
172+
* **And**
173+
* Can be used when layering any givens, whens, or thens.
174+
175+
176+
========================
177+
Writing python test code
178+
========================
179+
Feature steps used in the feature file scenarios are implemented in python files stored in the “steps” directory. As the python implementation code grows, fewer changes to the code base will be needed in order to add new tests. If you simply want to write feature files, you are free to do so using the existing predefined feature steps.
180+
181+
The behave implementation files are named '*<component>_impl.py*' and the utilities are named '*<action>_util.py*' in the steps directory.
182+
183+
Python implementation steps are identified using decorators which match the keyword from the feature file: 'given', 'when', 'then', and 'and'. The decorator accepts a string containing the rest of the phrase used in the scenario step it belongs to.
184+
185+
186+
.. sourcecode:: python
187+
188+
>>> from behave import *
189+
>>> @given('it is a new day')
190+
... def step_impl(context):
191+
... # Do some work
192+
... pass
193+
>>> @when('the day is {weather}')
194+
... def step_impl(context, weather):
195+
... weatherMap = {'rainy': 'an umbrella',
196+
... 'sunny': 'shades',
197+
... 'cold': 'a coat'}
198+
... context.accessory = weatherMap.get(weather, "nothing")
199+
>>> @then('we should bring {accessory}')
200+
... def step_impl(context, accessory):
201+
... assert context.accessory == accessory, "You're taking the wrong accessory!"
202+
203+
204+
====================
205+
Docker-Compose Files
206+
====================
207+
These docker composition files are used when setting up and tearing down networks of different configurations. Different tests can use different docker compose files depending on the test scenario. We are currently using `version 2 docker compose`_ files.
208+
209+
.. _version 2 docker compose: https://docs.docker.com/compose/compose-file/compose-file-v2/
210+
211+
212+
How to execute Feature tests
213+
----------------------------
214+
There are multiple ways to execute behave tests.
215+
* Execute all feature tests in the current directory
216+
* Execute all tests in a specific feature file
217+
* Execute all tests with a specified tag
218+
* Execute a specific test
219+
220+
221+
**Executes all tests in directory**
222+
::
223+
224+
$ behave
225+
226+
**Executes specific feature file**
227+
::
228+
229+
$ behave mytestfile.feature
230+
231+
**Executes tests labelled with tag**
232+
::
233+
234+
$ behave -t mytag
235+
236+
**Executes a specific test**
237+
::
238+
239+
$ behave -n 'my scenario name'
240+
241+
242+
Helpful Tools
243+
-------------
244+
Behave and the BDD ecosystem have a number of `tools`_ and extensions to assist in the development of tests. These tools include features that will display what feature steps are available for each keyword. Feel free to explore and use the tools, depending on your editor of choice.
245+
246+
.. _tools: http://behave.readthedocs.io/en/latest/behave_ecosystem.html
247+
248+
249+
Helpful Docker Commands
250+
-----------------------
251+
* View running containers
252+
* ``$ docker ps``
253+
* View all containers (active and non-active)
254+
* ``$ docker ps -a``
255+
* Stop all Docker containers
256+
* ``$ docker stop $(docker ps -a -q)``
257+
* Remove all containers. Adding the `-f` will issue a "force" kill
258+
* ``$ docker rm -f $(docker ps -aq)``
259+
* Remove all images
260+
* ``$ docker rmi -f $(docker images -q)``
261+
* Remove all images except for hyperledger/fabric-baseimage
262+
* ``$ docker rmi $(docker images | grep -v 'hyperledger/fabric-baseimage:latest' | awk {'print $3'})``
263+
* Start a container
264+
* ``$ docker start <containerID>``
265+
* Stop a containerID
266+
* ``$ docker stop <containerID>``
267+
* View network settings for a specific container
268+
* ``$ docker inspect <containerID>``
269+
* View logs for a specific containerID
270+
* ``$ docker logs -f <containerID>``
271+
* View docker images installed locally
272+
* ``$ docker images``
273+
* View networks currently running
274+
* ``$ docker networks ls``
275+
* Remove a specific residual network
276+
* ``$ docker networks rm <network_name>``

0 commit comments

Comments
 (0)