Repository files navigation

PTF is a Python based dataplane test framework. It is based on unittest, whichis included in the standard Python distribution.

This document is meant to provide an introduction to the framework, discuss thebasics of running tests and to provide examples of how to add tests.

Most of the code was taken from theOFTestframework. However, PTF focuses on thedataplane and is independent of OpenFlow. We also added severalnew features.

Before you start pushing new changes to this repository, you should noticethat the entiresrc/ code is automatically formatted with Black.Our GitHub Action pipeline will verify that code is correctlyformatted and fail if not.

Two separate targets in makefile were prepared to make our work easier.If you want to run a check, typemake format-check, but if you want toreformat your code, please usemake format.

Black is listed in therequirements-dev.txt. To install it locally, youcan usemake set-dev orpip install -r requirements-dev.txt.More information about Black, you find atBlack's GitHub Page

The following software is required to run PTF:

• Python 3.x

The following packages are optional for running PTF:

• Scapy 2.5.0 (you may also use the includedbf_pktpy module instead)
• pypcap (VLAN tests will fail without this)
• tcpdump (Scapy will complain if it's missing)

Root/sudo privilege is required on the host, in order to runptf.

The default packet manipulation module forptf isScapy. To install it use:

pip install scapy==2.5.0

To enable VLAN tests, you need to installpypcap:

pip install pypcap

For developer purpose, you should installrequirements-dev.txt with:

pip install -r requirements-dev.txt

Thetcpdump is optional, but to install it use:

# on CentOSyum install tcpdump# on Debian baseapt-get install tcpdump

The Python modulebf_pktpy is included as part of the ptf package.It was developed as an alternative toscapy. The tradeoffs of usingbf_pktpy vs.scapy are:

• scapy implements more functionality, but is licensed under thecopyleft GNU General Public License v2.0 (seehttps://github.com/secdev/scapy/blob/master/LICENSE), so may beundesirable in use cases where you wish your tests to be releasedunder a different license.
• bf_pktpy implements only a small subset of the functionality ofscapy, but it does include support for very commonly-used packetheaders. It is released under an Apache 2.0 license.

If you want to usebf_pktpy when running the commandptf from thecommand line, provide the-pmm option as shown below.

ptf -pmm bf_pktpy.ptf.packet_pktpy<othercommand line arguments>

If you want to write a Python program that importsptf and causes itto usebf_pktpy instead of the defaultscapy, you can do so asfollows in your Python code:

importptfptf.config["packet_manipulation_module"]="bf_pktpy.ptf.packet_pktpy"importptf.packet

The above methods are the highest precedence way of choosing thepacket manipulation module used byptf. If you do not use thosemethods, another way is to assign the packet manipulation module nameto the environment variablePTF_PACKET_MANIPULATION_MODULE, e.g. inBash:

export PTF_PACKET_MANIPULATION_MODULE="bf_pktpy.ptf.packet_pktpy"

When running such a program, you should see the following line printedto standard output confirming that it is usingbf_pktpy instead ofscapy:

Using packet manipulation module: bf_pktpy.ptf.packet_pktpy

If instead you see this line of output,ptf is usingscapy:

Using packet manipulation module: ptf.packet_scapy

Once you have written tests and your switch is running, you can run 'ptf'. Use--help to see command line switches.

For example:

sudo ./ptf --test-dir mytests/ --pypath $PWD \ --interface 0@veth1 --interface 1@veth3 --interface 2@veth5 \ --interface 3@veth7 --interface 4@veth9 --interface 5@veth11 \ --interface 6@veth13 --interface 7@veth15 --interface 8@veth17

This will run all the tests included in themytests directory. The--pypathoption can be used to easily add directories to the Python PATH. This is usefulif you use a Python interface to configure your data plane (as part of yourtests). The--interface option (or-i) can be used to specify the interfaceson which to inject packets (along with the corresponding port number).

PTF can be installed withpip:

# Install the latest versionpip install ptf# Install specific versionpip install ptf==0.9.1

You can also install a local copy of PTF withpip install ..

Take a look at theexample directory. This is not a working example as it is(the switch is not included), but it will show you how to write tests. Thisdirectory contains the following:

• run_client.sh: a wrapper aroundptf
• switch_sai_thrift: empty directory, this is where the Python bindings toprogram the switch's control plane would be copied
• mytests/sai_base_test.py: a wrapper Python class around PTF's BaseTestclass. It is the base class for all the tests we added tomytests/switch.py
• mytests/switch.py: some example tests

If you want to run the example, you will need to obtainp4factory. For the following, I willassume that you cloned the repository and installed the dependencies. I willassume that environment variableP4FACTORY contains the path to the clonedrepository.

First, you need to create the required veths:

cd $P4FACTORY/tools/sudo ./veth_setup.sh

The next step is to compile the target switch and to run it:

cd $P4FACTORY/targets/switch/make bm-switchsaisudo ./behavioral-model

Finally, you can run the example tests:

cd <ptf-dir>/example/sudo ../ptf --test-dir mytests/ \ --pypath $P4FACTORY/targets/switch/tests/pd_thrift/ --interface 0@veth1 --interface 1@veth3 --interface 2@veth5 \ --interface 3@veth7 --interface 4@veth9 --interface 5@veth11 \ --interface 6@veth13 --interface 7@veth15 --interface 8@veth17

We added the following features to the base OFTest framework:

They can be used to discard some of the packets received from the switch. Take alook atsai_base_test.py for an example. Youwill see the following codetestutils.add_filter(testutils.not_ipv6_filter)which tells PTF to discard received IPv6 packets. You can add your own filters(they have to be callable Python objects which take a Scapy packet as input).

A PTF test -just like an OFTest test- matches the received packets againstexpected packets. This is an exact match. However, sometimes one does not careabout all the fields in the packets. PTF introduces the Mask class which letsyou specified some field you do not care about when performing the match. Forexample:

import maskm = mask.Mask(expected_pkt)m.set_do_not_care_scapy(IP, 'ttl')verify_packets(<test>, m, <port list>)

A timeout for test cases can be specified using the--test-case-timeoutcommand line option. This timeout must be expressed in seconds. A timeout of 0is the same as no timeout (the default). If the timeout expires before the testis done executing, an exception will be raised and the test counts as anerror. A timeout can also be specified for each individual test case, using the@testtimeout decorator, which needs to be imported fromptf.testutils. Thistimeout takes precedence over the global timeout passed on the command line.

By default,ptf usesScapy as the packet manipulation module, but it canalso operate on a different one, e.g. the includedbf_pktpy module.

Such a modulemust define/implement the same symbols, as defined inScapyimplementation of packet. Most of them are just names of most common frameheaders (Ether, IP, TCP, UDP, ...).

The default implementation can be found in file/src/ptf/packet_scapy.py. It can be used as areference when implementing your own version.

To use another packet manipulation module, one needs toprovide it as argument-pmm or--packet-manipulation-module when running theptf binary.

sudo ./ptf <other parameters> -pmm foo.packet_foo

Please make sure that this module is loaded into the runtime before runningany tests.

You can achieve parallelization by splitting tests into N groups and running them with separate PTF processes.Each PTF instance will run disjoint subset of all selected tests.

For example to run specific set of tests across 3 PTF instances:

$ ssh mynode0 sudo ./ptf --test-dir mytests --num-shards 3 --shard-id 0 all ^other &$ ssh mynode1 sudo ./ptf --test-dir mytests --num-shards 3 --shard-id 1 all ^other &$ ssh mynode2 sudo ./ptf --test-dir mytests --num-shards 3 --shard-id 2 all ^other &

The "platform" is a configuration file (written in Python) that tells PTF how tosend packets to and receive packets from the dataplane of the switch.

The default platform,eth, uses Linux Ethernet interfaces and is configuredwith the-i option (or--interface). Pass the option as-i ofport@interface, for example-i 1@eth1. If no-i options are given the thedefault configuration uses vEths.

Another common platform,remote, provides support for testing of switches on adifferent host. This can be useful for cases where interfaces are not availableon one host (i.e. they're not bound to a Linux interface driver) or where PTFcannot run on the same host (unsupported OS, missing software, etc.).

This can be enable by modifying theplatforms/remote.py file to point to 4NICs on the host running PTF, like so:

remote_port_map = {    (0, 23) : "eth2", # port 23 of device 0 is connected to physical port on the server eth2    (0, 24) : "eth3", # port 24 of device 0 is connected to physical port on the server eth3    (0, 25) : "eth4",    (0, 26) : "eth5"}

We introduce a new platform,nn, which uses [nanomsg] (http://nanomsg.org/) tosend and receive packet to the switch. We support IPC and TCP nanomsgsockets. When using this platform, you need to make sure that the Python package[nnpy] (https://github.com/nanomsg/nnpy) is installed. Withnn, do not use--interface, instead use--device-socket. For each device, you need toprovide a list of enabled ports and a nanomsg socket address. For example:

--device-socket 0-{1,2,5-8}@ipc:///tmp/bmv2_packets_1.ipc

This command will enable ports 1, 2, 5, 6, 7, 8 on device 0. Packets for device0 will be captured and send on IPC socketipc:///tmp/bmv2_packets_1.ipc.

There is a facility for passing test-specific parameters into tests that works as follows. On the command line, give the parameter

--test-params="key1=17;key2=True"

You can then access these parameters in your tests' Pyhton code using thefollowing code:

import ptf.testutils as testutils# Returns a dictionary which includes all your parameterstest_params = testutils.test_params_get()# Returns the value of the parameter "param", or None if not foundparam_value = testutils.test_param_get("param")

Take a look atsai_base_test.py for anexample.

It is very easy to create groups of tests, using the providedgroup Pythondecorator. Simply decorate your test with@group(<name of group>).

Take a look atswitch.py for an example.

One given test can belong to several groups. You can choose to run only thetests belonging to a given group using a command like this one:

sudo ./ptf --test-dir mytests/ --pypath $PWD <name of group>

We also provide a convenientdisabled decorator for tests.

The original OFTest was meant to unit test a single OF-compliant switch. WithPTF, we tried to add support for testing a network of several devices. If you donot intend to use this multi-device feature, you do not need to worry about it,it should not impact you. If you want to leverage this feature, here is what youneed to do:

• when adding interfaces, instead of writing<port_number>@<interface_name>,you need to write<device_number>-<port_number>@<interface_name>
• when sending a packet, the port number becomes a tuple (device, port):send_packet(self, (<device_number>, <port_number>), pkt)
• theverify_* functions where also updated to include device information. Forexample:verify_packets(self, pkt, device_number=<device_number>, ports=<port_list>). For more information, you can take a look at thesefunctions insrc/ptf/dataplane.py.