Repository files navigation

This is a framework to benchmark the communication middleware supported byoslo.messaging. It's primary goal is to address the evaluation ofhttps://docs.openstack.org/performance-docs/latest/test_plans/massively_distribute_rpc/plan.html.

It is build on top of:

• EnOSlib. This library helps to describe the experimental workflow and enforce it : from the deployment to the performance metrics analysis.
• ombt. This will coordinate the benchmark once all the agents are up and running.

From a high level point of view the framework is able to deploy

• a communication bus (e.g RabbitMQ, qdr aka qpid-dispatch-router),
• a set of client/server that will communicate
• start a benchmark while gathering metrics

A typical test consists in the following components:

Client 1---------+      +----------------------+     +-----> Server 1                 |      |                      |     |                 +----> |  Communication       | ----+-----> Server 2Client 2--------------> |  Middleware          |     |                 +----> |  (e.g qdr, rabbitms) |     |...              |      |                      |     |                 |      +----------------------+     +------> Server nClient n---------+              |                             /  \                                                         /    \                           |                         /       \  --  --  --  --  -- Monitoring --  --  --  --  --

• Clone the repository:

git clone https://github.com/msimonin/ombt-orchestratorcd ombt-orchestrator

• Install the dependencies

pip install -U pippip install -e .

On Grid'5000 you can launch this command from any frontend.

The default configurations are currently defined in theconf.yaml file.

> ooUsage: oo [OPTIONS] COMMAND [ARGS]...Options:  --help  Show this message and exit.Commands:  backup       Backup environment logs [after test_case_*].  campaign     Perform a TEST according to the (swept)...  deploy       Claim resources from a PROVIDER and configure...  destroy      Destroy all the running dockers (keeping...  g5k          Claim resources on Grid'5000 (frontend).  inventory    Generate the Ansible inventory [after g5k,...  prepare      Configure available resources [after g5k,...  test_case_1  Run the test case 1: one single large...  test_case_2  Run the test case 2: multiple distributed...  test_case_3  Run the test case 3: one single large...  test_case_4  Run the test case 4: multiple distributed...  vagrant      Claim resources on vagrant (localhost).

• Deploying and launching the benchmark (default driverbroker is defined in the configuration file)

# default confs.yaml on $PWD will be read> oo deploy --driver=broker vagrant# Launch the one benchmark> oo test_case_1 --nbr_clients 10 --nbr_servers 2

Adapt to the relevant provider (e.gg5k)

• Real-time metrics visualisation

Grafana is available on the port 3000 of the control node (check the inventory file).

• Backuping the environment

> oo backup

The files retrieved by this action are located incurrent/backup dir by default.

• Some cleaning and preparation for the next run

# Preparing the next run by cleaning the environment> oo destroy> oo deploy vagrant# Next run> oo test_case_1 --nbr_clients 20 --nbr_servers 2

It's possible to force an experimentation dir with--env mydir

Note also that scripting from python is also possible using the function defined intask.py

## Workflow to run a campaign

• A campaign is a batch execution of several configurations for a given test case.Deployment and execution of a benchmark is read from a configuration file. For example,to run the first test case enabled on the framework run:

> oo campaign --provider g5k test_case_1

• Alternatively a campaign can be executed in aincremental mode in which deployments areperformed only when a differentdriver orcall_type is defined. Incremental campaignsare executed with a different semantic on the parameters defined in the configuration.With the incremental option the semantics is based on the combination of parameters bymeans of a dot product between a set of them in the configuration file (i.e., azipoperation between the lists of parameters). These parameters are defined by test caseas follows:

  • Test case 1:nbr_clients,nbr_servers andpause
  • Test case 2:nbr_topics andpause
  • Test case 3:nbr_clients,nbr_servers andpause (onlyrpc-cast calls)
  • Test case 4:nbr_topics andpause (onlyrpc-cast calls)

• To execute an incremental campaign be sure to use the ombt versionmsimonin/ombt:singletoninstead of the default and execute:

> oo campaign --incremental --provider g5k test_case_1

• Bound clients or servers to specific bus agents:

To bind ombt-clients to a specific bus instance you can declare the followingroles: [bus, bus-client].

Following the same idea ombt-servers can be bound to a specific bus instance usingroles: [bus, bus-server]