Repository files navigation

This repository contains all of the code for theBatfish enabled CI pipeline demonstration from Ansiblefest 2019, including a script to initialize the Gitlab repository from where the CI tests are run.

Each time a commit is made to the Gitlab repository, device configs are generated and the generated configs are evaluated against defined policies.

• The device configs are generated using jinja2 templates in thetemplates folder and input data in theinputs folder. The code for configuration generation is in thecode folder.
• Batfish supports both Ansible and Python (pytest) based policies. Thepolicies folder contains pytest-based policies, and theansible-policies folder contains Ansible-based policies. The setup for pytest policies (conftest.py) initializes a Batfish snapshot with generated device configs. Ansible policy playbooks assume that the snapshot has already been initialized.

The exact sequence of commands to generate configs and run policies are intemplate.gitlab-ci.yml file, which is uploaded to the Gitlab repository as its pipeline file (.gitlab-ci.yml) when the repository is initialized.

Network changes are proposed using Ansible playbooks in theplaybooks folder. These playbooks change the inputs used to generate device configs. When these changes are committed to Gitlab, new configurations are generated and evaluated as described above.

• Access to Gitlab server and runner. See below for local setup.
  • In Gitlab, create a new project
    • Recommend naming itaf19-template
  • Register the runner with the project
• BF server running on localhost
• Clone (or Download) this repository onto your local machine
• Install python dependencies listed inrequirements.txt file
  • pip install -r requirements.txt

Note:You can reduce execution time of the demo by leveraging an existing virtual environment that has the necessary dependencies install.To leverage an existing environment, edit thebefore_script stage of the Gitlab-CI pipeline filetemplate.gitlab-ci.yml.

The instructions below assume that you are installing a Gitlab server locally. If you use the online service, you can skip this part.

Make sure that you have Docker installed on your laptop

• Run the docker image

docker run --detach \  --publish 443:443 --publish 80:80 --publish 22:22 \  --hostname localhost \  --name gitlab \  --restart always \  --volume ${HOME}/gitlab/config:/etc/gitlab \  --volume ${HOME}/gitlab/logs:/var/log/gitlab \  --volume ${HOME}/gitlab/data:/var/opt/gitlab \  gitlab/gitlab-ce:latest

The initialization process may take a long time. You can track this process with the commandsudo docker logs -f gitlab

• Navigate tohttp://localhost/

  • Create an admin password

  • Register a new user

  • Create a new project

    • AF19-Template

    Initialize a README, but leave them private

  • Add an SSH key to your profile so you can clone the repos

    • Via user icon on top right -> settings -> ssh key

The demo requires a local clone of the GitLab repository that you just created.

Steps:

• Go to the project page
• Click onClone (towards the top right)

  • Copy theClone with SSH URL to clipboard

    

• Open a terminal window
  • Typegit clone git@localhost:samir-demo/af19-template.git (replace thegit@... with what you copied from the Project page)
  • All of the playbooks referenced in the demo below, will need to be executed from the directory of this local clone.

sudo curl --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-darwin-amd64sudo chmod +x /usr/local/bin/gitlab-runner

This requires getting a token from Gitlab for the project.Steps:

• Go to the project page

• Settings (left side menu) —> CI/CD -> Expand Runners

  

• Rungitlab-runner register

  • Enter the URL:http://localhost/
  • Enter the project specific Gitlab token from above
  • Accept default value for description
  • Accept the default value for tags
  • Entershell as the executor

• gitlab-runner run if the process is not already running
• gitlab-runner restart if the process is already running

Set the following environment variable:

• GIT_TEMPLATE - This is the Git Clone URL for the repo you setup in Gitlab
  • Example:git@localhost:samir-demo/af19-template.git

Then, initialize your Gitlab repo by running the following command from the local clone of thebatfish/af19-demo repository:

bash setup-gitlab-repo.sh

NOTE: This will reset the GitLab repository to the base state, so you can run the demo scenarios again.

To run the demos you will need to have a local clone of theaf19-template repo.Both demo scenarios require you to run an Ansible playbook from this directory

The first demo involves adding a new leaf to the existing DC fabric. Run the following command:

ansible-playbook -i playbooks/inventory playbooks/add_leaf.yml

Enter the following information when prompted:

• Enter the number for the new leaf router:90
• Enter the BGP AS number for the new leaf router:65009
• Enter commit message [adding leaf90 to fabric]:
  • this will accept the default commit message

And that's all you need to do to start the demo. In about 2-3 minutes you will see a new snapshot in Batfish.

Navigate to the Gitlab projectAF19-Template to see the pipeline execution logs.

• Thebuild stage of the pipeline builds all of the configurations.
• Thetest phase uploads the configurations to the BF server and evaluates the policies.
  • This is where you will see the results of the BF policy execution, underpolicy_eval

You should see 2 failures:

• Unique BGP AS
• All leaf routers have all host subnets

The leaf routers do not havebgp allow-as in configured, soleaf09 andleaf90 reject either others routes.

To correct the error, re-run the ansible playbook with the correct BGP ASN -65090.

• For correct operation each leaf should set the BGP AS to be equal to it's id/number

The first demo involves adding a new leaf to the existing DC fabric. Run the following command:

ansible-playbook -i playbooks/inventory playbooks/request_service.yml

Enter the following information when prompted:

• Direction of request flow. IN for opening up access to internal service from the internet. OUT for opening up access to external service:IN
• Enter the destination IP address/prefix:10.100.10.0/24
• Enter the source IP address/prefix:0.0.0.0/0
• Enter the IP protocol (TCP or UDP):tcp
• Enter the destination port:443
• Enter the name of the application:SSL
• Enter the change request id:CHG12345
• Enter commit message [Change request CHG123345 for access to 10.100.10.0/24 application SSL from 0.0.0.0/0]:
  • this will accept the default commit message

And that's all you need to do to start the demo. In about 2-3 minutes you will see a new snapshot in the BFE Dashboard with policy execution complete.

Navigate to the Gitlab projectAF19-Template to see the pipeline execution logs.

• Thebuild stage of the pipeline builds all of the configurations.
• Thetest phase uploads the configurations to the BF server and evaluates the policies.
  • This is where you will see the results of the BF policy execution, underpolicy_eval

You should see 1 test failure:

• Protect internal services (TCP)
  • Private subnets (new-TCP) reachability

This test fails because the network10.100.0.0/16 has been designated as internal only. So traffic from the internet is allowed to it.

NOTE: If you want to re-run the scenario and select another subnet, you must provide a newsource IP prefix andapplication since Capirca doesn't allow multiple entries for the same named object even if the definition is identical

To restart the demo, go back to the window where you ran the demo setup script and re-run the script.