Fabric¶

Fabric is a library for simplifying systemadministration tasks. While Chef and Puppet tend to focus on managing serversand system libraries, Fabric is more focused on application level tasks suchas deployment.

Install Fabric:

$ pip install fabric

The following code will create two tasks that we can use:memory_usage anddeploy. The former will output the memory usage on each machine. Thelatter will ssh into each server, cd to our project directory, activate thevirtual environment, pull the newest codebase, and restart the applicationserver.

fromfabric.apiimportcd,env,prefix,run,taskenv.hosts=['my_server1','my_server2']@taskdefmemory_usage():run('free -m')@taskdefdeploy():withcd('/var/www/project-env/project'):withprefix('. ../bin/activate'):run('git pull')run('touch app.wsgi')

With the previous code saved in a file namedfabfile.py, we can checkmemory usage with:

$ fab memory_usage[my_server1] Executing task 'memory'[my_server1] run: free -m[my_server1] out:              total     used     free   shared  buffers   cached[my_server1] out: Mem:          6964     1897     5067        0      166      222[my_server1] out: -/+ buffers/cache:     1509     5455[my_server1] out: Swap:            0        0        0[my_server2] Executing task 'memory'[my_server2] run: free -m[my_server2] out:              total     used     free   shared  buffers   cached[my_server2] out: Mem:          1666      902      764        0      180      572[my_server2] out: -/+ buffers/cache:      148     1517[my_server2] out: Swap:          895        1      894

and we can deploy with:

$ fab deploy

Additional features include parallel execution, interaction with remoteprograms, and host grouping.

Fabric Documentation

Salt¶

Salt is an open source infrastructure managementtool. It supports remote command execution from a central point (master host)to multiple hosts (minions). It also supports system states which can be usedto configure multiple servers using simple template files.

Salt supports Python versions 2.6 and 2.7 and can be installed via pip:

$ pip install salt

After configuring a master server and any number of minion hosts, we can runarbitrary shell commands or use pre-built modules of complex commands on ourminions.

The following command lists all available minion hosts, using the ping module.

$ salt'*' test.ping

The host filtering is accomplished by matching the minion id,or using the grains system. Thegrainssystem uses static host information like the operating system version or theCPU architecture to provide a host taxonomy for the Salt modules.

The following command lists all available minions running CentOS using thegrains system:

$ salt -G'os:CentOS' test.ping

Salt also provides a state system. States can be used to configure the minionhosts.

For example, when a minion host is ordered to read the following state file,it will install and start the Apache server:

apache:pkg:-installedservice:-running-enable:True-require:-pkg:apache

State files can be written using YAML, the Jinja2 template system or pure Python.

Salt Documentation

Psutil¶

Psutil is an interface to differentsystem information (e.g. CPU, memory, disks, network, users and processes).

Here is an example to be aware of some server overload. If any of thetests (net, CPU) fail, it will send an email.

# Functions to get system values:frompsutilimportcpu_percent,net_io_counters# Functions to take a break:fromtimeimportsleep# Package for email services:importsmtplibimportstringMAX_NET_USAGE=400000MAX_ATTACKS=4attack=0counter=0whileattack<=MAX_ATTACKS:sleep(4)counter=counter+1# Check the cpu usageifcpu_percent(interval=1)>70:attack=attack+1# Check the net usageneti1=net_io_counters()[1]neto1=net_io_counters()[0]sleep(1)neti2=net_io_counters()[1]neto2=net_io_counters()[0]# Calculate the bytes per secondnet=((neti2+neto2)-(neti1+neto1))/2ifnet>MAX_NET_USAGE:attack=attack+1ifcounter>25:attack=0counter=0# Write a very important email if attack is higher than 4TO="you@your_email.com"FROM="webmaster@your_domain.com"SUBJECT="Your domain is out of system resources!"text="Go and fix your server!"BODY=string.join(("From:%s"%FROM,"To:%s"%TO,"Subject:%s"%SUBJECT,"",text),"\r\n")server=smtplib.SMTP('127.0.0.1')server.sendmail(FROM,[TO],BODY)server.quit()

A full terminal application like a widely extended top which is based onpsutil and with the ability of a client-server monitoring isglance.

Ansible¶

Ansible is an open source system automation tool.The biggest advantage over Puppet or Chef is it does not require an agent onthe client machine. Playbooks are Ansible’s configuration, deployment, andorchestration language and are written in YAML with Jinja2 for templating.

Ansible supports Python versions 2.6 and 2.7 and can be installed via pip:

$ pip install ansible

Ansible requires an inventory file that describes the hosts to which it hasaccess. Below is an example of a host and playbook that will ping all thehosts in the inventory file.

Here is an example inventory file:hosts.yml

[server_name]127.0.0.1

Here is an example playbook:ping.yml

----hosts:alltasks:-name:pingaction:ping

To run the playbook:

$ ansible-playbook ping.yml -i hosts.yml --ask-pass

The Ansible playbook will ping all of the servers in thehosts.yml file.You can also select groups of servers using Ansible. For more informationabout Ansible, read theAnsible Docs.

An Ansible tutorial is also agreat and detailed introduction to getting started with Ansible.

Chef¶

Chef is a systems and cloud infrastructure automationframework that makes it easy to deploy servers and applications to any physical,virtual, or cloud location. In case this is your choice for configuration management,you will primarily use Ruby to write your infrastructure code.

Chef clients run on every server that is part of your infrastructure and these regularlycheck with your Chef server to ensure your system is always aligned and represents thedesired state. Since each individual server has its own distinct Chef client, each serverconfigures itself and this distributed approach makes Chef a scalable automation platform.

Chef works by using custom recipes (configuration elements), implemented in cookbooks. Cookbooks, which are basicallypackages for infrastructure choices, are usually stored in your Chef server.Read theDigital Ocean tutorial serieson chef to learn how to create a simple Chef Server.

To create a simple cookbook theknife command is used:

knife cookbook create cookbook_name

Getting started with Chefis a good starting point for Chef Beginners and many community maintained cookbooks that canserve as a good reference or tweaked to serve your infrastructure configuration needs can befound on theChef Supermarket.

• Chef Documentation

Puppet¶

Puppet is IT Automation and configuration managementsoftware from Puppet Labs that allows System Administrators to define the stateof their IT Infrastructure, thereby providing an elegant way to manage theirfleet of physical and virtual machines.

Puppet is available both as an Open Source and an Enterprise variant. Modulesare small, shareable units of code written to automate or define the state of asystem.Puppet Forge is a repository formodules written by the community for Open Source and Enterprise Puppet.

Puppet Agents are installed on nodes whose state needs to be monitored orchanged. A designated server known as the Puppet Master is responsible fororchestrating the agent nodes.

Agent nodes send basic facts about the system such as to the operating system,kernel, architecture, ip address, hostname etc. to the Puppet Master.The Puppet Master then compiles a catalog with information provided by theagents on how each node should be configured and sends it to the agent. Theagent enforces the change as prescribed in the catalog and sends a report backto the Puppet Master.

Facter is an interesting tool that ships with Puppet that pulls basic factsabout the system. These facts can be referenced as a variable while writingyour Puppet modules.

$ facter kernelLinux

$ facter operatingsystemUbuntu

Writing Modules in Puppet is pretty straight forward. Puppet Manifests togetherform Puppet Modules. Puppet manifest end with an extension of.pp.Here is an example of ‘Hello World’ in Puppet.

notify{'This message is getting logged into the agent node':    #As nothing is specified in the body the resource title    #the notification message by default.}

Here is another example with system based logic. Note how the operating systemfact is being used as a variable prepended with the$ sign. Similarly, thisholds true for other facts such as hostname which can be referenced by$hostname

notify{'Mac Warning':message=>$operatingsystem?{'Darwin'=>'This seems to be a Mac.',default=>'I am a PC.',},}

There are several resource types for Puppet but the package-file-serviceparadigm is all you need for undertaking majority of the configurationmanagement. The following Puppet code makes sure that the OpenSSH-Serverpackage is installed in a system and the sshd service is notified to restarteverytime the sshd configuration file is changed.

package{'openssh-server':ensure=>installed,}file{'/etc/ssh/sshd_config':source=>'puppet:///modules/sshd/sshd_config',owner=>'root',group=>'root',mode=>'640',notify=>Service['sshd'], # sshd will restart                                  # whenever you edit this                                  # filerequire=>Package['openssh-server'],}service{'sshd':ensure=>running,enable=>true,hasstatus=>true,hasrestart=>true,}

For more information, refer to thePuppet Labs Documentation

Blueprint¶

Buildout¶

Buildout is an open source software build tool.Buildout is created using the Python programming language. It implements aprinciple of separation of configuration from the scripts that do the setting up.Buildout is primarily used to download and set up dependencies in Python eggsformat of the software being developed or deployed. Recipes for build tasks in anyenvironment can be created, and many are already available.

Shinken¶

Shinken is a modern, Nagios compatiblemonitoring framework written in Python. Its main goal is to give users a flexiblearchitecture for their monitoring system that is designed to scale to largeenvironments.

Shinken is backwards-compatible with the Nagios configuration standard, andplugins.It works on any operating system, and architecture that supports Pythonwhich includes Windows, GNU/Linux, and FreeBSD.