Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings
This repository was archived by the owner on Feb 4, 2018. It is now read-only.
/fullstackPublic archive

Full-stack DevOps demo

License

NotificationsYou must be signed in to change notification settings

miketheman/fullstack

Repository files navigation

Full-stack DevOps demo

The purpose of this is to demonstrate some great automation tools in orchestra.

Things used

(in alphabetic order)

Some of the more exotic pieces

  • bluepill is a process manager, similar to SysV init, Upstart, supervisord, runit, etc.
  • Bottle.py is a web micro-framework written in python.
  • Siege creates web requests based on an input file for load testing.
  • Spiceweasel generates Chef's knife commands from a config file

A picture

Diagram

Thanks todiagrammr.

Application

The Bottle.py application is a simplistic word counter, acts like a REST interface, where/insert/<someword> will add the word to the database and increment its counter.

The/get/<someword> will retrieve the word, the unique object ID, and the count of times this word was hit.

A call to/toplist will bring back the top 10 words that have been hit.

Customizations

All cookbooks used are released on the Chef Community site.The only cookbooks here are a personalizedmongodb cookbook, due to a currently faulty community cookbook, and thefullstack cookbook, that performs the customized functions this stack needs.

Prep work

Some EC2 security group work:

ec2-create-group fullstack -d "Full Stack Demo"# Allow pingsec2-authorize fullstack --protocol icmp --icmp-type-code=-1:-1 --source-or-dest-group fullstack# Could be shorter: ec2-authorize fullstack -P icmp -t=-1:-1 -o fullstackec2-authorize fullstack -P tcp -p 0-65535 -o fullstackec2-authorize fullstack -P udp -p 0-65535 -o fullstackec2-authorize fullstack -P tcp -p 22    # SSHec2-authorize fullstack -P tcp -p 80    # HTTP# Optional, don't use in a production environment unless neededec2-authorize fullstack -P tcp -p 22002 # HAProxy Statsec2-authorize fullstack -P tcp -p 8080  # Webapp node

A chef server (open source or hosted) must exist, andknife.rb must be set up correctly with AWS credentials.My personal one is excluded from the repo.

I recommend using a dedicated server/organization since the cleanup actions are destructive.

current_dir = File.dirname(__FILE__)log_level                :infolog_location             STDOUTnode_name                "<my username>"client_key               "#{current_dir}/<my username>.pem"validation_client_name   "<organization-name>-validator"validation_key           "#{current_dir}/<organization-name>-validator.pem"chef_server_url          "https://api.opscode.com/organizations/<organization-name>"cache_type               'BasicFile'cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )cookbook_path            ["#{current_dir}/../site-cookbooks"]# AWS credentialsknife[:ssh_user]              = "ec2-user"knife[:ssh_identity_file]     = "#{current_dir}/../.aws/<key pair cert>.pem"knife[:aws_access_key_id]     = "<some key id>"knife[:aws_secret_access_key] = "<some secret string>"### END ###

A Users Databag item must be placed indata_bags/users/<username>.json. An example is:

{  "id": "bofh",  "ssh_keys": "ssh-rsa AAAAB3Nz...yhCw== bofh",  "groups": "sysadmin",  "uid": 2001,  "shell": "\/bin\/bash",  "comment": "BOFH",}

See theusers cookbook for more help.

A Credentials databag item for Datadog monitoring is also excluded from the repo, since it contains API keys.Construct your own atdata_bags/credentials/datadog.json so:

{  "id": "datadog",  "api_key": "YOURAPIKEY",  "application_key": "APPLICATIONKEYFORCHEF"}

You can obtain your API key and Application Key from the Integrations => API page, or from the Agent Setup => Chef page, you will find a "Generate Application Key" button.

Launch

spiceweasel fullspice.yml | bash

Some cool tricks

Get the top list of words:

open http://`knife search node 'role:load_balancer' -a ec2.public_hostname |grep ec2.public_hostname | cut -f4 -d" "`/toplist

HAProxy web console:

open http://`knife search node 'role:load_balancer' -a ec2.public_hostname |grep ec2.public_hostname | cut -f4 -d" "`:22002/

Find the mongodb replset primary:

knife search node "fqdn:`knife ssh 'role:mongodb-replset-member' -a ec2.public_hostname 'curl http://localhost:28017/replSetGetStatus?text=1' | grep -B4 PRIMARY | grep name | awk '{print $4}' |cut -f1 -d":" | sed 's/^.\{1\}//' | uniq`" -i

NOTE: This is probably overly complicated, but awesome. Probably better to have chef-client update the node record with the current state.

Kill the primary:

knife ec2 server delete --purge -y <instance-id from previous command># or:knife ec2 server delete --purge -y `knife search node "fqdn:`knife ssh 'role:mongodb-replset-member' -a ec2.public_hostname 'curl http://localhost:28017/replSetGetStatus?text=1' | grep -B4 PRIMARY | grep name | awk '{print $4}' |cut -f1 -d":" | sed 's/^.\{1\}//' | uniq`" -a ec2.instance_id | grep instance_id | cut -f2 -d":"`

Launch a new mongodb replica:

spiceweasel fullspice.yml | grep --color=none rolemongodb-replset-member | uniq | bash

Add a webserver:

spiceweasel fullspice.yml | grep --color=none rolewebserver | uniq | bash

Show counts of servers:

knife status -r | awk '{print $9}' | sort | uniq -c

Do something on all nodes:

knife ssh '*:*' -a ec2.public_hostname 'hostname -f'

Cleanup

Spiceweasel, in reverse:

spiceweasel -d fullspice.yml | bashknife client bulk delete i-.*

That's all, folks!

Credits

Bitdeli Badge

Packages

No packages published

Contributors2

  •  
  •  
[8]ページ先頭
©2009-2025 Movatter.jp