• Tomaz Muraus
• Engineer @ Rackspace
• Cloud Monitoring
• Service Registry
• Apache Libcloud Project chair & committer
• Dude who likes open standards and open source
• Dude who likes to automate "all the things"

• Rackspace?
• Why I selected this topic?
• Traditional approach to delivering software
• What is Continuous Deployment?
• What is CI/CD/CD
• Benefits of Continuous Deployment
• Who Uses Continuous Deployment
• Tools for Continuous Deployment
• Continuous Deployment in practice
• Q&A

• Founded in 1998
• Started with traditional and managed hosting
• Fanatical support (focus on service and support)
• Managed hosting is still big and important
• Cloud is the future
• Founded OpenStack with Nasa

• A lot of other interesting topics (distributed systems, Cassandra, functional programming, devops, etc.)
• Not a lot of companies around here are aware of advantages or practice CD
• More widely applicable
• Biggest impact

• Disk in a box
• Longer iterations and release cycles
• Tests usually only ran before creating a release
• Usually not automated - single person who creates a release
• SaaS changed this and made some things easier

• Practice of merging work into main branch constantly so you can verify and test changes and that those changes work with other changes
• Build automation
• Automatic testing
• Every change should be built
• Test in a production-like environment

• Delivering of the code to an environment once a developer feels the code is ready

• Deployment of release code to production as soon as it's ready
• Should be automated (press the big green button!)
• Should be repeatable
• Everyone should be able to do it in a short time frame
• You should be able to deploy mainline at any time
• Requires Continuous Integration and Continuous Delivery

• Lower risk (smaller, incremental changes)
• Faster time to detection and recovery
• Roll new features and bug fixes out to customers faster
• Minimal manual overhead (less stress for developers)

• Fabric (Python)
• Capistrano (Ruby)
• Deployinator
• Dreadnot

• Lets get our hands dirty!
• Real life examples from two Rackspace products
  • Cloud Monitoring (cross dc / multi region, bare metal)
  • Service Registry (single dc, cloud only)
• Your Mileage May Vary

• Fully distributed monitoring system
• Highly available (we are ready for zombie apocalypse)
• Powerful alarm language
• Eventually consistent
• Remote checks (ping, http, smtp, ssh, ...)
• Agent checks (cpu, load, memory, write your own, ...)
• Historical data and metrics collection
• No official UI yet

• Easy to use REST based API
• Flexible API aka monitoring anything which has an IPv[4|6]
• Making it really reliable
  • Two alerts are (usually) better than no alerts (A & P over C)
  • No SPOF
• Horizontally scalable to many hundreds of servers per account
• Easy to operate and maintain

• Zero downtime
• Zero customer impact (maintaince windows FTL)
• Deployment should be fast, not a lot of manual overhead
• Deploying should be easy (click the green button!)
• Everyone should be able to do it at any point in time
• Developers should be initivied to deploy as often as possible

• Fail fast
• Always roll forward (code needs to be designed this way)
  • Code needs to be designed this way
  • Usually simpler (think schema changes, etc.)
  • Small changes make it possible

• Trigger build on Buildbot
• Set up PagerDuty override
• Disable Nagios alerts
• Modify LB rules to redirect traffic to other regions
• Deploy API servers
• Setup CQL schema (run schema migrations)
• Deploy backend servers
• Verify API version
• Verify API functionality
• Restore LB rules
• Enable Nagios

• REST API based product for keeping track of your services
• Middle tier load-balancing
• Events feed for automation
• Configuration storage with change notifications
• Currently in preview (free, go play with it)
• http://rax.io.rsr

• Cloud Servers
• Cloud Files
• Cloud Monitoring
• Cloud Load Balancers
• Mailgun

• REST API based product for keeping track of your services
• Middle tier load-balancing
• Events feed for automation
• Configuration storage with change notifications
• Currently in preview (free, go play with it)
• http://rax.io.rsr

• Trigger build on Buildbot
  • Runs tests
  • Builds documentation
  • Builds a release artifact (tarball with source code)
• Remove API server n from load-balancer rotation
• Deploy API server
  • Download the release artifact (code) from Cloud Files
  • Unpack the artifact
  • Run chef-solo
• Verify API version
• Verify API functionality
• Add API server n back to rotation
• Repeat for every API server

Deployment demo video

• Fully automated rollback
  • Hard especially when schema changes are involved
  • Not worth the effort atm

• Decreases risk
• Get new features and fixes out to customers faster
• Automation reducess stress and burden put on developers
• Developers can focus on things which actually provide value to business and customers
• You can start small and use iterative approach (automate small things, etc.)

• http://rackertalent.com/sanfrancisco
• [email protected]

Location: Corner pub

Come and talk about what we do

Drinks are on us