Apache Libcloud

Cloudstack Collaboration Conference, Amsterdam, November 2013

Tomaz Muraus / @kamislo

Who am I?

• Tomaz Muraus
• Apache Libcloud Committer & Project chair
• Previously: Cloudkick, Rackspace
• Currently: Hacking away on my startup (email analytics)
• Likes: Distributed systems, big data, open standards and open source systems, startups, ponies
• tomaz.me
• github.com/Kami
• @kamislo
• +TomazMuraus

Agenda

• What is Libcloud?
• Project History
• Libcloud APIs
  • Compute and Block Storage
  • Object Storage and CDN
  • Load Balancer
  • DNS
• CloudStack + Libcloud = ♥
  • Supported Functionality
  • Code Examples
  • Plans for the future

CloudStack + I

• Not a whole lot of experience
• Mostly got my hands dirty while reviewing patches and cross-checking the documentation
• First used it with a real provider while working on the code examples for this presentation
• ^^ Standard API exposed by Libcloud allows you to do that

Note About Code Examples

• Examples assume you are using Libcloud trunk
• Examples were executed against a CloudStack provider (exoscale), but should also work with other supported providers

What is Libcloud?

Libcloud is a Python library which hides differences between different cloud provider APIs and allows you to manage different cloud resources through a unified and easy to use API.

Turns this...

from pprint import pprint

from libcloud.compute.types import Provider
from libcloud.compute.providers import get_driver

from config import API_KEY, API_SECRET_KEY, API_HOST, API_PATH

cls = get_driver(Provider.CLOUDSTACK)
driver = cls(API_KEY, API_SECRET_KEY, host=API_HOST, path=API_PATH)

size = [size for size in driver.list_sizes() if size.name == 'Micro'][0]
image = [image for image in driver.list_images() if 'Ubuntu 12.04 LTS 64-bit'
         in image.name][0]

name = 'cloudstack-node'
node = driver.create_node(name=name, image=image, size=size)

pprint(node)

Into this

What is Libcloud?

• Similar projects for other languages
  • fog (Ruby)
  • Apache jclouds (Java)
  • Apache Deltacloud (Ruby / HTTP)

How it came about?

• We formed a committee and locked 10 people with a PhD into a room and tasked them with designing a standard interface
• Not really :-)
• Like many other "new era" projects and standards (think SPDY, etc.), Libcloud was developed as a pragmatic solution for our problem

How it came about?

How it came about?

How it came about?

• Developed as a solution for our problem with talking to many different providers at Cloudkick
• Released as an open-source Apache 2.0 licensed project
• Joined Apache Incubator in late 2009, graduated to a top-level project in 2011
• Mostly organic growth where interfaces and APIs have evolved over time

Supported Providers

Compute - Terminology

• Node - represents a cloud / virtual server
• NodeState - represents a state node is currently in (running, stopped, ...)
• NodeSize - represents size of the server (cpu, memory, disk size)
• NodeImage - represents an operating system

Compute - Base API

• list_images()
• list_sizes()
• list_nodes()
• create_node()
• deploy_node()
• destroy_node()
• + volume management methods (block storage)

CloudStack + Libcloud

• Standard Libcloud functionality
• IP address allocation and management
• Security group management
• Firewall rules / port forwarding management
• Keypair management
• but wait, there is more!

Example 1 - Create a node

import random
from pprint import pprint

from libcloud.compute.types import Provider
from libcloud.compute.providers import get_driver

from config import API_KEY, API_SECRET_KEY

cls = get_driver(Provider.EXOSCALE)
driver = cls(API_KEY, API_SECRET_KEY)

size = [size for size in driver.list_sizes() if size.name == 'Micro'][0]
image = [image for image in driver.list_images() if 'Ubuntu 12.04 LTS 64-bit'
         in image.name][0]

name = 'cloudstack-node-' + str(random.randint(0,100))
node = driver.create_node(name=name, image=image, size=size)

pprint(node)

Example 2 - Deploy a node

import random
from pprint import pprint

from libcloud.compute.types import Provider
from libcloud.compute.providers import get_driver
from libcloud.compute.deployment import ScriptDeployment

from config import API_KEY, API_SECRET_KEY

cls = get_driver(Provider.EXOSCALE)
driver = cls(API_KEY, API_SECRET_KEY)

size = [size for size in driver.list_sizes() if size.name == 'Micro'][0]
image = [image for image in driver.list_images() if 'Ubuntu 12.04 LTS 64-bit'
         in image.name][0]

with open('./script.sh', 'r') as fp:
    content = fp.read()

name = 'cloudstack-node-' + str(random.randint(0, 100))
script = ScriptDeployment(script=content)
node = driver.deploy_node(name=name, image=image, size=size, deploy=script,
                          ex_keyname='default',
                          ssh_key_file='/home/kami/.ssh/id_rsa_cloudstack')

pprint(node)
pprint(script.stdout)

CloudStack Driver - Plans for the future

• Better support for locations (zones)
• Better support for advanced zones
• Support for multiple API versions

Questions?

Thanks

https://libcloud.apache.org

https://libcloud.readthedocs.org

@kamislo / tomaz.me