Running Luvit and Rackspace Monitoring Agent on Raspberry Pi

I wanted to get Luvit and Rackspace Monitoring Agent (also known as Virgo) running on Raspberry Pi for quite a while now. Until recently, I haven’t owned a Raspberry Pi so I had a good excuse.

My little Raspberry Pi.

This past week, I have attended CloudStack Collaboration conference where I have received a Raspberry Pi as a speaker gift. Not owning a Raspberry Pi was now not a good excuse anymore and I needed a break from other things I was currently working on, so I decided to dive right in and get Luvit and Rackspace Monitoring Agent running on it :)

Luvit

Luvit is an open-source project, originally written by Tim Caswell, which exposes a Node.js-like API in Lua. It allows you to write asynchronous and high-performant applications with a low memory footprint.

In addition to asynchronous Node.js-like callback style API, Luvit also uses the power of Lua coroutines to expose synchronous like API through the fiber module.

High performance and low memory footprint makes it a great candidate for running on a Raspberry Pi and other embedded and / or memory constrained devices.

Under the covers it uses libuv for cross-platform asynchronous I/O and LuaJIT as a Lua Just-In-Time compiler. Libuv is the same library which powers Node.js and LuaJIT is one of the fastest JIT compilers for dynamic languages out there. Other popular and well known JIT’s for dynamic languages include PyPy JIT and V8 JIT.

Oh and yes, microbenchmarks are bad and don’t represent a real-life scenario, but if you are curious about the performance, I get around 300 / requests per second serving a static page from memory on Raspberry Pi using Luvit’s built in HTTP server.

Virgo and Rackspace Monitoring Agent

Virgo is an open-source framework which provides shared infrastructure for building host-based agents. It provides functionality such as:

  • Business logic can be written in a high level scripting language in Lua using Luvit
  • Secure (signed) automatic updates
  • Automatic generation of .zip bundles with the agent logic
  • Support for generating signed Linux packages and Windows installers
  • Crash report functionality using Google’s Breakpad
  • Low memory footprint
  • Cross platform support
Comparison of memory usage of agents written in different languages.

Rackspace Monitoring Agent is a host-based monitoring agent built on top of the Virgo framework used with the Rackspace Cloud Monitoring product.

Why

Before I dive into how you can get those projects running on a Raspberry Pi, lets have a look at why you would want to do that.

The first and the most important reason (at least for me) is that it’s a fun thing to do!

This should be a good enough reason by itself, but there are also a couple of other “real” reasons why you might want to write an application which runs on a Raspberry Pi in Luvit instead of Python, Ruby, Node.js or a similar language.

Depending on the model (model A vs model B), Raspberry Pi either has 256MB or 512MB of ram shared with GPU. This makes it a memory constrained device. If you can remember from the section above, Luvit has a low memory footprint which makes it a perfect candidate.

Getting Luvit to run on Raspberry Pi / ARM

Luvit apparently already ran on Raspberry Pi in the past and at some point back, Tim Caswell also provided pre-built binaries for Raspberry Pi.

It seems that with the recent versions, no one was really testing if the project still builds on Raspberry Pi, so things got out of sync. When I tried to build the latest version on a Raspberry Pi, I got a bunch of build errors.

Luckily, most of the errors were related to the build process, so with a couple of minor upstream changes to luvit/luvit and luvit/openssl I managed to get project to build and run on Raspberry Pi again.

Currently, Luvit uses Travis CI for continuous integration and running the tests. Travis CI sadly only supports x86_64 architecture. I will try to obtain another Raspberry Pi and dedicate it to Luvit test run. This way, similar problems won’t repeat again in the future.

Getting Virgo to run on Raspberry Pi / ARM

Unlike Luvit, Virgo never ran on Raspberry Pi / ARM before, so I thought that getting it running there will be a bit more challenging (but not too much, since it uses Luvit underneath).

To make it compile and run on Raspberry Pi, I needed to make two changes:

  1. Similar to Luvit, I needed to fix the build process for ARM. (#42)
  2. I disabled crash report functionality when running on ARM. (#42)

As far as the second point goes, Virgo uses Google Breakpad for crash reporting functionality. Current version of breakpad used in Virgo is slightly old and doesn’t support ARM. Because of that, I decided to disable crash report functionality when running on ARM.

Another solution would be to upgrade breakpad to a newer version which supports ARM, but this change would be more invasive and risky so for now, I decided to skip it.

Agent Installation - Requirements

This guide was tested on and assumes that you are running Raspbian (Debian based distro optimized for Raspberry Pi), but should also work with other Linux distributions which run on Raspberry Pi.

Besides the operating system, you also need the following packages to get the agent compiled:

  • build-essential - Compiler, and other build tools.
  • libssl-dev - OpenSSL development header files. Needed for SSL support in luvit and the agent.
  • git

Those packages can be installed using apt-get:

sudo apt-get -y install build-essential libssl-dev git

Keep in mind that on different distributions those packages might have a different name. Also, if you are using Raspbian, build-essential and git package already comes pre-installed.

Agent Installation - Installation process

All of the changes I made have already been merged upstream. This means that to install the agent, you can simply follow a standard agent installation guide which is outlined bellow.

git clone https://github.com/racker/virgo
cd virgo
git submodule update --init --recursive
./configure
make
sudo make install
Go grab a sandwich because the whole build process will take a while.

Keep in mind that the project includes quite a bit of dependencies, so compiling and linking will take a while. On my Raspberry Pi which is overclocked to 900MHz, a clean build process takes ~20 minutes.

Even tough Luvit does some smart stuff to only compile OpenSSL features which are commonly needed by default, the worst offender, which takes the most of time to compile, still (not so unexpectedly) is OpenSSL.

After you have compiled and installed the agent, you should have a look at the official guide for instructions on how to configure and start it.

Quick note about speeding up the compilation process

There are a couple of ways to speed up the whole compilation process. First and probably the best way is to use a Debian package / pre-built binary and avoid the compilation process all together (remember, the fastest works is always the one you don’t do!).

Sadly that’s not something I can really help with at the moment. If you want official Rackspace Monitoring Agent Debian package for ARM, you should bug Rackspace to do it :-)

Second way is to cross-compile for ARM on a single x84_64 or multiple machines using distcc. This should drastically speed up the compile time. The only problem is that, as far as I know, no one has tried that out before with the agent and the whole process would probably consume a lot of time and end up being a big PITA.

Third option would be to do a native compile on a cluster of Raspberry Pi’s using distcc. Sadly I don’t have a cluster of Raspberry Pi’s so I can’t do that either.

Conclusion

That’s it!

You can now monitor your Raspberry Pi using Rackspace Monitoring Agent and use Luvit to write performant applications with low memory footprint and run them on your Raspberry Pi.

To my knowledge, this is a first Raspberry Pi running Rackspace Monitoring Agent being monitored by Rackspace Cloud Monitoring! And yes, I totally fat-fingered "Raspberry".