About Archive Tags RSS Feed


I don't like this ending...

16 January 2010 21:50

I've talked before about the minimal way in which I've been using a lot of the available automation tools. I tend to use them to carry out only a few operations:

  • Fetch a file from a remote source.
    • If this has changed run some action.
  • Ensure a package is installed.
    • If this is carried out run some action.
  • Run a command on some simple criterion.
    • E.g. Every day at 11pm run a mirror.

In the pub I've had more than a few chats about how to parse a mini-language and carry these operations out, and what facilities other people use. It'd be almost trivial to come up with a mini-language, but the conclusion has always been that such mini-languages aren't expressive enough to give you the arbitrary flexibility some people would desire. (Nested conditionals and the ability to do things on a per-host, per-day, per-arch basis for example.)

It struck me last night that you could instead cheat. Why not run scripting langues directly on your client nodes? Assume you could write your automation in Ruby or Perl and all you need to do is define a few additional primitives.

For example:

#  /policies/default.policy - the file that all clients nodes poll.

#  Fetch the per-node policy if it exists.
FetchPolicy $hostname.policy ;

#  Ensure SSH is OK
FetchPolicy ssh-server.policy ;

#  Or explicitly specify the URL:
# FetchPolicy http://example.com/policies/ssh-server.policy ;

#  Finally a quick fetch of a remote file.
if ( FetchFile(
                Source => "/etc/motd",
                Dest => "/etc/motd",
                Owner => "root",
                Group => "root",
                Mode => "0644" ) )

    RunCommand( "id" );

This default policy attempts to include some other policies which are essentially perl files which have some additional "admin-esque" primitives. Such as "InstallPackage", "PurgePackage", and "FetchFile".

FetchFile is the only one I've fully implemented, but given a server it will fetch http://server/prefix/files/$FILENAME - into a local file, and will setup the owner/gid/mode. If the fetch succeeded and contents differ from the current contents of the named file (or the current file doesn't exist) it will be moved into place and the function will return true.

On the server side I just have a layout that makes sense:

|-- files
|   `-- etc
|       |-- motd
|       |-- motd.silver.my.flat
|       `-- motd.gold
`-- policies
    |-- default.policy
    |-- ssh-server.policy
    `-- steve.policy

Here FetchFile has been implemented to first request /files/etc/motd.gold.my.flat, then /files/etc/motd.gold, and finally the global file /files/etc/motd.

In short you don't want to be forced to write perl which would run things like this:

# install ssh
if ( -e "/etc/apt/sources.list" )
  # we're probably debian
  system( "apt-get update" );
  system( "apt-get install openssh-server" );

You just want to be able to say "Install Package foo", and rely upon the helper library / primitives being implemented correctly enough to be able to have that work.

I'll probably stop there, but it has given me a fair amount to think about. Not least of which : What are the minimum required primitives to usefully automate client nodes?

ObFilm: Moulin Rouge!



Comments on this entry

icon Andrew Pollock at 14:17 on 16 January 2010

Have you seen slack?

icon Steve Kemp at 14:22 on 16 January 2010

No I hadn't.

Looking over it briefly it seems very specific on what it does, in a way that I like.

The main job seems to be installing files and running shell scripts / commands once those files have been replaced/copied/installed.

I'd like to see some facility for running a DiskFree or sending an alert - but perhaps thats a level of complexity that people don't often need?

icon toupeira at 15:02 on 16 January 2010

We also use slack at work to automate server installations, together with a Bash "library" of helper functions for various tasks (e.g. installing packages, creating users, editing files, etc.), to keep the preinstall/postinstall scripts as simple as possible, and also support the different platforms we use (Debian, Nexenta/OpenSolaris, XenServer/RedHat).

For monitoring/alerts we use Zabbix with custom scripts, which simply get dropped by slack in the correct places, and then in Zabbix we have pre-defined templates for the various server roles.

icon C.J. Adams-Collier at 19:33 on 16 January 2010

Hey... What are you thoughts on this, mister? AoE root for guests seems like a cool idea... eases live migration, too. If you don't need super-fast disks...


icon Steve Kemp at 19:37 on 16 January 2010

I think it is very cool.

(I've also been looking at PXE Booting "working" systems recently, as opposed to merely using PXE/NFSRoot for rescue environments).

The only specific comment I'd have on your documentation/setup is that rather than copying pxelinux.0 I'd use a symlink.

icon Andrew France at 23:38 on 16 January 2010

Hi, a promising project that is still under heavy development (but works well for single clients so far) is Babushka at http://babushka.me.