Entries posted in February 2010

This is my land. All that pass through pay me tribute.

Monday, 1 March 2010

As previously mentioned I've switched my webserving over to a mixture of apache2 & thttpd.

I chose thttpd as it is simple to configure for my needs, and supports the execution of CGI scripts. Some of the other simple webservers available to Debian's current stable release (such as nginx) don't support CGI so they were ruled out.

Of course prior to choosing thttpd I looked at the state of the Debian package. Distressingly the package has no current maintainer and has several bugs open, including some that have been open for several years without comment.

I've just made my second upload fixing a couple of bugs, including ones that I could see affecting myself, but now I'm done with it.

In conclusion:

  • I've fixed a few bugs.
  • I suspect that many of the open bugs are 100% unreproducable and should be closed after checking with the submitter.
  • The package could do with a volunteer to maintain it.

On the one hand it is "just another webserver", on the other hand it is genuinely small, simple to configure, and has a couple of compelling features (CGI + throttling).

So. Go. Adopt. Maintain.

Pretty please...

ObFilm: Red Sonia

| 4 comments.

 

Fire and wind come from the sky, from the gods of the sky.

Sunday, 28 February 2010

Recently I was flirting with the idea of creating an online game, but I got distracted by wondering how to make the back-end more flexible.

To communicate the state of the game to N connected clients I figured I needed some kind of server which would accept "join"/"quit" requests and then make changes available.

To that end I came up with the idea that a client would make requests via HTTP such as:

http://example.com/server/game/chess/join

This would regard the originating client as part of a new chess game, for example, and return a UID identifying the "game channel".

http://example.com/server/changes/1-2-3-4

This will retrieve a list of all events which had occurred in the game which had not already been sent.

(Here 1-2-3-4 is obviously the UID previously allocated.)

http://example.com/server/submit/1-2-3-4/move

This would submit the move "move" to the server.

After mulling this over for a while it seemed like a great reusable solution, I'd make an initial "join" request, then repeated polling with the allocated UID would allow game moves to be observed. All using JSON over HTTP as the transport.

It was only this morning that I realised I'd have saved a lot of time if I'd just proxied requests to a private IRC server, as the functionality is essentially the same.

Still I'm sure this pattern of "join"/"poll"/"quit" could be useful for a lot of dynamic websites, even in the non-gaming world. So although the idea was mostly shelved it was an interesting thing to have experimented with.

D'oh.

ObFilm: Conan The Barbarian

| 3 comments.

 

Let go of the handle.

Sunday, 21 February 2010

I don't talk about SPAM publicly these days, for reasons that are probably self-explanatory.

However this is just insane:

  • Saturday 20th February 2010: Registered a new domain.
  • Sunday 21st February 2010: Received first spam.

Currently at 40+ SPAM mails and rising; all mails addressed to "postmaster@", rather than any past users of the domain. (I can see from http://archive.org that the domain was last active in 2008.)

ObSubject: The Goonies

| No comments

 

If you were a comic book character, what character would you be?

Friday, 19 February 2010

I've been overhauling the way that I am host a number of virtual websites upon my main box. Partly to increase security, and partly for a cleaner separation or roles, ownership, and control. (In general everything on my box is "mine", but some things are "ours"...)

After a fair amount of experimentation I decided that I wasn't willing or able to rewrite all my Apache mod_rewrite rules just yet. So my interim plan was to update each existing virtual host:

  • Add a dedicated user & group to run it under.
  • Launch it via a minimal server listening upon the loopback adapter.
  • Have Apache 2.x proxy through to it.
    • Expanding any mod_rewrite rules prior to the proxying.

To make it clear what the users were for I decided that every hosting-user would have an "s-" prefix. So the virtual host "static.steve.org.uk" was initially going to be served by the s-static user.

The thttpd configuration file would look like this, and would be located in /etc/thttpd/sites/static.steve.org.uk:

host=127.0.0.1
port=1008
dir=/home/www/static.steve.org.uk/htdocs/
chroot
user=s-static
throttles=/etc/thttpd/throttle.conf
logfile=/home/www/static.steve.org.uk/logs/thttpd.log
pidfile=/home/www/static.steve.org.uk/pid/file

(I wrote a trivial script to stop/start all the sites en mass, and removed the default thttpd init script, logrotation job, and similar things.)

How did I decide which port to run this instance under? By taking the UID of the user:

steve@skx:~$ id s-static
uid=1008(s-static) gid=1009(s-static) groups=1009(s-static)

With this in place I could then update the Apache configuration file from serving the site directly to merely proxying to the back-end server:

<VirtualHost *>
    ServerName  static.steve.org.uk

    # Proxy ACL
    <Proxy *>
        Order allow,deny
        Allow from all
    </Proxy>

    # Proxy directives
    ProxyPass          /   http://localhost:1008/
    ProxyPassReverse   /   http://localhost:1008/
    ProxyPreserveHost on
</VirtualHost>

So was that all there is to it? Sadly not. There were a couple of minor issues, some of which were:

cronjobs

I have various cron-jobs in my main steve account which previously updated blog indexes, etc. (I use namazu2 to make my blog searchable.)

I had to change the ownership of the existing indexes, the scripts themselves, and move the cronjob to the new s-blog user.

cross-user dependencies

I run a couple of sites which pull in content from other locations. For example a couple of list summaries, and archives. These are generally fed from a ~/.procmail snippet under my primary login.

Since my primary login no longer owns the web-tree it is no longer able to update things directly. Instead I had to duplicate a couple of subscriptions and move this work under the UID of the site-owner.

I'm no longer running apache

For a day or two I'd forgotten I was using the apache facility to include snippets in my site; such as links to my wishlist.

Since I'm not using Apache in the back-end server-parsed files no longer work. Happily I'm using a simple template-based setup for my main sites, so I updated the template-parser to understand "##include ./path/to/file". For example this source file produces my donation page.

The upshot is my "static" site is even more static, which is a good thing.

uploads are harder

Several of my domains host entirely static content which is generated on my main desktop machine, and then uploaded via rsync post-build.

I had to add some more accounts and configure SSH keys, then update the uploading routines/Makefiles appropriately. Not a major annoyance, but suddenly my sshd_config file has gone from "PermitUser steve,backup" to including many additional accounts.

The single biggest pain was handling my my mercurial repositories - overhauling that took a bit of creativity to ensure that nothing was broken for existing or new checkouts. I wish that a backport of mercurial-server was trivial because I'd love to be using that.

In general though watching the thttpd logs has been sufficient to spot problems. I had to tweak things a little to generate statistics properly, but otherwise all is good.

Why thttpd? Well small, lightweight, and the ability to run CGI scripts. Something missing from nginx for example.

I'm still aiming to remove apache2 from the front-end - it is mostly just a dumb proxy, but it does perform some ACL operations and expand mod_rewrite rules. I could port those to another engine .. but not today.

The most likely candidates are nginx, perlbal, or lighttpd - each of these should be capable of doing simple ACL checks, and performing mod_rewrite-like rules.

ObFilm: Mallrats

| 5 comments.

 

I am lightened, can we drop this?

Tuesday, 16 February 2010

As part of some house-keeping I've been checking over my systems and ensuring they're all tickity-boo for the past couple of days.

One thing that I'm getting increasingly tempted by is converting my kvm guest to a 64-bit system.

I've not quite sold myself on the prospect of what will be a fair amount of downtime, but I'm 90% there.

I do think that a lot of my setup needs an overhaul, for example:

  • Running all my websites under www-data is beginning to worry me.
  • Running services as root is beginning to make me more and more paranoid.

One possible plan is to wipe my system, and then restore data from backups. A perhaps saner approach is divide my guest into two smaller ones, and migrate services over one by one (e.g. website1, website2, .. websiteN, email, etc).

For the moment I've taken a complete dump of my existing guest, and I'm running it with an IP in the 10.0.0.0/24 range on my desktop. That's at least given me a clear idea of the amount of work involved.

I'm still a little unclear on how best to manage running N websites with the intention they'll each run under their own UID. I guess it comes down to having a few instances of nginx/lighttpd/apache and then proxy from *:80 to the actual back-end. Precisely which mixture of services to use is a little overwhelming. Though at some point soon I need to start enabling IPv6 support, and that changes things a little.

(Not least because nginx has no IPv6 support present in the Lenny release - I've got a backported package which I run on the Debian Administration website.)

It's possible I could hack mod_vhost_alias to redirect/proxy to a local port based upon the virtual hostname present in the request - that's pretty trivial and I've already done something similar for work purposes. Though something like that should presumably already exist? I would expect a map of some form:

example.org: 127.0.0.1:8080
example.net: 127.0.0.1:9090

That has to be about the minimum necessary information to make the decision; a pair of vhost name & local destination.

/me googles some..

Update

OK quick update I've added local users for some of my sites, and now have them running under thttpd.

skx:/etc/thttpd# ls -ltr /home/www/ | tail -n 4
drwxr-sr-x  4 s-static   s-static   4096 Jan 15 01:41 static.steve.org.uk
drwxr-sr-x  5 s-openid   s-openid   4096 Feb 16 21:31 openid.steve.org.uk
drwxr-sr-x  6 s-images   s-images   4096 Feb 16 21:52 images.steve.org.uk
drwxr-sr-x  5 s-packages s-packages 4096 Feb 16 22:03 packages.steve.org.uk

That seems to work well, with a small wrapper script to start N instances of thttpd instead of a single one. Minor issues are that I'm using mod_proxy to forward requests to the thtpd instances running upon the loopback - and it was initially logging 127.0.0.1 as the source IP. A quick patch later all is well.

I'll leave it running a couple of the simple sites for the next few days and see if it kills children. If it does I'll convert the rest.

Probably will aim to have nginx in front of thttpd, instead of Apache, but this way I don't have to worry about mod_rewrite rules just yet.

ObFilm: Cruel Intentions

| 11 comments.

 

You seem uncomfortable.

Saturday, 6 February 2010

I've been trying to remember to post the pictures I like online for the past few months. So this is a reminder to myself.

This image below didn't turn out quite how I wanted it to:

  • I was hoping for a nicer sihouet upon the lady's face.
  • The tree-branch on the left irritates me.

But that said I keep on coming back to look at it. I like the lighting, and I love the way that the brick wall on the right hand side angles towards the building on the horizon.

Enjoy. Or not.

Sunset

A similarly "not perfect" image is this outdoor shot. I have only one irritation with this shot - and that is that the trees are clipped at the top. Meh, such is life.

(I have two styles of photography; semi-random where I snap what is in front of me, and staged where I try to construct a particular picture - the two images above? One of each.)

ObFilm: Bound

| No comments

 

Revenge is not good. Once you're done. Believe me.

Wednesday, 3 February 2010

I was interested to see Adnan Hodzic discuss life without evolution in the GNOME environment recently.

I too use GNOME as my desktop environment (I sometimes toy with various tiling window managers before getting annoyed at something or other).

My solution to the GNOME problem is to purge the gnome-desktop-environment package and instead my own local package gnome-desktop-minimal. This package is a meta-package which includes a smaller selection of GNOME packages, notably ignoring several that the gnome-core package would pull in such as eog - why install that when I prefer qiv or feh?

If I believed we could agree on precisely which packages to include I would submit a bug to the gnome team "Please provide gnome-desktop-minimal" or similar. Still I suspect individual biases/preferences will make such a suggestion contentious at best and impossible to satisfy at worst.

ObTitle: Léon

| 12 comments.

 

Recent Posts

Recent Tags