Entries posted in January 2009

Just some kind of fairytale.

Saturday, 24 January 2009

Once upon a time I ran a Valentine-Matching site for Livejournal users. (Actually I did it for about three years in a row, but then got bored.)

The basic idea was simple, you'd visit a magical site and it would present you with the usernames of all your friends. You'd tick a few boxes, and that would nominate those users as your potential valentines.

Then you'd do nothing until February 14th, at which point you'd discover if you matched. A match being made if you chose bob, and bob chose you in return.

(Essentially the site is a big double-blind matching system, albeit with constrained membership, via lj.)

The major technical difficulty in early days was preventing spoofed accounts. I didn't want Evil Alice to be able to visit the site and pretend to be Bob. In the couple of years I did this I came up with a couple of fun schemes to minimize the chances of this occuring - anything from using email validation, to requiring that users taking part updated their profiles to include a magic cookie-string, or posting to a specific community which I could later screen-scrape.

These days the use of openid allows me to prevent spoofing in a trivial manner - and once I realised that I figured why the hell not?!

In short here I am and here we go again:

The motivation? Playing with openID and because I still like to "match people up". I am aware of one marriage that resulted from a match on my system back in 2003. There could be more.

If you're a long-time reader you might remember ctrl-alt-date - how time flies - I just let the domain name(s) expire this week. The timing is a little bit of a coincidence, but not entirely.

The only thing that I'm doing this time is not sending emails on matches, because I just can't. Unless I asked people to volunteer their mail addresses I don't have the data. Still I expect remembering to return for the results won't be a challenge.

Now, lets get viral. But not in an icky fashion ..

ObFilm: Daywatch

| No comments

 

I saw green fields and flowers. I could smell the grass.

Tuesday, 20 January 2009

Fabio Tranchitella recently posted about his new filesystem which really reminded me of an outstanding problem I have.

I do some email filtering, and that is setup in a nice distributed fashion. I have a web/db machine, and then I have a number of MX machines which process incoming mail rejecting spam and queuing good mail for delivery.

I try not to talk about it very often, because that just smells of marketting. More users would be good, but I find explicit promotion & advertising distasteful. (It helps to genuinly consider users as users, and not customers even though money changes hands.)

Anyway I handle mail for just over 150 domains (some domains will receive 40,000 emails a day others will receive 10 emails a week) and each of these domains has different settings, such as "is virus scanning enabled?" and "which are the valid localparts at this domain?", then there are whitelists, blacklists, all that good stuff.

The user is encouraged to fiddle with their settings via the web/db/master machine - but ultimately any settings actually applied and used upon the MX boxes. This was initially achieved by having MySQL database slaves, but eventually I settled upon a simpler and more robust scheme: Using the filesystem. (Many reasons why, but perhaps the simplest justification is that this way things continue to work even if the master machine goes offline, or there are network routing issues. Each MX machine is essentially standalone and doesn't need to be always talking to the master host. This is good.)

On the master each domain has settings beneath /srv. Changes are applied to the files there, and to make the settings live on the slave MX boxes I can merely rsync the contents over.

Here's an anonymized example of a settings hierarchy:

/srv/foo.com/
|-- basics
|   `-- enabled
|-- dnsbl
|   |-- action
|   `-- zones
|       |-- foo.example.com
|       `-- bar.spam-house.com
|-- language
|   `-- english-only
|-- mx
|-- quarantine
|   `-- admin_._admin
|-- spam
|   |-- action
|   |-- enabled
|   `-- text
|-- spamtraps
|   |-- anonymous
|   `-- bobby
|-- uribl
|   |-- action
|   |-- enabled
|   `-- text
|-- users
|   |-- bob
|   |-- root
|   |-- simon
|   |-- smith
|   |-- steve
|   `-- wildcard
|-- virus
|   |-- action
|   |-- enabled
|   `-- text
`-- whitelisted
    |-- enabled
    |-- hosts
    |-- subjects
    |   `-- [blah]
    |-- recipients
    |   `-- simon
    `-- senders
        |-- root@steve.orgy
        |-- @someisp.com
        `-- foo@bar.com

So a user makes a change on the web machine. That updates /srv on the master machine immediately - and then every fifteen minutes, or so, the settigngs are pushed accross to the MX boxes where the incoming mail is actually processed.

Now ideally I want the updates to be applied immediately. That means I should look at using sshfs or similar. But also as a matter of policy I want to keep things reliable. If the main box dies I don't want the machines to suddenly cease working. So that rules out remotely mounting via sshfs, nfs or similar.

Thus far I've not really looked at the possabilities, but I'm leaning towards having each MX machine look for settings in two places:

  • Look for "live" copies in /srv/
  • If that isn't available then fall back to reading settings from /backup/

That way I can rsync to /backup on a fixed schedule, but expect that in everyday operation I'll get current/live settings from /srv via NFS, sshfs, or something similar.

My job for the weekend is to look around and see what filesystems are available and look at testing them.

Obmovie:Alive

| 9 comments.

 

You didn't faint. That's always a good sign

Sunday, 18 January 2009

Joey has implemented blogspam support for ikiwiki, which you can find here.

He was also good enough to provide valuable & useful feedback so I can present a moderately updated API - most notably with the ability to train comments when/if they are incorrectly marked.

I can imagine several other useful additions, but mostly I think the code should be cleaned up (written in an evening, posted to keep myself honest, didn't really expect anybody to be terribly interested) and then it should just tick over until comment submitters adapt and improve to the extent that my sites suffer again ..

ObFilm: Kissed

| 5 comments.

 

Death is... whimsical... today.

Monday, 12 January 2009

I'm not sure how you can pre-announce something in a way that cannot be later faked.

The best I can imagine is you write it in a text file and post the size / hash of the file.

steve@skx:~$ ls -l 10-march-2009
-rw-r--r-- 1 steve users 234 Jan 12 21:40 10-march-2009
steve@skx:~$ sha1sum 10-march-2009
99d1b6d625ed4c15a3be2be5fec63c17941c370d  10-march-2009
steve@skx:~$ md5sum 10-march-2009
1a0e68b8fbb3b0fe30e5b4a9413ceeec  10-march-2009

I don't need anybody to keep me honest, but I welcome interesting suggestions on more neat ways to pre-confirm you have content that hasn't been changed between being written and being released...?

I guess you could use GPG and a disposible key-pair, and then post the secret key afterward, but that feels kinda wrong too.

Update of course you could post the detached signature. D'oh.

Shamir's Secret Sharing could be another option - posting just enough pieces of the secret to make recovery possible with the addition of one piece that was witheld until the later date. Jake wrote a nice introduction to secret sharing a couple of years ago.

ObFilm: Léon

| 12 comments.

 

There's something out there waiting for us, and it ain't no man.

Sunday, 11 January 2009

Things have turned a little morbid here.

I imagine that if I were to cease to be alive things would mostly keep ticking over for a while. But for how long exactly?

Assuming that you've got your hosting paid for, supported, or otherwise managed that would continue to exist. But after a while domain names would start to expire, and manual intervention would be required (that is assuming that manual intervention were not required in advance.)

So when I die, I'd have to assume everything I maintained myself would disappear within two years.

Is that depressing, or realistic? I'm not sure. But definitely morbid.

ObFilm: Predator

| 10 comments.

 

Some of us are just better at hiding it, that's all.

Saturday, 3 January 2009

Since I managed to get my blogspam plugin listed on wordpress.org I've seen a surge of interest.

In the past few hours capture stats are:

Good comments1
SPAM comments100

Almost exactly 1% of submitted comments are OK, and a SPAM rate of 99%. Either forum and blog spam are even more rife than I'd expected or I am over-zealous!

Anyway that's enough on this topic for a while, if you want to follow the work you know where to look and if you don't my writing about it again will just drive you mad..

ObFilm: The Breakfast Club

| No comments

 

Recent Posts

Recent Tags