Entries posted in August 2006

All being well by the time you read this I'll have prepared and issued my first security advisory!

I've documented the process a little.

I could probably tidy it up a little bit more more I'll share it now in case it is useful/interesting to people.

Update: This was only possible because Joey + Moritz both helped me work through the process.

No tags | No comments

In case anybody was ever curious I've explained why I'm skx.

I admit at times that I'm curious how/why some developers chose/use the logins or nicknames they do .. but never quite enough to ask.

If you want to share please feel free to comment, I'm sure people would be interested in learning more about their fellow developers…

No tags | No comments

I started doing some bug triaging yesterday night after going over bugs in some packages I'm interested in.

(I have a small list of packages which I can't use, or need to be fixed in some way, before I can start using them and every now and again I check on progress. eg. I want the nvidia-kernel-source module to work with Xen.)

Since I was tired I only did a few things, but I'm going to do some more today; merging duplicates and closing older ones, simple stuff.

None of the packages are ones that I can NMU (OK I guess I could, and fix a couple of minor bugs, but they won't be improved enough to make it worthwhile - if that makes sense?)

Talking of NMUs .. anybody is welcome to upload "my" packages if there is a need, and feel free to poke around at my bug list. I can sometimes be slow with them.

I've been semi-seriously considering the idea of dropping all my packages, and working exclusively on security stuff instead..

I haven't made a decision yet but I'll mention it here in case I do ..

No tags | No comments

Today I

• Asked the Apache2 package team to see if they'd mind my performing a NMU. Will wait and see what they say.
• Booked a holiday in York for my girl and I.
• Looked at more bugs.
• Made the first tarball release of xen-shell
  • Is this worth a package? I could make one and share it locally, not sure that it is generally useful…

No tags | No comments

Why does all mailing list software suck?

It seems like it would be trivial to write a simple one - almost certainly using SQLite to store list details - the actual actions are trivial:

• Maintain a list of subscribers.
• Handle subscribe / unsubscribe requests
  • Without being all sucky and evil with "password requests" like GNU Mailman
• Send out each message to the subscribers.

Sure there are probably complications relating to removing/deactivating bouncing addresses, and the ability to reject/hold messages from non-subscribers. But even so it should be a weekends work. Right?

So how come nobody gets it right?

In other news I'm preparing an NMU of Apache2 to SID. I figure that my mail asking for "permission" going unanswered means that there are no objections … Works for me ;)

No tags | No comments

I received a few interesting comments on my last post about mailing list software so I'm going to summerise my current thoughts.

Before covering the available software I'll say what I personally need from a mailing list manager:

• The ability to run multiple lists on a single host.
• The ability to use virtual hosts, such that [email protected] is distinct from [email protected].
• The ability to create different types of lists:
  • Open list - anybody can post a message which gets sent to all subscribers.
  • Closed list - Only list members may post
  • Admin-list - Anybody may join, but only "special" users may post.

When it comes to the handling of spam I mostly don't care. Messages are filtered via the mailserver at SMTP time (ditto for viruses) and although moderation is generally useful I prefer to either bounce or eat the messages which are not forwarded. (Either case is fine so long as logs are available). This simplifies things significantly since the pedning messages don't need to be stored/queued.

Here is a list of all the mailing list software which is available to users of Debian's Sarge release along with some of my thoughts/comments/impressions. (Some of these details may be wrong, I'd happily be educated …):

ecartis

The Ecartis package for Debian GNU/Linux is mostly broken. In sid the package is essentially unmaintained, and that is a worrying sign.

Ignoring that for the moment the software works reasonably well. It allows most of the kind of control I desire but the significant downside is that it is severaly broken with regard to character sets and encoding issues.

For a cromulent example see this message.

enemies-of-carlotta

This software wins double-plus bonus points for a most excellent name. Naming software is hard, and although the relationship between the film and the purpose of the software is random I like it!

eoc suffers from a virtually non-existant set of documentation. Whilst it might be simple to use I it isn't documented except in a single manpage. I seem to recall I had difficulty getting my Exim4 configuration to pass relevent environment variables ($SENDER + $RECIPIENT) passed into the script properly. The lack of documentation just made this more irritating.

fml

There is a lot of documentation for this, which is half in English and half in Japanese.

I admit I mostly didn't explore this software - although my impression is that it doesn't have support for virtual hosts.

mailman

Mailman is big, popular, and I dislike it with a passion.

The admin interface has a billion configuration options, but even if you know they exist finding the damn things are an impossible job.

The system requires about five python processes running even when it is "idle". In my mind a mailing list software doesn't need a daemon componant - just a program to spawn when a message "comes in". The whole password system is broken and gains you zero additional security.

Again to rail against mailman: it has integrated message archival, something i think should be distinct. (eg. mhonarc).

Virtual hosting is possible, but I believe that mailing list names must be distinct regardless of domain name. That may not be correct honestly I've detested the softwware with a passion which suprises me and I may not be 100% objective here.

mlmmj

Looks nice. Looks clean. doesn't look particularly joyful. (The name mlmmj means "mailing list managing made joyful".)

It doesn't appear to support virtual hosting, and it doesn't appear to support enough different list types.

quickml

Listens upon a network port (10025 by default) and not support different list types or virtual hosting.

smartlist

Complex. Icky.

sympa

Seems very featureful, but it is yet another system which has an integrated web-component like mailman == bad.

It doesn't support having the same named mailing list in multiple virtual hosts, although it appears to do everything else I desire.

In summery: There are several available packages, but none are good, and many are actively bad.

I still believe that creating another list server would be a useful thing to do. I've written a brief design overview elsewhere which I will post for comment later.

I'd like to support a couple of "advanced" features such as allowing the mailing list software to only accept GPG signed messages, or to act as a remailer such that messages posted to the list will be automatically GPG signed prior to delivery to all users.

Looking around I think most of the individual bits of code are available (in Perl). eg. Mail::Message, Mail::GPG, etc. Its just a matter of gluing the bits together in a modular fashion.

No tags | No comments

The Xen hosting is now setup, and complete. There were only two minor complications:

• Giving Xen guests multiple IPs when using a routed setup is not documented terribly well
  • Predictably it is different than how you'd do it in NAT mode in a non-obvious fashion
  • When doing google searches for how to do it properly I kept finding references to documentation I'd written which didn't apply. Ugh!
• Installing grub on a RAID root took a few attempts.
  • I'm sure its setup correctly now though, on both drives.

As another achievement for the day (?!) the Debian Administration website now has exactly 5000 registered users.

Finally I made the NMU of Apache2. Since mail to @debian.org seems to be broken (?!) I've not been able to file the patch. Interested people can find it here until I can file the bug.

No tags | No comments

The past two days have been fairly busy and now I just want to avoid all computery things for the next week or so. This will almost certainly fail to happen, but I have a low-tech weekend planned which will involve shopping for outdoor gear and loitering outdoors drinking ale.

Briefly I've:

• Spent a while documenting the software setup and RAID/hardware setup of the xen-hosting environment.
• Released a new version of debian-updates. (Freshmeat project hasn't updated yet, I'm sure it will shortly.)
• Designed a specification and stub implementation of a mailing list manager
  • With no setuid/setgid components
  • A plugin facility to peform semi-arbitary operations on messages in transit.
• Investigated a million and one host-monitoring solutions:
  • Which will produce pretty graphs
  • But which will not require agents, or any software, installed on the monitored system.
• Investigated and installed a million wiki solutions:
  • Looking for one which is pretty and secure.
  • Looking for one which allows me to hook in my own authentication code to test against a remote RPC server.
• Found some excellent new music to listen to. (Band I like but hard to find second-hand - and I refuse to buy new CDs these days…)

Still todo:

• Finish writing a sexy online Ajax-filled todo list.
• Plan the migration of a lot of web + mail + dns setups from my current host to a new Xen environment.

I'm almost tempted to use one of those fancy "control panels" for managing a bunch of domains when I move over to the Xen instance. However many of them appear fragile, and contain numerous obvious XSS attacks - these do not fill me with confidence.

Perhaps I will investigate more hosting setups soon. At the very least I wish to move towards an SQL-backed virtual hosting setup for email. Ideally exim4, but I'd settle for postfix at a push.

So .. lots of research happening, and more to follow.

Expect quiet from me for a while.

No tags | No comments

If you can see this I've not screwed up DNS too badly.

Tomorrow I have to move SMTP. Joy.

On the plus side moving SMTP should be trivial since all DNS hosts I manage use the same STMP server and there is just a single alias mail.steve.org.uk. I should be able to just setup the mailserver, update that record, then use rinetd, or similar, to proxy connections from the old IP to the new one until the DNS switch propogates.

Complications include hosting mailing lists, and having mailing list archives be written to a new machine…

Bed now.

Beer first.

PS for those people that like to argue with Joerg Schilling please stop. It is beyond obvious now that arguing semantics is getting nobody anywhere, for the Nth time.

I've come to the conclusion that when arguing discussing things with Joerg he will either wilfully ignore your point(s) or deliberately misintepret the evidence to support his stance. The chance of persuading him to change his mind? 0%. Give it up already. Think of the DVD-media children!!

It really is time to either drop the cdrecord or fork it.

Please. Somebody. Do it.

No tags | No comments

My migration to the new Xen host is 75% complete. DNS changes appear to have finished and I've moved all the websites that I intended to move successfully. (Including databases & etc.)

I've got three domains to move still, but I'm being very careful with them since they tie into "local" things like CVS repositories, or mailing lists + archive creation.

The website migrations appear to have been a success, although I only noticed a couple of problems just now. In all cases my problems were:

• Symbolic links not being copied over to the new host correctly, leading to breakage. (e.g. This fine blog had a broken feed link because a symlink got trashed.)
• Missing packages/configuration on the new host. eg. CGI::Application module not being installed.

I've tested all the areas I can think of testing now. (e.g. "find /home/www -name '*.cg' -print"; then executing each one in turn!)

The next job will be the mail setup. My plan:

• Install new mailserver.
  • Configure virtual hosts.
  • Setup mailing list software (ha!)
  • Setup mail list archive creation.
  • Converting the "original" mailserver to treat the "new" as the mail relay - as per suggestion from Martin Krafft
• Flip mx record(s) over.
• Setup forwarding from old host to new host for a couple of days.

Sounds simple enough. What could possibly go wrong? (Sadly I know all too well virtually everything could go wrong.)

No tags | No comments

Migration complete! I now have two distinct servers/hosts:

• A dedicated host: AMD 1600 + 1Gb memory + 40Gb disk. (Donated)
  • Runs www.debian-administration.org + the matching planet (Apache + Perl + MySQL + memcached)..
  • Other than SSH no other services are running.
• A 10Gb Xen instance with 256Mb memory
  • Runs multiple virtual hosts; Apache + Exim4.
  • Holds my CVS Repositories.

Both servers are backed up nightly to a "backup server" hosted in my house (Hostname: itchy.my.flat) via rsnapshot.

An interesting few days getting the split to occur nicely, which involved some DNS + Domain Registrar juggling due to obscure reasons.

Now that the split has been made I can start working on better caching, and performance tweaks on the d-a.org site. I'm currently investigating the use of fast_cgi, since I think that the code should be simple to modify for this.

Otherwise I'm now back in action. I have pending releases of several packages to upload, and a new upstream release of GNUMP3d to prepare.

No tags | No comments

As another reason why I dislike mailing list software consider the following error:

2006-08-14 21:40:25 1GCk6m-0006ZH-Vu == |/usr/lib/ecartis/ecartis -s <a href="http://lists.cvsrepository.org/">xen-tools-commits</a> 
R=vdom_aliases T=address_pipe defer (0): 
Child process of address_pipe transport returned 75 (could mean temporary error)
from command: /usr/lib/ecartis/ecartis

What does that mean? I see precisely one hit for this error in Polish.

I resorted to debugging via strace:

strace -ff -o /tmp/foo /var/lib/ecartis/ecartis -s xen-tools-commits
   < /tmp/msg.txt

The error? "Connection refused" when connecting to 127.0.0.1:25.

D'oh!

It should invoke the local mailer via /usr/{lib sbin}/sendmail in the case of socket error. Ideally chosing the local execution in preference to the socket.

In conclusion: I believe my mailing lists are fixed, however several messages have been dropped/ignored.

No tags | No comments

A busy night which resulted in a new release of xen-shell, this coincided with me noticing that is being used by a Dutch ISP. (Fun sometimes googling your own name, or the name of software projects!)

I've added the project to freshmeat, under the obvious name, but no Debian package yet. I will probably make one soon though.

After that I fixed many many bugs/feature suggestions reported against xen-tools by Henning Sprang - a name I know a lot from the Xen lists.

Assuming I don't get distracted there will be a new release of that tomorrow. All the tests pass, and the code seems to work so I'm pretty sure it'll go OK. I've noticed though that my release speed is almost inversely proportional to the number of interested users.

No tags | No comments

There is now a Debian package of xen-shell available for Sarge from my apt repository.

This means I no longer need to run CVS on the xen host machine, which is one more package which can be purged :)

I don't think that it is worth uploading to Sid, (although I thought that about xen-tools too), so if you want to upload it feel free…

(Hmmmm Etch is fast-approaching. Probably time I started thinking about having an etch repository too…)

No tags | No comments

If I wrote a new feed for this blog which returned two-hundred entries, of gibberish, would this:

• Be filtered by planetplanet
• Or get be banned?

Tempting .. tempting …

As a serious reponse to planet-flooding I'd like to suggest that planetplanet be updated such that only N consecutive posts by the same individual would be shown. That way a person could still make 10 entries a day if they wished, but they would only be shown if somebody else got a word in edgeways - otherwise only the latest N would be displayed.

This could be tunable, and would ideally default to something small like 3.

/me goes to look at the code. It seems that planet/init.py: items() is the place to make the change, but dear God I hate python…

No tags | No comments

Peter Nuttall is my personal hero for running with my idea to limit N consecutive posts on a planetplanet installation - if you'd like test it and apply it to planet.debian.org, or elsewhere, find it here. (Note: Patch is reversed).

Bad day at work today. I spent half the morning wrestling with xen on my laptop - it seems that linux-image-2.6.16-2-xen-686 or linux-modules-2.6.16-2-xen-686 panics on boot in some weird Xen way. (ie. before initrd.) I upgraded to these packages yesterday afternoon, and the problem only showed itself by a failure to boot this morning.

Since the laptop is fairly new none of my install CDs were capable of booting it with networking. In the end I downloaded knoppix, broke a window whilst waiting for it to download, and finally installed a non-Xen kernel so I could work.

I've reported a bug against the kernel package (with a bogus email address which I'll need to fixup afterwards) but it hasn't shown up in the bts yet. There isn't anything obvious in the bootup message just a panic after "disabling trace buffers" so I'm not sure the report will be terribly useful.

In other "worky" news four months after making the request I got a second phone line installed in my flat today, so I can get another network connection. Neat.

Now I need to track down somebody to replace / fix my window. Thats going to be pricy, but it will have to be done before the weather turns or I'll freeze in my house. The problem is frustratingly mechanical - the window "handle" is jammed in such a way that I can't turn the handle and close the window - so it is permananetly open about four inches.

The frustration comes primarily because if the window would open more, (it won't), I could use a screwdrive to unscrew the locking site - and that would allow me to close it. (Albeit leaving it unlocked).

Its half-open/half-closed state is just the worse way for it to be stuck.

No tags | No comments

I uploaded a new perl package to the archive today Net::HTTPServer a stunningly easy to use HTTP-server module.

The last time I did so it was rejected by the ftpmasters who suggested that I used the wrong license. I didn't so I'm hoping this upload will get approved!

(Understandable mistake; the package contains LICENSE.LGPL but is dual-licensed like perl as the code and readme says. It was rejected because I said it was "duel perl" and the reviewer believed it was LGPL.)

I've been using this module a lot recently at home and work and like it. I might write about using it for random purposes in my copious free time …

No tags | No comments

I'm suprised to see people calling each other sluts on the Debian Planet.

Thankfully I'm not going to talk about sexism, or mention hot-babe.

I was tempted. Briefly.

On the right I've got a guy coming round to my flat to take a look at my broken window on Tuesday. I've done a fair bit of work today, and fixed some more bugs, but it is almost painful to work in that room, just because of the cold.

I could use the heaters but it seems wasteful to turn them on knowing full well that the majority of the heat will just float straight out of the window.

If I wasn't such a slut I'd wear more clothes …. ;)

No tags | No comments

Amused to see a mail from Radu asking me if I were the Steve Kemp mentioned in relation to osCommerce - I'm not.

I know that names aren't unique because I've noticed collisions in the past, but this is the first time that I've seen it in relation to myself and software.

I remember a few months back an utterly surreal situtation where I checked into The Tower Hotel in London, and after being in my room for a short while a guy tried to open the door. I opened it said "Hi, this is my room". (My response might have been a little less polite. I was tired..)

He replied my "My name is Steve Kemp, and this is my room."

Turns out that the hotel had the two of us there that night and managed to believe that the two of us were the same person - so we'd both been given keys to the same room.

No tags | No comments

GNUMP3d

I've made a test version of GNUMP3d v3.0 available for people who wish to try it out in advance of a real release.

This will most likely be the last GNUMP3d release I make, so feedback is appreciated.

Once it is out I'll hand the project over to other people who want it. (So long as they agree to keep the basic goals of the project alive, can program perl, and have some familiarity with the code).

The project has quite a lot of users interested in it, so it deserves to live on.

This project was the first program I packaged for Debian, and was the reason I joined the project in the first place…

No tags | No comments

A tool suggestion for moreutils

haschanged

The intention of haschanged is to test whether a file has changed.

It will do this by computing a hash of a file contents and storing that in a file beneath ~/.haschanged.

If a new hash differs from the previously stored hash, or there is no recorded hash it will return 1 0.

If the new hash and the old hash remain the same then the tool will return 0 1.

The script is trivial, but fairly useful for a lot of things.

The only thing that I don't like is having to store the hash somewhere… (The alternative is to copy the file somewhere, or create "${file}.orig", and then run diff. The latter doesn't work for non-root users wanting to monitor a file in /etc.)

Tags: mybin, utilities | No comments

Remember that broken window of mine?

The one which has made my home office almost painfully cold for the past four/five days?

Today an engineer came round to fix it. It took him 30 second to locate a hidden button, and press it. Thus resetting the catches inside the window-frame.

D'oh.

Still at least I know how to do it myself now if I ever get into this situation in the future.

No tags | No comments