Entries posted in June 2006

I have a new desktop machine. An AMD3000 with 1Gb of memory. It is almost entirely setup. Only three Xen-related woes:

• If I run "shutdown" under a Xen kernel "wakeonlan" fails to wake the machine up. I guess ACPI related?
• If I run a Xen kernel I can't use the NVidia kernel module. So xine won't play DVDs comfortably (#367326)
• If I run a Xen kernel Qemu's serial port pass-through fails.

Also gnome-desktop-environment is uninstallable on Sid at the moment. Just when I'd switched all my machines to using it :(

So now I have:

• desktop.my.flat - Main development + desktop machine. To be turned off every night.
• itchy.my.flat - Previous desktop machine P4 + 1Gb ram. Runs DHCP + dnsmasq + xen instances + rsnapshot to backup both desktop, and my remote sites.
• lappy.my.flat - Generic Dell laptop.
• steve.my.flat - Generic Widescreen Dell laptop - supplied by work.

The two Dell Dimension L800s I used for backup purposes are retired. This is good, they had 128Mb of memory each and were always at 90-100% disk capacity. One to go to my sister in York, the other to the first person who claims it in Edinburgh.

Now to find another LCD display …. I have a 19" and a 15".

No tags | No comments

Headphones which assume non-customized ears suck.

No tags | No comments

I saw on Planet Debian that Enrico Zini had some trouble using xen-tools:

Installing the base system.   This will take a while!

Copying files from host to image.
Finished
Something went wrong with the debootstrap installation
Aborting

This basically means that the invocation of debootstrap failed to leave the system with a /bin/ls present - meaning it failed for some reason.

My immediate mad debugging skillz suggest retrying with either -verbose, or -cache=no. The first will hopefully show the problem, the second will most likely fix the problem.

I've seen this a few times, but haven't updated the code to check for it. When creating a new Xen instance of, say 4gb, we copy all the packages from the host systems /var/cache/apt/archives so that debootstrap doesn't redownload things and use up bandidth.

If these cached files are >4gb then the new image is 100% full and debootstrap won't run successfully!

In related news I'm currently working on a system called image-server - this is a distributed xen-create-image script.

The idea is that you have a central host which is responsible for creating Xen images of Debian, RedHat, Gentoo, and serving them to clients.

This isn't much use for a home user, but as a hosting company wanting to provide Xen instances of semi-arbitary Linux distributions it should come in handy.

In other news I want to work on modularising the Xen tools scripts and moving them onto xen-tools.org.

No tags | No comments

There are times I get annoyed by the trolling, the stubborn nature of some people, and the mailing list threads which never end.

But honestly I love being part of the Debian project.

Why say this now?

No particular reason. I did receive a couple of mails from developers today which were very helpful, and this made me reflect on the sheer improbability of a distributed project like Debian hanging together, growing, and surving for N years.

For all the personal differences, animosities, and stubborn people the project as a whole is undoubtably a success. I hope it continues to be one.

Thanks for letting me take part.

No tags | No comments

Last night I reinstalled Debian GNU/Linux upon my laptop so that I could have a clean starting point and a simple LVM setup. Why? Well xen-tools has had half-hearted LVM support for a few months, but not complete. And waiting for other people to add fixes has become unbearable.

So now I have a 7Gb root partition and a 12Gb LVM group which can be used to create new Xen instances upon. (Or for anything else I guess!)

Speaking of Xen Tools. Yesterday was a major day of work. I've split up the code from one monolithic script xen-create-image into three simpler programs which are called in turn:

• xt-install-image
  • This will install a new distribution either via a copy, an untarring operation, rpmstrap, or debootstrap. Adding new installation mechanisms should be trivial.
• xt-customise-image
  • This script determines the which collection of shell scripts (or "hooks") to execute, and runs them in order. These scripts can do arbitary things but all they are supposed to do is setup networking and install openSSH.
• xt-create-xen-config
  • This creates the Xen configuration file in /etc/xen.

All these changes took a few hours, but it was mostly a simple job of moving code around and adding consistant debugging output rather than adding anything new.

I'm going to have to improve documentation, write manpages, etc, before this becomes the 2.x release. Still I've been working on a pretty website and I've made some pretty webpages which will go live at the same time.

No tags | No comments

I've seen several people writing here about using different kinds of network configuration depending upon where they are. This is fine and dandy, but for the more general case is there a simple mechanism for switching system configurations depending on "environment"?

I use Xen a lot on my systems, but sometimes I boot into a normal kernel.

If I'm running a normal kernel I want to use the non-free nvidia driver for X, since this has better performance when viewing DVDs, etc.

If I'm running Xen I need to use the free nv driver, since the non-free driver isn't available.

Can I make this switch automatically?

I could add an init script to symlink the right file into place depending on whether xend is running, but that just seems fragile and suboptimal.

No tags | No comments

Today I shall mostly be working on initrd images.

We've got some software which will take a linux-kernel-vx.x.x..deb file and with one command turn that into a bootable initrd.img file complete with busybox-compiled tools, modules, and an init script.

This is used for network booting with PXE.

Pretty neat when it works. But the init script is a bash script which contains lots of code like:

mount -n proc /proc -t proc || abort "Failed to mount proc" 

This all needs converting to:

• Run the command.
• If it fails and the error looks transient repeat say five times.
• If that fails drop into /bin/sh

This would be easy with perl/ruby/something else. But to keep the size and hackability of the image down its gotta be done in sh.

No tags | No comments

Due to constant frustration that two unrelated projects were sharing the same domain name for mailing lists I finally bit the bullet and looked into what was required to setup a simple mailing list manager with easy virtual domain support. (ie. not mailman. which I am coming to loathe.)

"Enemies of Carlotta" looked the simplest, but documentation was scant and I had trouble making it work.

Instead I'm now using ecartis combined with an installation of hypermail to make archives.

The upshot of this is that the xen-tools-commits mailing list has now moved from shellcode.org over to lists.cvsrepository.org.

I've also created a xen-tools mailing list for users/questions/bugreports.

I'm still limping towards a new release with the pretty new webpages, but a bit more work to do. Although people who wish to live dangerously have the CVS repository to play with.

No tags | No comments

Ooops. The mailing list was refusing to subscribe anybody.

Should be fixed now.

No tags | No comments

Today I mostly packaged a current version of ecartis, to avoid content-transfer-encoding weirdness (#176667).

There is also a compiled version of the package for sarge should that be useful to anybody.

I also updated the "blog" facilities on the Debian Administration website - this was after I wrote a very long entry - and was annoyed that it seemed to spam the associated planet site. Entries can now be truncated by the use of the magic "<cut>" tag.

I imagine few people will need/want it, but I'm sure I'll find it handy.

No tags | No comments

Good News

My parents were in town for the day and I got to meet them for lunch.

After that we went shopping to a garden center, since they happened to mention they needed some trays for holding plants. I bought a couple of new plants for my house - including a nice looking Dragon Plant: Dracaena bicolor

Bad News

plonk

I'm suprised Mutt doesn't have integrated killfile support, but managed to add it via a couple of additions to my ~/.procmailrc file - one for senders, and one for message references to catch threads of disinterest.

No tags | No comments

Right it is 23:43 and I'm just back from a dodgy metal night - picture the winners of the Eurovision song contest but with a lesser budget.

Normally this would be described upon my real blog which contains adult language, and a mixture of random trivia, sadism, and geekyness in more detail than is displayed here. (You might think this is a hard mix, but I'll tell you it isn't suprising if you've ever programmed Gtk 2.x in raw C ;) )

Anyway the reason I went to the gig in the first place was that last year one of my partners was a bass-player, and she was often playing gigs in different locations. Unfortunately for the duration of our relationship I managed to never actually hear her play. Not deliberately - it was just the way things worked out. We kinda tailed off seeing each other because I was sleepy and she lived in a different city.

Actually this is a common theme with me. I once lived with a concert chello player and for the entire 2.5. years of our relationship I managed to never hear her play, or practise (and we lived together!). Again not deliberate, it was just the way things worked out. (Even now I don't think she's ever heard me play the guitar.) Later I had a relationship with a Swedish dentist and for the duration of our relationship I refused to let her look inside my mouth [mostly ;)]. It was only after we'd ceased being .. close .. that I let her extract some wisdom teeth of mine. Of course she was Swedish, living abroad, and all her friends were Swedish too - so I had a lot of Swedish "friends" around that time. Maybe one day I'll even meet a genuine blonde Swede. (I can say "hello", "goodbye", "I love you", "marry me", and "you're cute" in Swedish!)

(I'm not a slut; I just know what I like.)

Anyway .. enough side-tracking. The only reason this is coming up at all is because I think context is important when writing about people that aren't known - I hate seeing people in blogs write about "J", or "C", with no hint of the nature of the relationship, or why we should know/care about their actions - people are important, and if you're going to bring them up you should explain who they are, and why they are relavent. The fact that I've drunk 5 bottles of cider now is also helping extend this entry

If it helps I almost promise not to mention these people again.

So, to recap, I'm at this metal gig, and damn they were a fantastic band. Miranda (the woman I was "with", previosuly if not recently) was a total camera-slut, and she was technically extremely competant. I guess I shouldn't have been suprised but I was.

So, why do I mention any of this?

Well I was standing outside before the band started as it is illegal to smoke in public enclosed places in Scotland (since Marche this year) and this obviously rules out smoking in a pub in the basement bar. (The bar was unstaffed since the bands playing tonight were "too loud". Sucky having to walk upstairs to find beer.)

So .. stood outside, totally under-dressed for the occasion. Almost everybody else in the (very sweaty) basement bar was dressed in monochrome and I had shiny white shoes and colourful t-shirt. Still I was one of the tattoo'd + pierced people so all was good.

"Excuse me, you look like Steve Kemp".

For the first time ever somebody recognised me from my planet debian hackergotchie. Neat huh?

And now it is 23:58 and I'm going to listen to some nice Rammstein and go sleep. Tomorrow I must do some Debian work, and tidy a friends flat - since they just moved out and want their deposit back.

In the future I will try to keep the entries here more on-topic, unless people actually care about this kind of entry …

No tags | No comments

xen-tools 2.0b1 released. Please test and let me know what breaks.

After so much work, so many bugs, and so many fixes I'm reluctant to go straight from 1.6 -> 2.0, so a 2.0b1 release is in order.

Changes:

• Support for arbitary hook scripts.
• Improved command completion code.
• Better handling of errors.
• The ability to allow multiple IP addresses for xen guests.
• Possible to use "sparse" vs. "full" disk images.
• Parameterised Xen configuration files (via Text::Template.)
• Better documentation (?)
• More robust passwd + shadow file handling.
• Several installation methods; debootstrap, rpmstrap, copy, and tarfiles.

Probably more that I've forgotten about.

If I hear positive feedback expect a real release and upload to Sid this week. If not .. whenever I get round to it.

No tags | No comments

Yesterday was a day of hacking and shopping. I made the xen-tools 2.0 release, and made the beta website live.

Once that was done I folded in the changes I'd made for work - so it is now possible to configure Gentoo linux - simply download an image from jailtime.org and you can use that image as the source for a new installation:

 xen-create-image --hostname=gentoo.my.flat \
    --ip=192.168.1.234 \
    --dist=gentoo \
    --copy=/mnt/gentoo 

(More detailed instructions here.)

Here "/mnt/gentoo" is the location of the loopback mounted Gentoo image, and "-dist gentoo" is used to specify that the hooks should be run from the Gentoo directory. (The hooks setup /etc/conf.d/hostname, /etc/conf.d/net, etc.)

From start to finish a pristine installation of Gentoo running under Xen takes about five minutes. Neat.

In between working, releasing, and hacking I did a lot of shopping. I ordered some beautiful stainless-steel jewelry, some new toys, and a few weeks worth of food. All without leaving the house!

I think for the next week or two I'm going to design a distributed xen monitor/control panel. The Argo software has a reasonably good design - but it is fundamently single-server.

For work purposes, and for scalability, we need to be able to control N hosts each running X xen instances. (Rather than 1 host running X instances). I'm not sure what the best design for this is, but I think having an agent on each xen server reporting to a central server is best.

I've gotten a bit of mail about Argo recently inquiring about SSL support for the communication link - nice to read - but so far I've only considered SSH tunnels. Really handling certificates is going to require .. careful .. consideration.

No tags | No comments

Ugh. I'd forgotten how annoying it is to reinstall existing systems.

I've spent close to two hours reinstalling my "backup" PC. (This is the machine which used to be my primary desktop, now it mostly exists to host backups of my websites, and the primary machine which replaced it.)

The intention was to reinstall so that I could setup a clean system with no desktop packages, and switch to using LVM. The machine hosts several Xen instances which are used to provide services such as DHCP + Debian package building.

Backing up the data was easy, the Sarge installer is a pleasure to use, and I even remembered to copy all my SSH host keys.

So why the pain?

Well the machine is running Sid. (Will stick to Etch when it is available I think), and several packages which had been running happily for months such as apt-cacher were temporarily uninstallable. In the case of apt-cacher I was able to use approx instead - so that wasn't a huge deal.

But one thing tripping me up is sound. It was my main desktop system and had sound working fine. Now I have alsa setup, all the mixer volumes up to the max, and yet nothing.

XMMS is playing happily - but absolutely no sound coming out. Apart from checking the mixer settings I can't think of what else to do. Not a huge deal but it is a minor irritation since sometimes people come over and want a machine to use for surfing, etc, and I like to give them a standard desktop with links to my MP3 share, etc.

Still all the main services are back up and running, GDM is available remotely via VNC, the logged in desktop is also shared via x11vnc, and the backups are starting to trickle back in.

The current partition scheme looks something like this:

/dev/hda1                         on / type ext3 (rw,errors=remount-ro)
/dev/mapper/vol-home     on /home type ext3 (rw,noatime)
/dev/mapper/vol-backups on /backups type ext3 (rw,noatime)

Then I have a couple of xen domains:

skx@itchy:~$ sudo xm list
Name                              ID Mem(MiB) VCPUs State  Time(s)
Domain-0                           0      873     2 r-----   230.8
builder.my.flat                    1      128     1 -b----    39.8
dhcpdns.my.flat                  1       96      1 -b----    39.8
skx@itchy:~$ 

Now time to find food, and water my plants.

No tags | No comments

Debian is teh win!

I've been playing with other distributions recently, under Xen, and so far I have to say that Debian is sooo much easier to work with. (And not just because I'm more familiar with it).

Gentoo: Portage does rock, but damn it is slow. I don't mean the recompilation, because you'd expect that to be slow, but something as simple as "portage sync"; takes minutes to complete, and pegs the CPU whilst it is running.

A similiar story for CentOS 4.x - the yum tool seems to be one of the slowest network applications I've ever used! Updating the lists of available packages is just painfully slow, far too much "thinking" between the download operations.

Tags: gentoo, portage | No comments

Over the weekend I mostly been played pinball and reading the review copy of Nagios: System and Network Monitoring which I recently received.

This week I shall be mostly doing more code audits, and converting more systems to using LVM. I've got almost a complete article on LVM reading to be posted, but it still needs more work. Although I did draw a pretty picture!

No tags | No comments

I just saw a reminder about the Edinburgh debconf visit. As somebody who volunteered to help be involved I'm a little blindsided by this sudden announcement.

Then I realised my mistake. When I got the mail from Moray I subscribed to debconf6-localteam which appears to be a dead list. So I'm not sure what the status of anything is.

So tomorrow I will join the recent lists and poke around.

Since I'm local I'm happy to meetup for keysigning purposes to anybody visiting, although since I've not been involved in the bid at all its probably best I don't suddenly appear and confuse things. If we "win" then I'll do better.

Now I'm going to bed. I've spent six hours on trains today, and that involved 90 minutes of standing, so I'm a little worn out.

No tags | No comments

Annoyed that mod_security isn't going to be included in Debians Etch release I started work on my own replacement.

The way I use mod_referer is three-fold:

• Block particular referers
• Block particular user-agents
• Block accesses to particular URIs

Knowing that naming software is the hardest part I just looked for words beginning with mod - so please see mod_ifier!

• http://mod_ifier.cvsrepository.org/

Building:

apt-get install apache2-dev
cd mod_ifier/src
make
make install

Using:

LoadModule mod_ifier_module   /usr/lib/apache2/modules/mod_ifier.so
DropAgent lynx
DropAgent links
DropAgent wget

DropReferer gambling-sites.com
DropReferer sex.com

TODO:

• Allow logging / command executation (firewall dropping.)
• Allow to match on patterns in URIs (the third usage I use, eg /cgi-bin/formmail.cgi)

Comments welcome… especially since I'm only just getting to grips with the Apache 2.x API…

No tags | No comments