Entries tagged kernels

I'm in a position where I need to rebuild a Linux kernel for a number of distributions and architectures. Currently the distributions are:

• Debian Etch
• Ubuntu Dapper
• Ubuntu Edgy
• Ubuntu Feisty
• Ubuntu Gutsy

(For each distribution I need a collection of packages for both i386 and amd64.)

I've written a couple of scripts to automate the process - first of all running "make menuconfig" within a debootstrap-derived chroot of each arch & distribution pair. Then later using those stored .config files to actually produce the packages via make-kpkg.

This process, as you could imagine, takes several hours to complete. Then there's the testing ...

I'm sure there must be other people with this kind of need but I was suprised to see nothing in my search attempts.

ObRandom: I'm tempted to switch from song-lyrics to film names as post titles. Undecided as yet. I guess it doesn't really matter, just gives me a small amount of amusement. Even now.

Tags: debian, kernels, make-kpkg, ubuntu | No comments

So I've had a hectic few days, and I'm getting close to having caught up with the things that I've been sitting on whilst I've been away.

ObRandom: Several people, independantly, have told me within the past few days that "whilst" is not a real word. it is. End of ..

Some interesting things I've been working upon recently include a fun little firewall tool. Once upon a time I wrote a firewall script which worked like this:

firewall/
`-- incoming.d
    |-- smtp
    |-- ssh
    `-- www
`-- outgoing.d
    |-- ssh
    |-- smtp
    |-- dns
    `-- icmp

When you executed the magic firewall script it would scan the incoming.d directory, and for each file it found lookup the relevant port in /etc/services. These port numbers would then be opened. And at the end you'd just have a "-j DROP".

After a long phone conversation to a colleague on Thursday/Friday of last week I've now reworked this idea anew. There is still the notion of filenames referring to what is allowed for a pair of directories (incoming.d/ + outgoing.d/) but even more flexability and no hardwired use of /etc/servvices.

I guess some ideas are just too simple to give up ..?

Anyway there are a plethora of different firewall applications of varying sophistication and complexity in the world. I don't really want to go out of my way to promote this one - but at the same time it might be a useful idea for somebody?

The next (work) job I have is determining how to make a "kernel" + "kernel-dev" RPM package based on Debian sources. Joy. Actually the more I look around the more fiddly, annoying, and troublesome I suspect this is going to be. Sigh.

ObQuote: The Grudgy

Tags: bytemark, centos, firewalls, kernels, rpm | 3 comments

This week has been a little hectic, as I've been struggling with testing different versions of the GNU/Linux kernel.

Specifically I've been trying to solve a problem where a Phenom processor, when coupled with 8Gb, would kernel panic under heavy load.

After testing various patches, kernel versions, and random things I believe the problem is fixed in the kernel version 2.6.27RC4 - however nothing in the changelog appears relevant, so I guess only time will tell.

Now we need to solve the problem of Atom processors panicing when attempting to boot 64-bit kernels. That is still present in the 2.6.27RC4 kernel.

(ObRandom: If there are any interested parties I can provide remote serial console access to such a system.)

Finally I've also been playing with PAM, the plugabble authentication module. Again specific use-case here. At work we want to allow people to ssh to some systems (to access serial consoles, etc), and we wish their connections to be tested against our internal single-sign-on mechanism.

That could have meant a whole new PAM module, which would do XML-RPC-fu. Instead it meant packaging libpam-external - which is a neat PAM module allowing you to specify a shellscript to validate users & passwords.

(libpam-external is very similar to mod_authnz_external which is a similar pluggable Apache2 module)

So, this week "kernel hacking", & "pam hacking". Does that make me a real developer now?

ObQuote: Time Bandits

Tags: apache2, kernels, pam | 3 comments

Good people steal ideas, right? On that basis I setup a static domain to host the javascript and icons I use upon a few different sites & projects. This was preempted by the release of a new version of the excellent jQuery library.

I also managed to put together a tremendous hack to solve a pretty annoying problem running multiple distributions from a single external kernel under KVM.

Ubuntu users, in particular, will be well aware of dmesg SPAM coming from the use of CONFIG_SYSFS_DEPRECATED.

In short the way that the kernel presents information beneath the /sys tree has changed over the life of the kernel - and this has a knock-on effect to the userspace supplied by different distributions and releases of GNU/Linux.

Some distributions need an "old" kernel and an "old" udev with "old" udev rules in order to create the appropriate device nodes such that the kernel will boot & mount its filesystems. (i.e. These need CONFIG_SYSFS_DEPRECATED to be set.)

Conversely some distributions mandate a "new" minimum kernel version, and supply a "new" version of udev with "new" udev rules and they absolutely will not function when presented with an "old" kernel. (i.e. They must have kernels without CONFIG_SYSFS_DEPRECATED set.)

I've solved this problem via a kernel patch which is both evil and genius. The details are a little me-specific, but in short:

• devtmpfs is used to setup and mount an initial /dev tree before /sbin/init is launched..
• udev launches later and mounts a tmpfs over /dev such that it can start creating its own nodes.
• At this point evil begins: I've patched the kernel such that any attempt to mount a tmpfs filesystem at /dev is silently changed to mount a devtmpfss filesystem instead.
  • The alternative is that udev creates many nodes, but manages to fail to create the root & swap nodes such that the KVM guests fail to boot.

Ultimately udev doesn't get an empty /dev tree to play with, instead it finds one already pre-populated, such that any devices it cannot create are there regardless - because the devtmpfs implementation has already created them.

Genius. And evil. So very evil.

Meh.

Steal that idea. I dare you .. (I'm impressed at how well devtmpfs works, and how easy I was able to make my "patch of evil"^tm. Just a few lines in fs/namespace.c.)

ObSubject: The Last House On The Left

Tags: evil, jquery, kernels, kvm, linux kernel, stealing, udev | 2 comments

Tonight I upgraded my personal machine to run the recently released 3.5[.0] kernel.

On my personal machine(s) I'm usually loathe to change a running kernel, but this one was a good step forward because it allows me to experiment with seccomp filters.

I've tested the trivial "no new privileges" pctl and I followed along with the nice seccomp tutorial which gave me simple working code which I married to my javascript interpreter.

On top of that I upgraded node.js, which meant I had to clean up a little depreciated code in my node reverse proxy - which is the public face of the websites I run upon my box. (The proxy tunnels to about 10 different thttpd instances, each running upon 127.0.0.1:xx).

Happily however my weekend was not full of code, it was brightened by the opportunity to take pictures of Aurora and her long hair - more to come as I've still got about 350 images to wade through..

ObQuote: "Don't you think I make a remarkable queen? " - St. Trinian's (2007)

Tags: kernels, kvm, kvm-hosting, seccomp | No comments

A couple of months ago I was experimenting with adding no-new-privileges to various systems I run. Unfortunately I was surprised a few weeks later at unintended breakge.

My personal server has several "real users", and several "webserver users". Each webserver user runs a single copy of thttpd under its own UID, listening on 127.0.0.1:xxxx, where xxxx is the userid:

steve@steve:~$ id -u s-steve
1019

steve@steve:~$ sudo lsof -i :1019
COMMAND  PID    USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
thttpd  9993 s-steve    0u  IPv4 7183548      0t0  TCP localhost:1019 (LISTEN)

Facing the world I have an IPv4 & IPv6 proxy server that routes incoming connections to these local thttpd instances.

Wouldn't it be wonderful to restrict these instances, and prevent them from acquiring new privileges? Yes, I thought. Unfortunately I stumbled across a down-side: Some of the servers send email, and they do that by shelling out to /usr/sbin/sendmail which is setuid (and thus fails). D'oh!

The end result was choosing between:

• Leaving "no-new-privileges" in place, and rewriting all my mail-sending CGI scripts.
• Removing the protection such that setuid files can be executed.

I went with the latter for now, but will probably revisit this in the future.

In more interesting news recently I tried to recreate the feel of a painting, as an image which was successful. I think.

I've been doing a lot more shooting recently, even outdoors, which has been fun.

ObQuote: "You know, all the cheerleaders in the world wouldn't help our football team." - Bring it On

Tags: blog, kernels | 3 comments