I also managed to put together a tremendous hack to solve a pretty annoying problem running multiple distributions from a single external kernel under KVM.
Ubuntu users, in particular, will be well aware of dmesg SPAM coming from the use of CONFIG_SYSFS_DEPRECATED.
In short the way that the kernel presents information beneath the /sys tree has changed over the life of the kernel - and this has a knock-on effect to the userspace supplied by different distributions and releases of GNU/Linux.
Some distributions need an "old" kernel and an "old" udev with "old" udev rules in order to create the appropriate device nodes such that the kernel will boot & mount its filesystems. (i.e. These need CONFIG_SYSFS_DEPRECATED to be set.)
Conversely some distributions mandate a "new" minimum kernel version, and supply a "new" version of udev with "new" udev rules and they absolutely will not function when presented with an "old" kernel. (i.e. They must have kernels without CONFIG_SYSFS_DEPRECATED set.)
I've solved this problem via a kernel patch which is both evil and genius. The details are a little me-specific, but in short:
- devtmpfs is used to setup and mount an initial /dev tree before /sbin/init is launched..
- udev launches later and mounts a tmpfs over /dev such that it can start creating its own nodes.
- At this point evil begins: I've patched the kernel such that any attempt to mount a tmpfs filesystem at /dev is silently changed to mount a devtmpfss filesystem instead.
- The alternative is that udev creates many nodes, but manages to fail to create the root & swap nodes such that the KVM guests fail to boot.
Ultimately udev doesn't get an empty /dev tree to play with, instead it finds one already pre-populated, such that any devices it cannot create are there regardless - because the devtmpfs implementation has already created them.
Genius. And evil. So very evil.
Steal that idea. I dare you .. (I'm impressed at how well devtmpfs works, and how easy I was able to make my "patch of evil"tm. Just a few lines in fs/namespace.c.)
ObSubject: The Last House On The Left
Tags: evil, jquery, kernels, kvm, linux kernel, stealing, udev
22 July 2012 21:50
Tonight I upgraded my personal machine to run the recently released 3.5[.0]
On my personal machine(s) I'm usually loathe to change a running
kernel, but this one was a good step forward because it allows me to
experiment with seccomp
I've tested the trivial
"no new privileges" pctl and I followed along with the
nice seccomp tutorial
On top of that I upgraded node.js,
which meant I had to clean up a little depreciated code in my node reverse
proxy - which is the public face of the websites I run upon my box.
(The proxy tunnels to about 10 different thttpd instances, each running
Happily however my weekend was not full of code, it was brightened by
the opportunity to take pictures of Aurora
and her long hair - more to come
as I've still got about 350 images to wade through..
ObQuote: "Don't you think I make a remarkable queen? " - St.
Tags: kernels, kvm, kvm-hosting, seccomp
7 September 2012 21:50
A couple of months ago I was experimenting with adding no-new-privileges to various systems I run. Unfortunately I was surprised a few weeks later at unintended breakge.
My personal server has several "real users", and several "webserver users". Each webserver user runs a single copy of thttpd under its own UID, listening on 127.0.0.1:xxxx, where xxxx is the userid:
steve@steve:~$ id -u s-steve
steve@steve:~$ sudo lsof -i :1019
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
thttpd 9993 s-steve 0u IPv4 7183548 0t0 TCP localhost:1019 (LISTEN)
Facing the world I have an IPv4 & IPv6 proxy server that routes incoming connections to these local thttpd instances.
Wouldn't it be wonderful to restrict these instances, and prevent them from acquiring new privileges? Yes, I thought. Unfortunately I stumbled across a down-side: Some of the servers send email, and they do that by shelling out to /usr/sbin/sendmail which is setuid (and thus fails). D'oh!
The end result was choosing between:
- Leaving "no-new-privileges" in place, and rewriting all my mail-sending CGI scripts.
- Removing the protection such that setuid files can be executed.
I went with the latter for now, but will probably revisit this in the future.
In more interesting news recently I tried to recreate the feel of a painting, as an image which was successful. I think.
I've been doing a lot more shooting recently, even outdoors, which has been fun.
ObQuote: "You know, all the cheerleaders in the world wouldn't help our football team." - Bring it On
Tags: blog, kernels