I'm full of love? I'm not losing it?

Wednesday, 18 February 2009

I think I've made the decision that at some point in the next few months the xen-hosting.org setup I maintain will be going away, and will be replaced with kvm-hosting(.org).

The only issue I need to ponder is handling the migration with the minimum downtime.

The plan would probably involve upgrading the host machine to Lenny, then installing KVM and fiddling with filesystems until the guests boot. I suspect it wouldn't be a huge job, but there are a few issues that will need to be planned.

Most notably I expect that most of the current guests don't have grub installed, etc, so we'd be in the position to use an external kernel + initrd. That's not an insurmountable problem, but I know that externally supplied kernels have caused me problem in the past with KVM.

Perhaps the actual plan would be to wait until September at which point I could order a new machine and cancel the current one. That would mean another increase in spec and the migration process would be a lot simpler - instead of everybody being offline for a few hours I could migrate guests individually from the old host to the new.

Anyway decisions decisions ..

ObFilm: Buffy - But we'll pretend the TV series counts as a film, kthxbye?



Comments On This Entry

[gravitar] Charles Darke

Submitted at 20:38:37 on 18 february 2009

It would be interesting to hear how this goes. I did set up a Xen host in the end (you may recall I posted on your mailing list). It's sad that Xen never got integrated into the mainline kernel. There seems to be no luck in getting it ported to newer kernels so it seems it will die out. Kinda sad, since it works well and performance is good. I tried KVM a while ago and stability just wasn't there. Hopefully things have improved. Since I run only linux, paravirtualisation is great.
[author] Steve

Submitted at 20:44:49 on 18 february 2009

Xen is slowly getting integrated these days, but until it does fully then its a real challenge to get it up and running on "current" hardware.

The lack of hardware support is at least half of the motivation for seeking alternatives at the moment amongst people I've spoken to.

As for KVM stability, well so far I've not had any issues. I'm running it quite happily at home & work.

Its pretty simple to get up and running, and having the console running inside tscreen makes control easy - in a very similar fashion to xen-shell.

(I do have a modified version of xen-shell which works for KVM, but it is slightly less featureful. It basically has "start", "stop", "console", and "reboot". I'll need to fix that up as part of the migration too.)

[gravitar] Aigars Mahinovs

Submitted at 20:55:16 on 18 february 2009

Why is everyone moving from Xen to KVM? Have I missed some development due to which Xen is no longer good? Does KVM have live migration and other more advanced Xen features?
[author] Steve

Submitted at 21:01:33 on 18 february 2009

Xen is here, and it works well enough.

The reason that a lot of people are abandonining it is because of the relative complexity and the fact that its integration into the Linux kernel has been glacial (at best).

When you're buying new hardware for use as virtualization hosts you do not want to be forced to run old kernels.

As for KVM, much like lguest, it just works. There are no funky python layers, you're just dealing with a single process and a relatively simple kernel module.

KVM does support live migration these days - over an SSH pipe at least. (I'm pretty sure the migration options of KVM exceed those of Xen, but I admit I've only toyed with migrations, and mostly do them off-line for my own sanity.)

Other advanced features? You'd have to be more specific, me? I just care that I can get a guest up and running with a fake-disk, and some fake-nics. More than that I don't much care. The wikipedia comparison page probably covers most of the highlights.

[gravitar] Charles Darke

Submitted at 01:42:40 on 19 february 2009

Your xen-tools / xen-shell are a godsend, so I'd be interested in seeing kvm-tools too! ;) Did you consider openvz, vserver as alternatives to KVM? If so, why did you go for KVM instead?
[author] Steve

Submitted at 05:49:36 on 19 february 2009

I didn't consider openvz at all - The limitation of everybody sharing a single kernel, and the resource-limitation ideas just don't map well for my personal use.

[gravitar] Bill Boughton

Submitted at 19:16:26 on 19 february 2009

If you wait to get new hardware, try and find a system that supports either AMD NPT(Nested Page Tables) or Intel EPT (Extended Page Tables). I have also had fun witht he -kernel and -initrd options, atleast in KVM-83 I am unable to reboot such guests without restarting qemu. The reason for using this was the lack of partition tables in the disk images of guests that formely ran under Xen.
[author] Steve Kemp

Submitted at 23:30:47 on 19 february 2009

Thanks for the tip Bill.

The failure to exit with external kernel was exactly the problem I saw - even when fiddling with the code and adding the "-no-reboot" flags.

That problem went away when I :

  • Enabled AHCI.
  • Switched to the installed local kernel.

I suspect the latter was the solution rather than the former, but I've not retested it recently.

[gravitar] Charles Darke

Submitted at 15:26:50 on 20 february 2009


What's the performance like with NPT and IOMMU (is this even out yet)?

The Xen approach is limited, but performance is pretty good. I'm wondering whether 2nd gen hardware virtualisation closes the gap.

Charles Darke

[gravitar] Alex

Submitted at 18:28:17 on 20 february 2009

I've been considering this myself, I've been a fairly long user of Xen (since 2.0) - but the progress towards getting it in the mainline kernel is just too slow. The main reason I've stuck with Xen so far is due to the lack of VT/SVM support from the cheap servers I've rented, making paravirtualisation the only option. However now I can get servers with SVM support, KVM looks like an appealing option.

I'm not quite satisfied at the maturity of KVM just yet though. I'll probably reconsider my position once RHEL/CentOS 6.0 is released, as that's my preferred distribution for running host servers - even though I tend to run Debian inside most of the virtual machines!

If decent paravirtual support was included about a year ago along with good distribution support for installing virtual machines then it would've been far more successful, but sadly everything Xen related still feels like a hack for different distributions.

[gravitar] Enzo

Submitted at 21:53:42 on 26 february 2009

Xen will NEVER be integrated into the Linux kernel. Xen IS a kernel and Linux runs on top of Xen.
[author] Steve Kemp

Submitted at 22:55:18 on 26 february 2009

Enzo - I suggest you learn more about Xen.

[gravitar] Andre Lu\xEDs Lopes

Submitted at 23:41:42 on 18 february 2009

Hello Steve,
Please, let us know how well this project goes by blogging about it, just like you've done already :-)
I'am also interested in using KVM in production, but couldn't find too much info on people doing that already.
I've a couple of servers running Xen and even if it's running quite well, I don't feel it's the right answer in the long term judging byt what I'm seeing currently.
Knowing success stories is half the way for me to get the confidence that I need to go ahead with my migration plans :-)


Comments are closed on posts which are more than ten days old.

Recent Posts

Recent Tags