Look are you gonna step outside or do I have to drag you?

Sunday, 25 October 2009

Over the past couple of months the machine which hosts the Debian Administration website has been struggling with two distinct problems:

The dreaded scheduler bug/issue

The machine would frequently hang with the messages of the form:

Task xxx blocked for more than 120 seconds

This would usually require the application of raised elephants to recover from.

OOM-issues

The system would exhaust the generous 2Gb of memory it possessed, and start killing random tasks until the memory usage fell - at which point the server itself stopped functioning in a useful manner.

Hopefully these problems are now over:

The combination of these two changes should resolve the memory issues, and I've installed a home-made 2.6.31.4 kernel which appears to have corrected the task-blocking scheduler issue.

ObTitle: Bridget Jones: The Edge of Reason

| 2 comments.

 

Comments On This Entry

[gravitar] schmichael

Submitted at 19:32:20 on 25 october 2009

I'll admit it: I've used nightly cron scripts to restart Apache to get around memory leaks.

To make matters worse, I fully stop & start Apache as stories around the interwebs of graceful restarts not cutting it have left me paranoid.

(FWIW, I'm sure these leaks have something to do with php or another non-core Apache module... still its enough to make me prefer deploying applications with builtin http servers and just proxying.)

[gravitar] Bob Proulx

Submitted at 08:08:07 on 26 october 2009

Your OOM-killer problems can be solved by ensuring that enough swap space is available and then setting
sysctl -w vm.overcommit_memory=2
to turn off memory overcommit.

 

Comments are closed on posts which are more than ten days old.

Recent Posts

Recent Tags