Over the past couple of months the machine which hosts the Debian Administration website has been struggling with two distinct problems:
- The dreaded scheduler bug/issue
The machine would frequently hang with the messages of the form:
Task xxx blocked for more than 120 seconds
This would usually require the application of raised elephants to recover from.
- OOM-issues
The system would exhaust the generous 2Gb of memory it possessed, and start killing random tasks until the memory usage fell - at which point the server itself stopped functioning in a useful manner.
Hopefully these problems are now over:
- Bytemark have generously increased the memory installed upon this (donated) machine from 2Gb to 4Gb.
- I've cleaned up some of the site code and switched to using a combination of nginx & apache 2 to serve the site. (I've documented this upon the site itself : Speeding up dynamic websites via an nginx proxy.)
The combination of these two changes should resolve the memory issues, and I've installed a home-made 2.6.31.4 kernel which appears to have corrected the task-blocking scheduler issue.
ObTitle: Bridget Jones: The Edge of Reason
Tags: apache2, bytemark, debian-administration, nginx, performance 2 comments
I'll admit it: I've used nightly cron scripts to restart Apache to get around memory leaks.
To make matters worse, I fully stop & start Apache as stories around the interwebs of graceful restarts not cutting it have left me paranoid.
(FWIW, I'm sure these leaks have something to do with php or another non-core Apache module... still its enough to make me prefer deploying applications with builtin http servers and just proxying.)