I understand volunterering is hard

Saturday, 5 October 2013

The tail end of this week was mostly spoiled by the discovery that libbeanstalkclient-ruby was not included in Wheezy.

Apparently it was removed because the maintainer had no time, and there were no reverse dependencies - #650308.

Debian maintainers really need to appreciate that no official dependencies doesn't mean a package is unused.

Last year I needed to redesign our companies monitoring software, because we ran out of options that scaled well. I came up with the (obvious) solution:

  • Have a central queue containing jobs to process.
    • e.g. Run a ping-test on host1.example.com
    • e.g. Run an SSH-probe on host99.example.com
    • e.g. Fetch a web-page from https://example3.net/ and test it has some text or a given HTTP status code.
    • (About 15 different test-types are available).
  • Have N workers each pull one job from the queue, execute it, and send the results somewhere.

I chose beanstalkd for my central queue precisely because it was packaged for Debian, had a client library I could use, and seemed to be a good fit. It was a good fit, a year on and we're still running around 5000 tests every minute with 10 workers.

The monitoring tool is called Custodian Custodian, and I think I've mentioned it before here and on the company blog.

It looks like we'll need to re-package the Ruby beanstalk client, and distribute it alongside our project now. That's not ideal, but also not a huge amount of work.

In summary? Debian you're awesome. But libraries shouldn't be removed unless it can't be helped, because you have more users than you know.

| 4 comments.

 

Comments On This Entry

[gravitar] paolog

Submitted at 09:47:29 on 5 october 2013

popcon data for libbeanstalkclient-ruby show 9 users at its peak. Probably many people do not allow popcon stats to be collected on their systems, but that is a good way to let Debian know about its users.

[author] Steve Kemp

Submitted at 10:43:03 on 5 october 2013

Yes, that's very true. I'll check on my systems later today and make sure I'm sending such data.

[gravitar] Paul Wise

Submitted at 04:12:31 on 7 october 2013

You might want to reintroduce this package to Debian:

http://www.debian.org/doc/manuals/developers-reference/pkgs.html#reintroducing-pkgs

How about packaging Custodian and adding it to Debian? If it was in Debian that would have prevented the removal of beanstalk.

I also wonder why you didn't notice this issue until now, beanstalk was removed a long time ago.

Some thoughts on running testing that can also apply to running stable.

http://bonedaddy.net/pabs3/log/2012/10/29/thoughts-on-debian-testing/

[author] Steve Kemp

Submitted at 05:18:10 on 7 october 2013

We didn't notice because this is the first time we've tried to install custodian on a pristine Debian system, until now it has been on hosts that were upgraded from Squeeze where the orphaned client-package was still present.

Custodian is already packaged as .deb releases, and early development moved so quickly there seemed little point in having it in Debian - of course if a Debian-developer wished to upload it we'd support it, happily.


 

Comments are closed on posts which are more than ten days old.

Recent Posts

Recent Tags