Entries tagged work

I've spent a few hours recently looking at building RPM packages of GNU/Linux kernels, which has been a frustrating process.

There are many many online guides which give the impression that this is actually a pretty complex process. For example How To Compile A Kernel - The CentOS Way guide. (Did I mention how bad most of the howtoforge guides are recently?)

So, after fiddling around for an afternoon and getting lost I decided to abandon the process.

Here is a tested process for building a binary RPM kernel package:

cd linux-2.6.24.7/
make rpm

Yes this works just fine upon a Centos 5.x machine - I'm used to using make-kpkg to make a Debian kernel package, but it seems that if you just visit kernel.org and download the latest version you can build a RPM without any extra effort thanks to native support. Cool.

Now I need to work out how to create, host, and update a YUM repository. That looks fiddly and annoying too. XML. Eww. Any guides are most welcome - ultimately I need to package and host a "recent" kernel for Centos 4.x, Centos 5.x and Fedora Core 6-9 - each for i386 + amd64.

ObQuote: Spiderman

Tags: rpm, work, yum | 3 comments

It is nice when you work for a company where you can say:

"Ice-lolly break..."

The response?

"Me too!"

Tonight has been a productive evening, I guess the ice-lolly helped!

I managed to optimize the storage of rejected SPAM mail for my commercial service. That is something I've been obsessing over recently since the volume of SPAM is currently hovering around 2.5 million messages.

Still I suspect it is only a matter of weeks before I need to expand. The current setup has me using three machines:

• Primary machine runs:
  • Web Application
  • SMTP processing/filtering/delivery
• Secondary machine runs:
  • SMTP processing/filtering/delivery
• Offsite machine:
  • Runs the blog.
  • Runs the support system.
  • Runs the service status page

Ideally I'd like to split that up further so that I have a single machine running the web application (the part the user interacts with), a pair of MX machines, and the offsite machine doing the minimal work it does.

That way the incoming mail will not affect the application at all directly.

Thankfully the split should be trivial. The only hard part is finding a fast webhost that can offer me ~1Gb of RAM, ~1000Gb of disk space, and won't charge much. Ideally around £15/$30 a month. (hahaha! hahaha! ha!)

ObQuote: Léon

Tags: hosting, mail-scanning, random, work | 4 comments

It was interesting to see Clint Adams describe love and dissatification with configuration management.

At work I've got control of 150(ish) machines which are managed via CFEngine. These machines are exclusively running Debian Lenny. In addition to these hosts we also have several machines running Solaris, OpenBSD, and various Ubuntu releases for different purposes.

Unfortunately I made a mistake when I setup the CFEngine infrastructure and when writing all the policies, files, etc, I essentially said "OK CFEngine controlled? Then it is Debian". (This has been slowly changing over time, but not very quickly.)

But in short this means that the machines running *BSD, Solaris, and non-Debian distributions haven't been managed as well via CFEngine as the rest, even though technically they could have been.

A while back I decided that it was time to deal with this situation. Looking around the various options it seemed Puppet was the way of the future and using that we could rewrite/port our policies and make sure they were both cleanly organised and made no assumptions.

So I setup a puppetmaster machine, then I installed the client on a range of client machines (openbsd, debian lenny, ubuntu, solaris) so that I could convince myself my approach was valid, and that the tool itself could do everything I wanted it to do.

Unfortunately using puppet soon became painful. It has primitives for doing various things such as maintaining local users, working with cronjobs, and similar. Unfortunately not all primitives work upon all platforms, which kinda makes me think "what's the point?". For example the puppet client running upon FreeBSD will let you add a local user, setup a ~/.ssh/authorized_keys file but will not let you setup a password. (Which means you can add users who can login, but then cannot use sudo. Subpar)

At this point I've taken a step back. As I think I've mentioned before I don't actually do too much with CFEngine. Just a few jobs:

• Fetch a file from the master machine and copy into the local filesystem. (Making no changes.)
• Fetch a file from the master machine, move it to the local system after applying a simple edit. (e.g "s/##HOSTNAME##/`hostname`/g")
• Install a package.
• Purge a package.
• Setup local user accounts, with ~/.ssh handled properly.
• Apply one-line sed-style edits to files. (e.g. "s/ENABLED=no/ENABLED=yes/" /etc/default/foo)

(i.e. I don't use cron facilities, I add files to cron directories. Similarly I don't use process monitoring, instead I install the monit package and drop /etc/monit/monitrc into place.)

There is a pretty big decision to make in the future with the alternatives being:

• Look at Chef.
• Stick with CFEngine but start again with a better layout, with more care and attention to portability things.
• Replace the whole mess with in-house-fu.

If we ignore the handling of local users, and sudo setup, then the tasks that remain are almost trivial. Creating a simple parser for a "toy-language" which can let you define copies, edits, and package operations would be an afternoons work. Then add some openssl key authentication and you've got a cfengine-lite.

For the moment I'm punting the decision but I'm 90% certain that the choice is CFEngine vs. Chef vs. In-House-Fu - and that puppet is no longer under consideration.

Anyway despite having taken months to arrive at this point I'm going to continue to punt. Instead my plan is to move toward using LDAP for all user management, login stuff, and sudo management. That will be useful in its own right, and it will coincidentally mean that whatever management system we do end up using will have on less task to deal with. (Which can only be a good thing.)

ObFilm: Terminator II

Tags: cfengine, chef, ldap, openldap, puppet, work | 14 comments

Recently an ex-colleague of mine changed jobs and suggested that I write something about the pros and cons of working from home. I've thought about this subject, off and on, for a few years and frustratingly I think most of the pros and the cons are the same:

• When you work from home you're working from home.

I live in a two-bedroom flat in Edinburgh. (Having just spent thousands on a new bathroom I don't expect I'll be moving any time soon. A rough budget of £40-50,000 would let me convert my attic into two/three rooms. So there is growth potential!)

In my flat I have made one of the bedrooms an office. The office contains:

• A huge desk with two PCs, and two telephones on it.
• Several book-cases.
• A wall-mounted fan.
• Very little else.

One PC is for work. One PC is for me. One phone is for work. One phone is my own.

Every working day I switch on the work router, the work phone, the work PC around 09:30. I then work, taking a lunch-break between either 12:00-13:00 or 13:00-14:00, until 18:00 at which point I switch off the work toys.

I ignore my personal PC during the working day with the exception that it is the source of my music. I can reach across and hit the appropriate multi-media keys to select Play/Pause/Next Track/Previous Track/Volume Up/Volume Down. (When thenever the work-phone rings the first ring is ignored as I scramble to hit "Mute" or "Pause"..!)

So what are some of the advantages/drawbacks? Well I'm at home. So the environment is one that I've made myself, and enjoy. The music is mine. The colour-scheme is mine. The pictures on the walls are mine. I have a Steve-loving chair. There is no soulless air-conditioning, no horrible cubicles, and no noisy people talking.

The downside? No people talking. If I didn't leave my house at lunchtime I'd speak to zero people face to face in an average working day. That took a while for me to notice, but it is not nice.

Since I work from home "the commute" takes seconds. I tend to get out of bed and wander straight to the desk. I'll work non-stop, then get dressed around lunchtime so that I can go out for lunch. Hail, Rain, Snow, or Sunshine I leave the house for lunch every single day (unless waiting for an atypical delivery). Because if I didn't I'd have no human contact. In the afternoon if it is a nice day I'll get undressed again, because I can, so why the hell not?

Providing you're focussed working from home has several advantages that I can think of - I've no qualms about setting the washing machine going before I start work knowing that I can "spare" five minutes to empty it later in the day. Similarly I've no concern about ordering (even large) items, because I know what time the postman comes, and I know I'm never going to be out and miss a delivery.

When I first started working from home I had a laptop instead of a PC and there were mornings when I worked, lazily, from my bed, or from my sofa whilst watching TV. That didn't last for long because I just didn't do a good job. I think I got away with it in the sense that I don't think people noticed, but I expect if it had lasted for longer it would have been quickly apparent. I stopped because the line was blurring between "home life" and "work life".

Having a dedicated working area is essential in keeping me focussed. I don't do "home things" when I'm in "work time" - with very rare exceptions. Yes I wander around and pace if I'm thinking, yes I make more tea and coffee than I would in a real office, and yes I might open windows, phone a friend, read my gas meter, washup dishes, or similar as I'm "making coffee". But on the whole it only works if I work when I'm working.

I could save money by using my work-internet instead of paying for personal-internet, but keeping the two links separate is another way of being focussed. I don't do "dodgy" things on the internet, on the whole (haha), but if I do I'd want to be damn sure that that was via my link and not the work-link - and having two PCs and two network links I know that is the case. There have been times when the work link has broken and I've used my personal link + openvpn to continue working, or at the very least re-join our internal chatroom and say "Internet down, brb".

As a system administrator there are times when I have to do things either early in the morning, late at night, or even during a weekend. I guess a final advantage is that this is not a struggle - providing I don't schedule such operations at times when I'm in the pub, meeting friends, or taking pictures of cute strangers, it isn't a struggle to say "I'll do this after 8pm tonight", or set the alarm early. No long walk to an office, and if I've already got food cooking for my tea I can eat it nearby whilst still configuring things and testing sites/services/machines.

So pros: I'm in my own environment, I don't worry about receiving parcels, meter-readings, and have wonderfully pleasant music all day. Ancillory bonuses are really side-effects of being in my environment: I have my good coffee, my nice cups, I can eat food I enjoy. etc, etc.

Cons: You must be dedicated. You must be focussed lest you give in to temptation and cease working for minutes/hours at a time. You lose part of your home space - I can't turn this room into a childrens bedroom, for example.

Nothing earth-shattering. I've done this for five years now, and although I was a little skeptical initially I thought "Why not?" It has worked out well and I think if I ever did need to leave my current position I'd have no hesitation about working from home in the future.

Finally it has to be said that when I've had partners in my life they've traditionally been the type to wake up later than me. I get significant brownie points for being able to wake them up around 10/11AM with a cup of hot coffee & breakfast in bed every morning. By virtue of having a separate space I can close the door and not be disturbed by them walking around.

I'm sure I've forgotten things - but as an initial pass the benefits and disadvantages of working from home are the same: You're in your own house.

ObQuote: "Explorers in the further regions of experience." - HellRaiser

Tags: flat, home, work | 7 comments

After seven years working from home I've resigned from my position at Bytemark.

Why? A combination of wanting to do something different coupled with the desire to reclaim my second bedroom, which is currently tied up as an office.

Working in an office in the future will be weird ("You mean I have to get dressed every day?!") but hopefully not unduly burdonsome.

My two-year plan still remains in effect: Pay off this flat as soon as possible, then purchase another and rent this one out. Giving me some income of my own, which I will need.

The "five" year plan involves me quitting work, so that I can stay home and raise children. That makes sense because sometime next year I'll become the partner who earns the least amount of monies, and I'll also be the partner with the lowest upper-bound on salary potential (short of moving to London/similar which I've always ruled out).

Having rental income for myself means I'm not utterly dependant on other money, and all being well this place will be 100% paid off within 18 months.

(After that lots of saving will take place for a deposit for the second place. We did bid on a couple of places locally, which were outstanding, but it is perhaps for the best we didn't win them. No more looking at ESPC!)

Bytemark now becomes a company I recommend 100% for hosting in the UK. In the past I've always said nice things, but I've not strongly recommended them/us, because I'm too biased.

All my personal hosting, except for one virtual machine, will remain at Bytemark indefinitely. Lovely, flexible, and great.

(I have one outside guest for the purposes of diversification. That currently lives at Mythic Beasts.)

Tags: bytemark, flat, life, work | 7 comments

Recently I wrote about docker, after a brief diversion into using runit for service management, I then wrote about it some more.

I'm currently setting up a new PXE-boot environment which uses docker for serving DHCP and TFTPD, which is my first "real" usage of any note. It is fun, although I now discover I'm not alone in using docker for this purpose.

Otherwise life is good, and my blog-spam detection service recently broke through the 11 million-rejected-comment barrier. The Wordpress Plugin is seeing a fair amount of use, which is encouraging - but more reviews would be nice ;)

I could write about work, I've not done that since changing job, but I'm waiting for something disruptive to happen first..

ObQuote: Dune. (film)

Tags: blogspam, docker, work | No comments

In November I resigned from Bytemark.

In December I started working for a local company, here in Edinburgh, in a real office (rather than working from home).

Unfortunately today I resigned from that new job, meaning I'm currently unemployed.

I plan to take a 1-2 week vacation, then look for another job as a matter of some urgency. (I can live off savings for the next half-year, or so, if I need to, but I'd go crazy if I had nothing to do for that long.)

It is unfortunate to have to resign from a new job after only five-six weeks, but much more honest to do so now than pretend everything was OK and do it at the point I'd passed my probationary period (of three months).

The people were lovely, the office was lovely, the coffee machine was excellent, the work was interesting, but the nature of a large corporate job with the associated bureaucracy made it a less good fit for me than it looked on paper.

I shall pretend that the next week or two of down-time is our honeymoon ;)

Tags: life, work | 2 comments

Just to recap my life since December:

I had worked with Bytemark for seven years and left for reasons which made sense. I started working for "big corp" with a job that on-paper sounded good, but ultimately turned out to be a poor fit for my tastes.

I spent a month trying to decide "Is this bad, or is this just not what I'm used to?", because I was aware that there would obviously be big differences as well as little ones.

At the point I realized some of the niggles could be fixed but most couldn't then I resigned, rather than prolong the initial probationary training period - because I knew I wouldn't stay, and it seemed unfair and misleading to stay for the full duration of the probationary period knowing full well I'd leave the moment it concluded - and the notice period switched from seven days to one month.

A couple of people were kind enough to get in touch and discuss potential offers, both locally, remotely in the UK, and from abroad (the latter surprised me, but pleased me too).

I spent a couple of days "contracting", by which I really mean doing a few favours for friends, some of whom paid me in Amazon vouchers, and some of whom paid me in beer.

e.g. I tweaked the upcoming death Knight site to handle 3000 simultaneous HTTP connections, then I upgraded some servers from Squeeze to Wheezy for some other folk.

That aside I've largely been idle for about 10 days and have now picked the company to work for - so I'm going to be a contractor with a day-rate for an American firm for the next couple of months. If that goes well then I'll become a full-time employee, hopefully.

Tags: life, random, work | 2 comments

It seems that several people replied to the effect that they would pay people to take care of applying security updates, or even configuring adhoc things such as wikis, graphite, and MySQL.

Not enough people to rely upon, but perhaps there is scope for remote stuff being done in exchange for folding-money. (Of course some of those that replied are in foreign countries which makes receiving payment an annoyance, that's a separate problem though.)

Food for thought.

In the meantime I've settled into my use of lighttpd, which I've recently migrated to.

One interesting thing is that you can set your own "Server Name" directive:

# Set server name/version
server.tag = "lighttpd/(steve)"

This value is used by mod_dirlisting, so for example if you examine a directory which doesn't contain an index.html file you see the server-name. Cute.

Well cute unless, or until, somebody sets:

# Set server name/version
server.tag = "<script>alert(3)</script>"

That does indeed show javascript to all your visitors. Not a security problem itself, as you need to be root on the remote site. If you're root in the remote server you could just modify the actual HTML pages being served to include your javascript. That said it's a little icky.

The following patch avoids the issue:

--- mod_dirlisting.c.org	2014-02-26 00:14:43.296373275 +0000
+++ mod_dirlisting.c	2014-02-26 00:16:28.332371547 +0000
@@ -618,7 +618,7 @@
 		} else if (buffer_is_empty(con->conf.server_tag)) {
 			buffer_append_string_len(out, CONST_STR_LEN(PACKAGE_DESC));
 		} else {
-			buffer_append_string_buffer(out, con->conf.server_tag);
+                        buffer_append_string_encoded(out, CONST_BUF_LEN(con->conf.server_tag), ENCODING_HTML);
 		}

 		buffer_append_string_len(out, CONST_STR_LEN(

Tags: lighttpd, random, work | 2 comments

A while back I mented github-backed DNS hosting.

Turns out NameCast.net does that already, and there is an interesting writeup on the design of something similar, from the same authors in 2009.

Fun to read.

In other news applying for jobs is a painful annoyance.

Should anybody wish to employ an Edinburgh-based system administrator, with a good Debian record, then please do shout at me. Remote work is an option, as is a local office, if you're nearby.

Now I need to go hide from the sun, lest I get burned again...

Good news? Going on holiday to Helsinki in a week or so, for Vappu. Anybody local who wants me should feel free to grab me, via the appropriate channels.

Tags: dns, work | 4 comments

Today I released version 0.25 of my console mail client, which is a release focussed upon portability (DragonFly BSD, and MacOSX specifically).

Over the past couple of weeks I've written a fair bit of code, wondering if I want to make the jump to a graphical email client, but the conclusion for the moment has to be no.

With the scripting support built into my client, and even before then using the hooks/hacks that mutt supported, I just process mail so much more quickly than via a GUI system.

I also benefit from reading the mail on the host to which it is delivered - mail gets filtered by something like procmail, and I read it in-situa. IMAP is available if I travel, but I rarely do so.

Having a GUI client might be fun, but it would mean I'd read mail on my desktop - pretty much the only system I don't backup (except for images, videos, and local media). It would also involve running imapsync, or similar, to pull the mail in, and relaying through the remote server to avoid my ISPs poor IP-reputation.

In short I believe if I use a GUI client I'll get slower, and I'll still need the remote host regardless.

It was this time last year when I thought it was functional, but now it is functional, battle-tested, and reliable.

So I guess I'm done with email for the next few years. Maybe in that time somebody will write something better - console based for preference, GUI as a last resort, and certainly not another webmail client.

In other news ..

I had a fun interview on Monday, it went well until they admitted they couldn't afford me - so their goal is to pay a junior member a small salary and hope to get somebody senior to work part-time for a similarly minimal salary. Might work for somebody else, but it wouldn't for me right now, so on that basis I declined.

The most annoying thing about interviewing is the waiting, between the early flirting about duties and expectations, to scheduling meetings, and then awaiting decisions.

On that note I'm half-way through building a new desk which is a nice physical job I can really concentrate upon. I'm currently waiting for the stain to dry on the legs, and then I'll get the damn thing finished. It probably looks more "rustic" than "modern", but it smells nice, so that's the main thing ;)

Expect pictures when it is finished.

Tags: diy, interviews, lumail, work | No comments

In my previous blog-post I mentioned, briefly, that I'd posted a couple of adverts on Reddit looking for work.

To give more detail I did three things:

• I made a brief blog-post on the Debian-Administration website, highlighting what I thought were interesting/useful/expected skills and experience I have.
• I updated the site to give that link a little prominance, because .. I can.
• I paid Reddit $10 to advertise links to that blog-post. ($5 being the minimum you could spend on any targetted advert.)

The advertisement was set to be shown in /r/edinburgh (where I live), and /r/sysadmin (where I thought some people might look if they were struggling for help).

The advertising on Reddit was painless to setup, and the traffic stats were interesting, but even though this worked out well I'm a little loathe to repeat the process - since the "non-sterling transaction fee" from my bank effectively doubled my budget.

I received a few (private) emails and comments, along with the expected grammar corrections. The end result was that I received contact from an American company founder who seemed interested.

He allowed me to write some code to solve a fun problem, appeared to enjoy the code I sent (Ruby code for dealing with (exim) email spam, that's as specific as I will be). The end result was a three month contract, which we obviously hope will lead to more permanent work.

Anyway I thought this was an atypical route to find a work, and was about a million times nicer than working with recruiters, so .. consider this documentation!

In other news it is now 10pm and I need to go to the gym and pub, in that order.

Tags: work | 2 comments

I'm not sure if I've talked about my job here, but I recently celebrated my one year anniversary - whilst on a company offsite trip to Sweden. When I joined the company there were approximately 100 people employed by it. Nowadays the numbers are much higher.

Having more people around is pretty awesome, but I realized that there were a lot of people wandering around the office who I didn't recognize so it occurred to me to make a game of it.

I had the idea I could write a slack bot to quiz me on my colleagues:

• Show a random face, using the Slack profile picture.
• Give a list of 5 names.
• Ask me which was correct.

I spent an hour messing around with various Slack APIs, and decided the whole thing was too much of a hassle. Instead I wrote a simple script to download the details of all members of the workspace:

• Name.
• Email address.
• Profile picture URL.

Then using that data, users.json, I hacked up a simple web application in Python, using the flask API. There only needed to be two pages:

• A page ("/") to show five random images, each with five random names beneath them.
• A page ("/quiz") to receive the HTTP POST, and score.

All in all this took only two hours or so. Old-school CGI is pretty awesome like that - Hidden values meant the whole thing could be stateless:

 <input type="hidden" name="1answer" value="Bob Smith" ..
 <input type="hidden" name="1profile" value="Sales" ..
 <input type="hidden" name="1url" value="https://.." ..

 <input type="hidden" name="2answer" value="Sally Smith" ..
 <input type="hidden" name="2profile" value="Sales" ..
 <input type="hidden" name="2url" value="https://.." ..

The only downside is that I don't have any authentication, so there is no ability to have a leaderboard. I've looked at the Okta samples and I think it would be easy to add, but I guess that would make it more complex and less portable. That said I'm not sharing the code this time, so who cares if it is tied to the company?

Anyway sometimes I forget how fast and easy it is to spinup a random virtual machine and present a HTTP(S) service for interactive use. This is one of those times when I remembered.

Tags: cgi, python, slack, work | 2 comments