About Archive Tags RSS Feed


Some things on DNS and caching

29 March 2014 21:50

Although there wasn't too many comments on my what would you pay for? post I did get some mails.

I was reminded about this via Mario Langs post, which echoed a couple of private mails I received.

Despite being something that I take for granted, perhaps because my hosting comes from the Bytemark, people do seem willing to pay money for DNS hosting.

Which is odd. I mean you could do it very very very cheaply if you had just four virtual machines. You can get complex and be geo-fancy, and you could use anycast on a small AS, but really? You could just deploy four virtual machines0 to provide a.ns, b.ns, c.ns, d.ns, and be better than 90% of DNS hosters out there.

The thing that many people mentioned was Git-backed, or Git-based DNS. Which would be trivial if you used tinydns, and no much harder if you used bind.

I suspect I'm "not allowed" to do DNS-things for a while, due to my contract with Dyn, but it might be worth checking...

ObRandom: Beat me to it. Register gitdns.io, or similar, and configure hooks from github to compile tinydns records.

In other news I started documenting early thoughts about my caching reverse proxy, which has now got a name stockpile.

I wrote some stub code using node.js, and although it was functional it soon became callback hell:

  • Is this resource cachable?
  • Does this thing exist in the cache already?
  • Should we return the server's response to the client, archive to memcached, or do both?

Expressing the rules neatly is also a challenge. I want the server core to be simple and the configuration to be something like:

is_cachable ( vhost, source, request, backened )
     * If the file is static, then it is cachable.
    if ( request.url.match( /\.(jpg|png|txt|html?|gif)$/i ) ) {
        return true;

     * If there is a cookie then the answer is false.
    if ( request.has_cookie? ) { return false ; }

     * If the server is alive we'll now pass the remainder through it
     * if not then we'll serve from the cache.
    if ( backend.alive? ) {
        return false;
    else {
        return true;

I can see there is value in judging the cachability based on the server response, but I plan to ignore that except for "Expires:", "Etag", etc ,etc)

Anyway callback hell does make me want to reexamine the existing C/C++ libraries out there. Because I think I could do better.



Comments on this entry

icon Steve Kemp at 19:27 on 29 March 2014

I realize I'm simplifying a little, as you have to cope with glue records, and some TLDs have weird requirements.

(I think it is .it who require your nameservers respond on TCP, rather than UDP.)

The biggest potential issue is that changing the IPs of nameservers is an annoyance, so you want long-term-static IPs, and ideally you want IPv6 too.

Still of all the "Run XXX as a service" DNS has to be the simplest...

icon Paul Tansom at 15:21 on 31 March 2014

I know what you mean about paying for DNS. Many domain registration agents offer the service for free (although some are pretty poor), and there are some very good free services around - I'm currently migrating, likely temporarily, to Hurricane Electric which have a nice setup and will even show you the zone file which is great for migrating to your own setup at some point.

I'd almost forgotten Dyn were still going. They bought up the service I was using a while back and, iirc, forced everyone across to a paid service. They also kicked me off the free dynamic DNS service I was using for my dad's connection because I hadn't logged into the web site - clearly a policy for jettisoning anyone not paying. The main pain there is that most consumer ADSL routers can be configured for Dyn, but aren't flexible enough to use other services.

On a more positive note, for a hosted service for only a few domains Hurricane Electric have a nice setup with good IPv6 support, proper SPF records and reverse DNS - setting up a reverse IPv6 zone was what prompted the move so I could get Google to accept mail from me over IPv6.

icon Pete Foster at 18:52 on 29 March 2014

I realised the exact same as you about DNS.

I have 4 VMs (I think a couple may actually be containers) from 4 separate providers in 4 separate locations. They cost peanuts. The only time I have had problems where DNS stopped working altogether is when it was a PEBKAC problem.