A colleague has pointed me at etcd - which looks like a similar system I'd never heard of.

Maybe I do need to look at go after all. Or learn how zeroconf/ahavi work.

With CARP you might be doing basically the same thing with ARP packets. Knowing that host 'db' is 10.112.0.1, it asks with a broadcast on that subnet, "at which ethernet address is this IP?" It gets a reply from db1, and so uses that.

Of course if your network doesn't allow the floating IP, or your DB nodes are not all on the same subnet, you might have to reinvent this at a higher layer like with avahi.

That's ARP balancing mode. In IP balancing mode, all traffic to 10.112.0.1 is multicast to all DB's. Exactly one node will answer, (hopefully) knowing which other db nodes are down, and standing in for them.

In both cases you have a choice of a designated 'active' DB with others on standby. Or a load-balanced setup, by taking a unique ID (or source address) of the client, modulo the number of active servers.

Interesting answer, and yes you're right that's almost exactly how ARP stuff works.

I'm only considering the case here of a private VLAN, because that means you're not having to cope with malicious local clients.

For the moment I've experimented with reimplementing a small subset of etcd - A node HTTP server that spits out JSON, and can be updated via multicast broadcasts.

• https://github.com/skx/mstore

That solves my immediate problem and looks like it will be an interesting tool for the future - even with the obvious caveats it possesses.

This idea is called 'service discovery', and there's lots of ways to skin this cat. At one end of the spectrum are things like avahi/zeroconf, and at the other are things like zookeeper. It depends on whether you're happy to have anything on the network announcing a service be canonical, or whether you'd prefer to have a gatekeeper in the form of an API driven service-discovery mechanism.

Cheers,

If mstore ran on every client, could it proxy connections/requests through to the 'active' DB node? That way your client merely try to connect to a DB at 'localhost' which has magically become an HA cluster.

It isn't a proxy.

But yes, in the general case you could wrap arbitrary TCP/IP services via something like rineted, so that all clients connected to localhost.

The big downside is, obviously, you'd have to update the configuration of that proxy when the master changed.

I don't see it as a great solution, and it seems tangential to the problem that mstore is solving.

I didn't mean for it to be configured statically. I meant, instead of mstore keeping track of the active node, and just telling clients where to find it; mstore could proxy TCP connections there directly, or even better, individual DB queries (if it understands the protocol) through to the db node it knows is active at that exact moment, during a single uninterrupted connection from the client. The client should need no modifications that way, not knowing what's happening behind the scenes.

And just to widen the scope even more; why aren't DB queries themselves multicast, where one of the cluster of DBs nodes chooses to answer, and/or replicate to the others...

I understood your point; I just was using "get the database hostname" as shorthand for querying many different types of data.

Yes, having a clever auto-updating proxy for MySQL would be useful, but that isn't the problem I'm trying to solve here :)

PS: See-also mysql-proxy, and spockproxy, and etc.