So status displays are cool. Seeing what is happening in real time is cool.
As a proof of concept I put together a trivial load-graph:
This is broken down into three parts:
- Load Client
The load client is a trivial script which reads /proc/loadavg, and sends the 1-minute entry to a remote server, via a single UDP packet.
- Load Server
The load-server is a service which listens for UDP traffic, and when it receives a new integer records that in a redis data-store.
- Load Display
This is a HTML page which has the values from the store in it, which is then plotted using javscript.
So the UDP-server which receives load will receive two things:
- load:N - The load figure. The text "load:" is literal, and present in case I decide to extend the stats..
- x.x.x.x - The IP address from which it received the message.
This is inserted into a Redis database as an array. This array could then be fetched via an AJAX script to update the HTML display in real-time, but at the moment I just have a shell script which updates it in near-real time.
The idea of having a UDP-server receive values from remote clients is interesting. We just need to define a mapping to redis. For me I've just done this:
receive a UDP packet with value "load:1.2" from source 18.104.22.168 append "1.2" to key "22.214.171.124-load". append the value "126.96.36.199" to the global "known_hosts"
The values received can be truncated (i.e. keep only the most recent 60 entries) with ease, due to the available Redis primitives, and we can easily graph these using the qjplot library.
Adding more metrics just means updating the clients to send "memfree:400m", "disk-free:50%", "users:2", "uptime:12345s", or similar. The storage is wonderfully abstract - all you need to do is get the graph-drawing code to a) Know which source to display, and b) which metric.
For example, if we did extend the client to send that data I could draw a graph of the memory on host foo.example.com just by selecting "memfree" against the origin "188.8.131.52".
ObQuote: "Come here, damn you, I want to touch you. " - Hellraiser