Entries tagged cloud

(This post was accidentally made live before it was completed; it is now complet.)

I'll keep this brief and to the point.

syslog indexing and searching

Jason Hedden suggested using swish-e to index and then search syslog files which are stored on disk - rather than inserting the log entries into mysql.

I have 120+ machines writing to a central server, and running a search of 'sshd.*refused' takes less than a second to complete now.

(To be fair using php-syslog-ng was fast, it was just ugly, hard to manage, and the mysql database got overloaded)

Cloud Storage .. but on my machines

I've become increasingly interested in both centralised hosting, and reliable backups.

Cloud storage, where I contrl all the nodes, allows good backups.

So far I've experimented with both mogilefs and peerfuse. Neither setup is entirely appropriate for me, but I love the idea of seamless replication.

ice-creams

Many ice-creams bought in supermarkets come in packs of three. Annoying:

• One for madam x.
• one for me.
• Who gets the spare? (Me, when she's gone ;)

It happens too often to be a coincidence: my cynicism wonders if it is designed to ensure people buy two boxes..?

new software releases

skxlist, the simple mailing list manager, got a couple of new options after user-submitted suggestions.

asql got a bugfix.

My todolist code is now running on at least one other site!

Nothing else much to say mostly because I'm suffering from poor sleep at the moment. In part because I've got a new clock on my bedroom windowsill and the ticking is distracting me (not to mention the on-the-hour chime!)

Still I'm sure it will pass. I grew up in York in a house that had the back yard abutting the local convent. Every hour, on the hour, they'd have bell ring! We moved house when I was about 11, but for months after the move I'd still wake up at midnight confused that the bells hadn't rung...

ObFilm: Mamma Mia!

Tags: cloud, filesystems, logs, random, syslog | 5 comments

Recently two companies have suffed problems due to compromised AWS credentials:

• Code Spaces
  • The company has effectively folded. Thier AWS account was compromised, and all their data and backups were deleted.
• Bonsai
  • Within two minutes all their instances were terminated.
  • This is still live - watch updates of the recovery process.

I'm just about to commit to using Amazon for hosting DNS for paying customers, so this is the kind of thing that makes me paranoid.

I'll be storing DNS-data in Git, and if the zones were nuked on the Amazon-side I could re-upload them, but users would be dead regardless - because they'd need to update the nameservers in whois before the re-uploaded data would be useful.

I suspect I need to upload to two DNS providers, to get more redundency.

Currently I have a working system which allows me to push DNS records to a Git repository, and that seamlessly triggers a DNS update (i.e. A webhook trigged by github/bitbucket/whatever).

Before I publish anything I need to write more code, more documentation, and agree on pricing details. Then I'll setup a landing-page at http://dns-api.com/.

I've been challenged to find paying customers before launching, and thus far have two, which is positive.

The DHCP.io site has now been freed. I'm no longer going to try to commercialize it, instead I will only offer the Git-based product as a commercial service. On that basis I upped the service so users could manage up to five names per account, more if you mail me privately and beg ;)

(ObRandom: Google does hosted DNS with an API. They're expensive. I'm surprised I'd not heard of them doing this.)

Tags: amazon, aws, cloud | No comments