22 January 2022 10:00
So in my previous post I mentioned that we were going to spend the Christmas period in the UK, which we did.
We spent a couple of days there, meeting my parents, and family. We also persuaded my sister to drive us to Scarborough so that we could hang out on the beach for an afternoon.
Finland has lots of lakes, but it doesn't have proper waves. So it was surprisingly good just to wade in the sea and see waves! Unfortunately our child was a wee bit too scared to ride on a donkey!
Unfortunately upon our return to Finland we all tested positive for COVID-19, me first, then the child, and about three days later my wife. We had negative tests in advance of our flights home, so we figure that either the tests were broken, or we were infected in the airplane/airport.
Thankfully things weren't too bad, we stayed indoors for the appropriate length of time, and a combination of a couple of neighbours and online shopping meant we didn't run out of food.
Since I've been back home I've been automating AWS activities with aws-utils, and updating my simple host-automation system, marionette.
Marionette is something that was inspired by puppet, the configuration management utility, but it runs upon localhost only. Despite the small number of integrated primitives it actually works surprisingly well, and although I don't expect it will ever become popular it was an interesting research project.
The aws-utilities? They were specifically put together because I've worked in a few places where infrastructure is setup with terraform, or cloudformation, but there are always the odd thing that is configured manually. Typically we'll have an openvpn gateway which uses a manually maintained IP allow-list, or some admin-server which has a security-group maintained somewhat manually.
Having the ability to update a bunch of rules with your external IP, as a single command, across a number of AWS accounts/roles, and a number of security-groups is an enormous time-saver when your home IP changes.
I'd quite like to add more things to that collection, but there's no particular rush.
Tags: covid, travel, uk
2 December 2021 17:00
I realize it has been quite some time since I last made a blog-post, so I guess the short version is "I'm still alive", or as Granny Weatherwax would have said:
I ATE'NT DEAD
Of course if I die now this would be an awkward post!
I can't think of anything terribly interesting I've been doing recently, mostly being settled in my new flat and tinkering away with things. The latest "new" code was something for controlling mpd via a web-browser:
This is a simple HTTP server which allows you to minimally control
mpd running on
localhost:6600. (By minimally I mean literally "stop", "play", "next track", and "previous track").
I have all my music stored on my desktop, I use
mpd to play it locally through a pair of speakers plugged into that computer. Sometimes I want music in the sauna, or in the bedroom. So I have a couple of bluetooth speakers which are used to send the output to another room. When I want to skip tracks I just open the
mpd-web site on my phone and tap the button. (I did look at android mpd-clients, but at the same time it seemed like installing an application for this was a bit overkill).
I guess I've not been doing so much "computer stuff" outside work for a year or so. I guess lack of time, lack of enthusiasm/motivation.
So looking forward to things? I'll be in the UK for a while over Christmas, barring surprises. That should be nice as I'll get to see family, take our child to visit his grandparents (on his birthday no less) and enjoy playing the "How many Finnish people can I spot in the UK?" game
Tags: golang, mpd, mpd-web, not dead
4 May 2021 18:00
Like many I use pass for storing usernames and passwords. This gives me easy access to credentials in a secure manner.
I don't like the way that the metadata (i.e. filenames) are public, but that aside it is a robust tool I've been using for several years.
The last time I talked about
pass was when I talked about showing the age of my credentials, via the integrated
That then became a pass-plugin:
frodo ~ $ pass age
6 years ago GPGemail@example.com
6 years ago GPGfirstname.lastname@example.org.OLD.gpg
4 years, 8 months ago Domains/Domain.fi.gpg
4 years, 7 months ago Mobile/dna.fi.gpg
1 year, 3 months ago Websites/netlify.com.gpg
1 year ago Financial/ukko.fi.gpg
1 year ago Mobile/KiK.gpg
4 days ago Enfuce/sre.tst.gpg
Anyway today's work involved writing another plugin, named
env. I store my data in pass in a consistent form, each entry looks like this:
# Extra data
The keys vary, sometimes I use "login", sometimes "username", other times "email", but I always label the fields in some way.
Recently I was working with some CLI tooling that wants to have a username/password specified and I patched it to read from the environment instead. Now I can run this:
$ pass env internal/cli/tool-name
That's ideal, because now I can source that from within a shell:
$ source <(pass env internal/cli/tool-name)
$ echo username
Or I could directly execute the tool I want:
$ pass env --exec=$HOME/ldap/ldap.py internal/cli/tool-name
you are steve
TLDR: If you store your password entries in "
key: value" form you can process them to export
$KEY=$value, and that allows them to be used without copying and pasting into command-line arguments (e.g. "
~/ldap/ldap.py --username=steve --password=secrit")
Tags: git, pass, passwordstore, shell
26 April 2021 18:00
In my previous post I wrote about how I'd been running CP/M on a Z80-based single-board computer.
I've been slowly working my way through a bunch of text-based adventure games:
- The Hitchhiker's Guide To The Galaxy
- Zork 1
- Zork 2
- Zork 3
Along the way I remembered how much fun I used to have doing this in my early teens, and decided to write my own text-based adventure.
Since I'm not a masochist I figured I'd write something with only three or four locations, and solicited facebook for ideas. Shortly afterwards a "plot" was created and I started work.
I figured that the very last thing I wanted to be doing was to be parsing text-input with Z80 assembly language, so I hacked up a simple adventure game in C. I figured if I could get the design right that would ease the eventual port to assembly.
I had the realization pretty early that using a table-driven approach would be the best way - using structures to contain the name, description, and function-pointers appropriate to each object for example. In my C implementation I have things that look like this:
desc: "A small generator.",
A bit noisy, but simple enough. If an object cannot be picked up, or dropped, the corresponding entries are blank:
edesc: "The desk looks solid, but old."},
Here we see something that is special, there's no description so the item isn't displayed when you enter a room, or LOOK. Instead the
edesc (extended description) is available when you type
Anyway over a couple of days I hacked up the C-game, then I started work porting it to Z80 assembly. The implementation changed, the easter-eggs were different, but on the whole the two things are the same.
Certainly 99% of the text was recycled across the two implementations.
Anyway in the unlikely event you've got a craving for a text-based adventure game I present to you:
Tags: cpm, retro, single-board computer, z80
17 April 2021 14:00
In the past, I've talked about building a Z80-based computer. I made some progress towards that goal, in the sense that I took the initial (trivial steps) towards making something:
- I built a clock-circuit.
- I wired up a Z80 processor to the clock.
- I got the thing running an endless stream of NOP instructions.
- No RAM/ROM connected, tying all the bus-lines low, meaning every attempted memory-read returned
0x00 which is the Z80 NOP instruction.
But then I stalled, repeatedly, at designing an interface to RAM and ROM, so that it could actually do something useful. Over the lockdown I've been in two minds about getting sucked back down the rabbit-hole, so I compromised. I did a bit of searching on tindie, and similar places, and figured I'd buy a Z80-based single board computer. My requirements were minimal:
- It must run CP/M.
- The source-code to "everything" must be available.
- I want it to run standalone, and connect to a host via a serial-port.
With those goals there were a bunch of boards to choose from, rc2014 is the standard choice - a well engineered system which uses a common backplane and lets you build mini-boards to add functionality. So first you build the CPU-card, then the RAM card, then the flash-disk card, etc. Over-engineered in one sense, extensible in another. (There are some single-board variants to cut down on soldering overhead, at a cost of less flexibility.)
After a while I came across https://8bitstack.co.uk/, which describes a simple board called the the Z80 playground.
The advantage of this design is that it loads code from a USB stick, making it easy to transfer files to/from it, without the need for a compact flash card, or similar. The downside is that the system has only 64K RAM, meaning it cannot run CP/M 3, only 2.2. (CP/M 3.x requires more RAM, and a banking/paging system setup to swap between pages.)
When the system boots it loads code from an EEPROM, which then fetches the CP/M files from the USB-stick, copies them into RAM and executes them. The memory map can be split so you either have ROM & RAM, or you have just RAM (after the boot the ROM will be switched off). To change the initial stuff you need to reprogram the EEPROM, after that it's just a matter of adding binaries to the stick or transferring them over the serial port.
In only a couple of hours I got the basic stuff working as well as I needed:
- A z80-assembler on my Linux desktop to build simple binaries.
- An installation of Turbo Pascal 3.00A on the system itself.
- An installation of FORTH on the system itself.
- A couple of simple games compiled from Pascal
- The Zork trilogy installed, along with Hitchhikers guide.
I had some fun with a CP/M emulator to get my hand back in things before the board arrived, and using that I tested my first "real" assembly language program (
cls to clear the screen), as well as got the hang of using the wordstar keyboard shortcuts as used within the turbo pascal environment.
I have some plans for development:
- Add command-line history (page-up/page-down) for the CP/M command-processor.
- Add paging to
TYPE, and allow terminating with
Nothing major, but fun changes that won't be too difficult to implement.
Since CP/M 2.x has no concept of sub-directories you end up using drives for everything, I implemented a "search-path" so that when you type "FOO" it will attempt to run "A:FOO.COM" if there is no file matching on the current-drive. That's a nicer user-experience at all.
I also wrote some Z80-assembly code to search all drives for an executable, if not found in current drive and not already qualified. Remember CP/M doesn't have a concept of sub-directories) that's actually pretty useful:
I've also written some other trivial assembly language tools, which was surprisingly relaxing. Especially once I got back into the zen mode of optimizing for size.
I forked the upstream repository, mostly to tidy up the contents, rather than because I want to go into my own direction. I'll keep the contents in sync, because there's no point splitting a community even further - I guess there are fewer than 100 of these boards in the wild, probably far far fewer!
Tags: cpm, retro, single-board computer, z80
5 January 2021 13:00
Nothing too much has happened recently, largely as a result of the pandemic killing a lot of daily interests and habits.
However as a result of Brexit I'm having to do some paperwork, apparently I now need to register for permanent residency under the terms of the withdrawal agreement, and that will supersede the permanent residency I previously obtained.
Of course as a UK citizen I've now lost the previously-available freedom of movement. I can continue to reside here in Helsinki, Finland, indefinitely, but I cannot now move to any other random EU country.
It has crossed my mind, more than a few times, that I should attempt to achieve Finnish citizenship. As a legal resident of Finland the process is pretty simple, I just need two things:
- Prove I've lived here for the requisite number of years.
- Pass a language test.
Of course the latter requirement is hard, I can understand a lot of
spoken and written Finnish, but writing myself, and speaking a lot is
currently beyond me. I need to sit down and make the required effort to
increase my fluency. There is the alternative option of learning Swedish, which is a hack a lot of immigrants use:
- Learning Swedish is significantly easier for a native English-speaker.
- But the downside is that it would be learning a language solely to
"cheat" the test, it wouldn't actually be useful in my daily life.
Finland has two official languages, and so the banks, the medical world,
the tax-office, etc, are obliged to provide service in both. However
daily life, ordering food at restaurants, talking to parents in the
local neighborhood? Finnish, or English are the only real options. So
if I went this route I'd end up in a weird situation where I had to
learn a language to pass a test, but then would continue to need to
learn more Finnish to live my life. That seems crazy, unless I were
desperate for a second citizenship which I don't think I am.
Learning Finnish has not yet been a priority, largely because I work in
English in the IT-world, and of course when I first moved here I was
working (remotely) for a UK company, and didn't have the time to attend
lessons (because they were scheduled during daytime, on the basis that
many immigrants are unemployed). Later we had a child, which meant that
early-evening classes weren't a realistic option either.
(Of course I learned a lot of the obvious things immediately upon
moving, things like numbers, names for food, days of the week were
essential. Without those I couldn't have bought stuff in shops and
would have starved!)
On the topic of languages a lot of people talk about how easy it is for
children to pick up new languages, and while that is broadly true it is
also worth remembering just how many years of correction and repetition they
have to endure as part of the process.
For example we have a child, as noted already, he is spoken to by
everybody in Finnish. I speak to him in English, and he hears his
mother and myself speaking English. But basically he's 100% Finnish
with the exception of:
- Me, speaking English to him.
- His mother and I speaking English in his hearing.
- Watching Paw Patrol.
If he speaks Finnish to me I pretend to not understand him, even when I
do, just for consistency. As a result of that I've heard him tell
strangers "Daddy doesn't speak Finnish" (in Finnish) when we've been
stopped and asked for directions. He also translates what some other
children have said into English for my benefit which is adorable
Anyway he's four, and he's pretty amazing at speaking to everybody in the
correct language - he's outgrown the phase where he'd mix different
languages in the same sentence ("more leipä", "saisinko milk") - when I
took him to the UK he surprised and impressed me by being able to
understand a lot of the heavy/thick accents he'd never heard before.
(I'll still need to train him on Rab C.
Nesbitt when he's a wee
bit older, but so far no worries.)
So children learn languages, easily and happily? Yes and no. I've
spent nearly two years correcting his English and he still makes the
same mistake with gender. It's not a big deal, at all, but it's a
reminder that while children learn this stuff, they still don't do it as
easily as people imagine. I'm trying to learn and if I'd been corrected
for two years over the same basic point you'd rightly think I was
"slow", but actually that's just how it works. Learning languages
requires a hell of a lot of practice, a lot of effort, and a lot of
Specifically Finnish doesn't have gendered pronouns, the same word is
used for "he" and "she". This leads to a lot of Finnish people, adults
and children, getting the pronouns wrong in English. In the case of our
child he'll say "Mommy is sleeping, when he wake up?" In the case of
adults I've heard people say "My girlfriend is a doctor, he works in a
hospital", or "My dad is an accountant, she works for a big firm". As I
say I've spent around two years making this correction to the child, and
he's still nowhere near getting it right. Kinda adorable actually:
- "Mommy is a woman we say "when she wakes up"..."
- "Adriana is a girl we say "her bike".."
Tags: bilingual, brexit, childcare, english, finland, finnish, languages, oiva
1 November 2020 13:00
Back in 2017 I announced that the https://Debian-Administration.org website was being made read-only, and archived.
At the time I wrote a quick update to save each requested page as a flat-file, hashed beneath
/tmp, with the expectation that after a few months I'd have a complete HTML-only archive of the site which I could serve as a static-website, instead of keeping the database and pile of CGI scripts running.
Unfortunately I never got round to archiving the pages in a git-repository, or some other store, and I usually only remembered this local tree of content was available a few minutes after I'd rebooted the server and lost the stuff - as the reboot would reap the contents of
Thinking about it today I figured I probably didn't even need to do that, instead I just need to redirect to the wayback machine. Working on the assumption that the site has been around for "a while" it should have all the pages mirrored by now I've made a "final update" to Apache:
RewriteRule ^/(.*) "http://web.archive.org/web/https://debian-administration.org/$1" [R,L]
Assuming nobody reports a problem in the next month I'll retire the server and make a simple docker container to handle the appropriate TLS certificate renewal, and hardwire the redirection(s) for the sites involved.
Tags: apache, cgi, debian-administration, wayback-machine
20 October 2020 13:00
For the past few years I've had a bunch of virtual machines hosting websites, services, and servers. Of course I want them to be available - especially since I charge people money to access at some of them (for example my dns-hosting service) - and that means I want to know when they're not.
The way I've gone about this is to have a bunch of machines running stuff, and then dedicate an entirely separate machine solely for monitoring and alerting. Sure you can run local monitoring, testing that services are available, the root-disk isn't full, and that kind of thing. But only by testing externally can you see if the machine is actually available to end-users, customers, or friends.
A local-agent might decide "I'm fine", but if the hosting-company goes dark due to a fibre cut you're screwed.
I've been hosting my services with Hetzner (cloud) recently, and their service is generally pretty good. Unfortunately I've started to see an increasing number of false-alarms. I'd have a server in Germany, with the monitoring machine in Helsinki (coincidentally where I live!). For the past month I've started to get pinged with a failure every three/four days on average, "service down - dns failed", or "service down - timeout". When the notice would wake me up I'd go check and it would be fine, it was a very transient failure.
To be honest the reason for this is my monitoring is just too damn aggressive, I like to be alerted immediately in case something is wrong. That means if a single test fails I get an alert, as rather than only if a test failed for something more reasonable like three+ consecutive failures.
I'm experimenting with monitoring in a less aggressive fashion, from my home desktop. Since my monitoring tool is a single self-contained golang binary, and it is already packaged as a docker-based container deployment was trivial. I did a little work writing an agent to receive failure-notices, and ping me via telegram - instead of the previous approach where I had an online status-page which I could view via my mobile, and alerts via pushover.
So far it looks good. I've tweaked the monitoring to setup a timeout of 15 seconds, instead of 5, and I've configured it to only alert me if there is an outage which lasts for >= 2 consecutive failures. I guess the TLDR is I now do offsite monitoring .. from my house, rather than from a different region.
The only real reason to write this post was mostly to say that the process of writing a trivial "notify me" gateway to interface with telegram was nice and straightforward, and to remind myself that transient failures are way more common than we expect.
I'll leave things alone for a moment, but it was a fun experiment. I'll keep the two systems in parallel for a while, but I guess I can already predict the outcome:
- The desktop monitoring will report transient outages now and again, because home broadband isn't 100% available.
- The heztner-based monitoring, in a different region, will report transient problems, because even hosting companies are not 100% available.
- Especially at the cheap prices I'm paying.
- The way to avoid being woken up by transient outages/errors is to be less agressive.
- I think my paying users will be OK if I find out a services is offline after 5 minutes, rather than after 30 seconds.
- If they're not we'll have to talk about budgets ..
Tags: go, monitoring, overseer, telegram
3 October 2020 13:00
Recently I've been writing a couple of simple compilers, which take input in a particular format and generate assembly language output. This output can then be piped through
gcc to generate a native executable.
Public examples include this trivial math compiler and my brainfuck compiler.
Of course there's always the nagging thought that relying upon
nasm) is a bit of a cheat. So I wondered how hard is it to write an assembler? Something that would take assembly-language program and generate a native (ELF) binary?
And the answer is "It isn't hard, it is just tedious".
I found some code to generate an ELF binary, and after that assembling simple instructions was pretty simple. I remember from my assembly-language days that the encoding of instructions can be pretty much handled by tables, but I've not yet gone into that.
(Specifically there are instructions like "
add rax, rcx", and the encoding specifies the source/destination registers - with different forms for various sized immediates.)
Anyway I hacked up a simple assembler, it can compile
a.out from this input:
.hello DB "Hello, world\n"
.goodbye DB "Goodbye, world\n"
mov rdx, 13 ;; write this many characters
mov rcx, hello ;; starting at the string
mov rbx, 1 ;; output is STDOUT
mov rax, 4 ;; sys_write
int 0x80 ;; syscall
mov rdx, 15 ;; write this many characters
mov rcx, goodbye ;; starting at the string
mov rax, 4 ;; sys_write
mov rbx, 1 ;; output is STDOUT
int 0x80 ;; syscall
xor rbx, rbx ;; exit-code is 0
xor rax, rax ;; syscall will be 1 - so set to xero, then increase
inc rax ;;
int 0x80 ;; syscall
The obvious omission is support for "JMP", "JMP_NZ", etc. That's painful because jumps are encoded with relative offsets. For the moment if you want to jump:
push foo ; "jmp foo" - indirectly.
nop ; Nothing happens
mov rbx,33 ; first syscall argument: exit code
mov rax,1 ; system call number (sys_exit)
int 0x80 ; call kernel
push bar ; "jmp bar" - indirectly.
I'll update to add some more instructions, and see if I can use it to handle the output I generate from a couple of other tools. If so that's a win, if not then it was a fun learning experience:
Tags: asm, assembly, github, go, golang
22 September 2020 13:00
So my previous post was all about implementing a simple FORTH-like language. Of course the obvious question is then "What do you do with it"?
So I present one possible use - turtle-graphics:
\ Draw a square of the given length/width
dup dup dup dup
4 0 do
\ pen down
\ move to the given pixel
100 100 move
\ draw a square of width 50 pixels
\ save the result (png + gif)
Tags: forth, github, go, golang, turtle