Entries tagged lumail

A massive marathon has resulted in my lumail mail client working well.

Functionally the application looks little different to the previous C-client, but it is a lot cleaner, neater, and nicer internally.

The configuration file luamail.lua gives a good flavour of the code, and the github repository has brief instructions.

Initially I decied that the navigation/index stuff was easy and the rest of the program would be hard; dealing with GPG-signatures, MIME-parts, etc.

But I'm stubborn enough to keep going.

If I can get as far as reading messages, with MIME handled properly, and replying then I can switch to using it immediately which will spur further development.

I'm really pleased with the keybinding code, and implementing the built-in REPL-like prompt was a real revelation. Worht it for that alone.

The domain name lumail.org was available. So I figured why not?

Tags: lumail | 3 comments

Lumail <http://lumail.org> received two patches today, one to build on Debian Unstable, and one to build on OpenBSD.

The documentation of the lua primitives is almost 100% complete, and the repository has now got a public list of issues which I'm slowly working on.

Even though I can't reply to messages I'm cheerfully running it on my mail box as a mail-viewer. Faster than mutt. Oddly enough. Or maybe I'm just biased.

Tags: github, lumail | No comments

Today my main machine was down for about 8 hours. Oops.

That meant when I got home, after a long and dull train journey, I received a bunch of mails from various hosts each saying:

• Failed to fetch slaughter policies from rsync://www.steve.org.uk/slaughter

Slaughter is my sysadmin utility which pulls policies/recipies from a central location and applies them to the local host.

Slaughter has a bunch of different transports, which are the means by which policies and files are transferred from the remote "central host" to the local machine. Since git is supported I've now switched my policies to be fetched from the master github repository.

This means:

• All my servers need git installed. Which was already the case.
• I can run one less service on my main box.
• We now have a contest: Is my box more reliable than github?

In other news I've fettled with lumail a bit this week, but I'm basically doing nothing until I've pondered my way out of the hole I've dug myself into.

Like mutt lumail has the notion of "limiting" the display of things:

• Show all maildirs.
• Show all maildirs with new mail in them.
• Show all maildirs that match a pattern.
• Show all messages in the currently selected folder(s)
  • More than one folder may be selected :)
• Shall all unread messages in the currently selected folder(s).

Unfortunately the latter has caused an annoying, and anticipated, failure case. If you open a folder and cause it to only show unread messages all looks good. Until you read a message. At which point it is no longer allowed to be displayed, so it disappears. Since you were reading a message the next one is opened instead. WHich then becomes marked as read, and no longer should be displayed, because we've said "show me new/unread-only messages please".

The net result is if you show only unread messages and make the mistake of reading one .. you quickly cycle through reading all of them, and are left with an empty display. As each message in turn is opened, read, and marked as non-new.

There are solutions, one of which I documented on the issue. But this has a bad side-effect that message navigation is suddenly complicated in ways that are annoying.

For the moment I'm mulling the problem over and I will only make trivial cleanup changes until I've got my head back in the game and a good solution that won't cause me more pain.

Tags: github, lumail, slaughter, sysadmin | 6 comments

Although I've still not got the ability to reply to messages, and composing new ones is ugly, my toy mail client is working nicely.

I've received a couple of patches, and given commit access to the repository to one other user.

Currently I'm still juggling primitives around and working out what is missing. The big exceptions are the obvious:

• Cannot reply to a message.
• Cannot move a message to a new folder.
• When composing a mail to be sent no copy is saved in "sent-mail", or similar.
• Thread-view is absent. Indefinitely.

But on the plus side the lua scripting is lovely:

precious ~/git/lumail $ rm /tmp/unread.log
precious ~/git/lumail $ ./lumail  --rcfile ./lumail.lua --eval "dump_unread();"
precious ~/git/lumail $ head /tmp/unread.log
Selected folder /home/skx/Maildir/.Automated.backups
	Folder has 10 unread messages
Selected folder /home/skx/Maildir/.Automated.bounces
	Folder has 3 unread messages
Selected folder /home/skx/Maildir/.CRM.Spam
	Folder has 7 unread messages
Selected folder /home/skx/Maildir/.facebook.com
	Folder has 4 unread messages
..

The website needs some love, most notably a logo. And there are several reported bugs/todo-items I need to work through.

Still for a toy program I'm using it daily. (Though still using mutt to reply to messages & view/save attachments.)

Tags: lumail | 2 comments

Give me a few days and I'll stop writing about Lumail, but tomorrow I intend to make the first stable source & binary release. From that point onwards you'll not need to track the github repository to getu pdates.

The website has had an overhaul in advance of the release, but could still benefit from a logo. As usual I've written the website using my templer static-site generator. I hacked up a couple of plugins to make it easy to generate the pages of Lua primitive documentation, and handle cross-links suitably. (The source is available for reference.)

The first binary release of the mail-client is obviously something of a big deal. I've been using the client daily for the past week or so, as a read-only mail-viewer. But now that the compose() and reply() primitives are present it is usable "for real". Having real scripting present is also allowing me to do interesting things, which are kinda/sorta demonstrated on the examples page.

Now its a case of fixing up a couple of display-related glitches, and implementing things that are both missing and desirable. Happily the list of missing things is actually surprisingly small.

I think the biggest outstanding issue is that the defaults are Steve-friendly, for example the colour-setup should probably be configurable.

Beyond the personal-defaults I think the next biggest issue is the lack of threading support. Messages are displayed in oldest->newest order. Always.

The other omission is that it is impossible to tag-things, in the mutt-sense, so you can't reply to two messages at once. That's a design decision I might have to revisit. The balance of course is that you can open multiple folders at once, and that rocks!

Happy days.

Tags: lumail | 4 comments

When people ask me why I chose to embed Lua in my mail-client I'll point to my on_idle() documentation.

Moving from a callback which runs once every second, or so, to allowing the user to schedule tasks on arbitrary boundaries is pretty cool - and obviously requires no explicit support from myself.

Now I've fixed a couple of bugs which went unspotted/unreported in the first release I'm ready for a new one "soon".

In the meantime I'm running the client exclusively, and loving the ability to view all unread mail, only, regardless of the parent folder.

Tags: lua, lumail | No comments

As part of writing a new mail client I'm wondering about how to change my email-life, and how other people process/handle their incoming email.

I sort my incoming email into folders at delivery-time using procmail. Mail is generally filtered into mailboxes on the basis of the company that sent it, the person that sent it, or the machine which generated it.

Because I manage a lot of machines personally I've split things up so that I have a folder per host. So on a morning I might have unread mail in the following folders:

machines.steve.org.uk/
machines.da-db1/
machines.da-db2/
machines.da-web1/
machines.da-web2/
machines.da-web3/
machines.da-web4/

The per-machine mailboxes usually contain a single mail every day from LogWatch, along with output from any cron-jobs. For example today I received the mail:

From: Cron Daemon
To: [email protected]
Subject: Cron [email protected] /home/steve/bin/download-check

URL http://nodejs.org/ - no longer matches v0.10.9

Generally speaking I don't need to read the per-machine messages. I'll keep the most recent 100 for reference, but only need to look if something seems "off" on a machine. But if I don't look I'd not see the node upgrade notice, so find that I do read them after all.

This suggest to me that email isn't the right way to handle this kind of thing. Instead I should use a notification system - at work we have a central service called MauveAlert (yes, Red Dwarf reference). Mauve receives "alerts" of various kinds, via UDP. The alerts are then fanned out to appropriate people via XMPP, Email, or SMSs.

I have a similarly-inspired system I use on my Debian Administration cluster. A (node) service runs non-stop collecting UDP messages and showing them on a dashboard. I look at it throughout the day to see when slaughter runs, etc.

Anyway in conslusion I get a lot of mail. Some of it is related to random projects, and all ends up in the steve.org.uk/ mailbox, some of it relates to machines, and gets filed away, and I have regular conversions with folk so I have a .people.kirsi/ folder which receives a lot of attention, for example.

ObRandom:daily() - Mark ~/Maildir/.machines.*, etc, read.

Tags: email, lumail, random | 8 comments

Recently I wanted to cleanup my "main" remote machine. It is a system I've had for many years, which started off as a i386 KVM-guest and was later migrated in place to an AMD64 installation.

These days the host runs:

• Mail for many domains, via QPSMTPD and ms-lite.
• Website hosting for many domains. Each site running under a dedicated per-UID thttpd instance, behind a node reverse proxy.
• IRC for kirsi.

The plan was originally to move the "mail stuff" to a new (wheezy) guest. I aborted that after discovering that the mutt-patched package has (IMHO) regressed.

So today I spun up a new virtual machine, and configured it to host websites.

Thus far I've migrated steve.org.uk, and lumail.org to the new host. Both simple sites that are built via my static-site-generator. Migration mostly involved configuring the proxy and the thttpd instances - then using rsync to migrate the content.

I've renamed my current host, which was previously www.steve.org.uk and is now ssh.steve.org.uk, and pushed DNS changes.

If all is smooth and happy I'll slowly migrate the rest of the sites. Fingers crossed this will be painless and I'll have a clean split between "login + mail" and "websites".

Tags: lumail, node-reverse-proxy, thttpd | 3 comments

Running a pristine operating system is fun. I keep going to run programs and finding they're not installed!

For example I'd planned to re-deploy the Debian Administration code-base this evening, but couldn't because fabric is not contained in Wheezy (#714421).

Thankfully backporting the package was trivial, but it was a minor stumbling block. I've hit a few of those recently.

Still in happier new my lumail mail client can now handle both adding attachments to outgoing mails and extracting them from incoming mails.

There are missing features which some people might expect and rely upon, such as:

• Threading support.
• GPG support.

But that said the scripting primitives allow interesting things to be done and I'm enjoying the experience of writing something so "major".

True I've got ~20 "stars" on github which isn't a great sign of popularity, but I have had some fun feedback and the client works for me.

I'm going to have to spend a few days working on TAB-completion code that plays-nice with curses, because that's a major irritation (and you can't mix/match curses & readline, annoyingly). But otherwise I think we're getting close to being complete enough I'll slow down.

Tags: fabric, lumail, wheezy | 5 comments

Having migrated my websites away from my ssh/mail box I'm going to upgrade that this weekend.

I've got my new mail client, lumail, working well enough to use exclusively, so the upgrade should be nice and simple.

I spent a few hours last night removing packages from my ssh/mail box to trim it down. Removing a bunch of Perl modules I used in my CGI coding, removing services such as nfs-common, portmapper, etc.

Today I'll have a stab at the upgrade. The only thing I have to be careful of is my backported/tweaked qpsmptd packages. I'll try the native wheezy version now it has caught up.

(I don't use any of the standard qpsmtpd plugins at all. Instead I have a separate tree of my own custom anti-spam and virtual-hosting aware plugins. They all work in a unified fashion. Using these plugins against a new version of qpsmtpd should be just fine. But obviously I need to test that.)

Work on Lumail is probably going to slow down now it is genuinely in use, but I'll keep an eye out for feature requests and missing primitives. Annoyingly I wasted 30 minutes just now implementing a plugin I'd already written: lumail issue #51.

I also need to step-back this weekend and reassess my hosting. When I was tweaking my slaughter setup I recently realized I have more hosts than I thought:

• Cluster running for Debian-Administration.org:
  • 4 x web nodes
  • 2 x database nodes
  • 2 x "planet" nodes
  • 1 x "misc" node.
• Personal stuff
  • 1 x ssh/mail host. (ssh.steve.org.uk/mail.steve.org.uk)
  • 1 x web host. (www.steve.org.uk, www.lumail.org, etc, etc.)
  • 1 x blogspam.net server (www.blogspam.net)
  • 1 x builder node - Runs buildd for producing my packages.

Total: 13 virtual machines. (+ one kvm-host)

Tags: lumail, wheezy | 2 comments

So, writing an email client, how did that turn out? Pretty damn well as it happens.

My views continue to range from "email is easy" to "email is hard". But I'm now using my home-grown mail-client for 100% of my personal mail-handling.

Writing a mail client did seem a little crazy when I started, but the problem breaks down into a few distinct steps and individually they're not so hard:

• Display a list of folders.
• Display a list of messages.
• Display a single message.

Once you've got that you're almost 80% complete. You're just missing things like "delete", "reply", "compose", and those things are pretty simple to implement.

I definitely made the right call in making it scriptable with Lua, because I've been able to write so many functions for working with mail. For example marking messages in all folders matching a regular expression.

The code is pretty well structured, and now I've got TAB-completion support on all primitives and user-additions I'm finding it a lot nicer to use.

The future? I'm not sure. But right now I'm very happy.

Tags: lumail | 15 comments

This week I made two releases of my mail client. Immediately after both releases I found bugs. Despite having been using the github source tree on my box for reading mail for days.

There must be a name for bugs that come up immediately after you've just made a release.

I'm torn between wanting to make a new release right now to fix the thing I spotted, or wait a few more days to fix a few other niggles.

Still I did write some cool code today:

1. If a mail is received on the list debian.security-announce
2. And the package in the Subject: is not installed on the current machine.
3. The mail is marked as read.

Sure this means that a package on my webserver won't be visible to me, but my upgrade tool will see that. It just decreases the odds I read about an update that doesn't apply to me.

ObQuote: "don't pay heed to temptation
for his hands are so cold"

Tags: bugs, lumail | 4 comments

Today I've made the 0.15 release of lumail, which has several fixups and cleanups.

The previous release included a rewrite of the scrolling code, courtesy of kain88-de. This release fixes a few corner cases in that update which caused empty messages/Maildirs to be highlighted - operating on such ghost-entries would cause a segfault. Oops.

I've received several more great contributions from 7histle, and trou and I'm very happy with the state of the code and the usefulness of the application.

The biggest outstanding issue is RFC 2047 header decoding. Converting subject/to/from fields to readable versions of their encoded form:

Subject: =?utf-8?Q?Blipfoto=20=2D=20Introducing=20the=20all=2Dnew=20Bli

This is annoying because I'm using mimetic for handling all MIME-related code, and this doesn't seem to offer the facilities that I need.

The current plan is to use the RFC-2047 handling from vmime, but I've fought with that library unsucessfully for two days now - and a further complication is that the library is included in Squeeze/Sid, but not the stable release of Debian.

In conclusion I still regard the client as complete, because I'm using it exclusively and I rarely get "foreign" mails. But there is one more push required to fix all the outstanding bugs which generall boil down to:

• Decode headers properly.
• Ensure all our input/output is in UTF-8.

Randomly I'm wondering if I can call out to Lua to do the header decoding. Add "on_header_field()" and display the results. So today I'll be looking at how sensible that is, probably not very.

Tags: lumail | 4 comments

I've spent the past few days overhauling the TAB-completion which is included in lumail.

Completing a single token is easy, if there is only one match, and you limit yourself to completing at the start of a line. But doing real completion is hard. Consider the case where you want to complete something like this:

unread_message_colour("re[TAB]

Clearly completing the first part "unread_[TAB]" is simple. But to complete "re" to "red" you need to split up your input line into tokens so that you can recognize a quote as a valid completion point.

Similarly you need to split on "(" to allow:

-- show the path to the editor
msg(edit[TAB]

To allow this to be changed/controlled by the user I defined completion_chars() which contains: SPACE, QUOTE, "(", etc.

I'm pleased with the user-callback for offering completion suggestions, my own is pretty basic and just includes all user-defined functions as well as an address book. The latter allows steve.org.uk[TAB] to complete to: "Steve Kemp" <[email protected]> - because we allow matches anywhere in the completion string, rather than just prefix-matching.

I struggled with resolving ambiguities, but now that is handled correctly too. Press "TAB" when there are multiple choices available and you can graphically TAB-through the available choices, or press Esc to cancel.

In conclusion I've spent a few days fighting with user-interface stuff and now the mail-client is better, but I've still to tackle RFC 2047 header decoding because that is really hard!

Tags: lumail | 1 comment

A hectic day and now we use GMime for all MIME-fu.

This has allowed me to decode headers correctly, setup MIME parts properly in outgoing mails with attachments, and cleanup the code-base.

The next release of Lumail will contain basically just this change, as it is pretty drastic. But first I need to work out how to make binaries for Squeeze compiled against the back-ported version of gmime-2.6.x.

(Previously we used libmimetic. Which was awesome in its way, but caused me some pain with RFC 2047 header-decoding.)

Tags: lumail | No comments

This morning I made a new release of Lumail, which recently completed the transition from using mimetic to GMime for all its MIME needs.

I was happy with mimetic, except it didn't have the facility to decode encoded header-values. I wrote some code, but it was broken, so I made the decision that we should move to something that made this easier, and GMime was chosen.

Jeffrey Stedfast was very helpful in answering my questions with near-perfect code samples.

Beyond that this release features some more Lua primitives and a couple of bug-fixes.

The only annoyance is that the version of GMime I'm using, 2.6.x, isn't available to users of the Squeeze release of Debian GNU/Linux. It is available as a backport, but that means building binaries with sbuild is a pain - due to #700522.

So for the moment I've only built binaries for Wheezy users.

ObQuote: "You know who I am, I've not been off TV for that long!" - Alan Patridge, Alpha Papa

Tags: backports, lumail | 3 comments

This weekend I'm mostly alternating between reading, writing, and trying to avoid death by the plague.[*]

I've switched Lumail to using a UTF-8 aware string library, which means we can now handle the obvious case:

--
-- Prove keybindings work.
--
keymap['global']['π'] = 'msg("UTF-8 rocks!")'

Similarly we can stuff input into the buffer:

--
-- Pretend the user typed ":msg ...\n"
--
stuff( ":msg('π is pie')\n" );

This transition was annoying to handle, but wasn't too difficult. There is only one more major update required, according to the development roadmap, which is to double check that UTF-8 output is correct.

Otherwise I think I'm almost done. In the sense that I don't see anything obvious missing, barring things that won't ever happen such as mutt-style "tag" support.

I've updated the online examples to include some nice code:

• A nice menu for viewing/opening attachments: view_attachments().
• A nice system for marking messages as read based upon their subjects: mark_read_by_pattern().

I can't claim to have many users, so far the development has been carried out by myself and approximately four other people. But that matters not. I genuinely believe this is a good client and it really suits the way that I handle (large volumes of) email:

• Show folders with unread mail.
• Quickly read it.

Allowing you to open multiple folders at once means you get a great view into your currently-unread mail, regardless of where procmail has placed it.

The overriding feeling having "completed" the client is that Lua rocks. I'm torn between wanting to sleep some more, and wondering what other system/package/tool can be extended by Lua. As epiphanies go my on_idle() update takes some beating.

* - I do not have the black death, but I'm not well.

Tags: lua, lumail | 2 comments

This week I have mostly:

Knocked up a forum ("gathering")

It is a simple clone of hacker news, storing things in Redis.

None of the aging though, entries are first->last and my tagging support is basic.

I'm in two minds about releasing it. I'm in three minds about deploying it at the location I'd written it for - forums.lumail.org - since there's nothing worse than a forum with no posts, unless it is a forum that used to be popular and is now a wasteland, or circlejerk. (c.f. slashdot. ahem.)

Updated Slaugther

I found a hidden dependency when installing a new slaughter controlled host.

A new release is imminent.

Setup a remote host for backups

Using BigV I configured a system with LUKS-encrypted disks to act as a "remote dropbox" via git-annex, and rsync.

An encrypted volume is manually mounted post-boot. It stays mounted (oops that's bad) but provides security when the guest is offline, or has been retired.

Wrestled with Graphite

Because damn that software is hard to install.

Turned down two weddings

Because I will never shoot a wedding again. And if I did I'd not do it for free for "exposure".

Merged my (digital) music archive with that belonging to my partner

Which took more discussion than a) moving in together, or b) opening a shared bank account.

Secretly decided I do like my kindle

Even though I bought two books from a charity shop tonight I'm almost certainly going to file them away and look for the epub online instead.

He fought with the goblins! He battled the trolls! He riddled with Gollum! The magic ring he stole!

This weekend I'll be mostly offline. Saturating my home broadband while I sync backups.

Tags: gathering, lumail | No comments

Now it is time for me to go silent for a while, and not talk about jobs, unemployment, or puppies.

This past week has also been full of software releases. Some of the public ones include:

Lumail - My console mail client, with integrated lua scripting

After three months of slow work I've issued a new release today. This release features several bugfixes for dealing with malformed MIME messages, and similar fun.

The core set of lua primitives hasn't changed very much for a good six months now, which means I guess rightly what kind of things would be useful.

Templer - My perl-based static-site generator.

This was recently updated to add two new plugins to the core:

• A redis plugin to allow you to set variables to values retrieved from redis.
• An RSS plugin to allow you to inline (remote) RSS feeds into your static HTML. Useful for building news-pages, etc.

Although there are a million static-site generators I still think mine has value, and I am consistently using it.

Months ago when I said "I'm writing a mail-client", all I need to do is handle three cases:

• Display a list of folders.
• Display index of messages.
• Display a single message.

Then some new things like "Compose", "Reply", "Forward", I remember somebody commented along the lines of "Yeah, but MIME will make you hate your life" I laughed. Now I know better. Still it works, it works well, and I'm glad I did it.

Tags: lua, lumail, software, templer | 2 comments

So I wrote a mail client, and this morning I added the ability for it to receive input from a Unix domain socket.

In one terminal I have my email client open. In another I run:

lumailctl /tmp/foo.sock "open('/home/skx/Maildir/.livejournal.2014/');"

That opens the unix domain socket, and pipes the following command to it:

open('/home/skx/Maildir/.livejournal.2014/');

The mail client has already got the socket open, and the end result is that my mail client suddenly opens the specified mail folder, and redraws itself.

Neat.

The "open" function is obviously a lua function, which builds upon the lua primitives the client understands:

function open( folder )
   clear_selected_folders()
   set_selected_folder(folder)
   index()
end

Obviously this would be woefully insecure if it were released like this. Later I'll wire up some lua function to establish the socket, such that the user specifies where the socket is created (their home directory, ideally), and it doesn't run by default.

Tags: lua, lumail | 2 comments

Today I took down my KVM-host machine, rebooting it and restarting all of my guests. It has been a while since I'd done so and I was a little nerveous, as it turned out this nerveousness was prophetic.

I'd forgotten to hardwire the use of proxy_arp so my guests were all broken when the systems came back online.

If you're curious this is what my incoming graph of email SPAM looks like:

I think it is obvious where the downtime occurred, right?

In other news I'm awaiting news from the system administration job I applied for here in Edinburgh, if that doesn't work out I'll need to hunt for another position..

Finally I've started hacking on my console based mail-client some more. It is a modal client which means you're always in one of three states/modes:

• maildir - Viewing a list of maildir folders.
• index - Viewing a list of messages.
• message - Viewing a single message.

As a result of a lot of hacking there is now a fourth mode/state "text-mode". Which allows you to view arbitrary text, for example scrolling up and down a file on-disk, to read the manual, or viewing messages in interesting ways.

Support is still basic at the moment, but both of these work:

  --
  -- Show a single file
  --
  show_file_contents( "/etc/passwd" )
  global_mode( "text" )

Or:

function x()
   txt = { "${colour:red}Steve",
           "${colour:blue}Kemp",
           "${bold}Has",
           "${underline}Definitely",
           "Made this work" }
   show_text( txt )
   global_mode( "text")
end

x()

There will be a new release within the week, I guess, I just need to wire up a few more primitives, write more of a manual, and close some more bugs.

Happy Thursday, or as we say in this house, Hyvää torstai!

Tags: jobs, kvm, lumail, misc, spam | 1 comment

I'm pondering a rewrite of my console-based mail-client.

While it is "popular" it is not popular.

I suspect "console-based" is the killer.

I like console, and I ssh to a remote server to use it, but having different front-ends would be neat.

In the world of mailpipe, etc, is there room for a graphic console client? Possibly.

The limiting factor would be the lack of POP3/IMAP.

Reworking things such that there is a daemon to which a GUI, or a console client, could connect seems simple. The hard part would obviously be working the IPC and writing the GUI. Any toolkit selected would rule out 40% of the audience.

In other news I'm stalling on replying to emails. Irony.

Tags: lumail, pondering | 8 comments

Today I released version 0.25 of my console mail client, which is a release focussed upon portability (DragonFly BSD, and MacOSX specifically).

Over the past couple of weeks I've written a fair bit of code, wondering if I want to make the jump to a graphical email client, but the conclusion for the moment has to be no.

With the scripting support built into my client, and even before then using the hooks/hacks that mutt supported, I just process mail so much more quickly than via a GUI system.

I also benefit from reading the mail on the host to which it is delivered - mail gets filtered by something like procmail, and I read it in-situa. IMAP is available if I travel, but I rarely do so.

Having a GUI client might be fun, but it would mean I'd read mail on my desktop - pretty much the only system I don't backup (except for images, videos, and local media). It would also involve running imapsync, or similar, to pull the mail in, and relaying through the remote server to avoid my ISPs poor IP-reputation.

In short I believe if I use a GUI client I'll get slower, and I'll still need the remote host regardless.

It was this time last year when I thought it was functional, but now it is functional, battle-tested, and reliable.

So I guess I'm done with email for the next few years. Maybe in that time somebody will write something better - console based for preference, GUI as a last resort, and certainly not another webmail client.

In other news ..

I had a fun interview on Monday, it went well until they admitted they couldn't afford me - so their goal is to pay a junior member a small salary and hope to get somebody senior to work part-time for a similarly minimal salary. Might work for somebody else, but it wouldn't for me right now, so on that basis I declined.

The most annoying thing about interviewing is the waiting, between the early flirting about duties and expectations, to scheduling meetings, and then awaiting decisions.

On that note I'm half-way through building a new desk which is a nice physical job I can really concentrate upon. I'm currently waiting for the stain to dry on the legs, and then I'll get the damn thing finished. It probably looks more "rustic" than "modern", but it smells nice, so that's the main thing ;)

Expect pictures when it is finished.

Tags: diy, interviews, lumail, work | No comments

Until recently I was very happy with my console mail client, Lumail, thinking I'd written it in a flexible manner, with lots of Lua-callable primitives.

Now I'm beginning to suspect that I might have gone down the wrong path at some point.

The user interface, at startup consists of a list of mailboxes. The intention being that you select a mailbox, and open it. That then takes you to a list of messages. (There is a cool and simple to use option to open the union of multiple mailboxes, which is something I use daily.)

Now the list of mailboxes is sorted alphabetically, so the user interface looks something like this:

Now the issue that triggered my rethink:

• Can it be possible for Lua to sort the maildir list? So I could arbitrarily have the Maildir .people.katy at the top of the list, always?

Sure you think. It's just a list of strings. You could pass an array to a lua on_sort_maildirs function, and then use the returned array/table as teh display order. Simple.

Simple until you realize the user might want to do more than operate solely on the list of strings. Perhaps they wish to put folders with unread messages at the top. At which point you need a "count_unread( maildir )" function. (Which does exist.)

Anyway the realisation I had is that the CMaildir object, on the C++ side, isn't exposed to the Lua-side. So the (useful) member functions which should be exported/visible are not.

Really what I'm trying to say is that I think I've implemented and exported useful Lua primitives, but actually many more parts of the implementation could be usefully exported - but are not, and that comes from the choice I made to expose "things" not "objects". If I'd exposed objects right from the start I'd have been in a better place.

Oh well.

I continued to toy with a basic GUI mail-client last week, but I've pretty much written that off as a useful way to spend my time. For the moment I'll leave email alone, I've done enough and despite niggles what I have is absolutely the best mail client for me.

(It is a shame that Mutt is so heavyweight and hard to deal with, and that notmuch never really took off.)

Tags: lua, lumail | 3 comments

I've continued to ponder the idea of reimplementing the console mail-client I wrote, lumail, using a more object-based codebase.

For one thing having loosely coupled code would allow testing things in isolation, which is clearly a good thing.

I've written some proof of concept code which will allow the following Lua to be evaluated:

-- Open the maildir.
users = Maildir.new( "/home/skx/Maildir/.debian.user" )

-- Count the messages.
print( "There are " .. users:count() .. " messages in the maildir " .. users:path() )

--
-- Now we want to get all the messages and output their paths.
--
for k,v in ipairs( users:messages()) do
    --
    -- Here we could do something like:
    --
    --   if ( string.find( v:headers["subject"], "troll", 1, true ) ) then v:delete()  end
    --
    -- Instead play-nice and just show the path.
    print( k .. " -> " .. v:path() )
end

This is all a bit ugly, but I've butchered some code together that works, and tried to solicit feedback from lumail users.

I'd write more but I'm tired, and intending to drink whisky and have an early night. Today I mostly replaced pipes in my attic. (Is it "attic", or is it "loft"? I keep alternating!) Fingers crossed this will mean a dry kitchen in the morning.

Tags: lumail | No comments

Once upon a time I wrote a mail-client, which worked in the console directly via Maildir manipulation.

My mail client was written in C++, and used Lua for scripting unlike clients such as mutt, alpine, and similar alternatives which don't have complete scripting support.

I've pondered several times whether to restart this project, but I think it is the right thing to do.

The original lumail client has a rich API, but it is very ad-hoc and random. Functions were added where they seemed like a good idea, but with no real planning, and although there are grouped functions that operate similarly there isn't a lot of consistency. The implementation is clean in places, elegant in others, and horrid in yet more parts.

This time round everything is an object, accessible to Lua, with Lua, and for Lua. This time round all the drawing-magic is will be written in Lua.

So to display a list of Maildirs I create a bunch of objects, one for each Maildir, and then the Lua function Maildir.to_string is called. That function looks like this:

--
-- This method returns the text which is displayed when a maildir is
-- to be show in maildir-mode.
--
function Maildir.to_string(self)
   local total  = self:total_messages()
   local unread = self:unread_messages()
   local path   = self:path()

   local output = string.format( "[%05d / %05d] - %s", unread, total, path );

   if ( unread > 0 ) then
      output = "$[RED]" .. output
   end

   if ( string.find( output, "Automated." ) ) then
      output = strip_colour( output )
      output = "$[YELLOW]" .. output
   end

   return output
end

The end result is something that looks like this:

[00001 / 00010 ] - Amazon.de
[00000 / 00023 ] - Automated.root

The formatting can thus be adjusted clearly, easily, and without hacking the core of the client. Providing I implement the apporpriate methods to the Maildir object.

It's still work in progress. You can view maildirs, view indexes, and view messages. You cannot reply, forward, or scroll properly. That said the hard part is done now, and I'm reasonably happy with it.

The sample configuration file is a bit verbose, but a good demonstration regardless.

See the code, if you wish, online here:

• https://github.com/lumail/lumail2/

Tags: lua, lumail, lumail2 | 2 comments

A couple of years ago I was dissatisfied with mutt, mostly because the mutt-sidebar patch was dropped from the Debian package. That lead to me thinking "How hard can it be to write a modal, console-based mail-client?"

It turns out writing a client is pretty simple if you limit yourself solely to Maildirs, and as I typically read my mail over SSH on the mailhost itself that suited me pretty well.

Recently I restarted the mail-client. Putting it together from scratch to simplify the implementation, and unify a lot of the adhoc scripting which is provided by Lua. People seem to like the client, but the single largest complaint was "Can't use it - no IMAP."

This week I've mostly been adding IMAP support, and today I'll commit the last few bits that mean it is roughly-functional:

• Connecting to a mail-server works.
• Getting the folders works.
• Getting the messages works.

The outstanding niggles will be relating to getting/setting the new/read/seen/unseen flags, and similar. But I'm pleased that the job wasn't insurmountable.

I've used libcurl to provide the IMAP functionality because most of the IMAP libraries I looked at were big, scary, and complex. Using curl to access IMAP is pretty neat, simple, and straightforward. The downside is you're making a lot of "http" requests. So I might need to revisit things.

Happily my imap wrapper doesn't need much functionality. So if I can find a better library swapping it out will be simple.

In conclusion: Lumail almost has IMAP support, and that might mean it'll be more useful to others.

Tags: imap, lumail, lumail2 | No comments

So after living here in Finland for 6 months I've now bought a flat.

We have a few days to sort out mortgage paperwork, and assuming there are no problems we'll be moving into the new place on/around the 1st of March.

Finally I'll be living in Finland, with a sauna of my very own.

Interesting times.

In more developer-friendly news I made a new release of Lumail with the integrated support for IMAP. Let us hope people like it.

Tags: helsinki, life, lumail | 4 comments

The past few days I've been working on my mail client which has resulted in a lot of improvements to drawing, display and correctness.

Since then I've been working on adding GPG-support. My naive attempt was to extract the signature, and the appropriate body-part from the message. Write them both to disk then I could validate via:

gpg --verify msg.sig msg

However that failed, and it took me a long to work out why. I downloaded the source to mutt, which can correctly verify an attached-signature, then hacked lib.c to neuter the mutt_unlink function. That left me with a bunch of files inside $TEMPFILE one of which provided the epiphany.

A message which is to be validated is indeed written out to disk, just as I would have done, as is the signature. Ignoring the signature the message is interesting:

Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Mon, 27 Jun 2016 08:08:14 +0200

...

--=20
Bob Smith

The reason I'd failed to validate my message-body was because I'd already decoded the text of the MIME-part, and I'd also lost the prefixed two lines "Content-type:.." and Content-Transfer:.... I'm currently trying to work out if it is possible to get access to the RAW MIME-part-text in GMIME.

Anyway that learning aside I've made a sleazy hack which just shells out to mimegpg, and this allows me to validate GPG signatures! That's not the solution I'd prefer, but that said it does work, and it works with inline-signed messages as well as messages with application/pgp-signature MIME-parts.

Changing the subject now. I wonder how many people read to the end anyway?

I've been in Finland for almost a year now. Recently I was looking over websites and I saw that the domain steve.fi was going to expire in a few weeks. So I started obsessively watching it. Today I claimed it.

So I'll be slowly moving things from beneath steve.org.uk to use the new home steve.fi.

I also setup a mini-portfolio/reference site at http://steve.kemp.fi/ - which was a domain I registered while I was unsure if I could get steve.fi.

Finally now is a good time to share more interesting news:

• I've been reinstated as a Debian developer.
• We're having a baby.
  • Interesting times.

Tags: finland, gpg, lumail, mutt, updates | 7 comments