Entries tagged lua

This is just a quick post to remind myself in the morning, as soon as I've made it I intend to turn my computer off and leave it off until I can re-organize my office.

I've been using mutt for my email for the past few years. Nothing compares to the flexibility of procmail/sieve for organizing server-side mail, and then mutt is ideal for reading them.

With the addition of the mutt-patched sidebar mode you can even go for a few days before realizing you're not in a graphical environment. But one thing I do long for is the ability to execute scripts at various times.

Thus far I've not actually planned what I'd like to do, but as a starting point imagine being able to execute a hook when new mail arrives? Or when you send a message matching a pattern in some fashion?

There are some things out there, such as the various hacks which are designed to abort sending a message if you mention "See attachment" in a message body but fail to add one before sendign the message. These hacks generally abuse the sendmail configuration such that they're extremely ad-hoc and hard to chain/nest.

I've mellowed out over the years and I have no interest in attempting to write a mail-client (though at the same time how hard can it be? Just restrict yourself to using inotify on ~/Maildir and offload delivery to exim and you're almost done? I guess the hard part is the UI, though I do like the mutt + sidebar layout. Write the whole thing in some scripty language?)

I'll re-examine notmuch and gnus over the next week or two, but I suspect both will continue to disappoint in various ways.

Anyway, for the moment I'm just pondering. But threading is an obvious concern. Most current mutt hooks relate to the local folder, or the local message. If I were viewing a message in one directory and a new mail notification fired for a delivery to both ~/Maildir and ~/Maildir/.people.foo I'd need to either serialise them or thread them.

Ponder ponder.

In other news I've been doing more photography recently. Nothing cohesive except for my recent experiment with shooting a "street-girl" outdoors in falling light, but that was an interesting challenge and the results were sufficient to make me want to try shooting outdoors in an organized fashion again. (Some random images have been linked to from my wee twitter page.)

ObFilm: "She doesn't get eaten by the eels at this time " - The Princess Bride

Tags: lua, mutt, perl, random | 5 comments

To recap: given the absence of other credible alternatives I had two options:

• Re-hack mutt to give me a sidebar that will show only folders containing new messages.
• Look at writing a "simple mail client". Haha. Ha. Hah.

I think there is room for a new console client, because mutt is showing its age and does feel like it should have a real extension language - be it guile, lisp, javascript(!), Lua, or something else.

So I distilled what I thought I wanted into three sections:

• mode-ful. There would be a "folder-browsing mode", a "message-browsing mode" and a "read-a-single-message" mode.
• There would be scripting. Real scripting. I chose Lua.
• You give it ~/Maildir as the configuration. Nothing else. If the damn computer cannot find your mailboxes something is wrong.

So how did I do? I wrote a ncurses-based client which has Lua backed into it. You can fully explore the sidebar-mode - which lets you select multiple folders.

From there you can view the messages in a list.

What you can't do is anything "real":

• Update a messages flags. new -> read, etc.
• GPG-validation.
• MIME-handling.
• Attachment viewing.

For a two-day hack it is remarkably robust, and allowing scripting shows awesomeness. Consider this:

--
-- show all folders in the Maildir-list.
--
function all()
   -- ensure that the sidebar displays all folders
   sidebar_mode = "all";
   -- we're going to be in "maildir browsing mode"
   cmail_mode = "sidebar";
   reset_sidebar();
   refresh_screen();
end

--
-- Test code, show that the pattern-searching works.
--
-- To use this press ":" to enter the prompt, then enter "livejournal".
--
-- OR press "l" when in the sidebar-mode.
--
function livejournal()
   sidebar_pattern = "/.livejournal.2";
   sidebar_mode = "pattern";
   reset_sidebar();
   refresh_screen();
end

--
-- There is a different table for each mode.
--
keymap = {}
keymap['sidebar'] = {}
keymap['index']   = {}
keymap['message'] = {}

--
-- In the sidebar-mode "b" toggles the sidebar <-> index.
--
-- ":" invokes the evaluator.
-- "q" quits the browser and goes to the index-mode.
-- "Q" quits the program entirely.
--
keymap['sidebar'][':'] = "prompt-eval"
keymap['sidebar']['b'] = "toggle"
keymap['sidebar']['q'] = "toggle"
keymap['sidebar']['Q'] = "exit"

-- show all/unread/livejournal folders
keymap['sidebar']['a'] = "all"
keymap['sidebar']['u'] = "unread"
keymap['sidebar']['l'] = "livejournal"

Neat, huh? See the cmail.lua file on github for more details.

My decision hasn't really progressed any further, though I can see that if this client were complete I'd love to use it. Its just that the remaining parts are the fiddly ones.

I guess I'll re-hack mutt, and keep this on the back-burner.

The code is ropey in places, but should you wish to view:

• https://github.com/skx/cmail

And damn C is kicking my ass.

Tags: cmail, console, development, lua, mail | 4 comments

When people ask me why I chose to embed Lua in my mail-client I'll point to my on_idle() documentation.

Moving from a callback which runs once every second, or so, to allowing the user to schedule tasks on arbitrary boundaries is pretty cool - and obviously requires no explicit support from myself.

Now I've fixed a couple of bugs which went unspotted/unreported in the first release I'm ready for a new one "soon".

In the meantime I'm running the client exclusively, and loving the ability to view all unread mail, only, regardless of the parent folder.

Tags: lua, lumail | No comments

This weekend I'm mostly alternating between reading, writing, and trying to avoid death by the plague.[*]

I've switched Lumail to using a UTF-8 aware string library, which means we can now handle the obvious case:

--
-- Prove keybindings work.
--
keymap['global']['π'] = 'msg("UTF-8 rocks!")'

Similarly we can stuff input into the buffer:

--
-- Pretend the user typed ":msg ...\n"
--
stuff( ":msg('π is pie')\n" );

This transition was annoying to handle, but wasn't too difficult. There is only one more major update required, according to the development roadmap, which is to double check that UTF-8 output is correct.

Otherwise I think I'm almost done. In the sense that I don't see anything obvious missing, barring things that won't ever happen such as mutt-style "tag" support.

I've updated the online examples to include some nice code:

• A nice menu for viewing/opening attachments: view_attachments().
• A nice system for marking messages as read based upon their subjects: mark_read_by_pattern().

I can't claim to have many users, so far the development has been carried out by myself and approximately four other people. But that matters not. I genuinely believe this is a good client and it really suits the way that I handle (large volumes of) email:

• Show folders with unread mail.
• Quickly read it.

Allowing you to open multiple folders at once means you get a great view into your currently-unread mail, regardless of where procmail has placed it.

The overriding feeling having "completed" the client is that Lua rocks. I'm torn between wanting to sleep some more, and wondering what other system/package/tool can be extended by Lua. As epiphanies go my on_idle() update takes some beating.

* - I do not have the black death, but I'm not well.

Tags: lua, lumail | 2 comments

Now it is time for me to go silent for a while, and not talk about jobs, unemployment, or puppies.

This past week has also been full of software releases. Some of the public ones include:

Lumail - My console mail client, with integrated lua scripting

After three months of slow work I've issued a new release today. This release features several bugfixes for dealing with malformed MIME messages, and similar fun.

The core set of lua primitives hasn't changed very much for a good six months now, which means I guess rightly what kind of things would be useful.

Templer - My perl-based static-site generator.

This was recently updated to add two new plugins to the core:

• A redis plugin to allow you to set variables to values retrieved from redis.
• An RSS plugin to allow you to inline (remote) RSS feeds into your static HTML. Useful for building news-pages, etc.

Although there are a million static-site generators I still think mine has value, and I am consistently using it.

Months ago when I said "I'm writing a mail-client", all I need to do is handle three cases:

• Display a list of folders.
• Display index of messages.
• Display a single message.

Then some new things like "Compose", "Reply", "Forward", I remember somebody commented along the lines of "Yeah, but MIME will make you hate your life" I laughed. Now I know better. Still it works, it works well, and I'm glad I did it.

Tags: lua, lumail, software, templer | 2 comments

So I wrote a mail client, and this morning I added the ability for it to receive input from a Unix domain socket.

In one terminal I have my email client open. In another I run:

lumailctl /tmp/foo.sock "open('/home/skx/Maildir/.livejournal.2014/');"

That opens the unix domain socket, and pipes the following command to it:

open('/home/skx/Maildir/.livejournal.2014/');

The mail client has already got the socket open, and the end result is that my mail client suddenly opens the specified mail folder, and redraws itself.

Neat.

The "open" function is obviously a lua function, which builds upon the lua primitives the client understands:

function open( folder )
   clear_selected_folders()
   set_selected_folder(folder)
   index()
end

Obviously this would be woefully insecure if it were released like this. Later I'll wire up some lua function to establish the socket, such that the user specifies where the socket is created (their home directory, ideally), and it doesn't run by default.

Tags: lua, lumail | 2 comments

Until recently I was very happy with my console mail client, Lumail, thinking I'd written it in a flexible manner, with lots of Lua-callable primitives.

Now I'm beginning to suspect that I might have gone down the wrong path at some point.

The user interface, at startup consists of a list of mailboxes. The intention being that you select a mailbox, and open it. That then takes you to a list of messages. (There is a cool and simple to use option to open the union of multiple mailboxes, which is something I use daily.)

Now the list of mailboxes is sorted alphabetically, so the user interface looks something like this:

Now the issue that triggered my rethink:

• Can it be possible for Lua to sort the maildir list? So I could arbitrarily have the Maildir .people.katy at the top of the list, always?

Sure you think. It's just a list of strings. You could pass an array to a lua on_sort_maildirs function, and then use the returned array/table as teh display order. Simple.

Simple until you realize the user might want to do more than operate solely on the list of strings. Perhaps they wish to put folders with unread messages at the top. At which point you need a "count_unread( maildir )" function. (Which does exist.)

Anyway the realisation I had is that the CMaildir object, on the C++ side, isn't exposed to the Lua-side. So the (useful) member functions which should be exported/visible are not.

Really what I'm trying to say is that I think I've implemented and exported useful Lua primitives, but actually many more parts of the implementation could be usefully exported - but are not, and that comes from the choice I made to expose "things" not "objects". If I'd exposed objects right from the start I'd have been in a better place.

Oh well.

I continued to toy with a basic GUI mail-client last week, but I've pretty much written that off as a useful way to spend my time. For the moment I'll leave email alone, I've done enough and despite niggles what I have is absolutely the best mail client for me.

(It is a shame that Mutt is so heavyweight and hard to deal with, and that notmuch never really took off.)

Tags: lua, lumail | 3 comments

Recently I was updating my dotfiles, because I wanted to ensure that media-players were "always on top", when launched, as this suits the way I work.

For many years I've used devilspie to script the placement of new windows, and once I googled a recipe I managed to achieve my aim.

However during the course of my googling I discovered that devilspie is unmaintained, and has been replaced by something using Lua - something I like.

I'm surprised I hadn't realized that the project was dead, although I've always hated the configuration syntax it is something that I've used on a constant basis since I found it.

Unfortunately the replacement, despite using Lua, and despite being functional just didn't seem to gell with me. So I figured "How hard could it be?".

In the past I've written softare which iterated over all (visible) windows, and obviously I'm no stranger to writing Lua bindings.

However I did run into a snag. My initial implementation did two things:

• Find all windows.
• For each window invoke a lua script-file.

This worked. This worked well. This worked too well.

The problem I ran into was that if I wrote something like "Move window 'emacs' to desktop 2" that action would be applied, over and over again. So if I launched emacs, and then manually moved the window to desktop3 it would jump back!

In short I needed to add a "stop()" function, which would cause further actions against a given window to cease. (By keeping a linked list of windows-to-ignore, and avoiding processing them.)

The code did work, but it felt wrong to have an ever-growing linked-list of processed windows. So I figured I'd look at the alternative - the original devilspie used libwnck to operate. That library allows you to nominate a callback to be executed every time a new window is created.

If you apply your magic only on a window-create event - well you don't need to bother caching prior-windows.

So in conclusion :

• devilspie is dead.
• there is a devils2pie2 project.
• I wrote kpie

I think my code is better than devilspie2 because it is smaller, simpler, and does things more neatly - for example instead of a function to get geometry and another to set it, I use one. (e.g. "xy()" returns the position of a window, but xy(3,3) sets it.).

kpie also allows you to run as a one-off job, and using the simple primitives I wrote a file to dump your windows, and their size/placement, which looks like this:

shelob ~/git/kpie $ ./kpie --single ./samples/dump.lua
-- Screen width : 1920
-- Screen height: 1080
..
if ( ( window_title() == "Buddy List" ) and
     ( window_class() == "Pidgin" ) and
     ( window_application() == "Pidgin" ) ) then
     xy(1536,24 )
     size(384,1032 )
     workspace(2)
end
if ( ( window_title() == "feeds" ) and
     ( window_class() == "Pidgin" ) and
     ( window_application() == "Pidgin" ) ) then
     xy(1,24 )
     size(1536,1032 )
     workspace(2)
end
..

As you can see that has dumped all my windows, along with their current state. This allows a simple starting-point - Configure your windows the way you want them, then dump them to a script file. Re-run that script file and your windows will be set back the way they were! (Obviously there might be tweaks required.)

I used that starting-point to define a simple recipe for configuring pidgin, which is more flexible than what I ever had with pidgin, and suits my tastes.

Bug-reports welcome.

Tags: devilspie, kpie, lua | No comments

Once upon a time I wrote a mail-client, which worked in the console directly via Maildir manipulation.

My mail client was written in C++, and used Lua for scripting unlike clients such as mutt, alpine, and similar alternatives which don't have complete scripting support.

I've pondered several times whether to restart this project, but I think it is the right thing to do.

The original lumail client has a rich API, but it is very ad-hoc and random. Functions were added where they seemed like a good idea, but with no real planning, and although there are grouped functions that operate similarly there isn't a lot of consistency. The implementation is clean in places, elegant in others, and horrid in yet more parts.

This time round everything is an object, accessible to Lua, with Lua, and for Lua. This time round all the drawing-magic is will be written in Lua.

So to display a list of Maildirs I create a bunch of objects, one for each Maildir, and then the Lua function Maildir.to_string is called. That function looks like this:

--
-- This method returns the text which is displayed when a maildir is
-- to be show in maildir-mode.
--
function Maildir.to_string(self)
   local total  = self:total_messages()
   local unread = self:unread_messages()
   local path   = self:path()

   local output = string.format( "[%05d / %05d] - %s", unread, total, path );

   if ( unread > 0 ) then
      output = "$[RED]" .. output
   end

   if ( string.find( output, "Automated." ) ) then
      output = strip_colour( output )
      output = "$[YELLOW]" .. output
   end

   return output
end

The end result is something that looks like this:

[00001 / 00010 ] - Amazon.de
[00000 / 00023 ] - Automated.root

The formatting can thus be adjusted clearly, easily, and without hacking the core of the client. Providing I implement the apporpriate methods to the Maildir object.

It's still work in progress. You can view maildirs, view indexes, and view messages. You cannot reply, forward, or scroll properly. That said the hard part is done now, and I'm reasonably happy with it.

The sample configuration file is a bit verbose, but a good demonstration regardless.

See the code, if you wish, online here:

• https://github.com/lumail/lumail2/

Tags: lua, lumail, lumail2 | 2 comments

I'm pleased with the way that Lumail2 development is proceeding, and it is reaching a point where there will be a second source-release.

I've made a lot of changes to the repository recently, and most of them boil down to moving code from the C++ side of the application, over to the Lua side.

This morning, for example, I updated the handing of index.limit to be entirely Lua based.

When you open a Maildir folder you see the list of messages it contains, as you would expect.

The notion of the index.limit is that you can limit the messages displayed, for example:

• See all messages: Config:set( "index.limit", "all")
• See only new/unread messages: Config:set( "index.limit", "new")
• See only messages which arrived today: Config:set( "index.limit", "today")
• See only messages which contain "Steve" in their formatted version: Config:set( "index.limit", "steve")

These are just examples that are present as defaults, but they give an idea of how things can work. I guess it isn't so different to Mutt's "limit" facilities - but thanks to the dynamic Lua nature of the application you can add your own with relative ease.

One of the biggest changes, recently, was the ability to display coloured text! That was always possible before, but a single line could only be one colour. Now colours can be mixed within a line, so this works as you might imagine:

Other changes include a persistant cache of "stuff", which is Lua-based, the inclusion of at least one luarocks library to parse Date: headers, and a simple API for all our objects.

All good stuff. Perhaps time for a break in the next few weeks, but right now I think I'm making useful updates every other evening or so.

Tags: lua, lumail2 | No comments

Recently Antirez made a post documenting a simple editor in 1k of pure C, the post was interesting in itself, and the editor is a cute toy because it doesn't use curses - instead using escape sequences.

The github project became very popular and much interesting discussion took place on hacker news.

My interest was piqued because I've obviously spent a few months working on my own console based program, and so I had to read the code, see what I could learn, and generally have some fun.

As expected Salvatore's code is refreshingly simple, neat in some areas, terse in others, but always a pleasure to read.

Also, as expected, a number of forks appeared adding various features. I figured I could do the same, so I did the obvious thing in adding Lua scripting support to the project. In my fork the core of the editor is mostly left alone, instead code was moved out of it into an external lua script.

The highlight of my lua code is this magic:

  --
  -- Keymap of bound keys
  --
  local keymap = {}

  --
  --  Default bindings
  --
  keymap['^A']        = sol
  keymap['^D']        = function() insert( os.date() ) end
  keymap['^E']        = eol
  keymap['^H']        = delete
  keymap['^L']        = eval
  keymap['^M']        = function() insert("\n") end

I wrote a function invoked on every key-press, and use that to lookup key-bindings. By adding a bunch of primitives to export/manipulate the core of the editor from Lua I simplified the editor's core logic, and allowed interesting facilities:

• Interactive evaluation of lua.
• The ability to remap keys on the fly.
• The ability to insert command output into the buffer.
• The implementation of copy/past entirely in Lua_.

All in all I had fun, and I continue to think a Lua-scripted editor would be a neat project - I'm just not sure there's a "market" for another editor.

View my fork here, and see the sample kilo.lua config file.

Tags: kilo, lua | No comments

I recently mentioned that I'd forked Antirez's editor and added lua to it.

I've been working on it, on and off, for the past week or two now. It's finally reached a point where I'm content:

• The undo-support is improved.
• It has buffers, such that you can open multiple files and switch between them.
  • This allows this to work "kilua *.txt", for example.
• The syntax-highlighting is improved.
  • We can now change the size of TAB-characters.
  • We can now enable/disable highlighting of trailing whitespace.
• The default configuration-file is now embedded in the body of the editor, so you can run it portably.
• The keyboard input is better, allowing multi-character bindings.
  • The following are possible, for example ^C, M-!, ^X^C, etc.

Most of the obvious things I use in Emacs are present, such as the ability to customize the status-bar (right now it shows the cursor position, the number of characters, the number of words, etc, etc).

Anyway I'll stop talking about it now :)

• https://github.com/skx/kilua

Tags: kilo, lua | No comments