Entries tagged tags

I've been re-reading RFC 2822 again over the weekend, for obvious reasons, and I'm amused I've not noticed this section in the past:

3.6.5. Informational fields

The informational fields are all optional. The "Keywords:" field contains a comma-separated list of one or more words or quoted-strings. The "Subject:" and "Comments:" fields are unstructured fields as defined in section 2.2.1, and therefore may contain text or folding white space.

subject  = "Subject:" unstructured CRLF

comments = "Comments:" unstructured CRLF

keywords = "Keywords:" phrase *("," phrase) CRLF

Now we all know that emails have subjects, but how many people have ever used the Keywords: header, or the Comments: one?

It'd be nice if we could use these fields in mails - I can immediately think of "keywords" as tags, and I'm sure I'm not alone.

I've looked at multiple "tags for mutt" systems, but all of them fall down for the same reason. I can add tags to a mail, and limit a folder to those mails that contain a given tag. But I cannot do that for multiple folders and that makes them useless :(

Has anybody worked on a multi-folder tag system for Mutt? If so pointers welcome. If not I'd be tempted to create one.

I guess implementation would be very simple. There are three caeses:

• Adding a tag
• Deleting a tag
• Finding all messages with agiven tag.

The first two are easy. The second could be done by writing a cronjob to scan messages for Keyword: headers, and writing a simple index. That could then be used to populate an "~/Maildir/.tag-results" folder, via hardlinks, of all matching messages.

Better yet you could pre-populate ~/Maildir/.tag-$foo containing each message with a given tag. Then theres no searching required! (Although your cronjob would need to run often enough when the tag were added to a message it would appear there within a reasonable timeframe.

Update: I've written the indexer now. It works pretty quickly after the initial run, and is quite neat! tagging messages with mutt.

Tags: mail-scanning, mutt, rfc, tags | 6 comments

So, as I previously mentioned I want to be able to tag messages in Mutt.

There exist folder-based solutions already, using the X-Label header. There doesn't appear to be any existing solution allowing you to view all messages with a given tag across mailboxes.

So I wrote a simple shell script to create virtual mailboxes, such as ~/Maildir/tags-debian for all messages with a debian tag, using hardlinks.

My conclusion is that this solution will not work properly in practise, primarily because of deficiencies in mutt.

The simple case works just fine. I add a tag to a message, and later when the indexing job runs the virtual folder is created. I can open it and work on it just fine.

So where's the problem? Well in my case I tend to tag messages with a label such as "todo". Once I've done whatever I was supposed to I can remove the tag.

Using this hardlinking scheme I cannot remove the tag(s) in the virtual folder - I have to remove it in the original message which is a real pain.

Why? Well quite simply mutt will not let me work on my virtual message without destroying the hardlink.. If I use the edit function, for example, I am presented with a copy of the mail for editing - and the hardlink is replaced when that copy is saved.

Even the edit-label patch which allows you to edit the X-Label header from within mutt ends up replacing the hardlink with a new file!

So whats the solution? Well I guess I want to be able to run an external command against a message in mutt - passing the filename of the Maildir message as an argument. That way I can edit the live file.

Right now I don't believe that is possible, but I'd love to be told different.

If anybody has any solutions of editing, or even just deleting, a header from a message within mutt - in such a way that the hardlink isn't destroyed please do let me know.

Simple reproducer:

mkdir -p ~/Maildir/.foo/cur
mkdir    ~/Maildir/.foo/new
mkdir    ~/Maildir/.foo/tmp
cp 'validmessage' ~/Maildir/
ln validmessage ~/Maildir/.foo/cur

Now edit the message - start mutt open the message in the index and press 'e' - the hardlink is now gone. Replaced by a new file with the contents, so the original mail message is unchanged.

Update: I've got an "edit-inplace" primitive working, via the very hacky header-fu patch. It is not complete, but it demonstrates that it can be done. My world is now complete.

Tags: mutt, rants, tags | 7 comments

One final post then I'm done discussing mutt. (Primarily because too many people failed to understand the problem, I guess I wasn't being as good at explaining as I could have been.)

So the goal was to be able to tag individual messages within mutt, such that a short while later a new folder would be created containing all messages of that tag, regardless of which folder the message(s) are in.

Actually finding the tags and creating the virtual folders is easy to do. I wrote a simple indexer which scans for tags in messages and creates hardlinks as necessary.

The problem with this is that many operations on those hardlinked messags would create a new copy of the message and operate on that - trashing the hardlink. For some operations that's just fine. But I did specifically want to be able to untag a tagged message and not have to hunt around for the original. (e.g. remove the "todo" tag from messages where I'd done the relevant action.)

So after a bit of trial and error I came up with a patch which allows the editing of a message in-place, in the hardlinked folder, without trashing the hardlink. This patch just invokes "$EDITINPLACE <filename>" where "filename" is the name actual file the message I'm working on is stored on disk. (i.e. ~/Maildir/.people.thewomanmeg/cur/1.2.3.txt).

By setting the EDITINPLACE variable to point to my ~/bin/editlabel script I may easily remove any tags from the hardlinked-message and have it apply to the original. Job done!

There are some limitations with this approach that are worth mentioning though:

• IMAP? Ha! Nope. We only work on Maildirs.
• The header-cache facility of Mutt breaks my "find the filename of the Maildir message" function. Not sure why, so I just disabled it.
• Many operations on the virtual mailbox will still break the symlink; because they don't edit in place.

I've documented things in a rather random fashion, and I've made a backported package of mutt with my patch available online too. Primarily so I dont lose the source like I did with my mutt-ng backport.

I'm an amateur photographer, although these days I tend to drop the amateur prefix, given that I shoot people for cash at least once a month.

(It isn't my main job, and I'd never actually want it to be, because I'm certain I'd become unhappy hustling for jobs and doing the promotion thing.)

Anyway over the years I've built up a large library of images, mostly organized in a hierarchy of directories beneath ~/Images.

Unlike most photographers I don't use aperture, lighttable, or any similar library management. I shoot my images in RAW, convert to JPG via rawtherapee, and keep both versions of the images.

In short I don't want to mix the "library management" functions with the "RAW conversion" because I do regard them as two separate steps. That said I'm reaching a point where I do want to start tagging images, and finding them more quickly.

In the past I wrote a couple of simple tools to inject tags into the EXIF data of images, and then indexed them. But that didn't work so well in practise. I'm starting to think instead I should index images into sqlite:

• Size.
• date.
• Content hash.
• Tags.
• Path.

The downside is that this breaks utterly as soon as you move images around on-disk. Which is something my previous exif-manipulation was designed to avoid.

Anyway I'm thinking at the moment, but I know that the existing tools such as F-Spot, shotwell, DigiKam, and similar aren't suitable. So I either need to go standalone and use EXIF tags, accepting the fact that the tags I enter won't be visible to other tools, or I cope with the file-rename issues by attempting to update an existing sqlite database via hash/size/etc.

Tags: images, itag, tags | 15 comments