About Archive Tags RSS Feed

 

Entries tagged mutt

Will you lead me to your armchair

11 May 2007 21:50

Assume I'm using mutt for my mailclient and that I have all my mail stored beneath ~/Maildir.

Is there a simple way of limiting an index to only those folders containing New messages?

I know that I can use the index_format to display something like "N" next to folders containing new messages - but I cannot seem to limit the display to just those ones.

(At any given time I'll have something like 30+ mailboxes with unread messages, if not more.)

My current mutt looks something like this, which shows that I have 24 folders with unread messages - but it doesnt make it easy to tell which they are:

..
269       4096 May 11 22:49   ~/Maildir/.spam/
270       4096 May 11 23:36   ~/Maildir/.spam.bayes/
271       4096 May 11 22:49 N ~/Maildir/.spam.image/
272       4096 May 11 22:49   ~/Maildir/.spam.pyzor/
273       4096 May 11 22:49   ~/Maildir/.spam.razor/
274       4096 May 11 22:49   ~/Maildir/.spam.request/
275       4096 May 11 23:27   ~/Maildir/.spam.unsure/
276       4096 May 11 22:54   ~/Maildir/.steve_org_uk/
277       4096 May 11 22:49   ~/Maildir/.syswear_com/
..
-- Mutt: Mailboxes [24]

| No comments

 

Fed through the tube that sticks in me

12 May 2007 21:50

Thanks to everybody who told me that there wasn't a simple way to persuade Mutt to work with mailboxes containing unread messages.

I've now discovered the mutt-ng project, packaged in Debian's experimental branch. This contains the infamous mutt sidebar patch which caters to my needs fully:

  • I can toggle the display of the sidebar by pressing "b".
  • From there I can jump to the name of the next/previous mailbox with unread mail.
  • Once on a mailbox name I can open it easily.

Job done.

Expect a more involved writeup shortly ..

| No comments

 

Where troubles melt like lemon drops

1 March 2008 21:50

I've been re-reading RFC 2822 again over the weekend, for obvious reasons, and I'm amused I've not noticed this section in the past:

3.6.5. Informational fields

The informational fields are all optional. The "Keywords:" field contains a comma-separated list of one or more words or quoted-strings. The "Subject:" and "Comments:" fields are unstructured fields as defined in section 2.2.1, and therefore may contain text or folding white space.

subject  = "Subject:" unstructured CRLF

comments = "Comments:" unstructured CRLF

keywords = "Keywords:" phrase *("," phrase) CRLF

Now we all know that emails have subjects, but how many people have ever used the Keywords: header, or the Comments: one?

It'd be nice if we could use these fields in mails - I can immediately think of "keywords" as tags, and I'm sure I'm not alone.

I've looked at multiple "tags for mutt" systems, but all of them fall down for the same reason. I can add tags to a mail, and limit a folder to those mails that contain a given tag. But I cannot do that for multiple folders and that makes them useless :(

Has anybody worked on a multi-folder tag system for Mutt? If so pointers welcome. If not I'd be tempted to create one.

I guess implementation would be very simple. There are three caeses:

  • Adding a tag
  • Deleting a tag
  • Finding all messages with agiven tag.

The first two are easy. The second could be done by writing a cronjob to scan messages for Keyword: headers, and writing a simple index. That could then be used to populate an "~/Maildir/.tag-results" folder, via hardlinks, of all matching messages.

Better yet you could pre-populate ~/Maildir/.tag-$foo containing each message with a given tag. Then theres no searching required! (Although your cronjob would need to run often enough when the tag were added to a message it would appear there within a reasonable timeframe.

Update: I've written the indexer now. It works pretty quickly after the initial run, and is quite neat! tagging messages with mutt.

| 6 comments

 

And time keeps dragging on

2 March 2008 21:50

So, as I previously mentioned I want to be able to tag messages in Mutt.

There exist folder-based solutions already, using the X-Label header. There doesn't appear to be any existing solution allowing you to view all messages with a given tag across mailboxes.

So I wrote a simple shell script to create virtual mailboxes, such as ~/Maildir/tags-debian for all messages with a debian tag, using hardlinks.

My conclusion is that this solution will not work properly in practise, primarily because of deficiencies in mutt.

The simple case works just fine. I add a tag to a message, and later when the indexing job runs the virtual folder is created. I can open it and work on it just fine.

So where's the problem? Well in my case I tend to tag messages with a label such as "todo". Once I've done whatever I was supposed to I can remove the tag.

Using this hardlinking scheme I cannot remove the tag(s) in the virtual folder - I have to remove it in the original message which is a real pain.

Why? Well quite simply mutt will not let me work on my virtual message without destroying the hardlink.. If I use the edit function, for example, I am presented with a copy of the mail for editing - and the hardlink is replaced when that copy is saved.

Even the edit-label patch which allows you to edit the X-Label header from within mutt ends up replacing the hardlink with a new file!

So whats the solution? Well I guess I want to be able to run an external command against a message in mutt - passing the filename of the Maildir message as an argument. That way I can edit the live file.

Right now I don't believe that is possible, but I'd love to be told different.

If anybody has any solutions of editing, or even just deleting, a header from a message within mutt - in such a way that the hardlink isn't destroyed please do let me know.

Simple reproducer:

mkdir -p ~/Maildir/.foo/cur
mkdir    ~/Maildir/.foo/new
mkdir    ~/Maildir/.foo/tmp
cp 'validmessage' ~/Maildir/
ln validmessage ~/Maildir/.foo/cur

Now edit the message - start mutt open the message in the index and press 'e' - the hardlink is now gone. Replaced by a new file with the contents, so the original mail message is unchanged.

Update: I've got an "edit-inplace" primitive working, via the very hacky header-fu patch. It is not complete, but it demonstrates that it can be done. My world is now complete.

| 7 comments

 

Feeling with your skin

3 March 2008 21:50

One final post then I'm done discussing mutt. (Primarily because too many people failed to understand the problem, I guess I wasn't being as good at explaining as I could have been.)

So the goal was to be able to tag individual messages within mutt, such that a short while later a new folder would be created containing all messages of that tag, regardless of which folder the message(s) are in.

Actually finding the tags and creating the virtual folders is easy to do. I wrote a simple indexer which scans for tags in messages and creates hardlinks as necessary.

The problem with this is that many operations on those hardlinked messags would create a new copy of the message and operate on that - trashing the hardlink. For some operations that's just fine. But I did specifically want to be able to untag a tagged message and not have to hunt around for the original. (e.g. remove the "todo" tag from messages where I'd done the relevant action.)

So after a bit of trial and error I came up with a patch which allows the editing of a message in-place, in the hardlinked folder, without trashing the hardlink. This patch just invokes "$EDITINPLACE <filename>" where "filename" is the name actual file the message I'm working on is stored on disk. (i.e. ~/Maildir/.people.thewomanmeg/cur/1.2.3.txt).

By setting the EDITINPLACE variable to point to my ~/bin/editlabel script I may easily remove any tags from the hardlinked-message and have it apply to the original. Job done!

There are some limitations with this approach that are worth mentioning though:

  • IMAP? Ha! Nope. We only work on Maildirs.
  • The header-cache facility of Mutt breaks my "find the filename of the Maildir message" function. Not sure why, so I just disabled it.
  • Many operations on the virtual mailbox will still break the symlink; because they don't edit in place.

I've documented things in a rather random fashion, and I've made a backported package of mutt with my patch available online too. Primarily so I dont lose the source like I did with my mutt-ng backport.

Pointless Work

I spent about 30 minutes rebuilding mutt to use a bubble-sort on the folder list which is displayed in the sidebar.

Only when I was just about to install this patched build did I realise that the mailboxes were displayed in the order they are listed in my .muttrc file. One quick edit with Emacs and the folder list was sorted properly.

Boy is my face red..

ObRandom: I'm loving my deaf-friendly alarm clock. It rocks.

Place it under your pillow and it vibrates to wake you up. Simple. Effective, and above all it doesn't disturb my partner up when I get up to catch an early morning train.

(Thought to be fair I usually wake her up anyway; can't leave without a goodbye kiss!)

I'm still waiting for the vibrating ring, but this is good enough in the meantime.

| 2 comments

 

Cecile, this is what I like to call "quiet time"

11 March 2008 21:50

So early this evening I looked for more bugs to fix in the Debian packages I use the most, instead of doing security work. (Because I'm waiting for buildds..)

Anyway I figured I'd take a peak at the mutt-patched package - since I have my own patched backport for Etch these days. I'd like to get into the habit of making sure it stays current, but honestly I'll probably forget in a few months.

One bug #464189 caught my eye. It is asking for a couple of patches, neither of which I'd heard of. One of them is obviously extremely useful though - it is designed to change the sidebar in such a way that it only displays mailboxes with unread messages in them.

I hunted high and low for the patch and had to admit defeat.

So I wrote my own, and now my backported package contains a suitable patch.

In other news I made a new release of the chronicle blog compiler, which I'm now using in a few more places.

All in all it has been a busy day and I got a fair amount of hacking done!

The other thing, that I can speak about, which I did today was package the Perl Lingua::Identify module so that my spam filtering service can offer users the opportunity to reject mails written in, say, Russian, or Italian.

I'm tempted to give them up to Sarah for adoption, as she was making noises at the weekend about joining Debian (then again so was Jacob Appelbaum). The only issue is that I think she should use the Debian Perl group to get involved, and I've not seen any sign of activity there from her since she first mentioned joining it. Sarah: Consider this a reminder!

Update: Feel free to nominate any *crash* bugs you'd like me to attempt to fix. Providing they're not in video or audio playback code! Could be a fun challenge..

ObQuote: Cruel Intentions.

| 1 comment

 

You know how to use candles?

5 April 2008 21:50

Mutt

François Marier has recently been posting some interesting entries about the Mutt mail client upon his blog.

His tips are pretty basic, but that doesn't make them less useful. So here's my tip of the day: reply_regexp.

When you're viewing a mail, and you choose to reply it the subject of that mail is the basis of your message's subject. For example given a message with "Subject: hello" your reply will typically have the subject "Subject: Re: hello".

This is real rocket science here, people.

Imagine you're using SPAM filtering which tags messages it isn't sure about with a prefix "UNS:". Suddenly things don't look so hot, as you might end up with a mail with the subject:

Subject: UNS: Re: Hello

Reply to that mail, whilst being half-asleep, and what do you get? You get this:

Subject: Re: UNS: Re: Hello

Bad. Ugly. Wrong.

The following snippet in my ~/.muttrc file correctly deals with this case:

set reply_regexp="^(((UNS:[ \t])|[rR][eE]:[ \t])*)+"

Neat. Cool. Nice.

Advertising

I've paid for some advertising upon the LWN.net site. (No link, I'm not trying to game things here.)

I didn't know what to expect, but I was willing to risk the expenditure as a way of saying thanks to them for their great content. (Because of my Debian Project membership I get a free LWN subscription, as do all developers. see here for details if you're a developer without a subscription.)

I've paid $30 and that has given me a months run of my advert, with clickthroughs hovering around 1%. Not horribly bad at all!

I probably won't repeat the experiment for the forseeable future, but I'm glad that I did it at least once.

If you have something appropriate for a Debian-based audience don't forget you're welcome to advertise upon the Debian Administration website for free - See the Advert FAQ for details.

ObQuote: The Craft

Update: jquery was accepted yesterday. Today I uploaded a new version to more closely align it with the javascript-policy.

| No comments

 

He said he'd think about it.

28 December 2008 21:50

Back a while I created an updated mutt package which builds upon the previous work with mutt-ng.

The package which I had constructed had two new features, well if you're generous. I guess there was only technically one new feature:

  • The ability to cause the sidebar to only display folders with new mail in them.
  • + Updated navigation to make next/prev work correctly with only new mail-containing folders.

I still hate quilt with a passion, but I've synced my patches with the current unstable Mutt - and there are now source & binary packages (x86 + amd64) in my apt-get repository for Etch. I can rebuild for Lenny if there is any interest. But it might take me a while to setup lenny chroots.

As a graphical example this is what you will end up with:

OK I've tweaked the settings a little, but only a little:

# set up the sidebar, default to being visible
set sidebar_width=45
set sidebar_visible=yes
set sidebar_delim='|'
color sidebar_new red default

# ctrl-n, ctrl-p to select next, prev folder
# ctrl-o to open selected folder
bind index \CJ sidebar-scroll-down
bind index \CK sidebar-scroll-up
bind index \CP sidebar-prev
bind index \CN sidebar-next
bind pager \CO sidebar-open
bind index \CO sidebar-open

# Remap bounce-message function to "B"
bind index B bounce-message

#
#  Toggle visability of the sidebar.
#
macro index b '<enter-command>toggle sidebar_visible<enter><refresh>'
macro pager b '<enter-command>toggle sidebar_visible<enter><redraw-screen>'

#
#  Show folders with new mail only
#
macro index E '<enter-command>toggle sidebar_newmail_only<enter>'
macro pager E '<enter-command>toggle sidebar_newmail_only<enter>'

ObFilm: Joan of Arc

| No comments

 

That's really one of the saddest things I've ever heard.

8 June 2009 21:50

Today I updated the package of mutt which is stored upon my apt-get repository - the Lenny repository now contains an updated copy of mutt & mutt-ng.

This package is synced from sid and contains the addition of a small patch to update the sidebar handling so that it is possible to show only folders with new mail (Before/After)

I've talked about this patch before, and the mutt sidebar generally, so I'll not repeat myself.

Instead I will share this simple mutt tip:

#
#  Specify which mails to show when changing folder:
#
folder-hook . push '<limit>((~N|~O)!~D)|(~d<1d!~Q)<enter>'

What does this do? When changing folder it limits the display of messages to those which match either pattern:

PatternMeaning
(~N|~O)!~D)

That is "New" or "Old" messages which haven't been deleted.

(~d<1d!~Q)

Messages received in the past day which haven't been replied to.

(The first pattern could be simplified but I like to be explicit and match "N"ew and "O"ld messages directly.)

I also have the following macros setup so I can type ".a" to view all messages in the current folder, ".t" to view only messages received today, ".n" to view only new messages, and ".y" to view all messages received yesterday:

macro index .n "l~N\n"
macro index .a "l~A\n"
macro index .t "l~d<1d\n"
macro index .y "l~d<2d ~d>1d\n"

ObFilm: Dead Like Me

| 4 comments

 

New backported packages!

10 July 2010 21:50

Since I'm I'm using real titles I guess I should make a real post, in which real things are mentioned. Unfortunately recently most of my time has been spent offline, doing things in and around Edinburgh.

However I have done a few things which are possibly worthy of mention. My Lenny repository has been updated a little:

The Gimp

There's a slightly newer version of The Gimp available now, corresponding to a recent upload to unstable.

gtk-gnutella

Once again I was forced to update the backported gtk-gnutella package, as my previous one was too old to connect to the network.

itag

Finally I added a Lenny package for the itag software which is now essentially complete.

Of those things I had a lot of fun with the itag software. Partly because it allows me to horde my images in a way that I appreciate, but also because it made me go over some older images and be pleasantly suprised.

My personal archive, ~/Images, is now just over 80Gb, and goes back about ten years. (Of course the older images were taken with random point and shoot digital cameras and each images is only a few hundred k in size. The newer images, saved at full-resolution, may be 5Mb each.)

Otherwise I've been slowly deploying OpenLDAP in anger, which has been educational. I've got a minor problem to solve which is that (posix)group definitions don't seem to be working reliably, but otherwise I've got Apache authenticating against groups, SSH logins working, and the little brother database using the LDAP server as an address book. (Mail clients? mutt is the one true mail client. notmuchmail.org will be interesting when further developed, but everything else I'm going to ignore with my stubborn Yorkshire nature ;)

ObQuote: "Oh no no no, dead broad OFF THE TABLE!", from Shrek.

| No comments

 

Upgrading from Lenny to Squeeze

16 February 2011 21:50

Rather than waiting for a few months, as I typically do, I decided to be brave and upgrade my main virtual machine from Lenny to Squeeze. That host runs QPSMTPD, Apache, thttpd, and my blogspam server; nothing too complex or atypical.

The upgrade was mostly painless; I was interrupted several times by debconf asking me if I wished to replace configuration files I'd modified, but otherwise there were only two significant messages in the process:

crm114

crm114 warned me that its spam database and/or configuration files had changed and would most likely result in brokenness, post-upgrade, and I should do something to stop avoiding lost mail.

Happily this was expected.

sysv-rc

It transpired I had a couple of local init scripts which didn't have dependency information succesfully encoded into them; so I couldn't migrate to dependency-based bootup.

Given that this server gets a reboot maybe once every six months that wasn't really worth telling me about; but nevermind. No harm done.

That aside there were no major surprises; all services seemed to start normally and my use of locally-compiled backports meant that custom services largely upgraded in a clean fashion. The only exception was my patched copy of mutt which was replaced unexpectedly. That meant my lovely mutt-sidebar was horribly full of mailboxes, rather than showing only new messages. I created a hasty backported mutt package for Squeeze and made it available. (This patch a) enables the side-bar, and b) allows you to toggle between the display of all mailboxes and those with only new mail in them. It is buggy if you're using IMAP; but works for me. I would not choose to live without it.)

Now that I've had a quick scan over the machine the only other significant change was an upgrade of the mercurial revision control system, the updated templates broke my custom look & feel and also required some Apache mod_rewrite updates to allow simple clones via HTTP. (e.g. "hg clone http://asql.repository.steve.org.uk/").

So in conclusion:

  • The upgrade from Lenny to Squeeze (i386) worked well.
  • Before you begin running "iptables -I INPUT -p tcp --dport 25 -j REJECT" will avoid some potential surprises
    • There are probably other services worth neutering, but I tend to only do this for SMTP.
  • Keeping notes of updated template files will be useful if you make such system-wide changes. (e.g. hgwebdir templates)

ObQuote - "Hmm, upgrades " - The Matrix Reloaded (shudder).

| 2 comments

 

Scriptable email clients

5 September 2011 21:50

This is just a quick post to remind myself in the morning, as soon as I've made it I intend to turn my computer off and leave it off until I can re-organize my office.

I've been using mutt for my email for the past few years. Nothing compares to the flexibility of procmail/sieve for organizing server-side mail, and then mutt is ideal for reading them.

With the addition of the mutt-patched sidebar mode you can even go for a few days before realizing you're not in a graphical environment. But one thing I do long for is the ability to execute scripts at various times.

Thus far I've not actually planned what I'd like to do, but as a starting point imagine being able to execute a hook when new mail arrives? Or when you send a message matching a pattern in some fashion?

There are some things out there, such as the various hacks which are designed to abort sending a message if you mention "See attachment" in a message body but fail to add one before sendign the message. These hacks generally abuse the sendmail configuration such that they're extremely ad-hoc and hard to chain/nest.

I've mellowed out over the years and I have no interest in attempting to write a mail-client (though at the same time how hard can it be? Just restrict yourself to using inotify on ~/Maildir and offload delivery to exim and you're almost done? I guess the hard part is the UI, though I do like the mutt + sidebar layout. Write the whole thing in some scripty language?)

I'll re-examine notmuch and gnus over the next week or two, but I suspect both will continue to disappoint in various ways.

Anyway, for the moment I'm just pondering. But threading is an obvious concern. Most current mutt hooks relate to the local folder, or the local message. If I were viewing a message in one directory and a new mail notification fired for a delivery to both ~/Maildir and ~/Maildir/.people.foo I'd need to either serialise them or thread them.

Ponder ponder.

In other news I've been doing more photography recently. Nothing cohesive except for my recent experiment with shooting a "street-girl" outdoors in falling light, but that was an interesting challenge and the results were sufficient to make me want to try shooting outdoors in an organized fashion again. (Some random images have been linked to from my wee twitter page.)

ObFilm: "She doesn't get eaten by the eels at this time " - The Princess Bride

| 5 comments

 

Symbiosis is wonderful

4 March 2012 21:50

Symbiosis

Symbiosis is the collective name given to a group of Debian GNU/Linux packages which implement simple virtual hosting. It is developed by my employers Bytemark.

Symbiosis is basically a collection of configuration snippets, code, and libraries which works to offer virtual hosting in a reliable consistent and easy to understand fashion.

You implement hosting for a new domain by merely creating a directory tree. So for example you might configure the hosting for the domain example.com by running:

mkdir -p /srv/example.com/public/htdocs
echo "hello, world" >> /srv/example.com/public/htdocs/index.html

mkdir -p /srv/example.com/mailboxes/webmaster
echo "super-secret" > /srv/example.com/mailboxes/webmaster/password

mkdir -p /srv/example.com/config
echo "3l33t" > /srv/example.com/mailboxes/config/ftp-password

There you are, now http://www.example.com/ and http://example.com/ will work, and you may login to check mail with the email address webmaster@example.com via POP3, IMAP, IMAPS, or POP3S. Finally you can FTP with username example.com and be dropped into the public directory.

The mail handling is very flexible, and the webhosting supports wonderful things.

I don't generally talk about work-stuff explicitly, but we've just made a major new release of the Symbiosis system such that it works upon Squeeze and has lots of IPv6 support out of the box. (Email, DNS, HTTP, Firewalling, FTP etc.)

All in all it is simple, well-documented, and open-source with a reasonably large user-base. More external testers, users, and developers would be a wonderful thing..

Mutt Mailboxes & Idle Hooks?

Mutt is wonderful but I'm starting to get annoyed by its lack of auto-mailbox discovery.

Assuming you use procmail you might deliver mail to ~/Maildir/.foo/ and mutt won't notice that if the directory is created once it starts.

(This is because generally mailboxes are defined via "mailboxes =one =two ..", even if you use a shell snippet it won't get updated unless you re-read configuration, or re-exec mutt).

I wish it were possible to use inotify/dnotify/something magic such that everything beneath ~/Maildir would just work.

(Re-reading mailboxes manually is one solution but it is .. nasty?)

I'm thinking that of all the possible solutions one of the most potentially interesting would be to define a new hook: "idle-hook command .."

That way "command" would be executed every time the client is idle. (This is a distinct state unrelated to IMAP IDLE times.)

Nopte: There is already "mail_check" & "timeout" options. Even running a defined command immediately following the code for mail_check would be reasonable.

Reverse Proxy

I continue to use, love, and enjoy my node.js-based reverse HTTP proxy, and pub discussions seemed to suggest it is a great idea (due to flexibility) but it will never take on because people don't trust node.

I'm almost tempted to re-code it in LUA & C. But I can't help but think that would be a waste of time which would not increase adoption - after all most people use "simple" reverse proxies, and they are well suited by Apache, nginx, or even varnish.

Still no rush I suppose.

In more personal news after living in this flat for 7 years, or so, I'm getting a new bathroom designed and deployed. Good times.

In the meantime I've been steadily watching Stargate SG-1 having recently purchased a box-set of series 1-10. I've just started series six this evening, and I'm enjoying it a lot.

ObQuote: "You have been recruited by the Star League to defend the frontier against Xur and the Ko-Dan armada. " - The Last Starfighter (1984). First film I ever saw at a cinema as a child.

| 9 comments

 

Modern console mail clients?

26 April 2013 21:50

I've recently started staging upgrades from Squeeze to Wheezy. One unpleasant surprise was that the mutt-patched package available to Debian doesn't contain the "sidebar-new-only" patch.

This means I need to maintain it myself again, which I'd rather avoid. Over time I've been slowly moving to standard Debian systems, trying to not carry too many local perversions around.

Unfortunately if you've kept all your mail since 1994 you have many mailboxes. having mutt-patched available at all, with the sidebar patch, is a great timesaver. But I don't want to see mailboxes I'm never going to touch; just mailboxes with new mail in them.

Also I find the idea of having to explicitly define mailboxes a pain. Just run inotify on ~/Maildir and discover the damn things yourself. Please computer, compute!

If you divide up "mail client" into distinct steps it doesn't seem so hard:

  • Show a list of folders: all, new-mail-containing only.
  • Viewing a list of mail-messages: all in folder, or folders.
  • Compose a new mail.
  • Reply to a mail.

Obviously there is more to it than that. Sending mail? exec( sendmail ). Filtering mail? procmail/sieve/etc. Editing mail? exec(vim).

I'm sure if I were to start a core of a program, suitable for myself, would be simple enough. Maybe with lua, maybe with javascript, but with a real language at the core.

Anyway I've thought this before, and working with quilt and some ropy patches has always seemed like the way to go. Maybe it still is, but I can dream.

(PS. Sup + Notmuch both crash on my archives. I do not wish to examine them further. Still some interesting ideas. It should be possible to say "maildirs are tags; view "~/Maildir/.livejournal.2003" and ~/Maildir/.livejournal.2007 at the same time. Why just a single directory in the "index-view? So 1994.)

Disjointed posts R Us.

Obquote: "How hard could it be?" -- Patrick.

| 7 comments

 

Recycling old ideas ..

9 April 2016 21:50

My previous blog post was about fuzzing and finding segfaults in GNU Awk. At the time of this update they still remain unfixed.

Reading about a new release of mutt I've seen a lot of complaints about how it handles HTML mail, by shelling out to lynx or w3m. As I have a vested interest in console based mail-clients I wanted to have a quick check to see how dangerous that could be. After all it wasn't so long ago that I discovered that printing a fingerprint of an SSH key could be dangerous, so the idea of parsing untrusted HTML is something I could see.

In fact back in 2005 I reported that some specific HTML could crash Mozilla's firefox. Due to some ordering issues my Firefox bug was eventually reported as a duplicate, and although it seemed to qualify for the Mozilla bug-bounty and a CVE assignment I never received any actual cash. Shame. I'd have been more interested in testing the browser if I had a cheque to hang on my wall (and never cash).

Anyway full-circle. Fuzzing the w3m console-based browser resulted in a bunch of segfaults when running this:

 w3m -dump $file.html

Anyway each of the two bugs I reported were fixed in a day or two, and both involved gnarly UTF-8/encoding transformations. Many thanks to Tatsuya Kinoshita for such prompt attention and excellent debugging skills.

And lynx? Still no segfaults. I'll leave the fuzzer running over the weekend and if there are no faults found by Monday I guess I'll move on to links.

| 2 comments

 

So I've been busy.

30 June 2016 21:50

The past few days I've been working on my mail client which has resulted in a lot of improvements to drawing, display and correctness.

Since then I've been working on adding GPG-support. My naive attempt was to extract the signature, and the appropriate body-part from the message. Write them both to disk then I could validate via:

gpg --verify msg.sig msg

However that failed, and it took me a long to work out why. I downloaded the source to mutt, which can correctly verify an attached-signature, then hacked lib.c to neuter the mutt_unlink function. That left me with a bunch of files inside $TEMPFILE one of which provided the epiphany.

A message which is to be validated is indeed written out to disk, just as I would have done, as is the signature. Ignoring the signature the message is interesting:

Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Mon, 27 Jun 2016 08:08:14 +0200

...

--=20
Bob Smith

The reason I'd failed to validate my message-body was because I'd already decoded the text of the MIME-part, and I'd also lost the prefixed two lines "Content-type:.." and Content-Transfer:.... I'm currently trying to work out if it is possible to get access to the RAW MIME-part-text in GMIME.

Anyway that learning aside I've made a sleazy hack which just shells out to mimegpg, and this allows me to validate GPG signatures! That's not the solution I'd prefer, but that said it does work, and it works with inline-signed messages as well as messages with application/pgp-signature MIME-parts.

Changing the subject now. I wonder how many people read to the end anyway?

I've been in Finland for almost a year now. Recently I was looking over websites and I saw that the domain steve.fi was going to expire in a few weeks. So I started obsessively watching it. Today I claimed it.

So I'll be slowly moving things from beneath steve.org.uk to use the new home steve.fi.

I also setup a mini-portfolio/reference site at http://steve.kemp.fi/ - which was a domain I registered while I was unsure if I could get steve.fi.

Finally now is a good time to share more interesting news:

  • I've been reinstated as a Debian developer.
  • We're having a baby.
    • Interesting times.

| 7 comments