About Archive Tags RSS Feed


They look like big, good, strong hands, don't they?

22 December 2008 21:50

Russ Allbery recently commented that it is really nice to receive patches for trivial scripts posted online.

I agree.

More than once I've posted a trivial script and had it be improved by people, or later included elsewhere.

So in the spirit of sharing here is my latest toy script:

This is a trivial script which searches a Maildir hierarchy and outputs a list of each email address which you've ever sent mail to.

Why would you want that? In my case my (personal) spam filtering makes use of whitelisting, and the assumption is that if I've ever mailed you in the past then I want to see your replies, and you get a break.

These days my (personal) mail filtering has a couple of broad rules:

  • If your mail is HTML it is junk. Unless I'm bored.
  • If your mail is GPG signed/encrypted I will see it.
  • If your mail address is on my whitelist then I want to see it.

After that then I see your message only if CRM119 decides I should.

# remove potentially spoofed header
:0 fhw
* ^X-whitelist:
| $FORMAIL -I "X-whitelist"

#  GPG-signed messages are OK and will be whitelisted
* < 1024000

| $FORMAIL -A "X-whitelist: yes" -A "X-GPG-Signed: Yes"

#  Get the sender of the message.
FROM=`formail -x From:| sed 's/^\([^@]*[ <]\)//' | sed 's/\([ >]\).*$//'`

# Add a whitelist tag if appropriate
:0 fhw
* !^X-whitelist: yes
* ? test -s $HOME/.procmail_whitelist
* ? echo $FROM| fgrep -qisf $HOME/.procmail_whitelist
| $FORMAIL -A "X-whitelist: yes" -A "X-Whitelist-Test: $FROM"

The net result of these tests is that I can now run the spam filter on non-whitelisted mails:

# Run CRM114 mailreaver
:0fw: .msgid.lock
* !^X-whitelist: yes
| /usr/bin/crm -u /home/steve/.crm /usr/share/crm114/mailreaver.crm

#  Spam.
* ^X-CRM114-Status: SPAM.*
* !^X-whitelist: yes

#  Unsure.
* ^X-CRM114-Status:.*UNSURE
* !^X-whitelist: yes

There is more to my setup than that, but that's the minimum you'd need to see.

Of course this is a reminder, once more, that the kind of filtering that you carry out for yourself is different from that that other people will do.

ObFilm: The NeverEnding Story



Comments on this entry

icon Alex at 18:20 on 22 December 2008
So what happens when spammers start GPG signing their mail? :P
icon Steve Kemp at 18:30 on 22 December 2008

If a few do it would suck.

If all spammers do then we start to be in a position to exploit the web of trust...

icon Thomas Bliesener at 21:02 on 22 December 2008
# Extract mail addresses for auto whitelist
# Thomas Bliesener <bli@melix.com.mx>
# 2008-09-23


# Extract from log
sed  -n '/status=sent/ s/^.*to=<\([[:alnum:]@.-]*\)>.*$/\1/p' \
/var/log/mail.log | sort -f | uniq -i | sed 's/$/\tok/' > $list_tmp

# Strip manually whitelisted domains
sed -i '/gmail.com/d' $list_tmp
sed -i '/hotmail.com/d' $list_tmp
sed -i '/yahoo.com.mx/d' $list_tmp

# Convert to lower case
tr 'A-Z' 'a-z' < $list_tmp >> $list

# Add to list
sort -f < $list | uniq -i > $list_tmp
mv $list_tmp $list
/usr/sbin/postmap $list