Entries tagged anti-spam

Fighting Spam

Lots of people get spam and don't want to fight it themselves.

Many of those people elect to use a service such as Google Mail, which does an adequate job of filtering most of the time.

For people who wish to host their own mailservers, and off-load their anti-spam support to another entity, there are big companies such as MessageLabs.

But if you can't/won't pay for somebody else how would you handle it?

Right now I run my own mailserver and mostly it rocks.

However I also manage a couple of servers for other people, and this mostly means that I have identical configuration(s) scattered across other machines. (Or as I recently discovered "mostly identical" setups scattered across N hosts.)

After viewing a recent thread upon the forums of my employers I'm wondering how hard it would be to setup an anti-spam mail host.

The idea being that users of domain example.org would continue to run their own mailserver (+optional POP/IMAP server). However this server wouldn't be listed in an MX record.

Instead the MX record would point at the antispam server, which would then process and forward the mail on. (I'm assuming that the mailserver at example.org would be firewalled such that it would only accept SMTP connections from the anti-spam host).

The only challenges I see would be:

• The anti-spam server knowing where to forward mail for domain X - since there would be no MX record. Maybe a A record 'incoming.example.org'; maybe a hardwired list.
• Having enough resources on the anti-spam server.
• The pain that would ensue if legitimate mail were dropped.

The last one is the killer point which makes me wonder if I'm wasting my time on the general case; I don't believe I could persuade people to pay for such a service (on the grounds that I wouldn't!) and if ever a mail were dropped I'd be the first person who would be blamed. And probably rightly so.

The simple solution would be to always accept mail - but just tag it. Then you still have the same issue, but you have the excuse that "I still delivered your mail; just tagged wrongly". The downside of that is that the recipient still has to filter the mail; and still eats the bandwidth cost. The main gain is the lack of spam-processing CPU that is eaten locally.

Anti-spam, and to a lesser extent anti-virus, topics are very interesting to me, and I'd love to hear any thoughts - or even what you use for yourself.

Right now I'm 50/50 on setting up my main mailserver as the proxy for a few domains I have to manage (family, friends) to see how it works, but I'm not quite convinced yet. (Single point of failure == bad. Though centralised configuration == good.)

Tags: anti-spam, anti-virus | No comments

Tonight I mostly installed Centos 5 upon dedicated servers in America, for a guy based in Germany. Sometimes people do pay me for work which is nice - this job paid for a small stirling engine :)

Yesterday I mostly implemented a web interface for controlling spam via a proxy server.

Several people seemed interested, so I'll explain what I've done:

• Coded several qpsmtpd plugins for different anti-spam mechanisms on a per-domain basis.
  • Because the stock server only supports global plugins.
• Coded a web interface for users to manage the settings on their domain:
  • Virus scanning.
  • Greylisting.
  • DNSBL.
  • Resolvable from hosts.
  • Reverse DNS

The only part that is missing is the final plugin integration, and some glue to propogate some of the GUI settings to the mail server - since adding a new domain is currently 50% GUI and 50% manual.

Each of the plugins supports whitelisting of addresses or domains, and there is a fair amount of logging detail - some is missing which I need to resolve.

Once that is done and the code is running upon my live server I'm happy to let people use it if they have a spare/non-critical domain they can expirement with...

The next thing to do would be to hook in spam assassin, spambayes, etc, on a per-domain basis with "training" addresses..

Tags: anti-spam, qpsmtpd | No comments

Recently the topic of spam on the Debian lists was revisited. I laugh at somebody who recieves 200 spam messages a day.

Here's my stats for yesterday:

                                          Total Mails    : 6399
                                          Total SPAM     : 6077
                                          Total Accepted : 322

                                          Spam Percentage: 94.97%

That's 6077 mails rejected at SMTP time via my filters, and only 322 mails accepted.

The breakdown of the spam rejected looks like this:

                                  Plugin      Count
--------------------------------------------------------------
                                   dnsbl       3755
                             hosts_allow        724
                             greylisting        661
                       check_earlytalker        303
                          check_spamhelo        238
             require_resolvable_fromhost        219
                           virus::clamav         79
                         check_badrcptto         75
                       check_badmailfrom         23
--------------------------------------------------------------

Tags: anti-spam, anti-virus, spam | No comments

I think I'm about ready to start running my SMTP proxy/filter for people now. I've got two users setup and it seems to be holding up just fine. I've managed to remove the need to manually create things, so I'm capable of adding new domains on the fly.

For my personal mail handling yesterday was the first day I've received over 10,000 messages. These are the stats from yesterday showing what happens at SMTP-time:

Total Mails    : 13302
Total SPAM     : 12620 (bounced back)
Total Accepted : 682
Spam Percentage: 94.87%

So I've dropped 95% of incoming mail at SMTP time, which is a big win. Of the mail which was actually delivered I'd guess that pyzor, razor, and spambayes marked a further 50% of it as spam.

So my proxy needs to do better ... but it's pretty impressive.

TODO: Find people to run as guinea pigs, register a domain, profit.

Tags: anti-spam, anti-virus | No comments

Friday Morning I ordered an 80Gb iPod classic.

Friday Afternoon I discover Apple Cuts Off Linux iPod Users

Saturday / Sunday I follow along on IRC the reverse engineering effort

Progress is being made, but still a little bit away from having a result. In the meantime I'm ripping Megan's cds.

In other news Debian Security Advisories are being held up by missing buildd machines for several archs.

Previously I supported the position that Debian should release for 14+ plus platforms, but given the regular frustrations and delays I'm now of the opinion that we should only release for platforms that have at least two buildds - one for security, and one for regular use.

(Benefit of two? If one dies we can both use the other.)

Finally I'm still doing more SMTP work, and it is going extremely well. The next job is to wire up a payment gateway which is tedious and dull.

Update - Looks like the ipod issue is fixed. kudos to wtbw + nopcode .. and everybody else who put in the work.

Tags: anti-spam, anti-virus, buildd, dsa, ipod | No comments

xen-tools has just got a new command:

xen-resize-guest --hostname=foo.my.flat --increase=5Gb

That will take the existing guest foo.my.flat and magically resize the main disk image to be 5Gb larger. This works for LVM & loopback images, but not yet for users of EVMS. Whilst doing this job manually isn't terribly difficult it can be troublesome to perform all the steps in order without screwing up. Hence the new command.

In other news I've managed to fix my broken greylisting implementation - so all the mails which were previously being queued upon klecker.debian.org should now be delivered/bounced.

I believe the blame here was 50/50 me and exim's back-off behaviour, but I'll know better in the future.

Remember me when you come to choose your next anti-spam service. It copes beautifully with a sustained delivery rate of 300-700 messages a minute when queues suddenly restart delivery ;)

TODO: Catchup on mail. Implement message tagging for mutt to better keep track of items which are pending/claimed by me. Until we get RT.

Tags: anti-spam, xen-tools | No comments

This week has mostly involved me getting my live mail filtering site up and running with a guineapig or two.

This uses a custom user interface to allow users to manage the filtering settings for an entire domain:

• spam filtering.
• virus scanning.
• greylisting.
• sender/recipient whitelisting.
• DNS-based blacklists

In terms of implementation this is an SMTP proxy which is built upon the qpsmtpd framework. I've got both the user interface and a collection of plugins reading all data from an MySQL database.

The practical upshot is that if you use the service you'll get less spam, and anything that has been rejected will appear in an online browsable quarantine for a period of times allowing you to view mistakes/rejected mails.

Any mail you didn't want, providing you've got the spam-filtering plugin enabled for your domain, you may send back to be trained as spam.

It scales nicely, doesn't appear to have lost any mail ever in real-world testing, and could be useful.

Tags: anti-spam, anti-virus, mail-scanning | No comments