Advanced Email Filtering to Control Spam

William Yang has been working on spam control methodologies since his first security publication, "Using Internet Standards to Control the Cost of Spam" in 1997. It's a serious problem, and it's received some very advanced and serious thought in the now more than two decades that he's been looking at ways to deal with junk email.


People sending junk email are going to be successful in bypassing any current or forseeable technolgy used to identify whether something is "okay" or not." The economic incentives and the asymmetry of effort are going to be difficult to overcome. Fundamentally, the paradigm we apply to fighting spam is just plain wrong.


Taking a page out of the information security playbook, there are two approaches when making a decision about something is permitted or not: "default permit" and "default deny". Any strategy based on "default permit" creates a massively asymmetric situation, where defenders have to expend more energy than attackers. Default permit stances (articulating what you don't want to receive, and accepting everything else) just don't work that well.


On WDY Enterprises systems, we believe that the most *effective* manner of stopping spam is to use the authorized sender feature, and list the email addresses (or domains, if the domain name is trustworthy) you want to receive email from. There are some real changes to how your mail works that fall out of doing this, so before you visit https://wdyllc.com/preferences/ and start making changes, really consider if the trade-off of slower legitimate email, having to watch a quarantine for legitimate messages, and is an okay trade-off for less spam in your inbox.


If you can accept the consequences, then the principle is pretty simple.

  • You maintain your list of email addresses (and domain names) that you DO want to correspond with. This "permit list" gives you the ability to ensure that the people you email with regularly can reach you.
  • You also can optionally maintain a list of email addresses (and domain names) that you DO NOT want to correspond with. The "block list" is reserved for people who really are seriously abusive -- day-to-day spammers change their email addresses with almost every outgoing message, so that it's not worth trying to listing them. The system does not accept email messages from an "block" listed sender by rejecting them before the message is received (this is an email technical protocol trick). It only works if you can identify a "bad" sender.
  • Any sender not on a permit or block list gets put into quarantine, and you get an emailed quarantine notification no more frequently than once every 15 minutes, with a description of the message (sender, date/time, subject, size, and clickable links (mobile friendly!).

Your workflow, if you use it, looks like this:

  • SETUP (all done at https://wdyllc.com/prefs).
    • Set up a list of email addresses or domain names you want to receive mail from.
    • Optionally, set up a list of email addresses and domain names you do not want to receive mail from.
    • enable the advanced filtering function "require authorized senders"
    • other recommended settings:
      • set the system to remove quarantined messages after some period (30, 60, 90, or 120 days)
      • turn off spamassassin filtering
     
  • Day-to-day:
    • Receive emailed quarantine notices (which gives you NEW quarantined messages only) and review the messages. Use the interface in the email message to release quarantined messages, add a sender to your permit list, or view the message.
    • Modify permit and block lists as needed.

All of this is still a little rough around the edges and isn't what anyone would call pretty, but it's effective. That may give users a leg up on what you're trying to manage.


This was part of a message William Yang sent to users requesting information about the blocking service:


I've been doing this doing this in one of my business accounts since
late 2014.  I went from 2500 spam messages a day to zero except for
when I make a mistake and releases a message accidentally.  It's
consistently stopped better than 99% of incoming spam.  Of course, I
still know I'm getting it, and I have to read subject lines and the
like still, but it's dramatically more effective in terms of managing
the time commitment that spam imposes.

The settings I use (and thus can recommend, but only if you're willing to
maintain your safe and unsafe lists):

Permit list:
	(list of email addresses that you regularly correspond with
Block list:
	(list of email addresses who you actively don't want to
         correspond with -- note that you do not have to put anything here
         and will still get the benefits)
Advanced spam filtering functions:  PICK ONE:
	* Use Greylisting, quarantine service errors and require
	  authorized senders
Quarantine incoming messages as spam: NEVER
Delete incoming messages as spam: NEVER
When should quarantined messages be removed by the system: 120 days (or less)
If malware is detected: quarantine in INBOX.spam.virus-quarantine.

Of course, your mileage may vary. There are tradeoffs involved in using this approach. Understand what you can accept before you start changing your settings.