Verify IMF Tune with Keyword Reporting - Part 1

Most often filtering software is allowed to quietly do its job. However, when troubleshooting, we need to look under the hood and answer questions of the type: Is this blacklist rejecting any emails? Are there any weak whitelists causing spam to bypass filtering? Why isn't an email being blocked despite having rules that should block it? Keyword Reporting is where we seek answers to such questions.

Where is the Keyword Report?

Keyword Reporting is so useful that we provide the same information twice:

1. Standalone Keyword Reporting - This is enabled from the IMF Tune configuration, Keyword Reporting category. There isn't much to configure beyond turning it on/off.

   

   

   IMF Tune produces one flat HTML file containing multiple reports. Each report shows information on a different email. The HTML file can be a bit hard to read. Containing reports on a large number of emails, we need to search the file and identify the relevant information.

2. IMF Tune Moderator Interface - An optional ASP.NET application providing access to the emails IMF Tune processes. At the moderator each email includes the Keyword Report relevant to it.

   

   Searching emails from the Moderator is easy. Hence it is easier to get to the relevant Keyword Report from the Moderator than from the Standalone Keyword Report. The downside here is that the Moderator is a separate installation and introduces its own installation requirements. So running the Moderator involves extra setup work.

In this article we will work with the Standalone Keyword Report since it is immediately available as soon as IMF Tune is installed.

IMF Tune Basics

Before looking at the Keyword Report, we need to arm ourselves with some basics on how IMF Tune works. With this knowledge the Keyword Report will be easy to interpret.

Initial and Final SCL Ratings - IMF Tune works in tandem with one of these Microsoft anti-spam filters:

• Intelligent Message Filter (Exchange 2003)
• Content Filter Agent (Exchange 2007 and later)
• Forefront Protection 2010 for Exchange

An email is first scanned by one of the Microsoft anti-spam filters. These assign an initial SCL rating to the email. IMF Tune process the email next and depending on the scanning result it may retain or change the initial SCL.

The final SCL rating is the result of combining the initial rating and the IMF Tune processing result. The final rating will be the same as the initial rating if no other filter within IMF Tune changes it.

The final SCL rating determines whether IMF Tune accepts or blocks an email.

Prioritizing Multiple Matches - IMF Tune is made up of multiple filters. Each of these could match an email and change the SCL rating.

But what happens when more than one filter matches the same email? For example an email could match both a whitelist and a blacklist. In these cases IMF Tune applies its own priority rules as explained in the FAQs.

The rule of thumb is that IMF Tune always gives priority to the lowest SCL rating. So an email matching both a whitelist and a blacklist gets whitelisted.

Apart for whitelisting and blacklisting, some filters can be configured to assign any SCL value from 0 to 9. Here is an example of a Simple SCL Rule configured to set the SCL to 8:

Applying the same priority logic, whitelisting takes the highest priority next comes SCL 0, SCL 1, ... SCL 8, SCL 9, and blacklisting which comes last.

IMF Tune also provides the option for filters to push up/down the current SCL value. Instead of specifying a value to replace the SCL, the filter specifies an increment/decrement. Here is a Simple SCL Rule configured to increment the SCL by 4:

These SCL changes take the lowest priority of all. The change is only applied if the email does not match any other filter configured to replace the initial SCL.

Last thing to consider is the case when an email matches multiple filters, each contributing an increment/decrement. For example an email with an initial SCL of 4 matches two rules:

• Rule 1 applies an increment of 3
• Rule 2 applies a decrement of 1

Final SCL = 4 + 3 - 1 = 6

For more on prioritizing multiple matches check Setting up a Low Priority Whitelist in IMF Tune.

What goes into the Standalone Keyword Report?

The Keyword Report does not log information on each and every email. This is an important detail that sometimes causes administrators to incorrectly believe that some emails are not being processed.

Keyword Reporting only reports on emails that triggered some action by one of the IMF Tune filters. Just because an email does not show up here does not mean that it went through unfiltered. Keep in mind that IMF Tune may choose to retain the Initial SCL rating assigned by one of the Microsoft spam-filters.

End of Part 1

We conclude the first part of this article here. Today we covered the basics necessary to effectively read the Keyword Report. In the next part we will see how to use this information to verify and troubleshoot IMF Tune.