Configuring IMF Tune v5.5 Moderator/Reporting

The IMF Tune v5.5 server configuration includes the necessary interface for administering the Web Moderator/Reporting component. This document goes through the configuration elements relevant to this component.

Index

1. Initial Setup
2. Configuration Storage
3. IMF Tune Server Configuration
4. Publishing Emails to the Database Server
5. Archive/Quarantine Maintenance
   
   • Email files moved to backup will also be deleted from quarantine
   • Delete files from archive on resubmitting/deleting quarantines
   • Retain quarantine information for reporting purposes for (days)
6. The Quarantine Database Configuration Node
   
   • IMF Tune Server Registration/Un-Registration
   • Resubmit approved emails through this directory
   • Store information on all processed emails for reporting purposes
   • Process Moderator approve/delete operations every (min.)
   • Upload new quarantines to database every (min.)
7. Quarantine Users
8. User Custom Access Control Rights
   
   • Email Access Rights
   • Reporting

1. Initial Setup

In Installing IMF Tune v5.5 Moderator/Reporting we walked through the setup of the new Moderation and Reporting functionality. The setup discussed is depicted below:

Once the setup is completed, we can proceed with configuring the IMF Tune server(s) for them to start publishing emails at the database server.

The interaction between an IMF Tune server and the database starts from the registration process. This is done from the Configuration |Quarantine | Quarantine Database category and establishes a connection between an IMF Tune server and a database server.

We already discussed the Registration process in Installing IMF Tune v5.5 Moderator/Reporting. So we won't discuss this process any further here.

 

back to index

2. Configuration Storage

The web component does not include its own configuration interface. Instead, all settings are available from the IMF Tune server configuration. Thus, even if the IMF Tune server and Web interface are installed on different machines, we are still able to manage everything from a single interface.

Before delving into the specific configuration options, let's have a look at how configuration settings are being stored.

Each IMF Tune server has its own local database where most settings are stored. At the database server we store the list of Users that are allowed to access the web interface. Being centralized, all IMF Tune servers share the same User configuration.

 

back to index

3. IMF Tune Server Configuration

As already discussed the Moderator/Reporting is controlled from the IMF Tune server configuration interface. We will now go through the various setting relevant to email quarantining and reporting. The following image highlights the configuration categories relevant to our discussion.

The categories Archiving and Disk Maintenance | Archives have been renamed to Archiving/Quarantine and Disk Maintenance | Archives/Quarantine. The new names highlight the changes made to accommodate additional options for email Quarantining. Furthermore we also have two new categories:
Quarantine |Quarantine Database
Quarantine | Users

 

back to index

4. Publishing Emails to the Database Server

One of the most significant design changes introduced in IMF Tune v5.5 is the shift from email disk archiving to SQL database quarantining. Disk archiving is still 100% backwards compatible. We can easily implement archiving exactly in the same manner as we did in previous releases. However we now have the option to do more by publishing archived emails to a central database server.

The updated Archiving category, now renamed to Archiving/Quarantining, is the place where we identify which emails are to be published to the database server.

Select the Archiving/Quarantining category and click Add to open a new Archiving Profile.

This dialog is nearly identical to what we used to have in earlier versions with a single addition, the checkbox 'Also add archived email to quarantine database'

If we do not set this checkbox, we have the traditional disk archiving where emails are saved to the specified HDD location. On selecting this option, we instruct IMF Tune to also publish emails to the database server.

IMF Tune has always been able to archive all emails, both those blocked as spam and those accepted as legitimate. This feature is now inherited by the database server. At the Archive profile select the SCL configuration page:

Here we select the SCL ratings to which this Archiving profile applies. SCLs 7, 8, 9 and blacklisted are being blocked as spam. Thus we select their checkboxes so that emails with these ratings are published to the database for review.

Similarly, selecting the Whitelisted checkbox, instructs IMF Tune to also publish whitelisted emails. It really is totally up to us. We can choose to only publish emails blocked at the server, but we can also choose to publish emails that were delivered to the user mailboxes.

Note: The web interface includes a very simple switch that allows us to quickly hide accepted emails.

One reason why we might want to publish accepted emails to the database is to analyze the type of emails being assigned midrange SCL ratings, for example SCL4. It is not unusual for us to receive support questions on how to best handle emails assigned such ratings. By publishing these emails, an administrator may gain better understanding of the type of emails falling in this category and thus choose the most appropriate email handling action.

The Archiving interface should immediately highlight the tight coupling that exists between disk archiving and the database server. Indeed for an email to be published to the database a copy must also be archived to disk.

The database server is not fed with a full copy of the email. Most notably the database is only supplied with up to 32Kb of body text, no html body content and no file attachments. These potentially large pieces of data are kept at the disk archive.

Let's see what happens when a user, through the IMF Tune Web interface, releases a quarantined email for delivery. In this case, the IMF Tune server fulfills the request by fetching the email from the disk archive and submits the email for delivery.

 

back to index

5. Archive/Quarantine Maintenance

Furthering the twinning between disk archiving and the database server we now look at the changes available under the Maintenance | Archives/Quarantine category.

 

back to index

5.1 Email files moved to backup will also be deleted from quarantine

At the top of Maintenance | Archives/Quarantine we find a Warning saying:
Email files moved to backup will also be deleted from quarantine

If the Archive Maintenance option 'Compress and backup files after' is enabled, emails at the disk archive reaching the specified age limit are moved to backup. At this point, the email may no longer be resubmitted for delivery from the Web Moderator. Thus IMF Tune also deletes the emails from the database server.

 

back to index

5.2 Delete files from archive on resubmitting/deleting quarantines

At the bottom of Maintenance | Archives/Quarantine category we have a new checkbox:
Delete files from archive on resubmitting/deleting quarantines

This option determines whether archived emails should be immediately deleted from disk as soon as these are moderated from the Web interface. Consider a user resubmitting and deleting emails at the moderator. IMF Tune on fulfilling these operations removes the emails from the database. In addition it checks this setting to determine whether or not a copy of the email should be retained on disk.

Retaining emails on disk is useful in a multi-layer backup system where a predictable audit trail is required. You can choose to keep all emails on disk until the archives are backed up and later purged by disk maintenance. This renders the availability of emails archived to disk predictable. If we need to go back and dig some old email from the disk archive, it will still be available.

 

back to index

5.3 Retain quarantine information for reporting purposes for (days)

The last option at the bottom of Maintenance | Archives/Quarantine category is:
Retain quarantine information for reporting purposes for (days)

This is the age limit for email retention at the database server. Emails reaching this limit are purged completely. Note how there is no way to disable this purging and the maximum value here is 999 days.

Some might be tempted to employ the IMF Tune database as some kind of permanent email archive. However the Quarantine functionality has not been designed for this purpose. Thus we discourage employing the system in this manner.

The option reads 'Retain quarantine information for reporting purposes' because this age limit determines the number of days reporting information is retained within the database. In other words the number of days configured here will determine the span of time covered by all reports. As an example let's say we set this to 30 days. The bar chart showing the number of accepted, rerouted, rejected and deleted emails will show the totals for the last 30 days.

 

back to index

6. The Quarantine Database Configuration Node

We now move to the new Quarantine configuration nodes starting from the Quarantine Database category.

 

back to index

6.1 IMF Tune Server Registration/Un-Registration

The top most configuration elements are all related to the IMF Tune server registration/un-registration process. This is the process that connects/disconnects an IMF Tune server to/from the database.

At the top we have two read-only fields, SQL Server and Database. These identify the database against which IMF Tune is currently registered or was last registered with. On the right we have a button that will read 'Register' or 'Unregister' depending on the current IMF Tune server state. If the server is currently not registered the button will read 'Register' and vice versa. Here is the dialog that pops on clicking the button.

The registration process is discussed in detail in Installing IMF Tune v5.5 Moderator/Reporting. The un-registration dialog is identical, but performs a clean disconnection from the database server.

 

back to index

6.2 Resubmit approved emails through this directory

The 'Resubmit approved emails through this directory' field at the Quarantine Database category identifies the Exchange directory path to which emails are copied for SMTP delivery. This path is automatically initialized at installation time. However if the Exchange transport is reconfigured following the installation of IMF Tune we may need to update this field.

In Exchange 2003 the default path is that of the SMTP Virtual Server pickup directory:
<Exchange Server>\Mailroot\vsi 1\PickUp

In Exchange 2007 and Exchange 2010, IMF Tune is installed on the server running the Hub or Edge Transport role. In this case we set the path to the transport replay directory:
<Exchange Server>\TransportRoles\Replay

 

back to index

6.3 Store information on all processed emails for reporting purposes

The 'Store information on all processed emails for reporting purposes' checkbox instructs IMF Tune to publish at the database information on each processed email. This information is only used to generate reports. If we want to use the web interface for its reporting capabilities this checkbox has to be set. On activating this functionality IMF Tune immediately starts collecting information for real-time reporting.

In case we are only interested in using the Web interface as an email moderation tool, it is a good idea to disable this functionality by clearing the checkbox. IMF Tune will still generate reports however these will only take into consideration the set of emails published to the database through the Archiving profiles configured under the Archiving/Quarantining category. Thus the reports won't include the complete set of processed emails. Disabling this functionality will reduce the amount of data we load to the database server, hence reducing resource consumption.

Emails loaded to the database exclusively for reporting, will not be visible at the Web Moderator interface. This information is retained at the database until it reaches the age specified at:
Maintenance | Archives/Quarantine | Retain quarantine information for reporting purposes for (days)

On reaching the age limit the information is purged from the database.

 

back to index

6.4 Process Moderator approve/delete operations every (min.)

The Moderator Web interface allows users to Approve (resubmit) and Delete quarantined emails permanently. These tasks are fulfilled by the IMF Tune server that was responsible for publishing the email at the database. For the best performance, this processing is done in batches.

The 'Process Moderator approve/delete operations every (min.)' field specifies how often this operation is processed. Thus it determines how long it takes for an email resubmitted from the Moderator to reach the User's mailbox.

 

back to index

6.5 Upload new quarantines to database every (min.)

The publishing of new email information at the database server is also done in batches. The volume of information uploaded, depends on the Exchange server email load. If we enable 'Store information on all processed emails for reporting purposes', IMF Tune collects information on each processed email. So the volume also depends on the configuration settings.

'Upload new quarantines to database every (min.)' determines the maximum time that may elapse until an email is published at the database server.

 

back to index

7. Quarantine Users

The settings discussed so far control the interaction between an IMF Tune server and the database server. These are stored locally at the IMF Tune server machine. Installations running multiple IMF Tune servers will thus have a set of settings for each server.

We now take a look at the User configuration, a list of users stored at the database server.

Being stored at the database, this list only becomes accessible once IMF Tune is registered with the database server as discussed in Registering IMF Tune Servers with the Database Server.

Here users are granted access rights over the Web interface. To create a new user, click on Add.

This opens the Add/Edit User wizard. In the first step we specify the login credentials and the initial enablement status for the new user. If enabled, the user is allowed to login to the Web interface. Otherwise, it is blocked as if the user did not exist.

Clicking Next we move to the Rights assignment step.

IMF Tune includes a set of predefined Roles available from the top most combo box list. A role is simply a set of rights that you will most often want to assign:

Role | User

• Allowed viewing of emails addressed him
• Allowed Delete and Resubmit rights over emails addressed to him
• Denied report access

Role | User Read Only

• Allowed viewing of emails addressed to him
• Denied Delete and Resubmit rights
• Denied report access

Role | Admin (full access rights)

• Allowed viewing of all emails
• Allowed Delete and Resubmit rights over all emails
• Allowed access to all reports

Role | Admin Read Only

• Allowed viewing of all emails
• Denied Delete and Resubmit rights
• Allowed access to all reports

Reporting Only

• Denied viewing of emails
• Denied Delete and Resubmit rights
• Allowed access to all reports

Apart for the fixed Roles, it is also possible to assign a custom set of rights. To do this, from the roles list select Custom. This will enable all the other UI elements in this step, giving us full control on the rights assignments. Custom rights are discussed in detail in the section that follows.

Depending on the assigned rights, the Wizard may require us to go through another step. For example if we allow a user to only see emails addressed to him, then we need to identify the user's email addresses. On the other hand this is not necessary for an Administrator who is allowed to see all emails.

So in the former case the Next button will be enabled, leading us to the Email Addresses step:

Here we click Add for us to manually enter the user's addresses. We can enter multiple addresses separated by carriage returns. Otherwise, we can click on Import and select the user Active Directory object to import the address from there.

It is good to appreciate that by adding addresses we are granting the user rights over those emails where the address appears in the recipient list. If we wanted, we could allow a user to manage emails for other users. All we have to do is to include the addresses for the other users here as well.

This completes the wizard. Just click Finish for the new user to be added to the list.

If we need to change any settings, including resetting the password, click edit to launch the wizard again.

Once ready, click Apply at the top of the Configuration for all settings to be persisted. At this point the IMF Tune configuration will connect to the database server and save all changes in the user list.

 

back to index

8. User Custom Access Control Rights

As discussed in the Quarantine Users section of this document, on configuring users, it is possible to assign a custom set of rights instead of the predefined roles. We now take a closer look at the options we have when assigning custom rights.

The above image shows side-by-side the user rights selection and the web interface. The top most set of rights, controls access to the Web Moderator email list. The last two checkboxes control access to the General Spam Detection Report and the Detailed Spam Detection Report.

 

back to index

8.1 Email Access Rights

Email Viewing | View all emails - The user is granted the right to view all emails published at the database. We have to be careful with this right, especially when publishing emails having low SCL ratings, Whitelisted or Unprocessed. Quite obviously this right can easily grant access to confidential emails.

Email Viewing | View own emails only - The user is only allowed to see emails addressed to him. This right is combined with the list of email addresses configured on creating the User. The addresses will identify the emails owned by the user.

Message processing details viewing | View message processing details - For each published email, IMF Tune includes a lot of additional information. As an example here is an email as displayed at the moderator:

The email information is organized into a number of pages. The View message processing details right, grants access to the Keywords Report and IMF Tune Processing pages shown above. It also allows viewing the full recipients list under the SMTP Protocol page, include any BCC recipients.

Email delete rights - This set of rights controls the ability to delete emails from the Moderator. Here we have four options:

• Delete not allowed - Block user from deleting any emails. We may want to centralize the deletion of quarantined items, leaving it up to the Administrator. Users would then be granted read-only rights.

• Delete any email - Allow user to delete any email over which he has viewing rights as discussed in View all emails and View own emails only.

• Delete email if user is the only recipient - An email may of course have been sent to multiple recipients. In this case the Moderator only holds one copy. If deleted, it would be deleted for all recipients. With this right, a user is only allowed to delete if he is the only recipient.

• Delete email if user is one of the recipients - This right allows the user to delete emails that include multiple recipients as long as he is in the recipient list.

Note how it is also possible to grant a user the right to view all emails (with Email Viewing | View all emails) but only allow him to delete his own emails.

Email re-submit rights - the set of rights available to control resubmitting emails, is identical to that discussed in Email delete rights. However here we are controlling the ability to have an email delivered to the original set of recipients. Note that, when resubmitting an email, this is delivered to all addresses in the original email recipient list.

 

back to index

8.2 Reporting

Access to reporting is controlled by granting/denying the rights:
View general spam detection report
View detailed spam detection report

The general report includes a set of graphs showing the current filtering performance. Being generic, this information is usually not considered to be very sensitive. However this type of reporting is not very relevant to the casual user.

The detailed report includes lists of email addresses, IPs and other more specific information on the email traffic flowing through the server. The information here is more sensitive.

 

back to index