Email Archiving with MailStore Basics

A checked version of this page, approved on 21 June 2023, was based on this revision.

Working with Archiving Profiles

In MailStore Server, every archiving task is stored as an archiving profile. By executing such a profile the archiving process is started.

Such an archiving profile could contain the following information:

WHAT: Mailbox [email protected]
WHENCE: Exchange server EXCHANGE01
WHERE (Target archive in MailStore Server): peter.berten
SCOPE: All folders except drafts
DELETION RULE: Delete all archived emails from the Microsoft Exchange mailbox that are older than 3 months.

Creating an Archiving Profile

To create a new archiving profile proceed as follows:

Start the MailStore Client and click on Archive Email to create and execute an archiving profile.
In the Create Profile section of the application window, select the source from which the emails should be archived (e.g. Microsoft Outlook). A wizard opens.
During the first steps of the wizard, several different settings can be specified for the archiving profile. These include the selection of folders (e.g. "Inbox") or deletion rules (by default, no emails are deleted).
If you are logged on as MailStore Server administrator, you will be prompted for the target archive (user archive), in which the emails should be stored. If logged on as a regular user, the corresponding user archive will be used automatically.
In the last step, a name for the archiving profile can be specified. Archiving profiles of type E-mail Server provide an option to configure the automatic execution from here as well. After clicking Finish, the archiving profile will be listed under Saved Profiles and can be run immediately, if desired.

Please note: Only MailStore Server administrators can archive emails into a target archive other than their own. This is not possible for regular MailStore Server users.

Manual Start of Archiving Process

As long as the automatic execution of an archiving profile has not already been configured (type E-mail Servers only) during its creation, you can manually execute archiving profiles at any time. It is recommended to always start the initial archiving process manually in order to identify possible configuration issues (i.e. incorrect credentials) as early as possible.

To manually start the archiving process, select the archiving profile to be executed from the list under Saved Profiles and click on Run.

A progress windows appears. After the archiving process has been completed you can get further information about its course as well as possible errors by clicking on Details.... A debug log is also available here.

In addition to being executed manually, archiving profiles can also be run automatically according to a schedule. Additional information about this topic is available in the chapter Automating the Archiving Process.

Executing Archiving Profiles Multiple Times

Any archiving profile can be run any number of times without concern. MailStore Server only archives emails that are not yet stored in the target archive. MailStore Server also detects if any emails were moved to a different folder within the source application (e.g. Microsoft Outlook) and reproduces such moves in its archive accordingly. Already archived emails, that have been moved within the source from a folder that is categorized by MailStore as a folder containing sent messages, to a folder that is categorized by MailStore as a folder not containing sent messages, will be archived again. This also applies vice versa.

Editing Archiving Profiles

To edit an existing archiving profile, proceed as follows:

Select the archiving profile you want to edit from the list Saved Profiles and click on Properties.
Navigate between the profile's pages with Next and Back and change the settings as desired.
Save the new settings on the last page by clicking OK.
By clicking Cancel you can quit editing the archiving profile at any time without committing any changes.

Deleting Archiving Profiles

If no longer needed, you can delete an existing archiving profile.

Right-click on the archiving profile you want to delete or click on Commands and select Delete
A confirmation prompt appears. Acknowledge by clicking OK.

Please note: Deleting an archiving profile has no effect on emails that have already been archived. They remain in the archive.

Changing Owner of Archiving Profiles

As MailStore Server administrator (admin) you can change the owner of an archiving profile. That way you can prepare archiving profiles for your users which they can execute on their own afterwards.

Right-click on the profile you want to change and select Change Owner....
Choose the profile's new owner in the next dialog.
Confirm the selection by clicking on OK.

Note on foreign archives: If you as a MailStore Server administrator have selected a target archive that does not belong to the new profile owner, the execution of that profile by its new owner will fail if she is not a MailStore Server administrator herself.
Only MailStore Server administrators can archive emails into a target archive other than their own. This is not possible for regular MailStore Server users.

Note on logon privileges: If you revoke the logon privileges of the user to whom you assigned the profile, the profile will not be able to run.

Managing and Executing the Archiving Profiles of Other Users

As MailStore Server administrator (admin) you can manage the archiving profiles created by other users as well.

In the section Saved Profiles activate the option Show Profiles of All Users.
You can now edit, execute or delete these profiles just as your own.

Special Case: Executing the archiving profiles for email clients for other users

The execution of archiving profiles for email clients such as Microsoft Outlook or Mozilla Thunderbird usually makes only sense on the PC where the corresponding email client is installed because that client stores emails and data necessary for accessing them locally on that PC.

Example: If user A has an archiving profile for Microsoft Outlook, you can view and edit it as a MailStore Server administrator through your MailStore Client. However, executing this profile only makes sense if it is run using the MailStore Client that is installed on user A's computer because only there can the locally stored data be accessed.

This information only applies to archiving profiles listed in the E-mail Clients section.

Archiving Specific Folders

By default MailStore Server archives the emails from all folders of the email application or email mailbox except the following (the list may vary depending on the system):

Deleted Items - Emails that were deleted by the user
Drafts - Emails that are still being edited and that have not yet been sent
Junk Email - Emails that have been classified as spam
Outbox - Emails that are about to be sent and will be listed under Sent Items after sending

The standard settings can be customized; for every archiving profile, folders can be included or excluded as needed.

Please notice: Take care of upper and lower case when manually specifying folder names for inclusion or exclusion. These declarations are case-sensitive.

General Procedure

You can select the folders to be archived directly upon creating a new archiving profile. Alternatively, you can adjust the selection for already existing profiles. Additional information about creating and editing archiving profiles is available in the chapter Working with Archiving Profiles. The following screenshot shows how to access the folder settings using a Microsoft Outlook archiving profile as an example:

Selecting Folders for Archiving

You can specify the folders that MailStore Server should archive. In this case, MailStore Server limits archiving to the selected folders and ignores all others. Archiving profiles that archive multiple mailboxes do not have the Add button. You have to enter the folders to include manually.

In the folder selection under Include Folders and Subfolders, click on Add.

Select a folder.
With the Include Subfolders option you can specify if subfolders of the selected folder should be included in the archiving process as well.
Click on OK to add the folder to the list.

Example: You have created a folder Archive in the email application or the mailbox, into which the user moves all emails that should be archived. In this case, add the folder Archive to the inclusion list. That way, only this single folder will be archived.

Excluding Folders from the Archiving Process

MailStore Server can be configured to always exclude certain folders from the archiving process. Archiving profiles that archive multiple mailboxes do not have the Add button. You have to enter the folders to exclude manually.

In the folder selection in the Exclude Folders and Subfolders section, click on Add.

Select a folder.
With the Include Subfolders option you can specify if subfolders of the selected folder should be excluded from the archiving process as well.
Click on OK to add the folder to the list.

Example 1: Emails that are still in the Inbox should not be archived because they will either be deleted or moved to a project folder by the user. In this case, add the Inbox folder to the list of folders to be excluded.

Example 2: Emails that are in a folder named Private should not be archived as it may contain personal information that is prohibited by law (in some countries) to be archived.

Please note: Depending on the system, some folders (e.g. Deleted Items) are always excluded from the archiving process. To deactivate this automatic exclusion, clear the checkbox Always exclude <folder names>.

Using Regular Expressions to Include or Exclude Folders

In MailStore Server you can include or exclude folders using regular expressions. The syntax follows the Specification of Regular Expressions in the .NET Framework 4.5.1. To specify a regular expression simply prefix it with || or |i| (using case sensitivity/insensitivity).

Examples
The regular expression "||Folder\d/Folder\d\.\d" would for example include Folder1/Folder1.1 in the archiving.

The regular expression "||Folder\d/Folder\d\.\d/\w+" excludes further subfolders in the structure above from being archived. The result could look like this:

Defining Sent Items Folders

The message list in the archive displays the recipient on sent messages and the sender on received messages. Therefore a user always sees his communication partner in the message list.

When archiving user mailboxes, sent messages are identified through their containing folder. In case of Exchange- and Outlook archiving profiles, MailStore is able to detect the sent items folder automatically and it cannot be adjusted. When archiving from IMAP servers that implement the SPECIAL-USE-capability, MailStore is also able to detect the sent items folder if the option is enabled in the archiving profile. Since IMAP clients may not support that capability, these clients may have created their own sent items folder, with the result of multiple folders that hold send items. For MailStore to detect all sent items, the list of sent items folders can be extended in IMAP archiving profiles.

In case the wrong communication partner is displayed in the message list, the message type can be changed by an administrator in the MailStore Client by right clicking on the message and selecting Change Message Type.

Archive access on other users' messages has to be unblocked in case the message type of other users should be changed. The archiving profile should configured that manually changing the message type afterwards should not be necessary.

Showing the Folders Selected for Archiving

In the folder selection, click on Show Effective Folders to verify the selected settings. Hereby all folders that will effectively be archived are displayed. Folders displayed in gray will not be archived. In IMAP archiving profiles, folders that contain sent messages will have a small envelope icon attached. Archiving profiles that archive multiple mailboxes do not support this functionality.

Specifying Filter Criteria for Archiving

In MailStore Server, you can limit archiving to certain emails for most archiving profiles by specifying filter criteria. The following filter options are available:

Also archive unread messages
By default, MailStore Server archives emails without regard to their read/unread status. Deactivating this option limits archiving to emails with status read.
Only archive messages that are older than
- Date
  Here you can specify a particular date to which MailStore Server compares the send or receive date of the emails to be archived. Only emails that have been sent or received before this date will be archived. The UTC option defines whether this date is specified in Coordinated Universal Time (UTC) or takes the local time zone of the MailStore Server into account.
- Range
  With this option MailStore Server only archives emails that have been sent or received before the beginning of this range, measured from the date of archiving.

Deleting Emails after Archiving

MailStore Server can be used to reduce and to keep the amount of emails stored in a source system at a minimum. Source system can be email servers, email mailboxes, email applications as well as email files or email container files. Please notice that deletion of emails is not available in some archiving profiles due to missing interfaces of the source system.

The delete feature that has to be activated explicitly enables MailStore Server to delete emails from the source system autonomously, according to specified rules and time periods.

Please note: In order to not jeopardize ongoing business processes, emails that have been flagged will never be deleted by MailStore Server. Configured deletion rules do not apply to such emails. To get them deleted by MailStore Server you have to un-flag them in the source system directly.

Functionality

By default, MailStore Server only copies emails from the source system into its archive. The archived emails remain in the source system and will not be deleted.

The deletion process will be executed after the archiving of all emails by the archiving profile has been finished. During this process it is explicitly checked whether any email that is about to be deleted has really been archived. This ensures that only emails will be deleted from the source system that have been written into the archive before.

Please note: Emails can only be deleted from a source system after an archiving process, which means that such deletions are always tied to an archiving profile. Furthermore, deletion rules have to be configured explicitly in such a profile as described in further detail below.

Beyond this functionality, no other automatic processes exist to delete emails from their source systems.

Configuration

Delete conditions can be configured during setting up a new archiving profile or for existing profiles under Advanced Settings > Delete.

The following options are available:

never
Emails are never deleted from the source system. In effect, emails will be copied into the MailStore Server archive. This is the default setting.
immediately after archiving
Emails are deleted from the source system as soon as they have been successfully archived. In effect, emails will be moved into the MailStore Server archive.
at least [period] after archiving
Emails are deleted from the source if they have been in the MailStore Server archive for at least the given period of time. In effect, emails will be copied into the MailStore Server archive at first and also remain in the source system for the given period, before they will be deleted in a subsequent run of the archiving profile.
if message older than [period]
Emails are deleted from the source system if their send or receive dates predate the given period of time. In effect, emails with send or receive dates in the given range will be copied into the MailStore Server archive at first, while those with send or receive dates predating the given range will be moved into the MailStore Server archive. During subsequent runs of the archiving profiles, previously archived emails will be deleted from the source systems if they match the defined age criterion.

Please note: If a filter date or range is configured together with a deletion rule if message older than, the longer period of time will be applied to the deletion rule as well.
If, for example, the filter range is set to Only archive messages that are older than 1 month and the deletion rule is set to if message older than 1 day, only emails will be deleted that are older than one month because younger emails have not been archived yet.

With MailStore Server, it is not possible to delete emails from source systems that have not been previously archived.

Automating the Archiving Process

With MailStore, every archiving profile can be executed automatically. MailStore Server distinguishes mainly between two type of automatic executions. For archiving profiles from the category E-mail Server, a continuous server-side execution by MailStore Server is available, whereas profiles from the category E-mail Clients and E-mail Files are executed solely on the clients by means of the Windows Task Scheduler.

Important notice: The time-controlled execution of archiving processes cannot prevent users from deleting emails from their email applications or mailboxes prior to having been archived. In order to ensure that all emails are archived they should be automatically archived upon sending and receiving.

By deleting a scheduled task (through MailStore's Administrative Tools or the Windows Task Manager) neither the underlying archiving profile nor the emails already archived by the task are deleted.

Automating Execution of E-mail Server Archiving Profiles

If the automatic execution was not already configured while creating a new archiving profile, follow the below instructions to activate the automatic execution of archiving profiles from the E-mail Servers category.

Start MailStore Client
Log in to MailStore Server
Click on Archive Email.
Right-click on an existing archiving profile in the Saved Profiles list or click on Commands and then click on Automatic....
In the Automatic Execution dialog you can specify the properties of the continuous execution:
- Between executions, wait...seconds
  Defines how long MailStore Server should wait until the next execution of the archiving profile.
- Do not execute between
  This option allows to specify a period in which the archiving profile is not executed. This for example can be due to nightly backups or other maintenance tasks.
After defining all settings, click OK

MailStore immediately starts the continuous execution of the archiving profile in the background. In the Current Status section, the status of the selected archiving profile can be watched. The very last execution results can be shown by clicking on Details..., whereas a list of all previous executions results is shown after clicking on All Results....

Automating Execution of E-mail Client or E-mail Files Archiving Profiles

Archiving profiles from the category E-mail Clients (i.e Microsoft Outlook, Mozilla Thunderbird) and E-mail Files must be executed in the context of the Windows users whose emails are to be archived or who has access to the files to be archived. To automate the archiving process of these profiles please proceed as follows:

Please note: As MailStore administrator and using MailStore's user management, first verify that the user possesses the privilege to archive emails.

Start MailStore Client on the user's computer.
Log on to MailStore Server using the user's credentials.
Click on Archive Email.
Right-click on an existing profile or click on Commands and then select Create Task on <COMPUTERNAME>.
In the next window "Schedule Task" the following settings can be specified:
- Task - Name:
  The task name can be chosen freely. By default, the name of the archiving profile and its internal MailStore ID are used.
- Task - Password: The password of the user. If Multi-factor Authentication is enabled for the user, an app password has to be used.
- Trigger:
  The following triggers can be specified:
  - Daily at - Specify a time for the daily execution (once a day)
  - Every [xx] minutes - Specify an interval of minutes to execute the profile (i.e. every 60 minutes)
  - Other Trigger - If the option Other Trigger is chosen. the archiving task is still created as a Windows task but the execution times have to be specified separately. This has to be done using the Windows Task Scheduler and is described in Windows Help. All options offered by your version of Windows can be used.
After all settings have been specified, the task can be created by clicking on OK. Confirm the safety prompt
Confirm again by clicking OK.

Please notice: Modifications to the schedule of archiving profiles from the categories E-mail Clients or E-mail Files is only possible through the Windows Task Scheduler. The task scheduler can be opened directly from MailStore Client by right-clicking on one of the existing profiles from the above categories and selecting the option Open Windows Task Scheduler.

Showing the Command Line

If desired, the archiving process of E-mail Client and E-mail Files profiles can be incorporated into a larger procedure, such as one run by a batch file (.bat). To simplify this process, MailStore offers the option Copy Cmd Line in the Schedule Task window (see top of this page). Click on the button to view a command line that can be incorporated into the desired procedure. For example:

"C:\Programme\Mailstore\MailStore Server\MailStoreCmd.exe"
--h="localhost" --pkv3="A3774AD575BA6DE29844829697585EBE21B20B83"
--cred="[email protected]"
-c import-execute --id=4 --user="admin" --verbose

The command line is to be read as follows:

"...\MailStoreCmd.exe" - MailStoreCmd.exe is the MailStore Management Shell. It was developed for the automated execution of certain processes in MailStore. Additional information about MailStoreCmd is available in chapter MailStore Server Management Shell.

--h="localhost" - The computer name of MailStore Server to which MailStoreCmd.exe is to connect.

--pkv3="..." - Public Key Fingerprint. It guarantees that the MailStore server at the time of archiving is the same as it was at the time when the scheduled task was created (optimal safety feature).

--cred="[email protected]" - The password of the user admin of the server 127.0.0.1 will be read from the Windows Credential Manager, which has been stored there before by MailStore.

-c - The actual command follows (non-interactive mode).

import-execute --id=4 --user="admin" --verbose - Execute the archiving profile of user admin with the internal ID 4.