Difference between revisions of "Using External Archive Stores"

[checked revision][checked revision]
m (Bmeyn moved page Using Advanced Archive Stores to Using External Archive Stores without leaving a redirect)
 
(17 intermediate revisions by 4 users not shown)
Line 1: Line 1:
__NOTOC__
+
In MailStore there are two types of archive stores: ''Internal Archive Stores'' and ''External Archive Stores''.
  
MailStore distinguishes between two types of archive stores: Standard archive store and advanced archive store.
+
While, with interal archive stores, folder information, meta data, email headers and contents as well as the full text index are stored in the file system, external archive stores allow you to store some of these components in SQL databases.
  
When using standard archive stores, folder information, meta data, email headers and contents, and the full text index are all stored within a directory structure in the file system, while advanced archive stores allow you to store these components in different locations, such as SQL databases, for example. The 5,000,000 emails per archive store limit also applies to advances archive stores.
+
<span class="mswarning">Database servers where external archive stores reside on must not be turned off or put into standby mode at any time, as long as there is a MailStore Server service accessing them. Otherwise, database corruption may occur, which can lead to data loss. If database servers must be turned off or rebooted, for example due to maintenance, please set the corresponding external archive stores to ''disabled'' first.</span>
  
For most environments, using standard archive stores is recommended, which is described in detail in chapter [[Storage Locations]].
+
For most environments, using internal archive stores is recommended; these are described in detail in chapter [[Storage Locations]].
 
 
'''Important notice:''' If advanced archive stores use SQL databases to store data, the appropriate database server needs to be running before the MailStore Server service is started. Manual configuration of a service dependency may be required.
 
  
 
== Structure of an Archive Store ==
 
== Structure of an Archive Store ==
In MailStore, both standard an advanced archive stores always consist of the following three components:
+
In MailStore, both internal and external archive stores always consist of the following three components:
 
+
{{Archive_Stores_Structure|Contains all data needed for searching through emails and attachments. The full text index can be reconstructed at any time.<br/>MailStore always uses its own high-performance full text index and not the index of the SQL database, therefore the full text index always has to be stored in the file system. Additional information on full text indexes is available in chapter [[Search Indexes]].}}
:'''Folder Information and Meta Data'''
 
:: Contains all data needed for the construction of the directory structure and the email list, which in some cases is also used in search requests.
 
:'''Email Headers and Contents'''
 
:: Contains the actual payload of the archive.
 
:'''Full Text Index'''
 
:: Contains all data needed for browsing emails and attachments.
 
 
 
While there is a direct relationship between ''folder information and meta data'' and ''email headers and contents'', the full text index is derived from both and can be reconstructed at any time.  
 
 
 
Because of its special data structure and for performant access, the full text index must always be stored in the file system. Using MailStore's local file system is recommended. Additional information about full text indexes is available in chapter [[Search Indexes]].
 
 
 
Please notice, that the 5,000,000 emails per archive store limit also applies to advanced archive stores.
 
 
 
== Creating an Advanced Archive Store ==
 
To create an advanced archive store, please proceed as follows:
 
 
 
* Start MailStore Client and log on as MailStore administrator (admin).
 
* Click on ''Administrative Tools > Storage'' and then on ''Storage Locations''.
 
* In the menu bar at the bottom of the window click on ''Create...''.
 
* The dialog ''Create New Archive Store'' opens.
 
*: [[File:Tech_storageloc_adv_01.png|center|350px]]
 
* Enter a name for the new advanced archive store into the ''Name'' field, e.g. ''2012-05''.<br/>'''Please note:''' If you don't want MailStore to archive new emails in the new archive store, remove the checkmark from the box titled ''Archive new messages here''.
 
* Select ''Advanced Archive Store'' and click on ''Next''.
 
* Select the type of advanced archive store:
 
*: [[File:Tech_storageloc_adv_fs_01.png|center|350px]]
 
*: '''Directory (File System)'''<br/>The entire archive store is stored in the file system (local hard drive or network share).
 
*: '''External Microsoft SQL Server Database'''<br/>The archive store is stored in an external Microsoft SQL Server Database. Emails can be stored in the database or in the file system.
 
*: '''External PostgreSQL Database'''<br/>The archive store is stored in an external PostgreSQL Database. E-Mails can be stored in the database or in the file system.
 
* Click on ''Next''.
 
 
 
Depending on the type selected, different input is required. How each archive store type is configured is described in the following sections.
 
 
 
=== Advanced Archive Store Type: Directory (File System) ===
 
Using an advanced archive store of type ''Directory (File System)'' requires you to specify directories for the ''Folder Information and Meta Data'', the ''Email Headers and Contents'' and the ''Full Text Index''.
 
 
 
: [[File:Tech_storageloc_adv_fs_02.png|350px|center]]
 
  
Based on the name entered at the beginning of the wizard and the path of the master database MailStore recommends directories for the new advanced archive store. To change a proposed path, click on the respective button next to the ''Directory'' field or enter a path manually.
+
== Creating an External Archive Store ==
 +
Under ''Administrative Tools > Storage > Storage Locations'' you can create new archive stores and manage the archive's existing archive stores. To create an external archive store, please proceed as follows:
  
'''Important Notice:''' The directories are created automatically. If they already exist, they must not contain any files of subfolders.
+
* Below the list of archive stores, click on the ''Create...'' button.
 +
* [[File:Tech_storageloc_adv_01.png|right|350px]]The ''Create New Archive Store'' wizard opens.
 +
* Select the database type:
 +
** '''External Microsoft SQL Server Database'''<br/>The archive store is stored in an external Microsoft SQL Server Database. Emails can be stored in the database or in the file system.
 +
** '''External PostgreSQL Database'''<br/>The archive store is stored in an external PostgreSQL Database. E-Mails can be stored in the database or in the file system.
 +
* Click on ''Next''.<br clear=all/>
 +
Based on the database type additional parameters need to be configured in the next step.
  
Please note that distributing the individual components of an advanced archive store among local drives or network shares significantly increases the complexity of [[Backup and Restore]].
+
=== External Archive Store Type: External Microsoft SQL Server Database ===
 
 
=== Advanced Archive Store Type: External Microsoft SQL Server Database ===
 
 
Before you can set up the database connection in MailStore, an empty database has to be created on the database server. The MailStore user who is used for the connection should be the owner of the database. Please see the documentation of the database server for details.  
 
Before you can set up the database connection in MailStore, an empty database has to be created on the database server. The MailStore user who is used for the connection should be the owner of the database. Please see the documentation of the database server for details.  
  
 
''Folder information and meta data'' are always stored in the SQL database, while storing ''email headers and contents'' therein is optional.
 
''Folder information and meta data'' are always stored in the SQL database, while storing ''email headers and contents'' therein is optional.
  
<p class="msnote">'''Please note:''' MailStore supports all editions of Microsoft SQL Server Version 2005, 2008, 2012 and 2014. Please keep their respective size limits in mind and verify their suitability for managing the expected volume of data in your environment.</p>
+
<p class="msnote">'''Please note:''' MailStore supports all editions of Microsoft SQL Server Version 2008, 2012, 2014 and 2016. Please keep their respective size limits in mind and verify their suitability for managing the expected volume of data in your environment.</p>
  
Once an empty database has been created, please proceed as follows:
+
Once you have created an empty database, please proceed as follows:
  
 +
* Enter a name for the new external archive store in the ''Name'' field, e.g. ''2016-12''.
 +
* If you don't want MailStore to archive new emails in the new archive store, deselect the option ''Archive new messages here''.
 
* Specify the connection parameters for the ''Microsoft SQL Server Database Connection'':  
 
* Specify the connection parameters for the ''Microsoft SQL Server Database Connection'':  
*:[[File:Tech_storageloc_adv_mssql_01.png|350px|center]]
+
** [[File:Tech_storageloc_adv_mssql_01.png|right|350px]]'''Server Name:''' Enter the server name or the IP address of the SQL server on which a database has been created for MailStore. If you click on the arrow to the right of the input field, MailStore will return a list of all Microsoft SQL servers located on the network.
*: '''Server Name:''' Enter the server name or the IP address of the SQL server on which a database has been created for MailStore. If you click on the arrow to the right of the input field, MailStore will return a list of all Microsoft SQL servers located on the network.
+
** '''User Name:''' Name of the user with access to the database.
*: '''User Name:''' Name of the user with access to the database.
+
** '''Password:''' Password of the user listed under ''User Name''. Passwords may not start or end with whitespace characters.
*: '''Password:''' Password of the user listed under ''User Name''.
+
** '''Database:''' Name of the database to be used by MailStore. Click on the arrow to the right of the input field to obtain a list of all available databases on the server.
*: '''Database:''' Name of the database to be used by MailStore. Click on the arrow to the right of the input field to obtain a list of all available databases on the server.
+
* Under ''email headers and contents'' select the appropriate storage location.<br/><br/>''Microsoft SQL Server Database'' is the default suggestion. When choosing ''Directory (File System)'', the input field ''Directory'' is activated. MailStore derives a directory based on the name entered and the path of the master database. To choose a different directory, click on the button next to the ''Directory'' field or enter a path manually.<br/>The specified directory is created automatically. If it already exists, it must not contain any files or subfolders.
* Under ''email headers and contents'' select the appropriate storage location.<br/><br/>''Microsoft SQL Server Database'' is the default suggestion. When choosing ''Directory (File System)'', the input field ''Directory'' is activated. MailStore recommends a directory based on the name entered at the beginning of the wizard and the path of the master database. To choose a different directory, click on the button next to the ''Directory'' field or enter a path manually.<br/><br/> '''Important Notice:''' The specified directory is created automatically. If it already exists, it must not contain any fields of subfolders.
+
* A directory for the full text index is also derived based on the name entered and the path of the master database.
* A directory for the full text index is also recommended based on the name entered at the beginning and the path of the master database.  
 
 
* Click on ''Finish''.
 
* Click on ''Finish''.
  
Please note that distributing the individual components of an advanced archive store among different local drives or network shares significantly increases the complexity of [[Backup and Restore]].
+
Please note that distributing the individual components of an external archive store among different local drives or network shares significantly increases the complexity of [[Backup and Restore]].
  
=== Advanced Archive Store Type: External PostgreSQL Database ===
+
=== External Archive Store Type: External PostgreSQL Database ===
 
Before you can set up the database connection in MailStore, an empty database has to be created on the database server. The MailStore user who is used for the connection should be the owner of the database. Please see the documentation of the database server for details.
 
Before you can set up the database connection in MailStore, an empty database has to be created on the database server. The MailStore user who is used for the connection should be the owner of the database. Please see the documentation of the database server for details.
  
 
''Folder information and meta data'' are always stored in the SQL database, while storing ''email headers and contents'' therein is optional.  
 
''Folder information and meta data'' are always stored in the SQL database, while storing ''email headers and contents'' therein is optional.  
  
<p class="msnote">'''Please note:''' MailStore supports PostgreSQL version 8.4.8 or newer.</p>
+
<p class="msnote">'''Please note:''' MailStore supports PostgreSQL version 10 or newer.</p>
  
 
Once an empty database has been created, please proceed as follows:
 
Once an empty database has been created, please proceed as follows:
 +
 +
* Enter a name for the new external archive store in the ''Name'' field, e.g. ''2016-12''.
 +
* If you don't want MailStore to archive new emails in the new archive store, deselect the option ''Archive new messages here''.
 
* Specify the connection parameters for the ''PostgresSQL Database Connection'':  
 
* Specify the connection parameters for the ''PostgresSQL Database Connection'':  
*: [[File:Tech_storageloc_adv_pgsql_01.png|350px|center]]
+
** [[File:Tech_storageloc_adv_pgsql_01.png|right|350px]]'''Server Name:''' Enter the server name or the IP address of the SQL server on which a database has been created for MailStore.  
*: '''Server Name:''' Enter the server name or the IP address of the SQL server on which a database has been created for MailStore.  
+
** '''Encrypted Connection:''' Enable encryption of connection to the database server.
*: '''User Name:''' Name of a user with access to the database.
+
** '''Accept all certificates:''' {{Option_Accept_all_certificates}}
*: '''Password:''' Password of the user specified under ''User Name''.
+
** '''User Name:''' Name of a user with access to the database.
*: '''Database:''' Name of the database to be used by MailStore. To obtain a list of all available databases on the server, click on the arrow to the right of the input field.
+
** '''Password:''' Password of the user specified under ''User Name''. Passwords may not start or end with whitespace characters.
* Under ''Email Headers and Contents'' select the appropriate storage location.<br/><br/>''PostgresSQL Database'' is the default suggestion. Selecting ''Directory (File System)'' activates the input field ''Directory''. MailStore proposes a directory based on the name entered at the beginning of the wizard and the path of the master database. To choose a different directory, click on the button next to the ''Directory'' field or enter a path manually.<br/><br/> '''Important Notice:''' The specified directory is created automatically. If it already exists, it must not contain any files or subfolders.
+
** '''Database:''' Name of the database to be used by MailStore. To obtain a list of all available databases on the server, click on the arrow to the right of the input field.
* MailStore also recommends a directory for the full text index based on the name entered at the beginning and the path of the master database.
+
* Under ''Email Headers and Contents'' select the appropriate storage location.<br/><br/>''PostgresSQL Database'' is the default suggestion. Selecting ''Directory (File System)'' activates the input field ''Directory''. MailStore derives a directory based on the name entered and the path of the master database. To choose a different directory, click on the button next to the ''Directory'' field or enter a path manually.<br/>The specified directory is created automatically. If it already exists, it must not contain any files or subfolders.
 +
* A directory for the full text index is also derived based on the name entered and the path of the master database.
 
* Click on ''Finish''.  
 
* Click on ''Finish''.  
  
 
Please note that distributing the individual components of an advanced archive store among different local drives or network shares significantly increases the complexity of [[Backup and Restore]].
 
Please note that distributing the individual components of an advanced archive store among different local drives or network shares significantly increases the complexity of [[Backup and Restore]].
  
[[de:Verwendung_erweiterter_Archivspeicher]]
+
[[de:Verwendung_externer_Archivspeicher]]
[[en:Using_Advanced_Archive_Stores&action]]
+
[[en:Using_External_Archive_Stores]]

Latest revision as of 12:21, 30 May 2023

In MailStore there are two types of archive stores: Internal Archive Stores and External Archive Stores.

While, with interal archive stores, folder information, meta data, email headers and contents as well as the full text index are stored in the file system, external archive stores allow you to store some of these components in SQL databases.

Database servers where external archive stores reside on must not be turned off or put into standby mode at any time, as long as there is a MailStore Server service accessing them. Otherwise, database corruption may occur, which can lead to data loss. If database servers must be turned off or rebooted, for example due to maintenance, please set the corresponding external archive stores to disabled first.

For most environments, using internal archive stores is recommended; these are described in detail in chapter Storage Locations.

Structure of an Archive Store

In MailStore, both internal and external archive stores always consist of the following three components:

  • Folder Information and Meta Data
    Contains all data needed for the construction of the folder structure and the email list.
  • Email Headers and Contents
    Contains the actual payload of the archive.
  • Full Text Index
    Contains all data needed for searching through emails and attachments. The full text index can be reconstructed at any time.
    MailStore always uses its own high-performance full text index and not the index of the SQL database, therefore the full text index always has to be stored in the file system. Additional information on full text indexes is available in chapter Search Indexes.


Creating an External Archive Store

Under Administrative Tools > Storage > Storage Locations you can create new archive stores and manage the archive's existing archive stores. To create an external archive store, please proceed as follows:

  • Below the list of archive stores, click on the Create... button.
  • Tech storageloc adv 01.png
    The Create New Archive Store wizard opens.
  • Select the database type:
    • External Microsoft SQL Server Database
      The archive store is stored in an external Microsoft SQL Server Database. Emails can be stored in the database or in the file system.
    • External PostgreSQL Database
      The archive store is stored in an external PostgreSQL Database. E-Mails can be stored in the database or in the file system.
  • Click on Next.

Based on the database type additional parameters need to be configured in the next step.

External Archive Store Type: External Microsoft SQL Server Database

Before you can set up the database connection in MailStore, an empty database has to be created on the database server. The MailStore user who is used for the connection should be the owner of the database. Please see the documentation of the database server for details.

Folder information and meta data are always stored in the SQL database, while storing email headers and contents therein is optional.

Please note: MailStore supports all editions of Microsoft SQL Server Version 2008, 2012, 2014 and 2016. Please keep their respective size limits in mind and verify their suitability for managing the expected volume of data in your environment.

Once you have created an empty database, please proceed as follows:

  • Enter a name for the new external archive store in the Name field, e.g. 2016-12.
  • If you don't want MailStore to archive new emails in the new archive store, deselect the option Archive new messages here.
  • Specify the connection parameters for the Microsoft SQL Server Database Connection:
    • Tech storageloc adv mssql 01.png
      Server Name: Enter the server name or the IP address of the SQL server on which a database has been created for MailStore. If you click on the arrow to the right of the input field, MailStore will return a list of all Microsoft SQL servers located on the network.
    • User Name: Name of the user with access to the database.
    • Password: Password of the user listed under User Name. Passwords may not start or end with whitespace characters.
    • Database: Name of the database to be used by MailStore. Click on the arrow to the right of the input field to obtain a list of all available databases on the server.
  • Under email headers and contents select the appropriate storage location.

    Microsoft SQL Server Database is the default suggestion. When choosing Directory (File System), the input field Directory is activated. MailStore derives a directory based on the name entered and the path of the master database. To choose a different directory, click on the button next to the Directory field or enter a path manually.
    The specified directory is created automatically. If it already exists, it must not contain any files or subfolders.
  • A directory for the full text index is also derived based on the name entered and the path of the master database.
  • Click on Finish.

Please note that distributing the individual components of an external archive store among different local drives or network shares significantly increases the complexity of Backup and Restore.

External Archive Store Type: External PostgreSQL Database

Before you can set up the database connection in MailStore, an empty database has to be created on the database server. The MailStore user who is used for the connection should be the owner of the database. Please see the documentation of the database server for details.

Folder information and meta data are always stored in the SQL database, while storing email headers and contents therein is optional.

Please note: MailStore supports PostgreSQL version 10 or newer.

Once an empty database has been created, please proceed as follows:

  • Enter a name for the new external archive store in the Name field, e.g. 2016-12.
  • If you don't want MailStore to archive new emails in the new archive store, deselect the option Archive new messages here.
  • Specify the connection parameters for the PostgresSQL Database Connection:
    • Tech storageloc adv pgsql 01.png
      Server Name: Enter the server name or the IP address of the SQL server on which a database has been created for MailStore.
    • Encrypted Connection: Enable encryption of connection to the database server.
    • Accept all certificates: If the certificate provided by the remote host cannot be verified (e.g. self-signed or signed by an unknown certificate authority), enable the option Accept all certificates to allow MailStore to establish a connection. As this option leads to an insecure configuration, warnings may appear in the summary and/or the dashboard.
    • User Name: Name of a user with access to the database.
    • Password: Password of the user specified under User Name. Passwords may not start or end with whitespace characters.
    • Database: Name of the database to be used by MailStore. To obtain a list of all available databases on the server, click on the arrow to the right of the input field.
  • Under Email Headers and Contents select the appropriate storage location.

    PostgresSQL Database is the default suggestion. Selecting Directory (File System) activates the input field Directory. MailStore derives a directory based on the name entered and the path of the master database. To choose a different directory, click on the button next to the Directory field or enter a path manually.
    The specified directory is created automatically. If it already exists, it must not contain any files or subfolders.
  • A directory for the full text index is also derived based on the name entered and the path of the master database.
  • Click on Finish.

Please note that distributing the individual components of an advanced archive store among different local drives or network shares significantly increases the complexity of Backup and Restore.