Difference between revisions of "Using External Archive Stores"

[unchecked revision]

[checked revision]

Latest revision as of 12:21, 30 May 2023

In MailStore there are two types of archive stores: Internal Archive Stores and External Archive Stores.

While, with interal archive stores, folder information, meta data, email headers and contents as well as the full text index are stored in the file system, external archive stores allow you to store some of these components in SQL databases.

Database servers where external archive stores reside on must not be turned off or put into standby mode at any time, as long as there is a MailStore Server service accessing them. Otherwise, database corruption may occur, which can lead to data loss. If database servers must be turned off or rebooted, for example due to maintenance, please set the corresponding external archive stores to disabled first.

For most environments, using internal archive stores is recommended; these are described in detail in chapter Storage Locations.

Structure of an Archive Store

In MailStore, both internal and external archive stores always consist of the following three components:

Folder Information and Meta Data
Contains all data needed for the construction of the folder structure and the email list.
Email Headers and Contents
Contains the actual payload of the archive.
Full Text Index
Contains all data needed for searching through emails and attachments. The full text index can be reconstructed at any time.
MailStore always uses its own high-performance full text index and not the index of the SQL database, therefore the full text index always has to be stored in the file system. Additional information on full text indexes is available in chapter Search Indexes.

Creating an External Archive Store

Under Administrative Tools > Storage > Storage Locations you can create new archive stores and manage the archive's existing archive stores. To create an external archive store, please proceed as follows:

Below the list of archive stores, click on the Create... button.
The Create New Archive Store wizard opens.
Select the database type:
- External Microsoft SQL Server Database
  The archive store is stored in an external Microsoft SQL Server Database. Emails can be stored in the database or in the file system.
- External PostgreSQL Database
  The archive store is stored in an external PostgreSQL Database. E-Mails can be stored in the database or in the file system.
Click on Next.

Based on the database type additional parameters need to be configured in the next step.

External Archive Store Type: External Microsoft SQL Server Database

Before you can set up the database connection in MailStore, an empty database has to be created on the database server. The MailStore user who is used for the connection should be the owner of the database. Please see the documentation of the database server for details.

Folder information and meta data are always stored in the SQL database, while storing email headers and contents therein is optional.

Please note: MailStore supports all editions of Microsoft SQL Server Version 2008, 2012, 2014 and 2016. Please keep their respective size limits in mind and verify their suitability for managing the expected volume of data in your environment.

Once you have created an empty database, please proceed as follows:

Enter a name for the new external archive store in the Name field, e.g. 2016-12.
If you don't want MailStore to archive new emails in the new archive store, deselect the option Archive new messages here.
Specify the connection parameters for the Microsoft SQL Server Database Connection:
- Server Name: Enter the server name or the IP address of the SQL server on which a database has been created for MailStore. If you click on the arrow to the right of the input field, MailStore will return a list of all Microsoft SQL servers located on the network.
- User Name: Name of the user with access to the database.
- Password: Password of the user listed under User Name. Passwords may not start or end with whitespace characters.
- Database: Name of the database to be used by MailStore. Click on the arrow to the right of the input field to obtain a list of all available databases on the server.
Under email headers and contents select the appropriate storage location.

Microsoft SQL Server Database is the default suggestion. When choosing Directory (File System), the input field Directory is activated. MailStore derives a directory based on the name entered and the path of the master database. To choose a different directory, click on the button next to the Directory field or enter a path manually.
The specified directory is created automatically. If it already exists, it must not contain any files or subfolders.
A directory for the full text index is also derived based on the name entered and the path of the master database.
Click on Finish.

Please note that distributing the individual components of an external archive store among different local drives or network shares significantly increases the complexity of Backup and Restore.

External Archive Store Type: External PostgreSQL Database

Before you can set up the database connection in MailStore, an empty database has to be created on the database server. The MailStore user who is used for the connection should be the owner of the database. Please see the documentation of the database server for details.

Folder information and meta data are always stored in the SQL database, while storing email headers and contents therein is optional.

Please note: MailStore supports PostgreSQL version 10 or newer.

Once an empty database has been created, please proceed as follows:

Enter a name for the new external archive store in the Name field, e.g. 2016-12.
If you don't want MailStore to archive new emails in the new archive store, deselect the option Archive new messages here.
Specify the connection parameters for the PostgresSQL Database Connection:
- Server Name: Enter the server name or the IP address of the SQL server on which a database has been created for MailStore.
- Encrypted Connection: Enable encryption of connection to the database server.
- Accept all certificates: If the certificate provided by the remote host cannot be verified (e.g. self-signed or signed by an unknown certificate authority), enable the option Accept all certificates to allow MailStore to establish a connection. As this option leads to an insecure configuration, warnings may appear in the summary and/or the dashboard.
- User Name: Name of a user with access to the database.
- Password: Password of the user specified under User Name. Passwords may not start or end with whitespace characters.
- Database: Name of the database to be used by MailStore. To obtain a list of all available databases on the server, click on the arrow to the right of the input field.
Under Email Headers and Contents select the appropriate storage location.

PostgresSQL Database is the default suggestion. Selecting Directory (File System) activates the input field Directory. MailStore derives a directory based on the name entered and the path of the master database. To choose a different directory, click on the button next to the Directory field or enter a path manually.
The specified directory is created automatically. If it already exists, it must not contain any files or subfolders.
A directory for the full text index is also derived based on the name entered and the path of the master database.
Click on Finish.

Please note that distributing the individual components of an advanced archive store among different local drives or network shares significantly increases the complexity of Backup and Restore.

@@ Line 1: / Line 1: @@
-__NOTOC__
+In MailStore there are two types of archive stores: ''Internal Archive Stores'' and ''External Archive Stores''.
-MailStore distinguishes between two types of archive stores: Standard archive store and advanced archive store.
+While, with interal archive stores, folder information, meta data, email headers and contents as well as the full text index are stored in the file system, external archive stores allow you to store some of these components in SQL databases.
-When using standard archive stores, folder information, meta data, email headers and contents, as well as the full text index are stored within a directory structure in the file system, while advanced archive stores allow you to store these components in different locations, such as SQL databases, for example.
+<span class="mswarning">Database servers where external archive stores reside on must not be turned off or put into standby mode at any time, as long as there is a MailStore Server service accessing them. Otherwise, database corruption may occur, which can lead to data loss. If database servers must be turned off or rebooted, for example due to maintenance, please set the corresponding external archive stores to ''disabled'' first.</span>
-For most environments, using standard archive stores is recommended, which is described in detail in chapter [[Storage Locations]].
+For most environments, using internal archive stores is recommended; these are described in detail in chapter [[Storage Locations]].
-= Structure of an Archive Store =
+== Structure of an Archive Store ==
+In MailStore, both internal and external archive stores always consist of the following three components:
+{{Archive_Stores_Structure|Contains all data needed for searching through emails and attachments. The full text index can be reconstructed at any time.<br/>MailStore always uses its own high-performance full text index and not the index of the SQL database, therefore the full text index always has to be stored in the file system. Additional information on full text indexes is available in chapter [[Search Indexes]].}}
-In MailStore, both standard an advanced archive stores always consist of the following three components:
+== Creating an External Archive Store ==
+Under ''Administrative Tools > Storage > Storage Locations'' you can create new archive stores and manage the archive's existing archive stores. To create an external archive store, please proceed as follows:
-:'''Folder Information and Meta Data'''
+* Below the list of archive stores, click on the ''Create...'' button.
-:: Contains all data needed for the construction of the directory structure and the email list, which in some cases is also used in search requests.
+* [[File:Tech_storageloc_adv_01.png|right|350px]]The ''Create New Archive Store'' wizard opens.
-:'''Email Headers and Contents'''
+* Select the database type:
-:: Contains the actual payload of the archive.
+** '''External Microsoft SQL Server Database'''<br/>The archive store is stored in an external Microsoft SQL Server Database. Emails can be stored in the database or in the file system.
-:'''Full Text Index'''
+** '''External PostgreSQL Database'''<br/>The archive store is stored in an external PostgreSQL Database. E-Mails can be stored in the database or in the file system.
-:: Contains all data needed for browsing emails and attachments.
+* Click on ''Next''.<br clear=all/>
+Based on the database type additional parameters need to be configured in the next step.
-While there is a direct relationship between ''folder information and meta data'' and ''email headers and contents'', the full text index is derived from both and can be reconstructed at any time.
+=== External Archive Store Type: External Microsoft SQL Server Database ===
+Before you can set up the database connection in MailStore, an empty database has to be created on the database server. The MailStore user who is used for the connection should be the owner of the database. Please see the documentation of the database server for details.
-Because of its special data structure and for performant access, the full text index must always be stored in the file system. Using MailStore's local file system is recommended. Additional information about full text indexes is available in chapter [[Search Indexes]].
+''Folder information and meta data'' are always stored in the SQL database, while storing ''email headers and contents'' therein is optional.
-= Creating an Advanced Archive Store =
+<p class="msnote">'''Please note:''' MailStore supports all editions of Microsoft SQL Server Version 2008, 2012, 2014 and 2016. Please keep their respective size limits in mind and verify their suitability for managing the expected volume of data in your environment.</p>
-To create an advanced archive store, please proceed as follows:
+Once you have created an empty database, please proceed as follows:
-* Start MailStore Client and log on as MailStore administrator (admin).
+* Enter a name for the new external archive store in the ''Name'' field, e.g. ''2016-12''.
-* Click on ''Administrative Tools > Storage'' and then on ''Storage Locations''.
+* If you don't want MailStore to archive new emails in the new archive store, deselect the option ''Archive new messages here''.
-* In the menu bar at the bottom of the window click on ''Create...''.
+* Specify the connection parameters for the ''Microsoft SQL Server Database Connection'':
-* The dialog ''Create new archive store'' opens.
+** [[File:Tech_storageloc_adv_mssql_01.png|right|350px]]'''Server Name:''' Enter the server name or the IP address of the SQL server on which a database has been created for MailStore. If you click on the arrow to the right of the input field, MailStore will return a list of all Microsoft SQL servers located on the network.
-*: [[File:Tech_storageloc_adv_01.png|center|350px]]
+** '''User Name:''' Name of the user with access to the database.
-* Enter a name for the new advanced archive store into the ''Name'' field, e.g. ''2012-05''.<br/>'''Please note:''' If you don't want MailStore to archive new emails in the new archive store, remove the checkmark from the box titled ''Archive new messages here''.
+** '''Password:''' Password of the user listed under ''User Name''. Passwords may not start or end with whitespace characters.
-* Select ''Advanced Archive Store'' and click on ''Next''.
+** '''Database:''' Name of the database to be used by MailStore. Click on the arrow to the right of the input field to obtain a list of all available databases on the server.
-* Select the type of advanced archive store:
+* Under ''email headers and contents'' select the appropriate storage location.<br/><br/>''Microsoft SQL Server Database'' is the default suggestion. When choosing ''Directory (File System)'', the input field ''Directory'' is activated. MailStore derives a directory based on the name entered and the path of the master database. To choose a different directory, click on the button next to the ''Directory'' field or enter a path manually.<br/>The specified directory is created automatically. If it already exists, it must not contain any files or subfolders.
-*: [[File:Tech_storageloc_adv_fs_01.png|center|350px]]
+* A directory for the full text index is also derived based on the name entered and the path of the master database.
-*: '''Directory (File System)'''<br/>The entire archive store is stored in the file system (local hard drive or network share).
+* Click on ''Finish''.
-*: '''External Microsoft SQL Server Database'''<br/>The archive store is stored in an external Microsoft SQL Server Database. Emails can be stored in the database or in the file system.
-*: '''External PostgreSQL Database'''<br/>The archive store is stored in an external PostgreSQL Database. E-Mails can be stored in the database or in the file system.
-* Click on ''Next''.
-Depending on the type selected, different input is required. How each archive store type is configured is described in the following sections.
+Please note that distributing the individual components of an external archive store among different local drives or network shares significantly increases the complexity of [[Backup and Restore]].
-== Advanced Archive Store Type: Directory (File System) ==
+=== External Archive Store Type: External PostgreSQL Database ===
+Before you can set up the database connection in MailStore, an empty database has to be created on the database server. The MailStore user who is used for the connection should be the owner of the database. Please see the documentation of the database server for details.
-Using an advanced archive store of type ''Directory (File System)'' requires you to specify directories for the ''Folder Information and Meta Data'', the ''Email Headers and Contents'' and the ''Full Text Index''.
+''Folder information and meta data'' are always stored in the SQL database, while storing ''email headers and contents'' therein is optional.
-: [[File:Tech_storageloc_adv_fs_02.png|350px|center]]
+<p class="msnote">'''Please note:''' MailStore supports PostgreSQL version 10 or newer.</p>
-Based on the name entered at the beginning of the wizard and the path of the master database MailStore suggests the directories of the new advanced archive store. To change a proposed path, click on the respective button next to the ''directory'' field or enter a path manually.
+Once an empty database has been created, please proceed as follows:
-'''Important Notice:''' The directories are created automatically. If they already exist, they must not contain any files of subfolders.
+* Enter a name for the new external archive store in the ''Name'' field, e.g. ''2016-12''.
+* If you don't want MailStore to archive new emails in the new archive store, deselect the option ''Archive new messages here''.
+* Specify the connection parameters for the ''PostgresSQL Database Connection'':
+** [[File:Tech_storageloc_adv_pgsql_01.png|right|350px]]'''Server Name:''' Enter the server name or the IP address of the SQL server on which a database has been created for MailStore.
+** '''Encrypted Connection:''' Enable encryption of connection to the database server.
+** '''Accept all certificates:''' {{Option_Accept_all_certificates}}
+** '''User Name:''' Name of a user with access to the database.
+** '''Password:''' Password of the user specified under ''User Name''. Passwords may not start or end with whitespace characters.
+** '''Database:''' Name of the database to be used by MailStore. To obtain a list of all available databases on the server, click on the arrow to the right of the input field.
+* Under ''Email Headers and Contents'' select the appropriate storage location.<br/><br/>''PostgresSQL Database'' is the default suggestion. Selecting ''Directory (File System)'' activates the input field ''Directory''. MailStore derives a directory based on the name entered and the path of the master database. To choose a different directory, click on the button next to the ''Directory'' field or enter a path manually.<br/>The specified directory is created automatically. If it already exists, it must not contain any files or subfolders.
+* A directory for the full text index is also derived based on the name entered and the path of the master database.
+* Click on ''Finish''.
-Please note that distributing the individual components of an advanced archive store among local drives or network shares significantly increases the complexity of [[Backup and Restore]].
+Please note that distributing the individual components of an advanced archive store among different local drives or network shares significantly increases the complexity of [[Backup and Restore]].
-== Erweiterter Archivspeichertyp: Externe Microsoft SQL Server-Datenbank ==
+[[de:Verwendung_externer_Archivspeicher]]
-Bevor Sie die Datenbankverbindung in MailStore einrichten können, müssen Sie auf dem Datenbankserver eine leere Datenbank erstellen. Der Benutzer den Sie beabsichtigen in MailStore zur Verbindung zu verwenden, sollte Besitzer der Datenbank sein. Die genaue Vorgehensweise entnehmen Sie bitte der Dokumentation des Datenbankserver.
+[[en:Using_External_Archive_Stores]]
-In der SQL-Datenbank werden bei diesem Archivspeichertyp immer die ''Ordnerinformationen und Metadaten'' abgelegt und optional auch die ''E-Mail-Kopfzeilen und -inhalte''.
-<p class="msnote">'''Hinweis:''' MailStore unterstützt alle Editionen des Microsoft SQL Server in den Versionen 2005, 2008 und 2012. Beachten Sie jedoch die jeweiligen Größenbeschränkungen der einzelnen Editionen und prüfen Sie deren Tauglichkeit auf die zu erwartende Datenmenge in Ihrer Umgebung.</p>
-Nachdem Sie eine leere Datenbank angelegt haben, fahren Sie wie folgt fort:
-* Legen Sie die Verbindungsparametern für die ''Microsoft SQL Server-Datenbankverbindung'' fest:
-*:[[File:Tech_storageloc_adv_mssql_01.png|350px|center]]
-*: '''Servername:''' Tragen Sie den Servernamen oder IP-Adresse des SQL Servers ein, auf welchem Sie eine Datenbank für die Verwendung von MailStore erstellt haben. Klicken Sie auf den Pfeil am rechten Rand des Eingabefeldes, wird MailStore versuchen alle im Netzwerk vorhandenen Microsoft SQL Server zu finden und aufzulisten.
-*: '''Benutzername:''' Benutzername mit Zugriff auf die Datenbank.
-*: '''Kennwort:''' Das Passwort der unter ''Benutzername'' angegebenen Benutzers.
-*: '''Datenbank:''' Name der Datenbank welche MailStore verwenden soll. Klicken Sie auf den Pfeil an der rechten Seite des Eingabefeldes um eine Liste der zur Verfügung stehenden Datenbanken des Servers abzurufen.
-* Wählen Sie unter ''E-Mail-Kopfzeilen und -inhalte'' den gewünschten Speicherort aus.<br/><br/>Standardmäßig wird ''Microsoft SQL Server-Datenbank'' vorgeschlagen. Wenn Sie ''Verzeichnis (Dateisystem)'' auswählen, aktiviert sich das Eingabefeld ''Verzeichnis''. MailStore erstellt aus dem eingegebenen Namen zu Beginn des Assistenten und dem Pfad der Masterdatenbank einen Vorschlag für das Verzeichnis. Klicken Sie auf die Schaltfläche hinter dem Feld ''Verzeichnis'' oder tragen Sie manuell einen Pfad ein um ein anderes Verzeichnis zu verwenden.<br/><br/> '''Wichtiger Hinweis:''' Das angegebene Verzeichnis wird automatisch angelegt. Falls es bereits existiert, dürfen sich darin keine Dateien und Unterverzeichnisse befinden.
-* Das Verzeichnis für den Volltextindex wird von MailStore ebenfalls anhand des zu Beginn eingegebenen Namens und dem Pfad zu Masterdatenbank vorgeschlagen.
-* Zum Fertigstellen klicken Sie auf ''Fertig''.
-Bitte beachten Sie, dass das Verteilen der einzelnen Bestandteile eines erweiterten Archivspeichers auf verschiedene lokale Laufwerke oder Netzwerk-Shares die Komplexität der [[Datensicherung und Wiederherstellung]] deutlich erhöht.
-== Erweiterter Archivspeichertyp: Externe PostgreSQL-Datenbank ==
-Bevor Sie die Datenbankverbindung in MailStore einrichten können, müssen Sie auf dem Datenbankserver eine leere Datenbank erstellen. Der Benutzer den Sie beabsichtigen in MailStore zur Verbindung zu verwenden, sollte Besitzer der Datenbank sein. Die genaue Vorgehensweise entnehmen Sie bitte der Dokumentation des Datenbankserver.
-In der SQL-Datenbank werden bei diesem Archivspeichertyp immer die ''Ordnerinformationen und Metadaten'' abgelegt und optional auch die ''E-Mail-Kopfzeilen und -inhalte''.
-<p class="msnote">'''Hinweis:''' MailStore unterstützt PostgreSQL ab Version 8.4.8 oder neuer.</p>
-Nachdem Sie eine leere Datenbank angelegt haben, fahren Sie wie folgt fort:
-* Legen Sie die Verbindungsparametern für die ''PostgresSQL-Datenbankverbindung'' fest:
-*: [[File:Tech_storageloc_adv_pgsql_01.png|350px|center]]
-*: '''Servername:''' Tragen Sie den Servernamen oder IP-Adresse des SQL Servers ein, auf welchem Sie eine Datenbank für die Verwendung von MailStore erstellt haben.
-*: '''Benutzername:''' Benutzername mit Zugriff auf die Datenbank.
-*: '''Kennwort:''' Das Passwort der unter ''Benutzername'' angegebenen Benutzers.
-*: '''Datenbank:''' Name der Datenbank welche MailStore verwenden soll. Klicken Sie auf den Pfeil an der rechten Seite des Eingabefeldes um eine Liste der zur Verfügung stehenden Datenbanken des Servers abzurufen.
-* Wählen Sie unter ''E-Mail-Kopfzeilen und -inhalte'' den gewünschten Speicherort aus.<br/><br/>Standardmäßig wird ''PostgresSQL-Datenbank'' vorgeschlagen. Wenn Sie ''Verzeichnis (Dateisystem)'' auswählen, aktiviert sich das Eingabefeld ''Verzeichnis''. MailStore erstellt aus dem eingegebenen Namen zu Beginn des Assistenten und dem Pfad der Masterdatenbank einen Vorschlag für das Verzeichnis. Klicken Sie auf die Schaltfläche hinter dem Feld ''Verzeichnis'' oder tragen Sie manuell einen Pfad ein um ein anderes Verzeichnis zu verwenden.<br/><br/> '''Wichtiger Hinweis:''' Das angegebene Verzeichnis wird automatisch angelegt. Falls es bereits existiert, dürfen sich darin keine Dateien und Unterverzeichnisse befinden.
-* Das Verzeichnis für den Volltextindex wird von MailStore ebenfalls anhand des zu Beginn eingegebenen Namens und dem Pfad zu Masterdatenbank vorgeschlagen.
-* Zum Fertigstellen klicken Sie auf ''Fertig''.
-Bitte beachten Sie, dass das Verteilen der einzelnen Bestandteile eines erweiterten Archivspeichers auf verschiedene lokale Laufwerke oder Netzwerk-Shares die Komplexität der [[Datensicherung und Wiederherstellung]] deutlich erhöht.