Difference between revisions of "Search Indexes"

[unchecked revision][checked revision]
(Created page with "= Administration of the Full-Text Search = MailStore Server offers users an extremely fast full-text search. All emails a user has read-access to are searched, in most cases in ...")
 
 
(42 intermediate revisions by 5 users not shown)
Line 1: Line 1:
= Administration of the Full-Text Search =
 
 
 
MailStore Server offers users an extremely fast full-text search. All emails a user has read-access to are searched, in most cases in only fractions of a second. To ensure this remarkable speed, MailStore Server sets up so-called search indexes during archiving. They work in a way similar to the indexes often found in the back of books: looking up something in an index gets results significantly faster than searching each single page.
 
MailStore Server offers users an extremely fast full-text search. All emails a user has read-access to are searched, in most cases in only fractions of a second. To ensure this remarkable speed, MailStore Server sets up so-called search indexes during archiving. They work in a way similar to the indexes often found in the back of books: looking up something in an index gets results significantly faster than searching each single page.
  
 
MailStore Server maintains one index file each   
 
MailStore Server maintains one index file each   
  
*per file group and
+
* per file group and
*per user.  
+
* per user.  
  
Normally, the creation, maintenance and usage of search indexes is completely transparent meaning that neither administrators nor MailStore users need to know of their existence or their internal workings. In some cases, however, some maintenance may become necessary. For example:
+
MailStore Server can index all file types for which a so-called IFilter is installed on the MailStore Server computer. Typically, IFilters exist at least for all applications which are installed on the respective machines.
  
*because of power outage,
+
For reasons of stability and performance, MailStore Server processes the following file types directly, regardless of the IFilters that are installed:
*because of unexpected termination of the MailStore Server service
 
*because of missing network connectivity (only with storage on an NAS) or
 
*because of changes to the index configuration (see below).
 
  
In these cases it may occur that archive, index and settings are no longer synchronous and that individual indexes must be rebuilt. Generally, a corresponding error message will be displayed.
+
* Text files (TXT)
 +
* HTML files (HTM and HTML)
 +
* PDF files (PDF)
  
 
Typical tasks regarding indexes are described in the following sections.
 
Typical tasks regarding indexes are described in the following sections.
  
== Setting Up Indexing for the Contents of File Attachments ==
+
== Install Missing IFilters ==
 +
Typically, IFilters exist at least for all applications which are installed on the respective machines. If, for example, Microsoft Office 2013 is installed, the corresponding IFilter for Microsoft Office documents are installed as well.
  
In the standard configuration, MailStore Server includes the file names of file attachments in the search indexes but not their contents. To enable MailStore Server to search the contents of file attachments, it has to be configured accordingly. Please proceed as follows:
+
Install the following IFilters on the MailStore Server computer to index the corresponding file types. Restart the MailStore Server service after the IFilter installation to let MailStore detect the newly installed IFilters.
  
*Start ''MailStore Client'' and log on as administrator (''admin'').
+
* '''Plain Text Files (TXT, CSV)'''<br/>The IFilter responsible for these file types is shipped with Windows by default. In case this option is disabled, a registry values might be wrong. Open the registry editor and verify that the ''Default'' value of the key ''HKEY_LOCAL_MACHINE\SOFTWARE\Classes\.csv\PersistentHandler'' is set to ''{5e941d80-bf96-11cd-b579-08002b30bfeb}'' and correct it if necessary.
*Click on ''Administrative Tools''.
+
* '''Microsoft Office (97-2003), Microsoft Office (2007 and later),'''<br/>All these file types are supported by the [https://www.microsoft.com/en-us/download/details.aspx?id=17062 Microsoft Office 2010 Filter Pack].
*Open the page ''Search Indexes''.
+
* '''Open Document Format (Libre Office/Open Office)'''<br/>These file types require a working installation of OpenOffice or [https://www.libreoffice.org/ LibreOffice]. Latest version tested was ''LibreOffice 5.1''.
*Click on ''Options''.
+
*: '''Notice:''' Though Microsoft's Office 2010 Filter Pack registers support for Open Document Format files, indexing does not work with that IFilters.
*Separated by space, enter the file types (file extensions) whose contents MailStore Server is to include in the search indexes.
 
*Click on OK to save the settings.
 
*The new settings apply to all emails that are archived as of now. To apply the settings to already archived emails, rebuild the appropriate search indexes as described below.
 
  
[[File:tech_index_02.png|center|450px]]
+
Additional information about IFilters can be found in the corresponding [[wikipedia:IFilter|Wikipedia article]].
  
MailStore Server can index all file types for which a so-called IFilter driver is installed. When archiving, the drivers must be installed on the MailStore Client machine, when rebuilding indexes they must be installed on MailStore Server. Typically, IFilters exist at least for all applications which are installed on the respective machines.
+
== Setting Up Indexing for the Contents of File Attachments ==
 +
In the standard configuration, MailStore Server includes the file names of file attachments in the search indexes but not their contents. To enable MailStore Server to search the contents of file attachments, it has to be configured accordingly. Please proceed as follows:
  
If, for example, Microsoft Office 2007 is installed, the corresponding IFilter drivers for Mirosoft Office documents are installed as well. IFilter drivers can also be installed independently from their applications. To learn more, please enter IFilter into any search engine.
+
* Start ''MailStore Client'' and log on as administrator (''admin'').
 +
* Click on ''Administrative Tools'' > ''Storage'' and then on ''Search Indexes''.
 +
* In the area ''Attachments'' click on ''Change...''
 +
* Select the file type groups or enter a space separated list of file extensions of attachment types you would like archive in the text field below Other File Extensions.
 +
*: [[File:tech_index_02-9_7.png|center]]
 +
* Click on ''OK'' to save the settings.
  
For reasons of stability and performance, MailStore Server processes the following file types directly, regardless of the IFilter drivers that are installed:
+
MailStore displays a notice, if file extensions were added for which no IFilter is installed on the MailStore Server machine.
  
*Text files (TXT),
+
The new settings apply to all emails that are archived as of now. To apply the settings to already archived emails, rebuild the appropriate search indexes as described below.  
*HTML files (HTM and HTML)
 
*PDF files (PDF).
 
  
 
== Rebuilding Search Indexes ==
 
== Rebuilding Search Indexes ==
 +
Normally, the creation, maintenance and usage of search indexes is completely transparent meaning that neither administrators nor MailStore users need to know of their existence or their internal workings. In some cases, however, some maintenance may become necessary. For example:
  
If new indexing settings are to be applied to all existing emails or a message appears indicating that search indexes need to be rebuilt, please proceed as follows:
+
* because of power outage,
 
+
* because of unexpected termination of the MailStore Server service
*Start MailStore Client and log on as administrator (admin).
+
* because of missing network connectivity (only with storage on an NAS),
*Click on Administrative Tools.
+
* because of changes to the index configuration or
*Open the page Search Indexes.
+
* when restoring restoring backups without search indexes.
*Check all search indexes to be rebuilt. Indexes with the status Please Rebuild are already checked for your convenience.
 
*Click on Rebuild Search Indexes.
 
 
 
[[File:tech_index_01.png|center|450px]]
 
 
 
== Frequently Asked Questions Regarding Search Indexes ==
 
 
 
'''Why are the contents of file attachments not indexed in the standard configuration?'''
 
 
 
Indexing the contents of file attachments slows down the archiving process and increases the size of the search indexes significantly. Because of this, the contents of file attachments are not indexed in the standard configuration.
 
 
 
'''Everything was set up correctly but I still cannot find some text within the archived file attachments. What could be the reason?'''
 
 
 
First, this may be because, due to performance and stability, MailStore Server only indexes the first 10,000 Words (not characters!) of file attachments. In most cases, keywords are contained within the first 10,000 Words. If an email has multiple file attachments, at most 10,000 words are indexed overall. Of course, the file attachments themselves are archived in their entirety and can be searched in full with the appropriate application.
 
  
Another reason could be that MailStore Server was unable to read the corresponding file because no IFilter driver was available or because the file was DRM protected (copy protection), for example.
+
In these cases it may occur that archive, index and settings are no longer synchronous and that individual indexes must be rebuilt. Generally, a corresponding error message will be displayed. In case multiple indexes should be rebuilt, we recommend to start MailStore Server in [[MailStore_Server_Service_Configuration#Controlling_the_Service|Safe Mode]] to avoid interference by background tasks such as archiving profiles, exporting profiles, jobs or user interaction.
  
'''In the Status column, ''OK'' (Compatibility mode) is displayed. What should be done?'''
+
To rebuild search indexes please proceed as follows:
  
The search index was created with an older version of MailStore Server and has an internal structure different from indexes which were built using the current version. No action is required. However, these indexes can be upgraded by rebuilding them. This significantly increases indexing and search performance.
+
* Start MailStore Client and log on as administrator (admin).
 +
* Click on ''Administrative Tools'' > ''Storage'' and then on ''Search Indexes''.
 +
* Check all search indexes to be rebuilt. Indexes with the status ''Please Rebuild'' are already checked for your convenience.
 +
* Click on ''Rebuild Search Indexes''.
 +
*: [[File:tech_index_01.png|center|450px]]
  
[[de:Suchindizez]
+
[[de:Suchindizes]]
 +
[[en:Search Indexes]]

Latest revision as of 10:24, 25 August 2021

MailStore Server offers users an extremely fast full-text search. All emails a user has read-access to are searched, in most cases in only fractions of a second. To ensure this remarkable speed, MailStore Server sets up so-called search indexes during archiving. They work in a way similar to the indexes often found in the back of books: looking up something in an index gets results significantly faster than searching each single page.

MailStore Server maintains one index file each

  • per file group and
  • per user.

MailStore Server can index all file types for which a so-called IFilter is installed on the MailStore Server computer. Typically, IFilters exist at least for all applications which are installed on the respective machines.

For reasons of stability and performance, MailStore Server processes the following file types directly, regardless of the IFilters that are installed:

  • Text files (TXT)
  • HTML files (HTM and HTML)
  • PDF files (PDF)

Typical tasks regarding indexes are described in the following sections.

Install Missing IFilters

Typically, IFilters exist at least for all applications which are installed on the respective machines. If, for example, Microsoft Office 2013 is installed, the corresponding IFilter for Microsoft Office documents are installed as well.

Install the following IFilters on the MailStore Server computer to index the corresponding file types. Restart the MailStore Server service after the IFilter installation to let MailStore detect the newly installed IFilters.

  • Plain Text Files (TXT, CSV)
    The IFilter responsible for these file types is shipped with Windows by default. In case this option is disabled, a registry values might be wrong. Open the registry editor and verify that the Default value of the key HKEY_LOCAL_MACHINE\SOFTWARE\Classes\.csv\PersistentHandler is set to {5e941d80-bf96-11cd-b579-08002b30bfeb} and correct it if necessary.
  • Microsoft Office (97-2003), Microsoft Office (2007 and later),
    All these file types are supported by the Microsoft Office 2010 Filter Pack.
  • Open Document Format (Libre Office/Open Office)
    These file types require a working installation of OpenOffice or LibreOffice. Latest version tested was LibreOffice 5.1.
    Notice: Though Microsoft's Office 2010 Filter Pack registers support for Open Document Format files, indexing does not work with that IFilters.

Additional information about IFilters can be found in the corresponding Wikipedia article.

Setting Up Indexing for the Contents of File Attachments

In the standard configuration, MailStore Server includes the file names of file attachments in the search indexes but not their contents. To enable MailStore Server to search the contents of file attachments, it has to be configured accordingly. Please proceed as follows:

  • Start MailStore Client and log on as administrator (admin).
  • Click on Administrative Tools > Storage and then on Search Indexes.
  • In the area Attachments click on Change...
  • Select the file type groups or enter a space separated list of file extensions of attachment types you would like archive in the text field below Other File Extensions.
    Tech index 02-9 7.png
  • Click on OK to save the settings.

MailStore displays a notice, if file extensions were added for which no IFilter is installed on the MailStore Server machine.

The new settings apply to all emails that are archived as of now. To apply the settings to already archived emails, rebuild the appropriate search indexes as described below.

Rebuilding Search Indexes

Normally, the creation, maintenance and usage of search indexes is completely transparent meaning that neither administrators nor MailStore users need to know of their existence or their internal workings. In some cases, however, some maintenance may become necessary. For example:

  • because of power outage,
  • because of unexpected termination of the MailStore Server service
  • because of missing network connectivity (only with storage on an NAS),
  • because of changes to the index configuration or
  • when restoring restoring backups without search indexes.

In these cases it may occur that archive, index and settings are no longer synchronous and that individual indexes must be rebuilt. Generally, a corresponding error message will be displayed. In case multiple indexes should be rebuilt, we recommend to start MailStore Server in Safe Mode to avoid interference by background tasks such as archiving profiles, exporting profiles, jobs or user interaction.

To rebuild search indexes please proceed as follows:

  • Start MailStore Client and log on as administrator (admin).
  • Click on Administrative Tools > Storage and then on Search Indexes.
  • Check all search indexes to be rebuilt. Indexes with the status Please Rebuild are already checked for your convenience.
  • Click on Rebuild Search Indexes.
    Tech index 01.png