User:Janhoy/FAST Enterprise Search Platform (ESP)

FAST ESP (Enterprise Search Platform) is Microsoft's flagship Enterprise Search solution for "Internet Business", i.e. needs not using Sharepoint.

ESP, being Fast Search & Transfer's main product, is a generic search engine capable of indexing structured as well as unstructured content from a multitude of sources and content repositories including the web.

Main Features[edit]

Full-text search
Structured search of Strings, Numbers, Dates
Faceted search (Navigators)
Boolean query logic
Range query
Hit Highlighting
Sorting of results, including multi level sort
Document Processing Pipeline
Lemmatization in >40 languages
Spell Checking (Did You Mean)
Entity extraction
Automatic language detection
Document conversion for >220 document types

Competitive edge[edit]

ESP's main strength has been its fast sub-second search response time as well as its linear scalability capabilities. When growing in content volume, new columns of servers are added. When growing in query volume, new rows of servers are added. In addition the strong linguistics capabilities with i.e. automatic language detection for >80 languages, as well as advanced entity extraction has often been a competitive edge.

The scalability and query response advantages are however not unique anymore, as even open source engines like Apache Solr are able to match these.

Technology[edit]

ESP is written mainly in C++, Java and Python programming languages. The Document Processing Pipeline can be extended by users, writing Python scripts. Users can also interact with ESP using Web Services.

Platform support in version 5.3 includes Windows, Linux and Solaris-X86. Starting from 2010, Microsoft will continue development for the Windows platform only, abandoning Linux and Unix. ^[1]

The ESP index is an Inverted index, using tf-idf as its main relevancy formula. Relevancy is tunable, and other factors in relevancy can be:

Static document boost
Term boost
Freshness boost
Proximity boost
Phrase boost
Boolean operator boost
GEO distance
Query based document boost
Fixed placement of top-10 per query

Licensing model[edit]

ESP has been licensed for base system, with additional license options being available for extra features (such as high availability, extra languages or alerting) as well as extra capacity (QPS or GB storage).

Microsoft may plan to align licensing to its other products.

ESP in MS Office Sharepoint[edit]

After Microsoft's acquisition of FAST in 2008, they started incorporating the ESP technology into Office Sharepoint. The first release with ESP included is expected in Sharepoint 2010. The ESP search will be an add-on for advanced customers.