A question I have been getting a lot lately is this: “With all the improvements in the out-of-the-box search experience in SharePoint 2010, why would I go with FAST Search for SharePoint? What are the main motivations to do that?”
I will give you my view on this topic, but first let’s have a look at the main differentiators for FAST Search for SharePoint as listed in the official SharePoint 2010 editions comparison based on Search features:
- Advanced Content Processing
- Advanced Sorting
- Contextual Search
- Deep Refinement
- Extensible Search Platform
- Extreme Scale Search
- Rich Web Indexing
- Similar Results
- Thumbnails and Previews
- Tunable Relevance with Multiple Rank Profiles
- Visual Best Bets
At a first glance you probably already recognize some of the main features that everyone talks about in regards to FAST Search for SharePoint (Extreme Scale Search, Thumbnails and Previews, Extensible Search Platform, etc.), so I will focus on some of the other features that I find important and try to highlight with examples when they can be helpful.
Advanced Content Processing
With FAST Search for SharePoint you get the same crawling capabilities also available in SharePoint Search in regards to obtaining data from multiple types of content sources (SharePoint sites, Web sites, File Shares, Exchange Public Folders, Databases through BCS, etc.), but with the added bonus of being able to apply custom processing rules to content coming from all of these distinct content sources.
For example, out-of-the-box FS4SP will attempt to extract a list of Companies mentioned in your content. Be it from the name of a company listed on a Web page or mentioned on a PDF document in a file share, FS4SP will extract this information during content processing and will expose this through the managed property companies, allowing your users to execute queries like: companies:”Microsoft”. This would return all documents for which FS4SP has identified mentions of the company Microsoft. To perform this extraction, FS4SP uses a combination of dictionaries and linguistics rules to determine that XYZ Corp. or ABC Inc. are company names.
Now take this same logic and imagine that you probably also have similar types of important metadata in your business that users mention in documents, web pages, etc. and it would be extremely helpful for your business to be able to allow users to search for this metadata without the need to effectively apply this metadata in the original content. Think of “Projects” for a Consulting company, or “Chemical substances” for a Pharmaceutical firm, or even “Legal Cases” for a Legal department. All of those are important information that your users already mentioned in their documents, they just haven’t been explicitly defined as metadata, but with FS4SP you can perform this extraction from one single location for all of your content, no matter where it’s coming from. This extraction can be a simple exact
/case sensitive match (with SP1 you can now do case insensitive match) using Property Extraction, or it can be more advanced types of extraction (including lookups in databases or web services, if necessary) through the Pipeline Extensibility.
For more information about these features, you can watch the Content Processing and Property Extraction training videos available on MSDN or read the documentation about the Pipeline Extensibility or about adding a Custom Property Extractor.
The beauty about the deep refiner is that it makes easier to drilldown through the resultset and discover content you probably wouldn’t otherwise (since most users don’t ever click past the first page of results).
Think for example about a scenario where a user is searching for “relevance tuning” and gets back the first page of results with the top 10 hits out of a total of 10000 results (10K items that match the search terms). Imagine now that the search center is configured to display a refiner by Author to make it easier for users to find what they are looking for. So here is what it would happen with either a “shallow” refiner or a “deep” refiner by Author:
- With SharePoint Search “shallow” refiners, only the values for the Author property on the top 50 items(default configuration) in the resultset would be considered for display. So even if an Author single-handedly wrote half of the documents in your resultset (5000), but for some reason none of his documents are in the top 50 results, he would not be displayed and the user would never know about this.
- With FS4SP deep refiners, the full resultset of 10000 documents is considered so that this author that wrote half of the documents will easily appear at the top of the Author refiner. The user then can look at this, notice “wow, this guy wrote half of the documents, he must be an expert on this topic” and click on it to filter to show only the documents with this Author.
As you noticed in the examples above, deep refiners can be a very powerful way to provide your users with a quick snapshot of the most important metadata in your content. And with FAST Search for SharePoint you can even go beyond the string-based refiner. You can also get dynamically populated refiners based on numerical or datetime properties (also based on your full resultset). But this is a topic for another post.
Advanced Query Language
Another very powerful feature available only with FS4SP is the ability to use the FAST Query Language (FQL). This may seem a little bit obscure at a first look, since none of the out-of-the-box web parts support the use of FQL, but it becomes a really powerful tool when you build custom search applications using the Query Web Service or even custom web parts using the Query Object Model.
FQL offers you a wide range of query operators to perform more advanced queries, and among those my absolute favorite is the XRANK operator. What this operator gives you is a way to exert very fine grain control over the ranking of documents, as you can see in these examples of FQL queries:
XRANK(“erp”,language:”en”,boost=10000) – This will send a query for “erp” and give an extra boost of 10000 points for documents with language “en” (English). Consider a scenario where you have a large intranet deployment with documents in multiple languages, but some of your users want to give priority to documents in their native language. Your search application could allow users to select which language to give priority to, or simply apply the priority based on the office location of the user. The important thing to note is that you are giving a priority to documents of a specific language, instead of filtering out all documents in other languages. That’s a big difference.
XRANK(“washington”,category:OR(“Fiction”,”Bestseller”),boost=15000) – Imagine an ecommerce site that has the attribute category for all their books and based on past purchases (or wishlist) it was identified that a specific user typically buy Fiction or Bestseller books. So, when this user executes a query for the term “washington” the search application adjusts the query to give a higher priority to books belonging to the user’s favorite categories, without excluding the other books, but recognizing that the user has a specific preference. Now imagine that the same could apply to an intranet that detects that a specific user has a preference for PowerPoint presentations or maybe a preference for documents of a specific Content Type, and the search application automatically applies an extra boost giving a higher priority for those types of content.
These are just a couple examples of the power of XRANK, and there are many other interesting FQL operators that give you a lot of customization options to build advanced search applications.
Tunable Relevance with Multiple Rank Profiles
To put it simply, a rank profile defines all the weights for the many parameters that are considered when computing the overall rank score of a search result. This is not something unique with FS4SP, as SharePoint Search also gives you some flexibility to change the ranking model. What is really unique about FS4SP is that it allows you to configure multiple rank profiles, therefore allowing you (and your users) to define which rank profile should be used according to the specific search needs at the moment.
Consider a scenario where a user may want to search for items giving priority for more recent content, but not necessarily sorting by date (in case there is a highly relevant document that is a little bit older than others, but still more relevant). Or another scenario where this same user wants to give priority to items that have high ratings.
With rank profiles you basically have detailed control over all factors related to ranking computation, including freshness (how recent is the document), context (where the match occurred in the document – e.g. title vs. body), proximity (when searching for multiple terms, how close they are to each other in this document), etc.. And with multiple rank profiles you can configure a few predefined settings and allow your users to select which one they want to use with just a click from the search results page.
There are many situations where FAST Search for SharePoint will go the extra mile to give you more flexibility and custom options to create great search applications, and you should know what are these situations, not only to be sure you will acquire FS4SP only when you need it, but also to make sure that you leverage all its power once you have acquired it.
Update: I just found out about this great article that also highlights “Three Main Reasons Why You Should Upgrade to FAST for SharePoint” (and two of them are the same reasons I pointed out )