Elasticsearch Query-Time Strategies and Techniques for Relevance: Part II

In this article, we're going to look at some of the built-in tools that Elasticsearch provides for impacting relevance scores at query time. By manipulating a document's relevance score, we can give the impression of precision regardless of what kind of retrieval strategy we used. Take a look at our previous article about precision and recall strategies to decide which works best for your situation.

Make a Top 10 List

How often do you hear "search is broken"?

Users will decide how well they think search is working by looking at the first handful of results they see from a query (usually only the first couple). By having a solid top 10 list, you can influence users' perception of the search results to be more positive. You'll be putting the content they want front and center for them - what's not to like?

Dialing in which results get ranked higher in the retrieval set is a great strategy regardless of whether you're retrieving to improve recall or precision. Below we'll look at some techniques for giving the impression of precision (and typically then a more positive outlook from users), even if true precision is actually low.

For our examples in this article, we'll be focusing on getting our top 3 results dialed in based on querying for potential candidates for a social media position. Note that we'll be using the 2.x Query DSL syntax in our examples. If you're on a 1.x version of Elasticsearch, you'll want to check the official documentation for the DSL syntax as it exists in the version of Elasticsearch you're running.

Built-In Scoring

As we mentioned in Part I, all query types have built-in mechanisms for impacting a document's score (except for filters which simply provide a "yes" or "no" for a match). For example, boolean query types take the hierarchical structure of the query and the number of conditionals into consideration when calculating the relevance score for matched documents. For the dismax query type, additionally the combination of term-field matches will impact the relevancy score. So, depending on the query type you choose and how you structure it, the Practical Scoring Function will be applied somewhat differently. Check out our article on how scoring works in Elasticsearch to familiarize yourself with the basics of the Practical Scoring Function.

Beyond the built-in mechanisms, there are tools we can choose to apply at query time to affect a document's relevance score. That's what we'll focus on here.

Give it a Boost

Boosting is the easiest way to impact a document's relevance score. There are a few different ways that boosts can be applied. Here we'll look at field boosts, the boosting query, and the constant score, but also note that if you're searching across multiple indexes, you can boost more important indexes for matches as well.

Field Boost

When using a query type that is looking for matches in more than one field, you can boost the weight of matches found in specific fields. For example, you can specify a field boost of 2 for the "title" field to indicate that matches found in the title are always twice as relevant as matches from other fields.

In our previous article, we talked about using dis_max to query multiple fields to retrieve more documents based on term-field combinations. Let's add a field boost to our original dismax example for the social media job candidates so that matches in the field "qualifications" will weigh more highly in the relevance score than matches in the "cover_letter" field:

 {
   "query" : {
      "dis_max" : {
          "queries" : [
              { "match" : 
                { "qualifications" :
                  {
                    "query" : "social media",
                    "boost" : 2
                  }
                }
              },
              { "match" : 
                { "cover_letter" : "social media" }
              }
          ]
      }
   }
 }

Below are the results (the "hits") of this query. Five candidates were retrieved:

{
   "took" : 26,
   "_shards" : {
      "successful" : 3,
      "failed" : 0,
      "total" : 3
   },
   "timed_out" : false,
   "hits" : {
      "max_score" : 0.738416200000001,
      "total" : 5,
      "hits" : [
         {
            "_score" : 0.738416200000001,
            "_type" : "application",
            "_index" : "candidates_index",
            "_id" : "3",
            "_source" : {
               "department" : "Marketing",
               "location" : "Colorado",
               "name" : "Susan White",
               "qualifications" : "Social media, Twitter, Facebook, Social campaigns, Social advertising",
               "rating" : "4",
               "cover_letter" : "I do social. Call me for an interview.",
               "application_date" : "2016-03-10"
            }
         },
         {
            "_score" : 0.648669840000001,
            "_type" : "application",
            "_index" : "candidates_index",
            "_id" : "4",
            "_source" : {
               "name" : "Maria Gutierrez",
               "rating" : "5",
               "qualifications" : "Social media, Email marketing, Direct marketing",
               "department" : "Marketing",
               "location" : "Colorado",
               "application_date" : "2016-03-27",
               "cover_letter" : "I am a social media marketing professional with experience also in email and direct marketing campaigns."
            }
         },
         {
            "_type" : "application",
            "_index" : "candidates_index",
            "_score" : 0.54055816,
            "_id" : "2",
            "_source" : {
               "rating" : "2",
               "name" : "Tim Jones",
               "qualifications" : "Event coordination, Public relations, Media, Social events, Sporting events",
               "department" : "Marketing",
               "location" : "Arizona",
               "application_date" : "2016-03-20",
               "cover_letter" : "I am an events coordinator who would be perfect for your current opening."
            }
         },
         {
            "_id" : "1",
            "_source" : {
               "location" : "Colorado",
               "department" : "Marketing",
               "name" : "David Thomas",
               "rating" : "4",
               "qualifications" : "Twitter Facebook Evangelism",
               "application_date" : "2016-01-20",
               "cover_letter" : "I am looking forward to meeting you to discuss the opening you have in social media."
            },
            "_score" : 0.23721096,
            "_type" : "application",
            "_index" : "candidates_index"
         },
         {
            "_id" : "5",
            "_source" : {
               "qualifications" : "Twitter, Facebook",
               "name" : "Anne Wu",
               "rating" : "3",
               "location" : "California",
               "department" : "Marketing",
               "cover_letter" : "Social, social, social. I know I can help you be more successful with your social presence and campaigns.",
               "application_date" : "2016-02-28"
            },
            "_type" : "application",
            "_index" : "candidates_index",
            "_score" : 0.0632303600000001
         }
      ]
   }
}

We can identify an obvious dividing line here based on the relevance scores. We have 2 candidates who mentioned "social media" in their qualifications and 1 other who happened to have the term "media" and the term "social" in their qualifications. These 3 candidates ended up with much higher relevance scores (from 0.738 to 0.540) than the 2 candidates who did not mention "social media" in their qualifications (scores from 0.237 to 0.063). The lowest score which received the boost (0.540) is more than twice as high as the highest score which did not receive a boost (0.237). This clearly demonstrates the impact of boosting the query match by 2 for the "qualifications" field. Pretty straightforward.

Note that the ranges in the scores on either side of the dividing line are a result of the Practical Scoring Function (using term frequency and inverse document frequency) and the built-in mechanisms associated with the dis_max query type (term-field combinations). For example, our highest scorer "Susan White" at 0.738 not only has "social media" in the "qualifications" field (so she got the boost), but also mentions "social" two additional times in the "qualifications" field" and one additional time in the "cover_letter" field. It's that term frequency across multiple fields that is also contributing to the high score besides the boost we applied.

Boosting Query

The boosting query type lets us use a negative boost to downgrade documents where certain terms are matched that we believe will cause the document to less relevant. For example, someone searching for "pumps" in a catalog might mean the tool, not the women's shoe. We could use a boosting query to retrieve all documents where there was a match for "pumps", but then give a negative boost to the ones containing terms like "shoes" or "heels". We'd need to be careful, though, because if we gave a negative boost to the term "women", we might inadvertently downgrade breast pumps.

Let's get back to our social media candidates. We'll use the same dis_max query as above, but let's also add a negative boost for the term "events" since we want to downplay possible matches on "social events". To do that, we'll add the boosting query at a higher level in the query hierarchy, tuck our dismax in the "positive" grouping, and add our new negative match and boost:

 {
   "query" : {
     "boosting" : {
       "positive" : {
          "dis_max" : {
            "queries" : [
                { "match" : 
                  { "qualifications" :
                    {
                      "query" : "social media",
                      "boost" : 2
                    }
                  }
                },
                { "match" : 
                  { "cover_letter" : "social media" }
                }
            ]
          }
        },
        "negative" : {
          "match" : { "qualifications" : "events" }
        },
        "negative_boost" : 0.5
      }
   }
 }

Let's look at the relevance scores of the 5 candidates returned again, now that we also have a negative boost on the term "events" in the "qualifications" field:

{
   "timed_out" : false,
   "took" : 29,
   "_shards" : {
      "failed" : 0,
      "total" : 3,
      "successful" : 3
   }
   "hits" : {
      "total" : 5,
      "max_score" : 0.738416200000001
      "hits" : [
         {
            "_type" : "application",
            "_score" : 0.738416200000001,
            "_index" : "candidates_index",
            "_id" : "3",
            "_source" : {
               "cover_letter" : "I do social. Call me for an interview.",
               "name" : "Susan White",
               "location" : "Colorado",
               "department" : "Marketing",
               "qualifications" : "Social media, Twitter, Facebook, Social campaigns, Social advertising",
               "rating" : "4",
               "application_date" : "2016-03-10"
            }
         },
         {
            "_score" : 0.648669840000001,
            "_type" : "application",
            "_source" : {
               "name" : "Maria Gutierrez",
               "location" : "Colorado",
               "cover_letter" : "I am a social media marketing professional with experience also in email and direct marketing campaigns.",
               "rating" : "4",
               "qualifications" : "Social media, Email marketing, Direct marketing",
               "application_date" : "2016-03-27",
               "department" : "Marketing"
            },
            "_id" : "5",
            "_index" : "candidates_index"
         },
         {
            "_type" : "application",
            "_score" : 0.27027908,
            "_index" : "candidates_index",
            "_id" : "2",
            "_source" : {
               "cover_letter" : "I am an events coordinator who would be perfect for your current opening.",
               "name" : "Tim Jones",
               "location" : "Arizona",
               "department" : "Marketing",
               "rating" : "2",
               "qualifications" : "Event coordination, Public relations, Media, Social events, Sporting events",
               "application_date" : "2016-03-20"
            }
         },
         {
            "_source" : {
               "name" : "David Thomas",
               "location" : "Colorado",
               "cover_letter" : "I am looking forward to meeting you to discuss the opening you have in social media.",
               "qualifications" : "Twitter Facebook Evangelism",
               "rating" : "4",
               "application_date" : "2016-01-20",
               "department" : "Marketing"
            },
            "_index" : "candidates_index",
            "_id" : "1",
            "_score" : 0.23721096,
            "_type" : "application"
         },
         {
            "_index" : "candidates_index",
            "_id" : "5",
            "_source" : {
               "name" : "Anne Wu",
               "cover_letter" : "Social, social, social. I know I can help you be more successful with your social presence and campaigns.",
               "location" : "California",
               "rating" : "3",
               "qualifications" : "Twitter, Facebook",
               "application_date" : "2016-02-28",
               "department" : "Marketing"
            },
            "_type" : "application",
            "_score" : 0.0632303600000001
         }
      ]
   }
}

What we can see here is that all our candidates scores stayed exactly the same, except for the candidate who mentioned "events" in the "qualifications" field. That candidate's relevance score was effectively cut in half by our negative boost. In the previous query that candidate had received a score of 0.540, but now the score is 0.270, which now puts this candidate below our dividing line. So, even though this candidate received the field boost from having a match to "social" and "media" in the "qualifications" field, the negative boost from also having "events" in the "qualifications" field knocked them down significantly. As you can see, the boosting query can be quite powerful in pushing down results that are less relevant.

Constant Score

constant_score lets you negate the built-in scoring mechanisms (such as those described in our article on scoring in Elasticsearch) for whatever query or filter it wraps. This lets you match documents with particular characteristics, but be able to manually set the score received for matches by using a boost. If you do not add a boost or you use a boost of 1, then documents that match the characteristics will not have their scores affected by the match. If you set a boost higher than 1, then the document's relevance score will be raised accordingly. Let's go ahead and add a constant score to our query so that candidates located in Colorado will be weighted more highly:

 {
   "query" : {
     "boosting" : {
       "positive" : {
          "dis_max" : {
            "queries" : [
                { "match" : 
                  { "qualifications" :
                    {
                      "query" : "social media",
                      "boost" : 2
                    }
                  }
                },
                { "match" : 
                  { "cover_letter" : "social media" }
                },
                { "constant_score" :
                   { "query" : 
                     { "term" : { "location" : "colorado" }},
                       "boost" : 2
                   }
                }
             ]
          }
        },
        "negative" : {
          "match" : { "qualifications" : "events" }
        },
        "negative_boost" : 0.5
      }
   }
 }

Let's see how this affects our 5 candidates:

{
   "_shards" : {
      "failed" : 0,
      "successful" : 3,
      "total" : 3
   },
   "timed_out" : false,
   "took" : 55,
   "hits" : {
      "max_score" : 0.738416200000001,
      "total" : 5,
      "hits" : [
         {
            "_score" : 0.738416200000001,
            "_id" : "3",
            "_source" : {
               "department" : "Marketing",
               "name" : "Susan White",
               "location" : "Colorado",
               "qualifications" : "Social media, Twitter, Facebook, Social campaigns, Social advertising",
               "cover_letter" : "I do social. Call me for an interview.",
               "rating" : "4",
               "application_date" : "2016-03-10"
            },
            "_index" : "candidates_index",
            "_type" : "application"
         },
         {
            "_index" : "candidates_index",
            "_source" : {
               "name" : "Maria Gutierrez",
               "department" : "Marketing",
               "location" : "Colorado",
               "qualifications" : "Social media, Email marketing, Direct marketing",
               "cover_letter" : "I am a social media marketing professional with experience also in email and direct marketing campaigns.",
               "rating" : "5",
               "application_date" : "2016-03-27"
            },
            "_id" : "4",
            "_score" : 0.648669840000001,
            "_type" : "application"
         },
         {
            "_type" : "application",
            "_id" : "1",
            "_score" : 0.5781061,
            "_index" : "candidates_index",
            "_source" : {
               "name" : "David Thomas",
               "department" : "Marketing",
               "rating" : "4",
               "application_date" : "2016-01-20",
               "location" : "Colorado",
               "qualifications" : "Twitter Facebook Evangelism",
               "cover_letter" : "I am looking forward to meeting you to discuss the opening you have in social media."
            }
         },
         {
            "_type" : "application",
            "_index" : "candidates_index",
            "_source" : {
               "location" : "Arizona",
               "cover_letter" : "I am an events coordinator who would be perfect for your current opening.",
               "qualifications" : "Event coordination, Public relations, Media, Social events, Sporting events",
               "rating" : "2",
               "application_date" : "2016-03-20",
               "name" : "Tim Jones",
               "department" : "Marketing"
            },
            "_score" : 0.27027908,
            "_id" : "2"
         },
         {
            "_score" : 0.0632303600000001,
            "_id" : "5",
            "_source" : {
               "application_date" : "2016-02-28",
               "rating" : "3",
               "cover_letter" : "Social, social, social. I know I can help you be more successful with your social presence and campaigns.",
               "qualifications" : "Twitter, Facebook",
               "location" : "California",
               "name" : "Anne Wu",
               "department" : "Marketing"
            },
            "_index" : "candidates_index",
            "_type" : "application"
         }
      ]
   }
}

What we see now is that all our candidates scores are the same as our last query except for the candidate named "David Thomas". Because he is located in Colorado, his score was boosted higher, now making him the third-ranked candidate. All the Colorado candidates are in positions 1-3 now.

A Note about Query Types and Query Structure

You may be asking yourself... "But why didn't the constant score affect the scores of the other two Colorado candidates?" It's because, in this example, the constant score is used within the dismax query and so, for their scores, the boost received from the constant score was not enough to alter their overall dismax score. If we moved the constant_score clause outside of the dismax and instead wrapped it in a boolean query alongside the dismax query, then we'd see all the Colorado candidates' scores being impacted by the constant score.

As we mentioned above, the query types you use and how you structure your queries hierarchically can have a big impact on the relevance scores of the documents retrieved. We recommend that you play around with the different query types and query structure to understand how they will affect ranking your result set.

Get Func-y

Function scoring is Elasticsearch's toolbox of functions that allows us to manipulate relevance scores beyond what we've already looked at. There are many functions, but let's look at a couple of the most useful for dialing in our top 10 list. Note that you can even write your own custom function if none of the predefined ones work for your situation!

field_value_factor

The field_value_factor lets us use the value of an indexed field in the document to impact the document's relevance score. For example, if we had blog articles where we tracked the number of "likes" each received, we could use the value of the "likes" field to increase the score of the most popular articles.

For our hypothetical candidate records, we've gone through and given a pre-interview rating to each record on a scale of 1-5, with 5 being the best rating. We can now use that to give higher relevancy to candidates with higher ratings:

 {
   "query" : {
     "boosting" : {
       "positive" : {
         "function_score" : {
           "query" : {
             "dis_max" : {
               "queries" : [
                   { "match" : 
                     { "qualifications" :
                       {
                         "query" : "social media",
                         "boost" : 2
                       }
                     }
                   },
                   { "match" : 
                     { "cover_letter" : "social media" }
                   },
                   { "constant_score" :
                      { "query" : 
                        { "term" : { "location" : "colorado" }},
                          "boost" : 2
                      }
                    }
                  ]
                }
              },
              "functions" : [
                { "field_value_factor" :
                  { "field" : "rating" }
                }
              ]
            }
          },
          "negative" : {
            "match" : { "qualifications" : "events" }
           },
           "negative_boost" : 0.5
       }
    }
 }

For adding the field_value_factor to our query, we're wrapping the dismax query in a query clause. We are then wrapping that in a function_score clause so that the function score contains the dismax query and then also contains the functions we'll add - the first of these is the field_value_factor.

In the field_value_factor function, we're using the "rating" field without a modifier or other additional parameters in our example, though there are many options to choose from for this function. Using the value of the field directly as we're doing here is the simplest approach, but you may want to test what works best with your documents.

How did our candidates fare with the addition of this function? Let's see:

{
   "_shards" : {
      "total" : 3,
      "successful" : 3,
      "failed" : 0
   },
   "timed_out" : false,
   "took" : 12,
   "hits" : {
      "max_score" : 3.243349,
      "total" : 5,
      "hits" : [
         {
            "_index" : "candidates_index",
            "_source" : {
               "department" : "Marketing",
               "location" : "Colorado",
               "cover_letter" : "I am a social media marketing professional with experience also in email and direct marketing campaigns.",
               "name" : "Maria Gutierrez",
               "qualifications" : "Social media, Email marketing, Direct marketing",
               "application_date" : "2016-03-27",
               "rating" : 5
            },
            "_score" : 3.243349,
            "_id" : "4",
            "_type" : "application"
         },
         {
            "_index" : "candidates_index",
            "_source" : {
               "name" : "Susan White",
               "cover_letter" : "I do social. Call me for an interview.",
               "department" : "Marketing",
               "location" : "Colorado",
               "rating" : 4,
               "application_date" : "2016-03-10",
               "qualifications" : "Social media, Twitter, Facebook, Social campaigns, Social advertising"
            },
            "_score" : 2.9536648,
            "_id" : "3",
            "_type" : "application"
         },
         {
            "_type" : "application",
            "_score" : 2.3124244,
            "_id" : "1",
            "_source" : {
               "name" : "David Thomas",
               "cover_letter" : "I am looking forward to meeting you to discuss the opening you have in social media.",
               "department" : "Marketing",
               "location" : "Colorado",
               "rating" : 4,
               "qualifications" : "Twitter Facebook Evangelism",
               "application_date" : "2016-01-20"
            },
            "_index" : "candidates_index"
         },
         {
            "_index" : "candidates_index",
            "_source" : {
               "department" : "Marketing",
               "location" : "Arizona",
               "name" : "Tim Jones",
               "cover_letter" : "I am an events coordinator who would be perfect for your current opening.",
               "qualifications" : "Event coordination, Public relations, Media, Social events, Sporting events",
               "application_date" : "2016-03-20",
               "rating" : 2
            },
            "_score" : 0.54055816,
            "_id" : "2",
            "_type" : "application"
         },
         {
            "_score" : 0.18969107,
            "_id" : "5",
            "_type" : "application",
            "_index" : "candidates_index",
            "_source" : {
               "name" : "Anne Wu",
               "cover_letter" : "Social, social, social. I know I can help you be more successful with your social presence and campaigns.",
               "location" : "California",
               "department" : "Marketing",
               "rating" : 3,
               "qualifications" : "Twitter, Facebook",
               "application_date" : "2016-02-28"
            }
         }
      ]
   }
}

The ratings have made a large impact on scores. "Maria Gutierrez", who has a rating of 5, now holds the number 1 position in our search results with a score of 3.243. "Susan White" has now moved into the 2nd position with a score of 2.953 and "David Thomas" is in the 3rd spot with a score of 2.312. The other two candidates are lagging far behind now.

Decay Functions

Decay functions (which can be applied as linear, exponential, or gaussian) can be used to adjust the relevancy score of a document based on a sliding scale. Common uses include:

Let's add a date decay function to our query. In this case we're using an exponential decay with our scale at 30 days and an offset of 7 days. Since the origin date will default to now, basically what this means is that candidates whose application dates are within the past week will not be affected by the decay function. Those that fall in the next 30 days will be weighted less in terms of relevancy and so on for every 30 day period. We're using a decay parameter of 0.8 for score impact:

 {
   "query" : {
     "boosting" : {
       "positive" : {
         "function_score" : {
           "query" : {
             "dis_max" : {
               "queries" : [
                   { "match" : 
                     { "qualifications" :
                       {
                         "query" : "social media",
                         "boost" : 2
                       }
                     }
                   },
                   { "match" : 
                     { "cover_letter" : "social media" }
                   },
                   { "constant_score" :
                      { "query" : 
                        { "term" : { "location" : "colorado" }},
                          "boost" : 2
                      }
                    }
                  ]
                }
              },
              "functions" : [
                { "field_value_factor" :
                  { "field" : "rating" }
                },
                { "exp" :
                  { "application_date" :
                    { "scale" : "30d", 
                      "offset" : "7d",
                      "decay" : 0.8 }
                  }
                }
              ]
            }
          },
          "negative" : {
            "match" : { "qualifications" : "events" }
           },
           "negative_boost" : 0.5
       }
    }
 }

Since our candidates applied on different dates, some of them will be affected by this function:

{
   "timed_out" : false,
   "_shards" : {
      "successful" : 3,
      "failed" : 0,
      "total" : 3
   },
   "took" : 12,
   "hits" : {
      "hits" : [
         {
            "_source" : {
               "cover_letter" : "I am a social media marketing professional with experience also in email and direct marketing campaigns.",
               "location" : "Colorado",
               "application_date" : "2016-03-27",
               "qualifications" : "Social media, Email marketing, Direct marketing",
               "rating" : 5,
               "name" : "Maria Gutierrez",
               "department" : "Marketing"
            },
            "_type" : "application",
            "_score" : 3.243349,
            "_id" : "4",
            "_index" : "candidates_index"
         },
         {
            "_id" : "3",
            "_index" : "candidates_index",
            "_type" : "application",
            "_source" : {
               "cover_letter" : "I do social. Call me for an interview.",
               "application_date" : "2016-03-10",
               "location" : "Colorado",
               "qualifications" : "Social media, Twitter, Facebook, Social campaigns, Social advertising",
               "rating" : 4,
               "department" : "Marketing",
               "name" : "Susan White"
            },
            "_score" : 2.6982698
         },
         {
            "_id" : "1",
            "_index" : "candidates_index",
            "_source" : {
               "qualifications" : "Twitter Facebook Evangelism",
               "application_date" : "2016-01-20",
               "location" : "Colorado",
               "cover_letter" : "I am looking forward to meeting you to discuss the opening you have in social media.",
               "department" : "Marketing",
               "name" : "David Thomas",
               "rating" : 4
            },
            "_type" : "application",
            "_score" : 1.4563811
         },
         {
            "_type" : "application",
            "_source" : {
               "cover_letter" : "I am an events coordinator who would be perfect for your current opening.",
               "application_date" : "2016-03-20",
               "location" : "Arizona",
               "qualifications" : "Event coordination, Public relations, Media, Social events, Sporting events",
               "rating" : 2,
               "name" : "Tim Jones",
               "department" : "Marketing"
            },
            "_score" : 0.5319489,
            "_index" : "candidates_index",
            "_id" : "2"
         },
         {
            "_type" : "application",
            "_source" : {
               "department" : "Marketing",
               "name" : "Anne Wu",
               "rating" : 3,
               "location" : "California",
               "application_date" : "2016-02-28",
               "cover_letter" : "Social, social, social. I know I can help you be more successful with your social presence and campaigns.",
               "qualifications" : "Twitter, Facebook"
            },
            "_score" : 0.15967517,
            "_id" : "5",
            "_index" : "candidates_index"
         }
      ],
      "total" : 5,
      "max_score" : 3.243349
   }
}

Since our top-ranked candidate "Maria Gutierrez" applied within the past week, her score isn't affected by our decay function and remains a solid 3.243. Our 2nd-ranked candidate "Susan White" did not apply during the offset period, but was within the first 30 days of the scale. Because of that her score has only been decayed to 2.698. Our 3rd-ranked candidate "David Thomas" applied more than two months ago so his score is much more significantly affected, getting knocked down to 1.45. Still, it is not enough of a decay to push him below "Tim Jones" and "Anne Wu" who both applied more recently than he did. The relevancy of the other matches that "David Thomas" has in the query are more important than the decay we set on the application date.

Rescoring

And finally, when all is said and done, we can take the top n results of our result set and rescore within just that subset to ensure the most relevant are the first ones shown to the user.

For us, though, we've been able to dial in our top 3 and rank them accordingly.

In Summary

We've covered a lot in this article series, but there are many more query-time options in Elasticsearch that we didn't look at. We hope, however, we've given you some food for thought as to how you could begin to tune your search results to better serve your users and give them what they're expecting.

Here's what to do next:

With some time spent upfront and some regular check-ins and maintenance tuning along the way, you're not going to hear "search is broken" very often anymore. Satisfied users = happy developers.

Be happy.