Addressing search misses due to search dropping

When a search query does not yield the expected results (commonly referred to as "Search Misses"), the system takes proactive measures to enhance your search experience.

Term dropping, a key strategy in this process, is applied based on the configured term-dropping priority. When term dropping occurs, a new query is executed, refining the search term to increase the likelihood of finding relevant products. This term dropping, combined with search term breakdown, is reflected in the response JSON data as part of metaData under searchExecution.

Identifying Term Dropping in `SearchExecution`

Each time a term-dropping operation is executed, a new node is added under searchExecution. This node includes all the search term breakdown information. As a result, the presence of multiple nodes within the searchExecution list signifies the iterations and adaptations made to the search term after each term dropping operation.

The following is an example of term-dropping in the searchExecution section of the response JSON:

{
    "metaData": {
        ...
        "searchExecution": [
            {
                "searchTerm": "red dress",
                "searchRule": {},
                "nlp": {
                    "pos": "ADJECTIVE --> [red]",
                    "ner": "CATEGORY --> [dress~dresses] (boosted by 100.0)",
                    "color": "COLOR --> [red]"
                },
                "customFields": {}
            },
            {
                "searchTerm": "dress",
                "searchRule": {},
                "nlp": {
                    "ner": "CATEGORY --> [dress~dresses] (boosted by 100.0)"
                },
                "customFields": {}
            }
        ]
    },
    ...

In this example, the search term red dress initially contains two components, COLOR --> [red] and CATEGORY --> [dress~dresses]. However, the term-dropping process identifies that "red" is a color with higher priority than the category "dress." Consequently, "red" is dropped first, resulting in a refined search term of "dress".

Note: Priorities are given in inverse numerical order. In this case, a priority value of 4 (representing 'COLOR') is higher in priority than a value of 6 (representing 'CATEGORY').

"termDroppingPriority": {
    "1": "FILTER",
    "2": "MEASUREMENT",
    "3": "BRAND",
    "4": "COLOR",
    "5": "ADJECTIVE",
    "6": "CATEGORY",
    "7": "NOUN"
}

Disabling term dropping

customize your NLP profile to disable term-dropping and return no results in case of search misses. Configure the termDroppingPriority node in the NLP profile configuration as follows:

Open your NLP profile configuration file. This can be the default profile or any custom profile you are using.
Locate the termDroppingPriority node.
Set termDroppingPriority to an empty value.
```
"termDroppingPriority": {}
```

By setting termDroppingPriority to an empty value, you indicate that no specific term-dropping priority is defined. This disables term-dropping for search misses No results are returned when a search term does not match any products.

Enhanced term-drop

Set the enableSearchTermDropOperator configuration properly to enhance the term-drop feature in the search functionality. By default, this configuration is disabled. To activate this feature, configure it to true in the Zookeeper. To view the configuration details, see the code snippet given below:

End point : PATCH |  {{baseUrl}}/api/v2/configuration?nodeName=component&envType=auth
{
    "extendedconfiguration": {
        "configgrouping": [
            {
                "name": "SearchConfiguration",
                "property": {
                    "name": "enableSearchTermDropOperator",
                    "value": "true"
                }
            }
        ]
    }
}

Search Term Drop Behaviour after enabling the above property

When the enableSearchTermDropOperator flag is enabled, and no results are found for a requested search term, the system initiates term-dropping based on the NLP (Natural Language Processing) profile termDroppingPriority section, and because the default_operator will be changed to OR and an additional search request will be performed.

Implemented behavior supports BRAND, ADJECTIVE, CATEGORY, and NOUN. This feature works when the search term contains multiple words and is classified within the same classification.

search term : lamp, bulb. Both words are considerd as NOUN.

Classification Priority

The classification priority is based on the priority assigned in the NLP profile. The system follows a specific process to refine the search.

Default Operator Update: Initially, if no results are found, the default_operator updates to OR for a classification based on the term-dropping priority and then a search is performed.
Search Performed Update: To determine the specific search term on which the search is performed, an additional parameter, searchPerformed is added in the metadata in searchExecution along with termDroppingPriority. This will assist the user in distinguishing whether the search is performed with default_operator, OR, or AND.
Subsequent Term-Dropping: If the above steps still yield no results for the same classification, term-dropping is executed precisely for that classification.

Search Term: lamp, bulb

Initially, the search is performed for the lamp AND bulb (when enableSearchTermDropOperator is enabled).
If the ES-query request responds with 0 results, an additional search will be performed.
The default_operator is set to OR during the term-dropping, and the search is performed.
The result for a lamp OR bulb is generated.

The following is an example of searchPerformed in the searchxecution section of the response JSON:

{
    "metaData": {
        ...
        "searchExecution": [
            {
                "searchTerm": "lamp bulb",
                "searchRule": {},
                "nlp": {
                    "pos": "NOUN --> [lamp, bulb]",
                },
                "customFields": {}
            },
            {
                "searchTerm": "lamp bulb",
                "termDropOperator": "{NOUN} --> or",
                "searchPerformed": "lamp or bulb",
                "searchRule": {},
                "nlp": {
                    "pos": "NOUN --> [lamp, bulb]"
                },
                "customFields": {}
            }
        ]
    },

Search term: Delissima, Albini dress less than 10 dollars
- Drop the price filter and search with the AND operator for brands Delissima and Albini.
- Utilizing OR as the default operator.
- This process results in obtaining results for Albini or Delissima dress rather than dropping both brands simultaneously.
Search Term: Delissima Sunshine short elegant dress less than 10 dollars
- Drop the price filter.
- Search with OR operator for brands Delissima and Sunshine.
- Drop both brands.
- Search with the OR operator for both adjectives short and elegant.
- The final results include short or elegant dresses.
Note: This feature adds a search request for a classification to perform the search with an OR condition.