Search rule experiment statistics
Understanding how the server collects and calculates statistics for the test elements in your search rule experiments can help you evaluate the results of the experiment.
Here is an example of the statistical data that is provided for a search rule experiment:
The following table describes the statistical data:
| Statistic | Description |
|---|---|
| Total number of unique customers | The number of individual customers who participated in the experiment. |
| Unique Customers | The number of individual customers who were assigned the path and reached the test element. |
| Clicks | The number of times any link on the first page
is clicked.1 of
search results was clicked, which indicates that the search results
had relevance. This does not necessarily mean that the customer clicked a link for a product that is directly related to the search rule action. For example, if your search rule action is to boost Brand A products in the search results, the customer does not need to click a Brand A product to increase the Clicks count. If the customer clicks a featured product that is displayed on the search results page, this click is not counted. |
| Click Orders | The number of orders that are placed by customers
who were influenced by the search results. For the order to count in the statistics, the customer must add any item that is listed in the first page1 of search results to the shopping cart from the product display page that displays right after the customer clicks a link on the search results page. |
| Click Revenue | For orders that counted as Click
orders, the total value of all those orders. For example, if a path has a count of 10 Click Orders, and each order totaled $10, the Click Revenue value would be $100. This value is for the entire order, not just for the specific items in the order that were listed on the first page1 of search results. |
1. By default, only the first page of search results is considered in click, order, and revenue statistics. However, a site administrator can change the default value of the SearchStatisticsResultPagesTrackingThreshold parameter to consider a different number of search result pages, for example, the first three pages. |
|
You can view these statistics while the experiment is running and after it ends. See Viewing the statistics of a search rule experiment.
Enabling the collection of revenue statistics
- ExperimentEvaluationEventListener
- OrderSubmissionEvent
Frequency of statistical updates
The following table identifies how often statistics are updated in the Statistics tab while an experiment is running.
| In the Clicks column | The server gathers and displays these numbers using the SaveMarketingStatistics scheduled job. By default, this job runs every 15 minutes. You can change the frequency of this job; see Scheduling the SaveMarketingStatistics job. |
| In the Click Orders and Click Revenue columns | The server gathers and displays these numbers
using the RaiseECEvent scheduled job. By default, this job runs every
5 minutes. You can change the frequency of this job; see Statistics considerations. |
Experiment statistics in a staging environment
Search rule statistics are captured on the production server. If you want to view experiment statistics on a staging server for analysis, a system administrator can transfer the statistical data from the production server to the staging server by running the ExportStats and ImportStats commands. When you run the commands, specify the DMEXPSTATS and DMELESTATS tables. See Copying marketing statistics from the production server.