Monitoring limit configuration

After you enabled logging of dimensions, you can monitor your limit configuration by checking the number of counts of the [Limit] value for the dimension in each hour of reports that are configured to include the dimension.

  • In the Report Builder, you add the dimension to the report and check the counts for the [Limit] row in the detail table.
    Note: The appearance of the [Limit] value consistently in your reports indicates that some values are not being factored into the dimension data for the hours where the limit is reached.

If you are consistently reaching the [Limit] value for the dimension, you should consider raising the limit. Caveats:

  • Raising the limit requires more disk space to store the values. These hourly entries in the database can grow quite large and may not be detected unless you careful monitor the counts of [Limit] values.
  • If you are consistently hitting the limit even after it was raised, the dimension may not be able to reflect an accurate sample of the real data set.

Suppose you configured a dimension to capture a maximum of 1000 values per hour, which is the default setting. In the table below, you can see the logged values are recorded for successive hours.

  • Assume that the values are detected and recorded in the sequence that is listed below. For Hour 0, values are detected in the following order: v0000, v0001, v0002, etc.
Table 1. Monitoring limit configuration
Hour Detected Values Captured Values Values Overwritten with [Limit]
0 v0000 - v1200 v0000 - v0999 v1000 - v1200
1 v1000 - v1200, v0000 - v0999 v1000 - v1200, v0000 - v0799 v0800 - v0999

In Hour 0, the first 1000 values (0-0999) are captured, and all subsequent values are stored as [Limit]. In Hour 1, new values (1000-1200) are detected and captured initially, but then the sequence of values from Hour 0 starts again. However, since the limit is capped at 1000, [Limit] is assigned to the last two hundred values for the hour, even though the values were already captured and recorded in the previous hour. As a result, contextual information for data that is already known to the system is not captured in Hour 1.

Note: If you enabled logging of a dimension that captures a high number of discrete values per hour, you may be challenged to capture and use a useful selection of them. For example, recording IP addresses for a high-volume site may result in explosive log growth and a varying set of values that are recorded as the dimension limit value.

A potential solution for addressing these issues is to do the following:

  • Enable logging for a day or two.
  • Export the list as a text file.
  • Change the dimension to use a whitelist only.
  • Import the export list as your whitelisted values.
    Note: This solution does not work for data that varies significantly over time.