Storage of dimension values

When values are detected, they may be stored based on the following configuration options, which operate independently of each other.

Logging of dimension values

If logging for the dimension is enabled, all observed values for the dimension are stored in the database. Each instance of each value is counted when detected.

Values stored in the dimension logs are removed after two weeks, so the logs can be kept to a manageable size.

These logs can be used to build up whitelists, blacklists, and group lists while keeping down data growth.

Whitelists versus observed values

For the Values to Record setting, you can choose one of the following options:

Option
Description
Whitelist Only
When this option is selected, only the detected values that are matched against the whitelist that you defined and uploaded are allowed to be recorded for the dimension. All other values are recorded as an [others] value or, if the maximum number of values per hour was reached (Max Values Per Hour), the [limit] value.
  • For more information about these dimension constant values, see Dimension constants. For the URL (Normalized) dimension, the goal is to build a representative whitelist of the URLs of the web application. The remainder of this page works towards building a useful whitelist for this dimension.
Whitelist + Observed Values
When this option is selected, values that match whitelisted values are recorded, as well as any other value detected in the capture stream. The net effect is that each URL value for each hit is captured by the URL (Normalized) hit attribute, which then populates the URL (Normalized) dimension. Each of these values is then recorded.
Note: For high-volume dimensions the Whitelist + Observed Values can generate unbounded growth of the dimension tables in the database. This setting can be dangerous to the overall health of the Discover system, as these observed values may not be purged for some time. Particularly for high-volume dimensions, this setting should be avoided. See Data Management for Dimensions.
Note: The maximum number of values that can be contained in a whitelist is 50,000. For a high-volume dimension, you may need to make some decisions about the sample of URLs of your web application that you want to track. Discover provides some guidance, as discussed later.